Data interpolation function treats missing values as zero instead of throwing an error

[chauenstein]

Hi @stefanpauliuk @CarrerF, the automatic data processing (commit 455e89f) is of great help. One issue, however, arises, if in the config file in the cell for 'Data processing method' the given argument refers to a year not provided in the input data file. E.g., if it set to ['interpolate_c_1900_2060_linear'], however, the last year data is provided for in the input file is 2020, it assumes that all values for the year 2060 are zero. This might be true in some specific cases, however, probably better would be if an error is thrown and the user requested to either provide the relevant data in the input file or change the data processing method for this parameter.

[CarrerF]

Hi @chauenstein,

Yes, interpolate refers only to filling the gaps in the given range, and does not perform extrapolation in the future.
For that, one can use the copy or replicate methods. One of them works at product level, the other can extrapolate in the both time t or c. I cannot remember which one would be correct here, but they were both documented at the time I implemented them.

One can then provide both the interpolation and extrapolation methods as a list in the config. I no longer have access to the data repositories, but if I remember correctly some datasets are already processed like this.

[chauenstein]

Thanks @CarrerF for your quick reply! Yes, there are examples for the extrapolation etc. Yet, what I meant with the issue above is, that in case an extrapolation would be the correct method, yet interpolation was chosen, the script does not throw an error if the target year of the interpolation does not exist.

[CarrerF]

I see.
To some extent, it was assumed that the config has been correctly set up and only a little proofreading has been added, and definitely not enough to cover all the possible deviations from the intended use.

The Read Parameter functions create an array of Os and then fill it with the values from the templates, before preprocessing.
The feature you propose could be readily implemented by adding a else instance to the if statement currently starting at line 1253.

if: (ValIns_b[indices[:ix_position] + (startIndex,) + indices[ix_position:]] and ValIns_b[indices[:ix_position] + (endIndex,) + indices[ix_position:]])

checks that the starting and ending indexes of interpolation belong to the values from the template.
The quickest implementation of the feature would be throwing an error in its else instance.

[chauenstein]

Thank you @CarrerF! Is on the todo list for next year :) Happy holidays to you!