TSA - filtered VS unfiltered

[thielfab]

Hi,

I was testing different filtering and interpolation settings in the force-higher-level TSA module. Primarily I compared filtered (ABOVE_NOISE = 3 BELOW_NOISE = 1) and unfiltered/nofilter (ABOVE_NOISE = 0 BELOW_NOISE = 0) TSS.
See an example time series below (one location/pixel; you can ignore the rbf interpolation). What I found strange is that there are dates where there is a value in the filtered TSS, but not in the unfiltered (i.e., "square only" symbol).

I assumed my R code was wrong, but I checked some samples in QGIS and the extracts seem to be valid.

Any idea what is happening there?
The parameter files of filtered and nofilter are identical except for DIR_HIGHER, ABOVE_NOISE and BELOW_NOISE. Dates and sensor info are derived from band description, not FORCE metadata domain (although that shouldn't make a difference?).

prm filtered:

FORCE_param=DIR_LOWER: /home/level2, DIR_HIGHER: /home/higher_level/tsa_test/tsa_filter, DIR_MASK: NULL, BASE_MASK: NULL, FILE_TILE: NULL, X_TILE_RANGE: 9 9, Y_TILE_RANGE: 6 6, RESOLUTION: 10.000000, BLOCK_SIZE: 0.000000, OUTPUT_FORMAT: 1, OUTPUT_EXPLODE: 0, NTHREAD_READ: 1, NTHREAD_WRITE: 1, NTHREAD_COMPUTE: 1, SENSORS: 5 6, SPECTRAL_ADJUST: 0, SCREEN_QAI: 0 1 2 3 4 5 10 11, DATE_RANGE: 2016-01-01 2020-12-31, DOY_RANGE: 1 365, REDUCE_PSF: 0, USE_L2_IMPROPHE: 0, ABOVE_NOISE: 3.000000, BELOW_NOISE: 1.000000, INDEX: 10, STANDARDIZE_TSS: 0, OUTPUT_TSS: 1, FILE_ENDMEM: NULL, SMA_SUM_TO_ONE: 1, SMA_NON_NEG: 1, SMA_SHD_NORM: 1, SMA_ENDMEMBER: 1, OUTPUT_RMS: 0, INTERPOLATE: 0, MOVING_MAX: 16, RBF_SIGMA: 8 16 32, RBF_CUTOFF: 0.950000, INT_DAY: 16, STANDARDIZE_TSI: 0, OUTPUT_TSI: 0, STM: 32 57 82 4, OUTPUT_STM: 0, FOLD_TYPE: 4, STANDARDIZE_FOLD: 0, OUTPUT_FBY: 0, OUTPUT_FBQ: 0, OUTPUT_FBM: 0, OUTPUT_FBW: 0, OUTPUT_FBD: 0, OUTPUT_TRY: 0, OUTPUT_TRQ: 0, OUTPUT_TRM: 0, OUTPUT_TRW: 0, OUTPUT_TRD: 0, OUTPUT_CAY: 0, OUTPUT_CAQ: 0, OUTPUT_CAM: 0, OUTPUT_CAW: 0, OUTPUT_CAD: 0, LSP_DOY_PREV_YEAR: 273, LSP_DOY_NEXT_YEAR: 91, LSP_HEMISPHERE: 0, LSP_N_SEGMENT: 4, LSP_AMP_THRESHOLD: 0.200000, LSP_MIN_VALUE: 500.000000, LSP_MIN_AMPLITUDE: 500.000000, LSP: 10 12 14 17 24 21, STANDARDIZE_LSP: 0, OUTPUT_SPL: 0, OUTPUT_LSP: 0, OUTPUT_TRP: 0, OUTPUT_CAP: 0, POL_START_THRESHOLD: 0.200000, POL_MID_THRESHOLD: 0.500000, POL_END_THRESHOLD: 0.800000, POL_ADAPTIVE: 1, POL: 15 14 17 23 37 32, STANDARDIZE_POL: 0, OUTPUT_PCT: 0, OUTPUT_POL: 0, OUTPUT_TRO: 0, OUTPUT_CAO: 0, TREND_TAIL: 1, TREND_CONF: 0.950000, CHANGE_PENALTY: 0, FILE_PYTHON: NULL, PYTHON_TYPE: 0, OUTPUT_PYP: 0`

prm unfiltered:

FORCE_param=DIR_LOWER: /home/level2, DIR_HIGHER: /home/higher_level/tsa_test/tsa_nofilter, DIR_MASK: NULL, BASE_MASK: NULL, FILE_TILE: NULL, X_TILE_RANGE: 9 9, Y_TILE_RANGE: 6 6, RESOLUTION: 10.000000, BLOCK_SIZE: 0.000000, OUTPUT_FORMAT: 1, OUTPUT_EXPLODE: 0, NTHREAD_READ: 1, NTHREAD_WRITE: 1, NTHREAD_COMPUTE: 1, SENSORS: 5 6, SPECTRAL_ADJUST: 0, SCREEN_QAI: 0 1 2 3 4 5 10 11, DATE_RANGE: 2016-01-01 2020-12-31, DOY_RANGE: 1 365, REDUCE_PSF: 0, USE_L2_IMPROPHE: 0, ABOVE_NOISE: 0.000000, BELOW_NOISE: 0.000000, INDEX: 10, STANDARDIZE_TSS: 0, OUTPUT_TSS: 1, FILE_ENDMEM: NULL, SMA_SUM_TO_ONE: 1, SMA_NON_NEG: 1, SMA_SHD_NORM: 1, SMA_ENDMEMBER: 1, OUTPUT_RMS: 0, INTERPOLATE: 0, MOVING_MAX: 16, RBF_SIGMA: 8 16 32, RBF_CUTOFF: 0.950000, INT_DAY: 16, STANDARDIZE_TSI: 0, OUTPUT_TSI: 0, STM: 32 57 82 4, OUTPUT_STM: 0, FOLD_TYPE: 32, STANDARDIZE_FOLD: 0, OUTPUT_FBY: 1, OUTPUT_FBQ: 0, OUTPUT_FBM: 0, OUTPUT_FBW: 0, OUTPUT_FBD: 0, OUTPUT_TRY: 0, OUTPUT_TRQ: 0, OUTPUT_TRM: 0, OUTPUT_TRW: 0, OUTPUT_TRD: 0, OUTPUT_CAY: 0, OUTPUT_CAQ: 0, OUTPUT_CAM: 0, OUTPUT_CAW: 0, OUTPUT_CAD: 0, LSP_DOY_PREV_YEAR: 273, LSP_DOY_NEXT_YEAR: 91, LSP_HEMISPHERE: 0, LSP_N_SEGMENT: 4, LSP_AMP_THRESHOLD: 0.200000, LSP_MIN_VALUE: 500.000000, LSP_MIN_AMPLITUDE: 500.000000, LSP: 10 12 14 17 24 21, STANDARDIZE_LSP: 0, OUTPUT_SPL: 0, OUTPUT_LSP: 0, OUTPUT_TRP: 0, OUTPUT_CAP: 0, POL_START_THRESHOLD: 0.200000, POL_MID_THRESHOLD: 0.500000, POL_END_THRESHOLD: 0.800000, POL_ADAPTIVE: 1, POL: 15 14 17 23 37 32, STANDARDIZE_POL: 0, OUTPUT_PCT: 0, OUTPUT_POL: 0, OUTPUT_TRO: 0, OUTPUT_CAO: 0, TREND_TAIL: 1, TREND_CONF: 0.950000, CHANGE_PENALTY: 0, FILE_PYTHON: NULL, PYTHON_TYPE: 0, OUTPUT_PYP: 0

PS: Pixel profile location is in southern india, so don't look for central european phenology here ;-)

[davidfrantz]

Hi Fabian,

the BELOW_NOISE parameter is an "inlier restoration" option. It restores data that are flagged as cloud if they appear to be similar to the bracketing valid observations (lowest wavelength band is used for this, thus in case of NDVI: red).

[thielfab]

Thanks David, now it all makes sense :-)