etna 2.2.0

Highlights

• Add extension with models from statsforecast
• Speed up metrics computation
• Speed up DeepARModel and TFTModel
• Add DeseasonalityTransform
• Add PatchTSModel
• Add new category mode into HolidayTransform
• Add documentation warning about using dill during loading
• Add inverse transformation into predict method of pipelines
• Fix CLI to work with pipeline ensembles

Full changelog

Added

• DeseasonalityTransform (#1307)
• Add extension with models from statsforecast: StatsForecastARIMAModel, StatsForecastAutoARIMAModel, StatsForecastAutoCESModel, StatsForecastAutoETSModel, StatsForecastAutoThetaModel (#1295)
• Notebook feature_selection (#875)
• Implementation of PatchTS model (#1277)

Changed

• Add modes binary and category to HolidayTransform (#763)
• Add sorting by timestamp before the fit in CatBoostPerSegmentModel and CatBoostMultiSegmentModel (#1337)
• Speed up metrics computation by optimizing segment validation, forbid NaNs during metrics computation (#1338)
• Unify errors, warnings and checks in models (#1312)
• Remove upper limitation on version of numba (#1321)
• Optimize TSDataset.describe and TSDataset.info by vectorization (#1344)
• Add documentation warning about using dill during loading (#1346)
• Vectorize metric computation (#1347)

Fixed

• Pipeline ensembles fail in etna forecast CLI (#1331)
• Fix performance of DeepARModel and TFTModel (#1322)
• mrmr feature selection working with categoricals (#1311)
• Fix version of statsforecast to 1.4 to avoid dependency conflicts during installation (#1313)
• Add inverse transformation into predict method of pipelines (#1314)
• Allow saving large pipelines (#1335)
• Fix link for dataset in classification notebook (#1351)

Removed

• Building docker images with cuda 10.2 (#1306)

etna 2.1.0

Highlights

• Add class etna.auto.Tune for tuning hyperparameters
• Extend functionality of class etna.auto.Auto to include a tuning stage
• Add notebook about AutoML
• Add utilities for estimating number of folds for backtesting and forecasting and integrate them into CLI
• Add parameter for setting the start of prediction into CLI
• Add etna.transforms.ExogShiftTransform to shift all exogenous variables
• Add etna.models.DeepStateModel
• Update requirements for holidays, scipy, ruptures, sqlalchemy, tsfresh
• Optimize make_samples of etna.models.RNNNet and etna.models.MLPNet
• Add parameter fast_redundancy in etna.analysis.feature_selection.mrmm and etna.transforms.MRMRFeatureSelectionTransform to speed it up

Full changelog

Added

• Notebook forecast_interpretation.ipynb with forecast decomposition (#1220)
• Exogenous variables shift transform ExogShiftTransform(#1254)
• Parameter start_timestamp to forecast CLI command (#1265)
• DeepStateModel (#1253)
• Function estimate_max_n_folds for folds number estimation (#1279)
• Parameters estimate_n_folds and context_size to forecast and backtest CLI commands (#1284)
• Class Tune for hyperparameter optimization within existing pipeline (#1200)
• Add etna.distributions for using it instead of using optuna.distributions (#1292)

Changed

• Set the default value of final_model to LinearRegression(positive=True) in the constructor of StackingEnsemble (#1238)
• Add microseconds to FileLogger's directory name (#1264)
• Inherit SaveMixin from AbstractSaveable for mypy checker (#1261)
• Update requirements for holidays and scipy, change saving library from pickle to dill in SaveMixin (#1268)
• Update requirement for ruptures, add requirement for sqlalchemy (#1276)
• Optimize make_samples of RNNNet and MLPNet (#1281)
• Remove to_be_fixed from inference tests on SpecialDaysTransform (#1283)
• Rewrite TimeSeriesImputerTransform to work without per-segment wrapper (#1293)
• Add default params_to_tune for catboost models (#1185)
• Add default params_to_tune for ProphetModel (#1203)
• Add default params_to_tune for SARIMAXModel, change default parameters for the model (#1206)
• Add default params_to_tune for linear models (#1204)
• Add default params_to_tune for SeasonalMovingAverageModel, MovingAverageModel, NaiveModel and DeadlineMovingAverageModel (#1208)
• Add default params_to_tune for DeepARModel and TFTModel (#1210)
• Add default params_to_tune for HoltWintersModel, HoltModel and SimpleExpSmoothingModel (#1209)
• Add default params_to_tune for RNNModel and MLPModel (#1218)
• Add default params_to_tune for DateFlagsTransform, TimeFlagsTransform, SpecialDaysTransform and FourierTransform (#1228)
• Add default params_to_tune for MedianOutliersTransform, DensityOutliersTransform and PredictionIntervalOutliersTransform (#1231)
• Add default params_to_tune for TimeSeriesImputerTransform (#1232)
• Add default params_to_tune for DifferencingTransform, MedianTransform, MaxTransform, MinTransform, QuantileTransform, StdTransform, MeanTransform, MADTransform, MinMaxDifferenceTransform, SumTransform, BoxCoxTransform, YeoJohnsonTransform, MaxAbsScalerTransform, MinMaxScalerTransform, RobustScalerTransform and StandardScalerTransform (#1233)
• Add default params_to_tune for LabelEncoderTransform (#1242)
• Add default params_to_tune for ChangePointsSegmentationTransform, ChangePointsTrendTransform, ChangePointsLevelTransform, TrendTransform, LinearTrendTransform, TheilSenTrendTransform and STLTransform (#1243)
• Add default params_to_tune for TreeFeatureSelectionTransform, MRMRFeatureSelectionTransform and GaleShapleyFeatureSelectionTransform (#1250)
• Add tuning stage into Auto.fit (#1272)
• Add params_to_tune into Tune init (#1282)
• Skip duplicates during Tune.fit, skip duplicates in top_k, add AutoML notebook (#1285)
• Add parameter fast_redundancy in mrmm, fix relevance calculation in get_model_relevance_table (#1294)

Fixed

• Fix plot_backtest and plot_backtest_interactive on one-step forecast (1260)
• Fix BaseReconciliator to work on pandas==1.1.5 (#1229)
• Fix TSDataset.make_future to handle hierarchy, quantiles, target components (#1248)
• Fix warning during creation of ResampleWithDistributionTransform (#1230)
• Add deep copy for copying attributes of TSDataset (#1241)
• Add tsfresh into optional dependencies, remove instruction about pip install tsfresh (#1246)
• Fix DeepARModel and TFTModel to work with changed prediction_size (#1251)
• Fix problems with flake8 B023 (#1252)
• Fix problem with swapped forecast methods in HierarchicalPipeline (#1259)
• Fix problem with segment name "target" in StackingEnsemble (#1262)
• Fix BasePipeline.forecast when prediction intervals are estimated on history data with presence of NaNs (#1291)
• Teach BaseMixin.set_params to work with nested list and tuple (#1201)
• Fix get_anomalies_prediction_interval to work when segments have different start date (#1296)
• Fix classification notebook to download FordA dataset without error (#1299)
• Fix signature of Auto.fit, Tune.fit to not have a breaking change (#1300)

etna 2.0.0

Breaking changes

• Transforms now works with TSDataset instead of DataFrames:
  • Methods fit, transform and inverse_transform of transforms expect TSDataset as input
  • Transforms are not stored inside TSDataset now and should be explicitly passed into the methods fit_transform, make_future, inverse_transform
  • Forecasts from the models should be inverse transformed by the user now
• New workflow for NNs from PyTorch Forecasting, see notebook for details
• Remove some classes and methods:
  • BinsegTrendTransform - replaced with ChangePointsTrendTransform
  • sample_acf_plot, sample_pacf_plot - replaced with acf_plot
  • CatBoostModelPerSegment, CatBoostModelMultiSegment - redundant classes, CatBoostPerSegmentModel, CatBoostMultiSegmentModel are still available
  • PytorchForecastingTransform - see new workflow for NNs from PyTorch Forecasting
• Remove support of Python 3.7

Highlights:

• Add forecast decomposition for all the classical ML models, see return_components parameter in methods forecast and predict. Notebook with examples will be published soon
• Part of transforms and models are now able to work on new segments and on future data without refitting
• New backtesting strategies, see parameters refit and stride in method backtest

Full changelog:

Added

• Target components logic into AutoRegressivePipeline (#1188)
• Target components logic into HierarchicalPipeline (#1199)
• predict method into HierarchicalPipeline (#1199)
• Add target components handling in get_level_dataframe (#1179)
• Forecast decomposition for SeasonalMovingAverageModel(#1180)
• Target components logic into base classes of pipelines (#1173)
• Method predict_components for forecast decomposition in _SklearnAdapter and _LinearAdapter for linear models (#1164)
• Target components logic into base classes of models (#1158)
• Target components logic to TSDataset (#1153)
• Methods save and load to HierarchicalPipeline (#1096)
• New data access methods in TSDataset : update_columns_from_pandas, add_columns_from_pandas, drop_features (#809)
• PytorchForecastingDatasetBuiler for neural networks from Pytorch Forecasting (#971)
• New base classes for per-segment and multi-segment transforms IrreversiblePersegmentWrapper, ReversiblePersegmentWrapper, IrreversibleTransform, ReversibleTransform (#835)
• New base class for one segment transforms OneSegmentTransform (#894)
• ChangePointsLevelTransform and base classes PerIntervalModel, BaseChangePointsModelAdapter for per-interval transforms (#998)
• Method set_params to change parameters of ETNA objects (#1102)
• Function plot_forecast_decomposition (#1129)
• Method forecast_components for forecast decomposition in _TBATSAdapter (#1133)
• Methods forecast_components and predict_components for forecast decomposition in _CatBoostAdapter (#1148)
• Methods forecast_components and predict_components for forecast decomposition in _HoltWintersAdapter (#1162)
• Method predict_components for forecast decomposition in _ProphetAdapter (#1172)
• Methods forecast_components and predict_components for forecast decomposition in _SARIMAXAdapter and _AutoARIMAAdapter (#1174)
• Add refit parameter into backtest (#1159)
• Add stride parameter into backtest (#1165)
• Add optional parameter ts into forecast method of pipelines (#1071)
• Add tests on transform method of transforms on subset of segments, on new segments, on future with gap (#1094)
• Add tests on inverse_transform method of transforms on subset of segments, on new segments, on future with gap (#1127)
• In-sample prediction for BATSModel and TBATSModel (#1181)
• Method predict_components for forecast decomposition in _TBATSAdapter (#1181)
• Forecast decomposition for DeadlineMovingAverageModel(#1186)
• Prediction decomposition example into custom_transform_and_model.ipynb(#1216)

Changed

• Add optional features parameter in the signature of TSDataset.to_pandas, TSDataset.to_flatten (#809)
• Signature of the constructor of TFTModel, DeepARModel (#1110)
• Interface of Transform and PerSegmentWrapper (#835)
• Signature of TSDataset methods inverse_transform and make_future now has transforms parameter. Remove transforms and regressors updating logic from TSDataset. Forecasts from the models are not internally inverse transformed. Methods fit,transform,inverse_transform of Transform now works with TSDataset (#956)
• Create AutoBase and AutoAbstract classes, some of Auto class's logic moved there (#1114)
• Impose specific order of columns on return value of TSDataset.to_flatten (#1095)
• Add more scenarios into tests for models (#1082)
• Decouple SeasonalMovingAverageModel from PerSegmentModelMixin (#1132)
• Decouple DeadlineMovingAverageModel from PerSegmentModelMixin (#1140)
• Remove version python-3.7 from pyproject.toml, update lock (#1183)
• Bump minimum pandas version up to 1.1 (#1214)

Fixed

• Fix bug in GaleShapleyFeatureSelectionTransform with wrong number of remaining features (#1110)
• ProphetModel fails with additional seasonality set (#1157)
• Fix inference tests on new segments for DeepARModel and TFTModel (#1109)
• Fix alignment during forecasting in new NNs, add validation of context size during forecasting in new NNs, add validation of batch in MLPNet (#1108)
• Fix MeanSegmentEncoderTransform to work with subset of segments and raise error on new segments (#1104)
• Fix outliers transforms on future with gap (#1147)
• Fix SegmentEncoderTransform to work with subset of segments and raise error on new segments (#1103)
• Fix SklearnTransform in per-segment mode to work on subset of segments and raise error on new segments (#1107)
• Fix OutliersTransform and its children to raise error on new segments (#1139)
• Fix DifferencingTransform to raise error on new segments during transform and inverse_transform in inplace mode (#1141)
• Teach DifferencingTransform to inverse_transform with NaNs (#1155)
• Fixed custom_transform_and_model.ipynb(#1216)

Removed

• sample_acf_plot, sample_pacf_plot, CatBoostModelPerSegment, CatBoostModelMultiSegment (#1118)
• PytorchForecastingTransform (#971)

Release 1.15.1

Full changelog:

Changed

• Impose specific order of columns on return value of TSDataset.to_flatten (#1095)

Fixed

• Fix bug in GaleShapleyFeatureSelectionTransform with wrong number of remaining features (#1110)

etna 1.15.0

Highlights:

• Add saving/loading for transforms, models, pipelines, ensembles; tutorial for saving/loading (#1068)
• Add hierarchical time series support(#1083)

Full changelog:

Added

• RMSE metric & rmse functional metric (#1051)
• MaxDeviation metric & max_deviation functional metric (#1061)
• Add saving/loading for transforms, models, pipelines, ensembles; tutorial for saving/loading (#1068)
• Add hierarchical time series support(#1083)
• Add WAPE metric & wape functional metric (#1085)

Fixed

• Missed kwargs in TFT init(#1078)

etna 1.14.0

Highlights:

• Add python 3.10 support (#1005)
• Add experimental module with TimeSeriesBinaryClassifier and PredictabilityAnalyzer (#985), see example notebook for the ditails (#997)
• Inference track results: add predict method to pipelines, teach some models to work with context, change hierarchy of base models, update notebook examples (#979)

Full changelog:

Added

• Add python 3.10 support (#1005)
• Add SumTranform(#1021)
• Add plot_change_points_interactive (#988)
• Add experimental module with TimeSeriesBinaryClassifier and PredictabilityAnalyzer (#985)
• Inference track results: add predict method to pipelines, teach some models to work with context, change hierarchy of base models, update notebook examples (#979)
• Add get_ruptures_regularization into experimental module (#1001)
• Add example classification notebook for experimental classification feature (#997)

Changed

• Change returned model in get_model of BATSModel, TBATSModel (#987)
• Add acf_plot, deprecated sample_acf_plot, sample_pacf_plot (#1004)
• Change returned model in get_model of HoltWintersModel, HoltModel, SimpleExpSmoothingModel (#986)

Fixed

• Fix MinMaxDifferenceTransform import (#1030)
• Fix release docs and docker images cron job (#982)
• Fix forecast first point with CatBoostPerSegmentModel (#1010)
• Fix hanging EDA notebook (#1027)
• Fix hanging EDA notebook v2 + cache clean script (#1034)

etna 1.13.0

Highlights:

etna.auto module for pipeline greedy search with default pipelines pool
wandb sweeps and optuna examples

Full changelog:

Added

• Add greater_is_better property for Metric (#921)
• etna.auto for greedy search, etna.auto.pool with default pipelines, etna.auto.optuna wrapper for optuna (#895)
• Add MinMaxDifferenceTransform (#955)
• Add wandb sweeps and optuna examples (#338)

Changed

• Make slicing faster in TSDataset._merge_exog, FilterFeaturesTransform, AddConstTransform, LambdaTransform, LagTransform, LogTransform, SklearnTransform, WindowStatisticsTransform; make CICD test different pandas versions (#900)
• Mark some tests as long (#929)
• Fix to_dict with nn models and add unsafe conversion for callbacks (#949)

Fixed

• Fix to_dict with function as parameter (#941)
• Fix native networks to work with generated future equals to horizon (#936)
• Fix SARIMAXModel to work with exogenous data on pmdarima>=2.0 (#940)
• Teach catboost to work with encoders (#957)

etna 1.12.0

Highlights:

• ETNA native MLPModel
• to_dict method in all the etna objects
• DirectEnsemble implementing the direct forecasting strategy
• Notebook about forecasting strategies

Full changelog:

Added

• Function to transform etna objects to dict(#818)
• MLPModel(#860)
• DeadlineMovingAverageModel (#827)
• DirectEnsemble (#824)
• CICD: untaged docker image cleaner (#856)
• Notebook about forecasting strategies (#864)
• Add ChangePointSegmentationTransform, RupturesChangePointsModel (#821)

Changed

• Teach AutoARIMAModel to work with out-sample predictions (#830)
• Make TSDataset.to_flatten faster for big datasets (#848)

Fixed

• Type hints for external users by PEP 561 (#868)
• Type hints for Pipeline.model match models.nn(#768)
• Fix behavior of SARIMAXModel if simple_differencing=True is set (#837)
• Bug python3.7 and TypedDict import (867)
• Fix deprecated pytorch lightning trainer flags (#866)
• ProphetModel doesn't work with cap and floor regressors (#842)
• Fix problem with encoding category types in OHE (#843)
• Change Docker cuda image version from 11.1 to 11.6.2 (#838)
• Optimize time complexity of determine_num_steps(#864)
• All warning as errors(#880)
• Update .gitignore with .DS_Store and checkpoints (#883)
• Delete ROADMAP.md ([#904]#904)
• Fix ci invalid cache (#896)

etna 1.11.1

Full changelog:

Fixed

• Fix missing constant_value in TimeSeriesImputerTransform (#819)
• Make in-sample predictions of SARIMAXModel non-dynamic in all cases (#812)
• Add known_future to cli docs (#823)

etna 1.11.0

Highlights:

• ETNA native RNN and base classes for deep learning models
• Lambda transform
• Prophet 1.1 support without c++ compiler dependency
• Prediction intervals for DeepAR and TFTModel
• Add known_future parameter to CLI

Full changelog:

Added

• LSTM based RNN and native deep models base classes (#776)
• Lambda transform (#762)
• assemble pipelines (#774)
• Tests on in-sample, out-sample predictions with gap for all models (#785)

Changed

• Add columns and mode parameters in plot_correlation_matrix (#726)
• Add CatBoostPerSegmentModel and CatBoostMultiSegmentModel classes, deprecate CatBoostModelPerSegment and CatBoostModelMultiSegment (#779)
• Allow Prophet update to 1.1 (#799)
• Make LagTransform, LogTransform, AddConstTransform vectorized (#756)
• Improve the behavior of plot_feature_relevance visualizing p-values (#795)
• Update poetry.core version (#780)
• Make native prediction intervals for DeepAR (#761)
• Make native prediction intervals for TFTModel (#770)
• Test cases for testing inference of models (#794)
• Wandb.log to WandbLogger (#816)

Fixed

• Fix missing prophet in docker images (#767)
• Add known_future parameter to CLI (#758)
• FutureWarning: The frame.append method is deprecated. Use pandas.concat instead (#764)
• Correct ordering if multi-index in backtest (#771)
• Raise errors in models.nn if they can't make in-sample and some cases out-sample predictions (#813)
• Teach BATS/TBATS to work with in-sample, out-sample predictions correctly (#806)
• Github actions cache issue with poetry update (#778)