sktime/sktime

Add `random_state` to `statsmodel` adapter

Open

#2.388 aberto em 4 de abr. de 2022

Ver no GitHub
 (10 comments) (1 reaction) (0 assignees)Python (1.192 forks)batch import
good first issuemodule:forecasting

Métricas do repositório

Stars
 (7.162 stars)
Métricas de merge de PR
 (Mesclagem média 26d 10h) (86 fundiu PRs em 30d)

Description

Requesting the addition of a random_state argument to the statsmodels adapter and passing it appropriately to underlying non-deterministic models such as ETS so that the predictions can be made reproducible. Currently, multiple runs result in different interval results for the same quantiles.

More on this below:


Many statsmodels models are non-deterministic, e.g., as they do some random sampling inside, especially when it comes to estimating variation. Having some intrinsic randomness there is not unusual and expected.

@fkiraly You are right, it looks like in this case, the intervals are based on simulations and those simulations are not exact or repeatable. This model has multiplicative components and hence exact method is not available

https://www.statsmodels.org/stable/_modules/statsmodels/tsa/exponential_smoothing/ets.html

if method is None:
    exact_available = ["ANN", "AAN", "AAdN", "ANA", "AAA", "AAdA"]
    if results.model.short_name in exact_available:
        method = "exact"
    else:
        method = "simulated"
self.method = method

However, they can be made repeatable by passing a random_state inside the sktime ets.py _predict_interval method. What do you think - maybe open an enhancement request for this?

def _predict_interval(self, fh, X=None, coverage=None):
    valid_indices = fh.to_absolute(self.cutoff).to_pandas()

   start, end = valid_indices[[0, -1]]
   # Maybe add random_state in get_predictions (and accept random_state in the AutoETS initialization)?
   # https://www.statsmodels.org/stable/_modules/statsmodels/tsa/exponential_smoothing/ets.html
   prediction_results = self._fitted_forecaster.get_prediction(
       start=start, end=end
   )

Originally posted by @ngupta23 in https://github.com/alan-turing-institute/sktime/issues/2320#issuecomment-1080029680

Guia do colaborador