sktime/sktime

Add `random_state` to `statsmodel` adapter

Open

#2,388 opened on Apr 4, 2022

View on GitHub
 (10 comments) (1 reaction) (0 assignees)Python (1,192 forks)batch import
good first issuemodule:forecasting

Repository metrics

Stars
 (7,162 stars)
PR merge metrics
 (Avg merge 26d 10h) (86 merged PRs in 30d)

Description

Requesting the addition of a random_state argument to the statsmodels adapter and passing it appropriately to underlying non-deterministic models such as ETS so that the predictions can be made reproducible. Currently, multiple runs result in different interval results for the same quantiles.

More on this below:


Many statsmodels models are non-deterministic, e.g., as they do some random sampling inside, especially when it comes to estimating variation. Having some intrinsic randomness there is not unusual and expected.

@fkiraly You are right, it looks like in this case, the intervals are based on simulations and those simulations are not exact or repeatable. This model has multiplicative components and hence exact method is not available

https://www.statsmodels.org/stable/_modules/statsmodels/tsa/exponential_smoothing/ets.html

if method is None:
    exact_available = ["ANN", "AAN", "AAdN", "ANA", "AAA", "AAdA"]
    if results.model.short_name in exact_available:
        method = "exact"
    else:
        method = "simulated"
self.method = method

However, they can be made repeatable by passing a random_state inside the sktime ets.py _predict_interval method. What do you think - maybe open an enhancement request for this?

def _predict_interval(self, fh, X=None, coverage=None):
    valid_indices = fh.to_absolute(self.cutoff).to_pandas()

   start, end = valid_indices[[0, -1]]
   # Maybe add random_state in get_predictions (and accept random_state in the AutoETS initialization)?
   # https://www.statsmodels.org/stable/_modules/statsmodels/tsa/exponential_smoothing/ets.html
   prediction_results = self._fitted_forecaster.get_prediction(
       start=start, end=end
   )

Originally posted by @ngupta23 in https://github.com/alan-turing-institute/sktime/issues/2320#issuecomment-1080029680

Contributor guide