Repository metrics
- Stars
- (7,162 stars)
- PR merge metrics
- (Avg merge 26d 10h) (86 merged PRs in 30d)
Description
Requesting the addition of a random_state argument to the statsmodels adapter and passing it appropriately to underlying non-deterministic models such as ETS so that the predictions can be made reproducible. Currently, multiple runs result in different interval results for the same quantiles.
Many statsmodels models are non-deterministic, e.g., as they do some random sampling inside, especially when it comes to estimating variation. Having some intrinsic randomness there is not unusual and expected.
@fkiraly You are right, it looks like in this case, the intervals are based on simulations and those simulations are not exact or repeatable. This model has multiplicative components and hence exact method is not available
https://www.statsmodels.org/stable/_modules/statsmodels/tsa/exponential_smoothing/ets.html
if method is None:
exact_available = ["ANN", "AAN", "AAdN", "ANA", "AAA", "AAdA"]
if results.model.short_name in exact_available:
method = "exact"
else:
method = "simulated"
self.method = method
However, they can be made repeatable by passing a random_state inside the sktime ets.py _predict_interval method. What do you think - maybe open an enhancement request for this?
def _predict_interval(self, fh, X=None, coverage=None):
valid_indices = fh.to_absolute(self.cutoff).to_pandas()
start, end = valid_indices[[0, -1]]
# Maybe add random_state in get_predictions (and accept random_state in the AutoETS initialization)?
# https://www.statsmodels.org/stable/_modules/statsmodels/tsa/exponential_smoothing/ets.html
prediction_results = self._fitted_forecaster.get_prediction(
start=start, end=end
)
Originally posted by @ngupta23 in https://github.com/alan-turing-institute/sktime/issues/2320#issuecomment-1080029680