How to compare GARCH volatility forecast performance?

Open
  • Is something wrong?
#607 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
25/100
Issue type
Documentation
Clarity
Needs clarification
Activity status
Stale
Tech stack
python
Domain
documentation

Research direction

Start with the linked volatility-forecasting documentation and the arch_model / ARCHModelForecast APIs named in the issue. Clarify what variance represents and what target the author should use for evaluating forecasts; done means the three questions about interpretation, MSE, and the reported fit are answered.

Written by the indexing model from the issue text.

Description

I have a question regarding the evaluation of volatility forecasts using the arch_model class. I am very new, so I sincerely apologise if this is obvious, or if this is the wrong place to ask.

I want to forecast the daily volatility (for a test set of roughly 1000 obs) of stock returns using a GARCH model, and concretely using the arch package. After doing so, I want to measure the forecasting performance using the mean squared error.

I construct a model using mean adjusted and scaled returns arch_model(x_var, mean="Zero", vol="Garch", p=1, q=1) and I fit the model using model.fit(last_obs="2018-07-03", update_freq=10) to produce a fixed window forecast as in the tutorial in the documentation.

What I am having trouble with is:

  1. Correctly understanding the ARCHModelForecast object. Is it correct that the variance attribute is the prediction for the daily volatility?
  2. If yes, what value do I compare it to to calculate the MSE? My understanding for GARCH models was that you use squared returns as a proxy for volatility, but my model forecasts are nowhere near the returns, squared returns, or any other feature I have in my data set. Is it correct that I would compare the forecast values to the squared returns at the according time step to evaluate prediction performance?
  3. If yes, is the model I fitted just garbage? I copied the output for model_fit.summary() below. Any sugestions for improvements?

Before I go fiddle with different parameters, I want to make sure that my understanding of the concepts is correct.

I can post more code or an example of my dataset if this helps. Many thanks for your help!

Zero Mean - GARCH Model Results                         
=================================================================================
Dep. Variable:     Returns mean adjusted   R-squared:                       0.000
Mean Model:                    Zero Mean   Adj. R-squared:                  0.000
Vol Model:                         GARCH   Log-Likelihood:               -8545.06
Distribution:                     Normal   AIC:                           17096.1
Method:               Maximum Likelihood   BIC:                           17115.1
                                           No. Observations:                 4110
Date:                   Thu, Sep 01 2022   Df Residuals:                     4110
Time:                           11:40:24   Df Model:                            0
                              Volatility Model                             
===========================================================================
                 coef    std err          t      P>|t|     95.0% Conf. Int.
---------------------------------------------------------------------------
omega          0.0454  6.394e-02      0.710      0.478 [-7.994e-02,  0.171]
alpha[1]       0.0612  5.064e-02      1.209      0.227 [-3.804e-02,  0.160]
beta[1]        0.9319  6.049e-02     15.406  1.500e-53    [  0.813,  1.050]
===========================================================================

Covariance estimator: robust
Dominant language
Python
Stars
1.6k
Forks
298
Avg merge
14h 46m
Merged PRs (30d)
106

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from bashtage/arch

All issues in bashtage/arch

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.