Bootstrap prediction interval

Open
  • Is something wrong?
#323 9 comments 0 reactions 1 assignee View on GitHub

Maintainers usually reply within 1 day

@bashtage is already working on this.

Since Nov 9, 2019.

Assessment

This issue has not been assessed yet.

Description

enhancement

I'm trying to bootstrap the prediction interval for an sklearn style non-parametric estimator (specifically catboost regression). I want the prediction interval for the sum of n predictions (the data has time order). I'm using the method from Davison and Hinkley "Bootstrap Methods and their Applications" algorithm 6.4 i.e.

fit the model to X,y with residuals e
repeat R times
  X* = X, e*=sample(e), y*=F(X*) + e*
  fit y*=F'(X*)
  repeat M times
     e+=sample(e,n)
     Yhat-Y+ = sum(F(X*)-(F'(X*)+e+))

As far as I can see, there's no convience wrapper for a prediction interval similar bs.conf_int()?

Also like in https://github.com/bashtage/arch/issues/165 I only want the first n samples for e+ - I assume I still have to use the workaround from this issue?

My main question though is how to implement the two loops? I don't think creating two seperate CircularBlockBootstrap() instances is correct though since the random state of the inner bootstrap will be reset to whatever the default seed is? I think what I should do is create one bootstrap instance and use it as follows:

bs=CircularBlockBootstrap(block_size,e)
R=10
M=10

it = bs.bootstrap(N*(M+1))

for r in range(R):
    pos, _ = next(it)
    estar = pos[0]
    
    for m in range(M):
        pos, _ = next(it)
        eplus = pos[0][:n]

My issue though is when I sample estar and eplus - because the data is blocked I need to ensure the block boundaries are the same - specifically if the block size is one week and X and Xplus don't both start on the same hour of the week then sampling from the same CircularBlockBootstrap() object will result in misalignment of the eplus samples?

Edit: As is usually the case, taking the time to write out the problem gave me enough time to think it out. I compute the offset from the start of e to the same time of week of the start of the X+ and then simply sample as follows:

    eplus = pos[0][offset:n+offset]
Dominant language
Python
Stars
1.6k
Forks
298
Avg merge
14h 46m
Merged PRs (30d)
106

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from bashtage/arch

All issues in bashtage/arch

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.