Forecasting incrementally with patsy
还没有人认领这个 Issue。
评估
调研方向
从 dmatrix 和 incr_dbuilder 入口点开始,重现所提供的不平衡面板示例。调查增量评估如何处理跨 chunk 的依赖关系,并确定行选择如何保留滞后行为和面板行为。当针对预测场景定义并演示了所请求的仅行评估时,即视为完成。
由索引模型根据 Issue 内容生成。
描述
This is either a feature request or a request for help with current functionality. I am doing some work with unbalanced panel data work that involves using patsy to forecast some series. Here's a basic example:
import io, pandas, patsy
#raw panel data indexed on ID, YEAR. Y is the forecast variable of interest. There are no gaps in the data for an individual entity but the panel is potentially unbalanced (meaning different start/end dates).
data = '''ID,YEAR,Y,B,C,D
1,1999,0,2,3,4
1,2000,.,2,3,4
1,2001,.,2,3,4
1,2002,.,2,3,4
2,1996,1,2,3,4
2,1997,.,2,3,4
3,1998,3,2,3,4
3,1999,3,2,3,4
3,2000,.,2,3,4
3,2001,3,2,3,4
'''
data = io.StringIO(data)
df = pandas.read_csv(data, index_col=['ID','YEAR'], na_values=['.'])
print(df)
def lag(series, n=1):
return series.groupby(level=0).shift(n)
formula = '1+lag(Y)+B+C+D' #This is the forecast equation for Y
x = patsy.dmatrix(formula,df, return_type='dataframe')
params = pandas.Series([1,2,3,4,5], index=x.columns) #these are the coefficients on the forecast vars
#Now forecast year by year
for yr in range(1997,2010):
ind = df.index.get_level_values('YEAR')==yr
x = patsy.dmatrix(formula,df, return_type='dataframe').reindex(df.index)
x = x.loc[ind]
df.loc[ind, 'Y'] = df.loc[ind, 'Y'].fillna(x@params)
print('================')
print(yr)
print(df)
Note that to produce the entire forecast we need to call dmatrix over and over. The problem that I'm having is that it is quite inefficient to have to call dmatrix on the entire DataFrame repeatedly, but because the forecast formula can contain arbitrary numbers of lags I can't just pass in a df filtered to the current year (or a set number of lags from the current year). What would be ideal is if I could replace
ind = df.index.get_level_values('YEAR')==yr
x = patsy.dmatrix(formula,df, return_type='dataframe').reindex(df.index)
x = x.loc[ind]
with version of dmatrix that takes a boolean rows and only evaluates and returns the rows that are needed
ind = df.index.get_level_values('YEAR')==yr
x = patsy.dmatrix(formula,df, return_type='dataframe', rows=ind) #evaluates only on rows there ind==True and returns a dataframe with only those rows
I thought incr_dbuilder might be able to handle this, but it seems that it expects each chunk returned is completely separate from previous chunks. That won't work in the time series/panel context.
- 主要语言
- Python
- 星标
- 990
- 派生
- 113
- 平均合并
- 2 天 8 小时
- 30 天内合并 PR
- 3
环境准备
这个项目没有提供开发容器、Dockerfile 或贡献指南,环境需要你自己搭建:先看它的 README,通用步骤见我们的新手贡献指南。
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
pydata/patsy 的其他 Issue
-
VarLookupDict should implement __iter__, causes CPython crash when using statsmodels in marimo notebook可能已有人在做 @kouyouqi123 于 173 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 72/100
-
Add link to license in readme可能已有人在做 @zheng-guannan577 于 9 天前认领。 未关闭
难度 1/5 1 小时以内 新手友好度 72/100
-
难度 1/5 1 小时以内 新手友好度 68/100
-
Nullable pandas dtypes crash in categorical handling: "Cannot interpret 'BooleanDtype' as a data type"可能已有人在做 @RaphaelFakhri 于 15 天前认领。 未关闭
难度 3/5 1-2 天 新手友好度 35/100
-
cc() truncates periodic coordinates for integer input with fractional bounds可能已有人在做 @ktz03 于 39 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 35/100
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 64/100
维护者通常 1 天内回复
-
area:cli area:security enhancement priority:medium size:small
难度 2/5 1-3 小时 新手友好度 76/100
RonaldHensbergen/composable-data-stack#1338 ·
维护者通常 1 天内回复
-
area:stdlib effort:S impact:med platform:linux type:bug
难度 2/5 1-3 小时 新手友好度 84/100
维护者通常 1 天内回复
-
automated endpoint-dead needs-curation
难度 2/5 1-3 小时 新手友好度 76/100
almanac-data/climate-almanac#139 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 82/100
UKGovernmentBEIS/inspect_ai#6695 · 1 条评论 ·
维护者通常 1 天内回复