Patsy loses DatetimeIndex freq information even if no NA values
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 45/100
Research direction
Start in patsy/missing.py at _handle_NA_drop and reproduce the issue with the pandas DatetimeIndex example in the report. Verify that handling values without missing entries preserves the index frequency, and confirm the example produces matching frequency information afterward.
Written by the indexing model from the issue text.
Description
(This is probably better described as a pandas bug, see https://github.com/pandas-dev/pandas/issues/21282, but maybe patsy wants to patch this too?)
Reproducible example:
(proceeding from the above code)
import pandas as pd
import patsy
index = pd.DatetimeIndex(start='1990', end='1994', freq='AS')
data = pd.Series([0, 1, 2, 3, 4], name='y', index=index)
print(data.index)
lhs, rhs = patsy.dmatrices('y ~ 1', data={'y':data}, return_type='dataframe')
print(lhs.index)
The first print statement yields:
DatetimeIndex(['1990-01-01', '1991-01-01', '1992-01-01', '1993-01-01',
'1994-01-01'],
dtype='datetime64[ns]', freq='AS-JAN')
Whereas the second yields:
DatetimeIndex(['1990-01-01', '1991-01-01', '1992-01-01', '1993-01-01',
'1994-01-01'],
dtype='datetime64[ns]', freq=None)
This is a consequence of https://github.com/pandas-dev/pandas/issues/21282 as it affects the following function in patsy/missing.py:
def _handle_NA_drop(self, values, is_NAs, origins):
total_mask = np.zeros(is_NAs[0].shape[0], dtype=bool)
for is_NA in is_NAs:
total_mask |= is_NA
good_mask = ~total_mask
# "..." to handle 1- versus 2-dim indexing
return [v[good_mask, ...] for v in values]
when v is the DatetimeIndex, the ellipses cause the index to lose frequency information.
- Dominant language
- Python
- Stars
- 990
- Forks
- 106
- Avg merge
- 7d 34m
- Merged PRs (30d)
- 1
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from pydata/patsy
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 1/5 Under an hour Newbie friendliness 72/100
-
Difficulty 1/5 Under an hour Newbie friendliness 68/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 55/100
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
Similar issues
-
documentation help wanted
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
simonw/sqlite-utils#872 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100