Intraday code gets confused by daylight savings timezone cutover

Open
#693 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
pandas, python
Domain
api, data

Research direction

Reproduce the get_dataframe call from the report and inspect tiingo/api.py around lines 212-217, along with _request_pandas where the index timezone is handled. Verify the behavior across the provided pre- and post-DST timestamps; done means the request no longer raises AttributeError and JSON and CSV data retain equivalent timezone handling.

Written by the indexing model from the issue text.

Description

bug has-workaround
  • Tiingo Python version: 0.5.0
  • Python version: 3.9
  • Operating System: Debian 11
Description

Was trying to download IEX intraday by walking backward through dates.

What I Did
In [111]: q = c.get_dataframe('AAPL', endDate='2021-04-09T12:16:00.000000000', startDate='2021-02-08 12:16:00', fmt='csv', frequency='1Min')
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-111-384784fbb5a0> in <module>
----> 1 q = c.get_dataframe('AAPL', endDate='2021-04-09T12:16:00.000000000', startDate='2021-02-08 12:16:00', fmt='csv', frequency='1Min')

~/Downloads/ib-historical-data/.venv/lib/python3.9/site-packages/tiingo/api.py in get_dataframe(self, tickers, startDate, endDate, metric_name, frequency, fmt)
    292         if pandas_is_installed:
    293             if type(tickers) is str:
--> 294                 prices = self._request_pandas(
    295                     ticker=tickers, params=params, metric_name=metric_name)
    296             else:

~/Downloads/ib-historical-data/.venv/lib/python3.9/site-packages/tiingo/api.py in _request_pandas(self, ticker, metric_name, params)
    203         # Localize to UTC to ensure equivalence between data returned in json format and
    204         # csv format. Tiingo daily data requested in csv format does not include a timezone.
--> 205         if prices.index.tz is None:
    206             prices.index = prices.index.tz_localize('UTC')
    207 

AttributeError: 'Index' object has no attribute 'tz'

This happens because when the code tries to convert the index to_datetime, pandas returns a plain Index:

https://github.com/hydrosquall/tiingo-python/blob/6b032a0219290e1c37878687f2316c6c4c6cdb10/tiingo/api.py#L212-L217

...presumably because it can't work out the correct timezone of the data, since it has the DST cutover, sample:

2021-03-12 15:55:00-05:00,120.78,121.085,120.78,120.94,22801.0
2021-03-12 16:00:00-05:00,120.94,120.94,120.94,120.94,0.0
2021-03-15 09:30:00-04:00,121.4,121.49,120.425,120.9,39581.0
2021-03-15 09:35:00-04:00,120.885,121.33,120.83,120.96,23986.0
Dominant language
Python
Stars
319
Forks
58
PR merge metrics
No merged PRs in 30d

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from hydrosquall/tiingo-python

All issues in hydrosquall/tiingo-python

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.