query_pandas() fails with HOUR_TIMESTAMP dimension due to timezone mismatch
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 74/100
Research direction
Start in src/shopifyql/results.py at ShopifyQLPandasResult.TYPES_MAP and from_table_data, especially lines 50-53 cited in the issue. Reproduce query_pandas() with an HOUR_TIMESTAMP dimension and inspect how sub-day timestamp strings are cast. Done means the query completes without the timezone ValueError and the returned timestamp columns retain a consistent timezone-aware representation.
Written by the indexing model from the issue text.
Description
Issue
client.query_pandas() raises a ValueError when the query groups by hour (or any sub-day timeseries dimension).
The library's ShopifyQLPandasResult.TYPES_MAP maps HOUR_TIMESTAMP → "datetime64[ns]" (timezone-naive), but the Shopify API returns timezone-aware ISO 8601 strings for that column (e.g. "2026-05-01T13:00:00+00:00"). When df.astype(data_types) is called inside ShopifyQLPandasResult.from_table_data, pandas refuses to cast a tz-aware string into a tz-naive dtype.
Swapping the dimension to day works without error, because DAY_TIMESTAMP values appear to be returned as date-only strings with no timezone offset.
Error
ValueError: cannot supply both a tz and a timezone-naive dtype (i.e. datetime64[ns]):
Error while type casting for column 'hour'
Root cause (results.py:50-53)
All *_TIMESTAMP types in TYPES_MAP are mapped to "datetime64[ns]". Sub-day granularities (HOUR_TIMESTAMP, MINUTE_TIMESTAMP, SECOND_TIMESTAMP, TIMESTAMP) include a UTC offset in the API response, making the cast incompatible with a tz-naive dtype.
Expected behavior
query_pandas() should handle tz-aware timestamp strings returned natively by the API
Workaround
- Override problematic types inline:
_SUB_DAY_TS = {'TIMESTAMP', 'SECOND_TIMESTAMP', 'MINUTE_TIMESTAMP', 'HOUR_TIMESTAMP'}
for _k in _SUB_DAY_TS:
ShopifyQLPandasResult.TYPES_MAP[_k] = pd.DatetimeTZDtype(tz='UTC')
- Subclass ShopifyQLPandasResult to override the from_table_data class method.
Generally:
class _FixedPandasResult(ShopifyQLPandasResult):
@classmethod
def from_table_data(cls, table_data):
data_types = cls._pandas_dtypes_from_columns(table_data["columns"])
column_names = [str(c.get("name", "")) for c in table_data["columns"]]
df = pd.DataFrame(table_data["rows"], columns=column_names)
ts_cols = {col for col, dtype in data_types.items() if dtype == "datetime64[ns]"}
df = df.astype({col: dtype for col, dtype in data_types.items() if col not in ts_cols})
for col in ts_cols:
df[col] = pd.to_datetime(df[col], utc=True).dt.tz_convert("America/Regina").dt.tz_localize(None)
return df
- Dominant language
- Python
- Stars
- 7
- Forks
- 2
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from Shopify/shopifyql-py
-
Implement Pagination Open
Difficulty 4/5 3-5 days Newbie friendliness 45/100
Shopify/shopifyql-py#5 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 52/100
Shopify/shopifyql-py#4 ·
All issues in Shopify/shopifyql-py
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100