Dots in column names raises assertion error in sqlachemy hive
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 68/100
Research direction
Start in pyhive/sqlalchemy_hive.py at HiveCompiler.visit_column, then reproduce the failure with the quoted dotted column examples from the issue. Check how the compiled identifier is parsed and verify that columns such as col.id.part compile successfully while retaining the table and database qualification shown in the examples.
Written by the indexing model from the issue text.
Description
Dots in column names get counted as separators between db name, table name and column name.
Reason is code
dot_count = result.count('.')
assert dot_count in (0, 1, 2), "Unexpected visit_column result {}".format(result)
do not take care about quotes. See pyhive.sqlalchemy_hive.HiveCompiler.visit_column
In order to reproduce column should have two dots table.`col.id.part`, or one dot plus explicit dbname prefix dbname.table.`col.id`
from databricks_dbapi.sqlalchemy_dialects.base import DatabricksDialectBase
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, Integer
from sqlalchemy.orm import Query
Base = declarative_base()
class Test(Base):
__tablename__ = 'test'
id = Column(Integer, name='col.id.part', primary_key=True)
q = Query(Test)
# SELECT test.`col.id.part` FROM test
q.statement.compile(dialect=DatabricksDialectBase())
Traceback (most recent call last):
File "/home/kotofos/.config/JetBrains/PyCharm2021.2/scratches/scratch_1.py", line 16, in <module>
q.statement.compile(dialect=DatabricksDialectBase())
File "<string>", line 1, in <lambda>
File "/home/kotofos/Documents/slashdb/slashdb_venv/lib64/python3.6/site-packages/sqlalchemy/sql/elements.py", line 481, in compile
return self._compiler(dialect, bind=bind, **kw)
File "/home/kotofos/Documents/slashdb/slashdb_venv/lib64/python3.6/site-packages/sqlalchemy/sql/elements.py", line 487, in _compiler
return dialect.statement_compiler(dialect, self, **kw)
File "/home/kotofos/Documents/slashdb/slashdb_venv/lib64/python3.6/site-packages/sqlalchemy/sql/compiler.py", line 592, in __init__
Compiled.__init__(self, dialect, statement, **kwargs)
File "/home/kotofos/Documents/slashdb/slashdb_venv/lib64/python3.6/site-packages/sqlalchemy/sql/compiler.py", line 322, in __init__
self.string = self.process(self.statement, **compile_kwargs)
File "/home/kotofos/Documents/slashdb/slashdb_venv/lib64/python3.6/site-packages/sqlalchemy/sql/compiler.py", line 352, in process
return obj._compiler_dispatch(self, **kwargs)
File "/home/kotofos/Documents/slashdb/slashdb_venv/lib64/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 96, in _compiler_dispatch
return meth(self, **kw)
File "/home/kotofos/Documents/slashdb/slashdb_venv/lib64/python3.6/site-packages/sqlalchemy/sql/compiler.py", line 2177, in visit_select
for name, column in select._columns_plus_names
File "/home/kotofos/Documents/slashdb/slashdb_venv/lib64/python3.6/site-packages/sqlalchemy/sql/compiler.py", line 2177, in <listcomp>
for name, column in select._columns_plus_names
File "/home/kotofos/Documents/slashdb/slashdb_venv/lib64/python3.6/site-packages/sqlalchemy/sql/compiler.py", line 1929, in _label_select_column
return result_expr._compiler_dispatch(self, **column_clause_args)
File "/home/kotofos/Documents/slashdb/slashdb_venv/lib64/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 96, in _compiler_dispatch
return meth(self, **kw)
File "/home/kotofos/Documents/slashdb/slashdb_venv/lib64/python3.6/site-packages/pyhive/sqlalchemy_hive.py", line 161, in visit_column
assert dot_count in (0, 1, 2), "Unexpected visit_column result {}".format(result)
AssertionError: Unexpected visit_column result `test`.`col.id.part`
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 545
- PR merge metrics
- No merged PRs in 30d
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from dropbox/PyHive
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
-
Difficulty 3/5 1-2 days Newbie friendliness 35/100
-
Difficulty 3/5 1-2 days Newbie friendliness 35/100
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
-
Difficulty 5/5 Over a week Newbie friendliness 20/100
Similar issues
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
use-agent-os/agent-os#3314 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
BasedHardware/omi#15662 · 1 comment ·
-
documentation help wanted
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
AiursoftWeb/AnduinOS-2#19 ·