delta-io/delta

Obligation to hardcode full path in local Delta Lake

Open

#555 建立於 2020年11月20日

在 GitHub 查看
 (6 留言) (12 反應) (0 負責人)Scala (2,100 fork)batch import
buggood first issue

倉庫指標

Star
 (8,807 star)
PR 合併指標
 (平均合併 7天 1小時) (30 天內合併 142 個 PR)

描述

Hello! I found out that you cannot query a local Delta Lake using SQL directly from the files if you don't use the full path (from root to the Delta Lake directory). I know that Delta Lakes are not supposed to be used in standalone clusters or non-distributed file system but for testing and (perhaps in the future, general public reachment) it's not a bad idea to generate a Delta Lake in a local directory to try it out when you are working in Visual Studio Code or another IDE, that change the working directory to the folder that you have opened and let you use 'relative paths'.

If you try to use data/delta-test (example) and in a PySpark session you try to make a query on that delta-lake, you get: `pyspark.sql.utils.AnalysisException: Unsupported data source type for direct query on files: delta;; But if you do /home/gabmartini/data/delta-test it works.

Very frustrating for beginners to be truly honest. Perhaps a very explicit remark in the documentation will help to clear that out.

Thanks and keep up the good work!

貢獻者指南