Unable to open log file: open("/databricks/driver/hyperd.log", 66): Permission denied

从 hyperleaup/hyper_file.py 和 hyperleaup/creator.py 开始，重点查看 Creator.create()、copy_data_into_hyper_file() 以及 traceback 中显示的 HyperProcess 调用。在 Databricks 环境中复现 HyperFile 示例，并追踪 Hyper server 如何接收其日志目录。当示例能够创建 extract，且不再出现 /databricks/driver/hyperd.log 权限错误时，即视为完成。

Appears to try to access databricks root directory during hyper extract creation, which is denied.
Example below:

"query" is accessing a delta table in Databricks Hive

hf = HyperFile("TPCH_Extract", query, is_dbfs_enabled=True, creation_mode='copy')

HyperException Traceback (most recent call last)
HyperException: The Hyper server process exited during startup with exit code: 1
Command-line: "/databricks/python/lib/python3.7/site-packages/tableauhyperapi/bin/hyper/hyperd run --date-style=MDY --date-style-lenient=false --init-user=tableau_internal_user --language=en_US --log-config=file,json,all,hyperd,0 --log-dir=/databricks/driver --no-password=true --skip-license=true --listen-connection tab.domain:///tmp/domain/auto --callback-connection tab.domain:///tmp/domain/31f6c5fb534749aa9c43e8b4afc17f98"
Child process' stderr:
Unable to open log file: open("/databricks/driver/hyperd.log", 66): Permission denied
FileListener in error state after initial rotate()

Context: 0x13cead20

The above exception was the direct cause of the following exception:

HyperException Traceback (most recent call last)
in
3 # Step 1: Create a Hyper File from Spark SQL
4
----> 5 hf = HyperFile("TPCH_Extract", query, is_dbfs_enabled=True, creation_mode='copy')

/local_disk0/spark-bfb001e1-256e-4232-a0ea-1d515d18a094/userFiles-5f9eb004-70b6-4fdd-a74b-097f172c405f/addedFile76068253217714208702c2c0_09f1_4d07_a0cf_5f1648f667c2_hyperleaup_0_0_5_py3_7_a9c46-53e02.egg/hyperleaup/hyper_file.py in init(self, name, sql, df, is_dbfs_enabled, creation_mode, null_values_replacement)
42 self.is_dbfs_enabled,
43 self.creation_mode,
---> 44 self.null_values_replacement).create()
45 self.luid = None
46

/local_disk0/spark-bfb001e1-256e-4232-a0ea-1d515d18a094/userFiles-5f9eb004-70b6-4fdd-a74b-097f172c405f/addedFile76068253217714208702c2c0_09f1_4d07_a0cf_5f1648f667c2_hyperleaup_0_0_5_py3_7_a9c46-53e02.egg/hyperleaup/creator.py in create(self)
217 # COPY data into a Tableau .hyper file
218 logging.info("Copying data into Hyper File...")
--> 219 database_path = copy_data_into_hyper_file(csv_path, self.name, table_def)
220
221 elif self.creation_mode.upper() == CreationMode.INSERT.value:

/local_disk0/spark-bfb001e1-256e-4232-a0ea-1d515d18a094/userFiles-5f9eb004-70b6-4fdd-a74b-097f172c405f/addedFile76068253217714208702c2c0_09f1_4d07_a0cf_5f1648f667c2_hyperleaup_0_0_5_py3_7_a9c46-53e02.egg/hyperleaup/creator.py in copy_data_into_hyper_file(csv_path, name, table_def)
117 """Helper function that copies data from a CSV file to a .hyper file."""
118 hyper_database_path = f"/tmp/hyperleaup/{name}/{name}.hyper"
--> 119 with HyperProcess(telemetry=Telemetry.DO_NOT_SEND_USAGE_DATA_TO_TABLEAU) as hp:
120 with Connection(endpoint=hp.endpoint,
121 database=Path(hyper_database_path),

/databricks/python/lib/python3.7/site-packages/tableauhyperapi/hyperprocess.py in init(self, telemetry, user_agent, hyper_path, parameters)
90 Error.check(hapi.hyper_instance_create(InteropUtil.string_to_char_p(hyper_path),
91 telemetry == Telemetry.SEND_USAGE_DATA_TO_TABLEAU,
---> 92 native_params.cdata, pp))
93 self.__cdata = pp[0]
94

/databricks/python/lib/python3.7/site-packages/tableauhyperapi/impl/dllutil.py in check(p)
108 # this will free the error when it goes out of scope
109 errp = Error(p)
--> 110 raise errp.to_exception()
111
112 @staticmethod

HyperException: Failed to start a new Hyper instance.
Context: 0x86a93465

Caused by:
The Hyper server process exited during startup with exit code: 1
Command-line: "/databricks/python/lib/python3.7/site-packages/tableauhyperapi/bin/hyper/hyperd run --date-style=MDY --date-style-lenient=false --init-user=tableau_internal_user --language=en_US --log-config=file,json,all,hyperd,0 --log-dir=/databricks/driver --no-password=true --skip-license=true --listen-connection tab.domain:///tmp/domain/auto --callback-connection tab.domain:///tmp/domain/31f6c5fb534749aa9c43e8b4afc17f98"
Child process' stderr:
Unable to open log file: open("/databricks/driver/hyperd.log", 66): Permission denied
FileListener in error state after initial rotate()

Context: 0x13cead20