Cannot connect to a Hive Metastore with Kerberos when the principal's host differs from the URI host
還沒有人認領這個 Issue。
評估
- 難度
- 2/5
- 預估耗時
- 1-3 小時
- 新手友好度
- 78/100
- Issue 類型
- 缺陷
- 描述清晰度
- 描述清楚
- 活躍度
- 冷清
- 技術堆疊
- python
研究方向
從 pyiceberg/catalog/hive.py#L167 的 _HiveClient._init_thrift_transport 開始,檢查如何從 URI 中選取 Kerberos 服務主機。加入提議的設定屬性,同時保留 URI 主機作為預設值,然後執行 issue 中提到的單元測試,並驗證 Kerberos 連線行為。
由索引模型根據 Issue 內容生成。
描述
Apache Iceberg version
0.11.0 (latest release)
Please describe the bug 🐞
When connecting to a Kerberos-enabled Hive Metastore, authentication fails if the service principal's host does not match the host in the connection URI, and there is no setting to correct it.
I connect to the metastore through three HA hosts:
from pyiceberg.catalog import load_catalog
catalog = load_catalog("hive", **{
"type": "hive",
"uri": "thrift://myhms1:9083,thrift://myhms2:9083,thrift://myhms3:9083",
"hive.kerberos-authentication": "true",
"hive.kerberos-service-name": "myservice",
})
catalog.list_namespaces()
The last line fails with:
Traceback (most recent call last):
File "test.py", line 24, in <module>
ns = catalog.list_namespaces()
File ".../pyiceberg/catalog/hive.py", line 769, in list_namespaces
with self._client as open_client:
File ".../pyiceberg/catalog/hive.py", line 180, in __enter__
self._transport.open()
File ".../thrift/transport/TTransport.py", line 382, in open
initial_response = self.sasl.process()
File ".../puresasl/client.py", line 148, in process
return self._chosen_mech.process(challenge)
File ".../puresasl/mechanisms.py", line 505, in process
kerberos.authGSSClientStep(self.context, '')
kerberos.GSSError
Enabling KRB5_TRACE shows that the failure happens when the client asks the KDC for a service ticket:
set-error: -1765328243: Did not find credential for myservice/myhms1@EXAMPLE.COM in cache FILE:...
set-error: -1765328377: Error from KDC: LOOKING_UP_SERVER while looking up 'myservice/myhms1@EXAMPLE.COM'
My Hive Metastore's service principal is myservice/hive-host@EXAMPLE.COM.
PyIceberg, however, requests a ticket using my HMS server hostname (myhms1) as the hostname component, which is not a valid service principal, so the request fails.
A Kerberos service principal has the form Service/Hostname@REALM (see 3.2 Principal in the Kerberos tutorial), and with SASL/GSSAPI those two components come from the service and host arguments passed to the client.
PyIceberg always derives that host from the connection URI, so it is forced to be whichever host you connect to.
hive.kerberos-service-name (added in #2141) makes the Service part configurable, but there is no equivalent for Hostname.
The relevant line is pyiceberg/catalog/hive.py#L167 (_HiveClient._init_thrift_transport):
return TTransport.TSaslClientTransport(socket, host=url_parts.hostname, service=self._kerberos_service_name)
There is no way to avoid this from the client side. The uri has to contain the hostnames I actually connect to, and the principal's hostname component is not one of them.
Listing multiple URIs does not help either, because the client is built from the first URI and the failure happens later, during authentication.
So I patch that line locally and hardcode the host:
-return TTransport.TSaslClientTransport(socket, host=url_parts.hostname, service=self._kerberos_service_name)
+return TTransport.TSaslClientTransport(socket, host="hive-host", service=self._kerberos_service_name)
With that one change everything works well.
PyIceberg requests myservice/hive-host@EXAMPLE.COM and catalog.list_namespaces() succeeds.
I would like to propose a new configuration property, hive.kerberos-service-host. It could be added in the same way as #2141.
When the property is not set, the URI host would be used as the default, so the current behavior is preserved for backward compatibility.
I have finished the implementation including unit tests, and verified it end-to-end against the same metastore.
If this is welcome, I would like to submit the PR myself.
Willingness to contribute
- I can contribute a fix for this bug independently
- I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- I cannot contribute a fix for this bug at this time
- 主要語言
- Python
- 星號
- 1.1k
- 分支
- 589
- 平均合併
- 2 天 4 小時
- 30 天內合併 PR
- 72
貢獻指南
這個儲存庫沒有索引到貢獻指南
從這裡開始
- 先讀完整個 Issue,再讀專案的貢獻指南。
- 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
- Fork 儲存庫,在一個分支上完成修改。
- 送出 Pull Request,並在描述裡引用這個 Issue 編號。
apache/iceberg-python 的其他 Issue
-
難度 2/5 1-3 小時 新手友好度 78/100
apache/iceberg-python#3996 ·
-
bug
難度 2/5 1-3 小時 新手友好度 72/100
apache/iceberg-python#3979 ·
-
難度 2/5 1-3 小時 新手友好度 78/100
apache/iceberg-python#3885 ·
-
[Bug] PyArrowFileIO fails to propagate s3.ssl.ca-cert to pyarrow.fs.S3FileSystem tls_ca_file_path 未關閉
難度 2/5 1-3 小時 新手友好度 76/100
apache/iceberg-python#3866 · 1 則留言 ·
-
難度 2/5 1-3 小時 新手友好度 78/100
apache/iceberg-python#3836 · 1 則留言 ·
查看 apache/iceberg-python 的全部 Issue
相似的 Issue
-
難度 2/5 1-3 小時 新手友好度 88/100
-
難度 2/5 1-3 小時 新手友好度 82/100
-
難度 2/5 1-3 小時 新手友好度 78/100
-
enhancement
難度 2/5 1-3 小時 新手友好度 72/100
-
難度 2/5 1-3 小時 新手友好度 74/100