Cannot list Trino tables from Hive metastore using derby
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 25/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- kubernetes
- Domain
- databases, distributed-systems
Research direction
Start with the SHOW TABLES IN hive.myschema path and the Hive metastore/Trino deployment described in the report; compare the Derby setup with the working Postgres setup and inspect the related Derby discussion in issue 154. Done means determining whether Derby can be supported for this use case and recording or implementing the agreed handling, with table listing verified.
Written by the indexing model from the issue text.
Description
I wanted to connect Superset to Trino (V403) using Hive(V3.1.3) connector and a shared S3 bucket.
Creating a schema in Trino worked, the schema could be listed as well:
CREATE SCHEMA IF NOT EXISTS hive.myschema WITH (location = 's3a://bucket/')
Creating a table in that schema and querying that (via Trino CLI or Superset SQL query) worked:
CREATE TABLE IF NOT EXISTS hive.myschema.mytable (
myInt BIGINT
)
WITH (
external_location = 's3a://bucket/foo/',
format = 'PARQUET'
);
Listing that table via:
SHOW TABLES IN hive.myschema
leads to a EXTERNAL_HIVE_EXCEPTION due to a socket timeout (in Trino). The Hive metastore pod was reachable from the Trino coordinator pod (curl).
In Superset, when creating a dataset from a Trino database, the tables cannot be listed either (and therefore no dataset created). This can be circumvented but is not great in terms of user experience.
Switching Hive to Postgres instead of Derby worked.
We have a related issue (comment) with Derby https://github.com/stackabletech/hive-operator/issues/154#issuecomment-1182968401 to document that this should not be used for HA or in production at all.
In demos and integration tests with hive, afaik we always use Postgres.
This needs some investigation and a decision on how to deal/support Derby in the future.
I did not test Hive 2.3.9 or other Trino versions.
- Dominant language
- Rust
- Stars
- 29
- Forks
- 2
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 12
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from stackabletech/hive-operator
-
type/bug
Difficulty 4/5 3-5 days Newbie friendliness 48/100
stackabletech/hive-operator#757 ·
-
customer-request
Difficulty 3/5 1-2 days Newbie friendliness 65/100
stackabletech/hive-operator#717 ·
-
Difficulty 5/5 Over a week Newbie friendliness 30/100
stackabletech/hive-operator#678 · 1 reaction ·
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
stackabletech/hive-operator#630 · 1 comment ·
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
stackabletech/hive-operator#573 ·
All issues in stackabletech/hive-operator
Similar issues
-
Browser (wasm) relay client cannot connect to relays whose URL has a trailing-dot FQDN hostname Open
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
n0-computer/iroh#4550 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
paritytech/zombienet-sdk#591 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
farion1231/cc-switch#7638 · 1 comment ·
-
onnx-ir re-exports ModelProto and GraphProto but not NodeProto, AttributeProto and AttributeType Open
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100