feat: add Tencent GooseFS Table Master namespace implementation (Python)

Aperta
#121 0 commenti 1 reazione 0 assegnatari Vedi su GitHub

@XuQianJin-Stars ci sta già lavorando.

Dal 17/6/2026.

  • #122 di @XuQianJin-Stars — aperta

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
35/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Tranquilla
Stack tecnologico
grpc, python
Ambito
api, backend, databases

Direzione di ricerca

Confronta le implementazioni esistenti di Hive, Glue, Iceberg, Polaris e Unity prima di lavorare nel pacchetto Python lance-namespace-impls. Esamina python/pyproject.toml, python/tests/test_goosefs.py, python/tests/test_goosefs_integration.py, docs/src/goosefs.md e docs/src/.pages; il lavoro è completo quando la registrazione di GooseFS, le operazioni, il packaging, la documentazione, il linting, i test unitari e i controlli di integrazione sono completati.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Summary

Add a new Lance Namespace implementation backed by Tencent Cloud GooseFS Table Master to the Python lance-namespace-impls package.

GooseFS Table Master is a Lance-native metadata service exposed over gRPC. Unlike general-purpose catalogs (Hive, Glue, Iceberg, Polaris, Unity), it is purpose-built for Lance — every registered table is a Lance table, so no table_type=lance marker filtering is required on the client side.

The new implementation acts as a thin pass-through layer that translates each Lance Namespace request into the corresponding *PRequest defined in the GooseFS Table Master gRPC schema and forwards it through the official goosefs-metastore-client library.

Motivation

  • Provide first-class Lance Namespace support for users running Lance on top of Tencent Cloud GooseFS / COS.
  • Cover the full Lance Namespace surface (basic CRUD + advanced operations such as indexing, tags, versioning, and transactions) without the overhead of mapping into a generic catalog model.
  • Match the integration pattern already established for Hive, Glue, Iceberg, Polaris, and Unity.

Scope

Core implementation
  • New GooseFSNamespace class registered as "goosefs" in LanceNamespaces
  • 2-level namespace hierarchy (database > table); identifier is forwarded verbatim to the Table Master
  • Configurable connection (uri / host / port / timeout / max_retries), authentication (authentication_enabled / username / impersonation_user), and namespace-level defaults (manifest_enabled / dir_listing_enabled)
Operations covered
  • Namespace: Create / List / Describe / Drop / Exists
  • Table: Create / CreateEmpty / Declare / Register / List / Describe / Drop / Deregister / Exists / Rename
  • Data plane: Insert / MergeInsert / Delete / Update / Query / CountRows
  • Schema evolution: AddColumns / AlterColumns / DropColumns / UpdateSchemaMetadata
  • Indexing: CreateIndex / CreateScalarIndex / ListIndices / DescribeIndexStats / DropIndex
  • Tags & versioning: Create/Update/Delete/List/Get tag, Create/List/Describe version, Restore, plus Batch variants
  • Transactions & query planning: AlterTransaction / DescribeTransaction / ExplainQueryPlan / AnalyzeQueryPlan / GetTableStats
Packaging & build
  • New optional extra goosefs in python/pyproject.toml (goosefs-metastore-client==0.1.7, grpcio>=1.78, grpcio-status>=1.78)
  • New Makefile targets: lint-goosefs, install-goosefs, test-goosefs, integ-test-goosefs
  • Unit tests: python/tests/test_goosefs.py
  • Integration tests: python/tests/test_goosefs_integration.py
Documentation
  • New page docs/src/goosefs.md describing background, configuration properties, object mapping, Lance-table identification, and per-operation semantics
  • docs/src/.pages updated to include Tencent GooseFS: goosefs.md in the navigation

Out of Scope

  • A Java implementation — this PR only adds the Python binding. A Java counterpart can be added in a follow-up if there is demand.
  • Bundling the GooseFS gRPC .proto files in this repo; they continue to ship inside goosefs-metastore-client.

Verification

  • ruff format --check and ruff check pass on the new files and across the whole python/ tree
  • python -m compileall succeeds for src/ and tests/
  • Unit tests in tests/test_goosefs.py pass with the goosefs extra installed
  • Integration tests in tests/test_goosefs_integration.py pass against a live GooseFS Table Master (skipped automatically when the service is not reachable)
Lingua principale
Java
Stelle
14
Fork
16
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di lance-format/lance-namespace-impls

Tutte le issue di lance-format/lance-namespace-impls

Issue simili

Altre issue su Java

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.