Catalog impls access request fields as attributes but pylance's Rust bridge passes a DictWithModelDump (AttributeError on describe_table write path)

Aperta
#181 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

@ZackFairTS ci sta già lavorando.

Dal 3/8/2026.

  • #182 di @ZackFairTS — aperta

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
58/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Tranquilla
Stack tecnologico
aws, python
Ambito
api, backend

Direzione di ricerca

Inizia con i metodi che ricevono le richieste in glue.py, hive2.py, hive3.py, unity.py, iceberg.py e polaris.py, quindi riproduci il fallimento isolato di DictWithModelDump descritto nell’issue. Confronta questi metodi con i wrapper in lance/namespace.py e verifica che ogni implementazione del catalogo gestisca la richiesta del bridge Rust e completi il percorso di scrittura di describe_table senza AttributeError.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Summary

When pylance writes a Lance table to a namespace-backed catalog (e.g. Glue) and its Rust storage-options provider calls back into the Python namespace impl's describe_table, it passes the request as a DictWithModelDump (a plain dict subclass exposing only model_dump(), no attributes). But GlueNamespace.describe_table (and the other catalog impls) access the request via attribute (request.load_detailed_metadata), which raises:

AttributeError: 'DictWithModelDump' object has no attribute 'load_detailed_metadata'

This breaks the write path (lance.dataset(...).write / lance_ray.write_lance) for every catalog impl in this package when driven through pylance's Rust namespace bridge.

Environment (a concrete, reproducing set)

  • pylance == 9.0.0 (pins lance-namespace>=0.8.5,<0.9)
  • lance-namespace == 0.8.6
  • lance-namespace-urllib3-client == 0.8.6
  • lance-namespace-impls == 0.4.1
  • lance-ray == 0.5.0

Root cause

pylance's PyO3 bridge (python/src/namespace.rs, present across 6.x–10.x) builds a request, serializes it to JSON, wraps it in a DictWithModelDump, and invokes the Python namespace impl's method with that dict. pylance's own wrappers (DirectoryNamespace/RestNamespace in lance/namespace.py) tolerate this because they call request.model_dump(). But the impls in this package access request fields as attributes.

describe_table in lance_namespace_impls/glue.py (v0.4.1, lines 353–358):

def describe_table(self, request: DescribeTableRequest) -> DescribeTableResponse:
    """Describe a table."""
    if request.load_detailed_metadata:          # <-- AttributeError on DictWithModelDump
        raise RuntimeError(
            "load_detailed_metadata=true is not supported for this implementation"
        )

The same attribute-access pattern (request.<field>) appears in all catalog impls: glue.py, hive2.py, hive3.py, unity.py, iceberg.py, polaris.py.

Reproduction (isolated)

from lance_namespace_impls.glue import GlueNamespace
class DictWithModelDump(dict):
    def model_dump(self): return dict(self)
ns = GlueNamespace.__new__(GlueNamespace)
ns.describe_table(DictWithModelDump({"id": ["default", "t"], "load_detailed_metadata": True}))
# -> AttributeError: 'DictWithModelDump' object has no attribute 'load_detailed_metadata'

End-to-end, this surfaces during a Ray → Glue write as:

OSError: LanceError(IO): Failed to fetch storage options: ... Python error in describe_table:
AttributeError: 'DictWithModelDump' object has no attribute 'load_detailed_metadata',
src/namespace.rs:1571 ; lance-io/src/object_store/storage_options.rs

Suggested fix

Make the impls accept a dict-style request (the shape pylance's Rust bridge actually passes), e.g. coerce at the top of each request-taking method:

from lance_namespace_urllib3_client.models import DescribeTableRequest

def describe_table(self, request) -> DescribeTableResponse:
    if not isinstance(request, DescribeTableRequest):
        request = DescribeTableRequest.from_dict(dict(request))
    ...

(Verified working as a monkeypatch shim against the environment above — coercing DictWithModelDump → the real Pydantic model makes attribute access succeed and describe_table returns the correct storage_options.) Alternatively, pylance's call_py_method could pass a real model instead of DictWithModelDump.

The other five impls (hive2, hive3, unity, iceberg, polaris) need the same treatment for their request-taking methods.

Interim workaround

A startup shim (installed via a .pth so it runs in every process) that wraps the impl methods and coerces dict → model before delegating. Happy to open a PR with the in-method coercion if that direction is preferred.

Lingua principale
Java
Stelle
14
Fork
16
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di lance-format/lance-namespace-impls

Tutte le issue di lance-format/lance-namespace-impls

Issue simili

Altre issue su Java

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.