Hacktoberfest 2026: as issues que os mantenedores marcaram para outubro, abertas e boas para iniciantes. Ver issues do Hacktoberfest

Make examples/ behave like examples: runnable, and consistent with the extension guide

Aberta
#1,723 0 comentários 0 reações 0 responsáveis Ver no GitHub

Ninguém assumiu esta issue ainda.

Avaliação

Dificuldade
4/5
Tempo estimado
3-5 dias
Facilidade para iniciantes
48/100
Tipo de issue
Refatoração
Clareza
Razoavelmente clara
Status de atividade
Ativa
Stack de tecnologia
python, rust

Direção de pesquisa

Comece por extension-guide/codecs.md e user-guide/data-sources.md:219 e, em seguida, inspecione as três árvores em examples/ e datafusion-ffi-example/logical_extension_codec.rs:148. Execute as suítes pytest existentes para estabelecer seus pontos de entrada atuais. Está concluído quando os exemplos puderem ser executados, o codec lógico não depender mais de um registro de provedores global ao processo, a quarentena do plano físico for explícita e o guia corresponder ao código.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

documentation enhancement

#1721 added examples/distributed as a worked example and, in doing so, exposed two structural problems with the older example trees that the new one throws into relief.

The guide contradicts the code it points at. extension-guide/codecs.md tells authors to encode durable metadata, then spends a paragraph explaining that our own examples do not. Readers copy examples; an example that contradicts the rule beside it is worse than no example. #1719 records the reference-implementation half of this as G9, which examples/distributed/storage-library now satisfies — but the crate a reader is most likely to open first still does the wrong thing.

Two of the three trees have no runnable entry point. datafusion-ffi-example is 13 pytest files and no script; datafusion-ffi-query-planner-example is one pytest file and no script. This is not only an aesthetic problem: user-guide/data-sources.md:219 calls datafusion-ffi-example a "user example" and routes Python users to it from the user guide. They arrive at a test suite.

Investigation while reviewing #1721 found the first problem is smaller and differently shaped than the guide claims. Each item below was checked against the source, not inferred:

  • datafusion-ffi-query-planner-example has no object registry at all — grep -rn "HashMap\|OnceLock\|TOKEN" src/ is empty. It claims node.is::<DistributedExec>() narrowly and writes a fixed durable marker. The guide named it anyway. Corrected in #1721.
  • datafusion-ffi-example's logical codec parks providers in a process-global HashMap, and this is cheaply fixable: it claims MemTable narrowly (logical_extension_codec.rs:148), and a MemTable is fully describable as schema plus batches.
  • datafusion-ffi-example's physical codec parks plans, and this is not fixable here. It is forced by the greedy ForeignExecutionPlan claim, which is load-bearing for upstream defect apache/datafusion#25152 — that is G2 in #1719. Once you claim a node you cannot introspect, parking it is the only remaining option. The work is to make it uncopyable, not to remove it.

So the end state is one quarantined arm carrying an issue number and a delete-when condition, rather than a systemic contradiction.

Decided while planning this: the example trees stay in examples/. Relocating the conformance beds to an integration-tests/ tree was considered and rejected — making them runnable resolves the "these aren't examples" complaint, and a move would churn roughly 40 doc, README, and agent-skill references, none of which any build step validates.

Sub-issues track the pieces. They are ordered: the prose rewrite describes the state the first two produce.

Linguagem predominante
Python
Estrelas
605
Forks
176
Merge médio
1d 23h
PRs com merge (30d)
8

Guia de contribuição

Nenhum guia de contribuição indexado para este repositório

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de apache/datafusion-python

Todas as issues de apache/datafusion-python

Issues semelhantes

Mais issues de Python

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.