Client hides plan-encoding errors behind an Internal error
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 78/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- rust
- Domain
- backend, distributed-systems
Research direction
Start in ballista/core/src/execution_plans/distributed_query.rs at execute_query_params and read how the two plan-encoding errors are mapped. Preserve each error as context so the cause remains available to DataFusionError::find_root and the message avoids the misleading Internal error text. Done when both errors retain their causes and the reported message includes the serialization context without the bug-report text.
Written by the indexing model from the issue text.
Description
Describe the bug
When the client can't encode a logical plan to send it to the scheduler, execute_query_params (ballista/core/src/execution_plans/distributed_query.rs) turns the error into a string:
let plan_message = T::try_from_logical_plan(plan, extension_codec).map_err(|e| {
DataFusionError::Internal(format!("failed to serialize logical plan: {e:?}"))
})?;
That loses the error chain, so DataFusionError::find_root can't reach the cause, and it formats the cause with Debug, escaping its quotes and newlines. Because it's Internal, DataFusion also appends "This issue was likely caused by a bug in DataFusion's code. Please help us to resolve this by filing a bug report", even when the cause is the user's, such as a table that no longer exists.
To Reproduce
With ballista-iceberg, read a view whose Iceberg table was dropped from its catalog (#2784). The error is:
Internal error: failed to serialize logical plan: Context("Error serializing custom table at …/datafusion-proto-…/src/logical_plan/mod.rs:1534", Execution("no composed codec could encode it: codec 0: This feature is not implemented: LogicalExtensionCodec is not provided; codec 1: IcebergError(TableNotFound)\ncaused by\nExecution error: TableNotFound => No such table: TableIdent { namespace: NamespaceIdent([\"ns\"]), name: \"t\" }")).
This issue was likely caused by a bug in DataFusion's code. Please help us to resolve this by filing a bug report in our issue tracker: https://github.com/apache/datafusion/issues
Expected behavior
The cause is kept, with context added rather than replacing it:
.map_err(|e| e.context("failed to serialize logical plan"))
The message then reads "failed to serialize logical plan" followed by the cause, without the bug-report text, and find_root returns the cause. The try_encode error two lines below has the same shape.
Additional context
Raised in review of #2784. The cause also reaches this point flattened by ComposedLogicalExtensionCodec, which joins its codecs' errors into one string; that's a TODO on its collect, to fix when it moves to DataFusion.
- Dominant language
- Rust
- Stars
- 2.2k
- Forks
- 348
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 166
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- Ships a Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/datafusion-ballista
-
Executor's default work dir is deleted at startup and recreated without `TempDir`'s permissionsPossibly taken @aetherxeg-source claimed this 16 days ago. Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
apache/datafusion-ballista#2767 ·
Maintainers usually reply within 1 day
-
Release a task's vcores as its partitions finishPossibly taken @Dandandan claimed this today. Openenhancement
Difficulty 5/5 Over a week Newbie friendliness 18/100
apache/datafusion-ballista#2905 ·
Maintainers usually reply within 1 day
-
Iceberg: DROP TABLE fails on a registered table its catalog can't loadPossibly taken @NoahKusaba claimed this 15 days ago. Open
Difficulty 4/5 3-5 days Newbie friendliness 43/100
apache/datafusion-ballista#2882 ·
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
apache/datafusion-ballista#2881 ·
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 20/100
apache/datafusion-ballista#2872 · 1 comment ·
Maintainers usually reply within 1 day
All issues in apache/datafusion-ballista
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
antithesishq/bombadil#473 ·
Maintainers usually reply within 2 days
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
libracoder/kiri#669 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 88/100
Maintainers usually reply within 5 days
-
autofixer: follow-up
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
tursodatabase/turso#12217 ·
Maintainers usually reply within 1 day
-
Declare the CMS schema library in the optional recipients extraPossibly taken A pull request linked to this issue is open or already merged. Openarea:python in-progress
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
yhay81/pyropdf#1005 · 2 comments ·
Maintainers usually reply within 1 day