Client hides plan-encoding errors behind an Internal error

Open Beginner friendly
#2,885 0 comments 1 reaction 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
78/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
rust

Research direction

Start in ballista/core/src/execution_plans/distributed_query.rs at execute_query_params and read how the two plan-encoding errors are mapped. Preserve each error as context so the cause remains available to DataFusionError::find_root and the message avoids the misleading Internal error text. Done when both errors retain their causes and the reported message includes the serialization context without the bug-report text.

Written by the indexing model from the issue text.

Description

Describe the bug

When the client can't encode a logical plan to send it to the scheduler, execute_query_params (ballista/core/src/execution_plans/distributed_query.rs) turns the error into a string:

let plan_message = T::try_from_logical_plan(plan, extension_codec).map_err(|e| {
    DataFusionError::Internal(format!("failed to serialize logical plan: {e:?}"))
})?;

That loses the error chain, so DataFusionError::find_root can't reach the cause, and it formats the cause with Debug, escaping its quotes and newlines. Because it's Internal, DataFusion also appends "This issue was likely caused by a bug in DataFusion's code. Please help us to resolve this by filing a bug report", even when the cause is the user's, such as a table that no longer exists.

To Reproduce

With ballista-iceberg, read a view whose Iceberg table was dropped from its catalog (#2784). The error is:

Internal error: failed to serialize logical plan: Context("Error serializing custom table at …/datafusion-proto-…/src/logical_plan/mod.rs:1534", Execution("no composed codec could encode it: codec 0: This feature is not implemented: LogicalExtensionCodec is not provided; codec 1: IcebergError(TableNotFound)\ncaused by\nExecution error: TableNotFound => No such table: TableIdent { namespace: NamespaceIdent([\"ns\"]), name: \"t\" }")).
This issue was likely caused by a bug in DataFusion's code. Please help us to resolve this by filing a bug report in our issue tracker: https://github.com/apache/datafusion/issues

Expected behavior

The cause is kept, with context added rather than replacing it:

.map_err(|e| e.context("failed to serialize logical plan"))

The message then reads "failed to serialize logical plan" followed by the cause, without the bug-report text, and find_root returns the cause. The try_encode error two lines below has the same shape.

Additional context

Raised in review of #2784. The cause also reaches this point flattened by ComposedLogicalExtensionCodec, which joins its codecs' errors into one string; that's a TODO on its collect, to fix when it moves to DataFusion.

Dominant language
Rust
Stars
2.2k
Forks
348
Avg merge
2d 4h
Merged PRs (30d)
166

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/datafusion-ballista

All issues in apache/datafusion-ballista

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.