Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

feat: query cancellation via `CancellationToken` on `SessionContext`

Open
#68 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
45/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Quiet
Tech stack
java, rust

Research direction

Start with the JNI blocking sites in native/src/lib.rs and inspect the Java SessionContext, DataFrame, and resource-handle entry points. Compare the proposed token lifecycle and collect/executeStream overloads with the cancellation.rs and query_tracker.rs references. Done means the listed APIs support cancellation and cleanup without changing the existing zero-token methods, with cancellation observable during collection and streaming.

Written by the indexing model from the issue text.

Description

enhancement
Is your feature request related to a problem or challenge?

A long-running DataFrame.collect(allocator) or DataFrame.executeStream(allocator) call blocks the calling Java thread for the entire duration of the query. Thread.interrupt() does nothing — the JNI thread is parked inside runtime().block_on(...) (native/src/lib.rs), and the interrupt flag is ignored by the Tokio runtime. There is no way to abort an in-flight query, free its native resources early, or unblock the calling thread short of waiting for the query to finish.

For any embedder running multi-tenant workloads — request timeouts, user-cancel actions, node shutdown, leader-election handover — this is a hard operational gap. The OpenSearch analytics backend (OpenSearch/sandbox/plugins/analytics-backend-datafusion/rust/src/cancellation.rs and query_tracker.rs) carries a CancellationToken-based wrapper precisely because upstream offers nothing.

This is complementary to issue #40 (close()/JNI use-after-free race) but distinct: #40 is about safely tearing down a finished handle; this is about signalling an in-flight future to stop. Both eventually share the atomic-handle scaffolding from #40's option 2, so coordination is worthwhile, but the surface lands cleanly without #40 having to merge first.

Describe the solution you'd like

A token-based cancellation API on SessionContext, modeled on Spark 4.0's interruptTag shape (cancel lives on the session, not on the DataFrame). The token is a separate handle from the DataFrame so cancel can fire from a thread that does not hold the DataFrame.

v1 surface
try (SessionContext ctx = new SessionContext();
     CancellationToken token = ctx.newCancellationToken();
     DataFrame df = ctx.sql("SELECT ... FROM big_table")) {

    Future<ArrowReader> fut = pool.submit(() -> df.collect(allocator, token));

    // from another thread (timeout watcher, user-cancel handler, ...):
    token.cancel();

    // fut completes with CancellationException
}

New methods:

  • SessionContext.newCancellationToken() -- returns a fresh CancellationToken bound to this session.
  • CancellationToken.cancel() -- fires the token; idempotent.
  • CancellationToken.isCancelled() -- non-blocking check.
  • CancellationToken.close() -- releases the native handle; the token is AutoCloseable so try-with-resources handles cleanup.
  • DataFrame.collect(BufferAllocator, CancellationToken) -- overload that takes a token. The existing zero-token collect(BufferAllocator) is unchanged.
  • DataFrame.executeStream(BufferAllocator, CancellationToken) -- same overload pattern. Token is held by the returned ArrowReader for its full lifetime; cancel mid-stream aborts the next loadNextBatch().
Describe alternatives you've considered

No response

Additional context
Out of scope
  • Tag form. Ship the token primitive first; tag is sugar that can land in a follow-up if a user actually asks for it.
  • Sync-API breakage. df.collect(allocator) keeps working unchanged; the new method is df.collect(allocator, token) (overload).
  • Per-operator cancel granularity. Today the cancel point is each block_on site; sub-operator cancellation is upstream-DataFusion territory.
Dominant language
Java
Stars
32
Forks
12
PR merge metrics
No merged PRs in 30d

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/datafusion-java

All issues in apache/datafusion-java

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.