Long RPCs ignore cancellation mid-operation; a done context only withholds the answer

Open
#2,337 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
38/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
go
Domain
api, backend

Research direction

Start with the existing context checks in internal/frontend/grpc/runs.go and the handler locations listed in the issue, then trace the long parse, convert, query, render, and document operations. Add bounded cancellation checks through those paths and update the contract comment in client/opensysml/inprocess.go. Add deterministic counting-context tests for mid-loop cancellation and deadlines; done means each listed RPC returns the context status promptly and concurrent calls remain unaffected.

Written by the indexing model from the issue text.

Description

Disclaimer, this issue was detected and written by an AI

Summary

Most engine calls honour a context: RunRuns, RunSweep, CheckBehavior,
CompareResults, Explore*, ImportData and SelfCheck notice ctx.Err() and
stop (or return it). The long parse/convert/query/render paths do not.
client/opensysml/inprocess.go:258 states the current contract:

The engine runs a call to completion; a done context withholds whatever it
produced, answer or error, as the wire does.

So a caller deadline returns DEADLINE_EXCEEDED while the engine keeps parsing,
converting or querying. On a memory-constrained host a handful of timed-out
requests stack full CPU and memory cost: the timeouts make the overload worse,
not better.

Where the checks already are

  • internal/frontend/grpc/runs.go:173,261 (RunRuns)
  • internal/frontend/grpc/sweep.go:143 (RunSweep)
  • internal/frontend/grpc/check.go:186 (CheckBehavior)
  • internal/frontend/grpc/compare.go:196 (CompareResults)
  • internal/frontend/grpc/explore.go:45 (Explore*)
  • internal/frontend/grpc/importdata.go:49 (ImportData)
  • internal/frontend/grpc/selfcheck.go:56 (SelfCheck)

These return ctx.Err(); connect maps context.Canceled/DeadlineExceeded, and
the in-process answer maps them through contextStatus (inprocess.go:271).

Where they are missing (the work)

Add the same pattern — check at phase boundaries and every N elements/documents,
then return ctx.Err() — to:

RPC File Where to check
ParseFile internal/frontend/grpc/service.go:824 before/after the parse
ParseSources service.go:857 per document in the set, and between analysis and the response build
Convert export.go:28 before preparation, between export.PrepareModel, toGraph, graph build and serialization; per document in convertModelOfDocuments (export.go:564)
Migrate export.go:189 per migrated document/phase
Query query.go:99 in candidates/collect and when building rows, every ~1024 elements
RenderView render_view.go before and after the render
RenderDocument documents.go (render path) before and after the render
ExportGraphs export_graphs.go before and after the export
RunDocumentQuery docquery.go per row
InstancePaths instancepaths.go per node during the walk
ElementProperties element_properties.go per document/element
Instantiate objects.go per behavior/object

Notes:

  • Check cheaply: if err := ctx.Err(); err != nil { return nil, err } at the top
    of each document/row and every few hundred elements; do not call ctx.Err()
    per property.
  • Return the context error unchanged, so the wire code is CANCELLED /
    DEADLINE_EXCEEDED on both transports. Do not turn it into an INVALID_ARGUMENT
    or a 200 with error.
  • After this, update the inprocess.go:258 comment: the engine checks the
    context where the work is long, and a done context returns the status.
  • ParseSources is the biggest win in-process: a 572 KB/13-document project
    takes ~2 s cold, and an abandoned tree read or export currently keeps it
    running.

Tests

Service tests (internal/frontend/grpc), calling the handlers directly like
affected_test.go does:

  • A counting context that answers nil for N Err() calls and
    context.Canceled afterwards. ParseSources and Convert over a large model
    return CodeCanceled (not a result), and return before the uncancelled call
    would have finished. Pin the exact behaviour: with N below the first check the
    call is untouched; with N inside the document loop it stops after the current
    document.
  • The same for Query over a large scope.
  • context.DeadlineExceeded yields CodeDeadlineExceeded, not CodeCanceled.
  • A concurrent call on another model still completes (no shared cancellation).
  • An in-process client test (client/opensysml) drives ParseDocuments with the
    counting context and asserts StatusError{Code: CodeCanceled} and that the
    partial model is not returned.

Avoid wall-clock-only tests: under a loaded 4-core box a 2 s parse is not a stable
oracle. The counting context makes the stop point deterministic.

Acceptance

  • Every long RPC above stops within a bounded number of elements/documents of
    a cancellation and returns CANCELLED/DEADLINE_EXCEEDED.
  • A pre-cancelled request still returns immediately (unchanged).
  • Concurrent calls are unaffected.
  • The answer comment and any docs that say "runs to completion" are updated.
  • Tests cover both cancellation and deadline, at a mid-loop check point.
Dominant language
Go
Stars
59
Forks
20
Avg merge
10h 23m
Merged PRs (30d)
897

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Open-MBEE/OpenSysML

All issues in Open-MBEE/OpenSysML

Similar issues

More Go issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.