Backend protocol: name the interface between Client and Backend

Đã đóng
  • Có gì không ổn?
#157 0 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

@SimonHeybrock đang làm issue này rồi.

Từ ngày 6/11/2026.

  • #158 của @SimonHeybrock — đã merge

Đánh giá

Độ khó
5/5
Thời gian dự kiến
Hơn một tuần
Mức phù hợp với người mới
40/100
Loại issue
Tái cấu trúc
Độ rõ ràng
Đặc tả rõ ràng
Mức độ hoạt động
Sôi nổi
Công nghệ
python
Lĩnh vực
backend, backend-api-design

Hướng nghiên cứu

Start by reading ess/apps/client.py, ess/apps/backend.py, and the lifetime and accumulator docs named in the issue to map the current interface and its guarantees. The proposal names ess/apps/protocol.py as the destination for BackendProtocol, Entry, Selection, FileSource, and Store; the concrete completion criteria are to type Client against the protocol, move local() beside Backend, and preserve public imports. Resolve the stated wait and watch_files design decisions, then move the rules into protocol docstrings and trim their duplicate documentation.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

Make the interface that Client calls on Backend an explicit typing.Protocol, and make it the one home of the rules that clients rely on. It is also the shape of the API that an HTTP server would expose for a hosted backend (#27).

Today

Client calls 27 methods of Backend. All of them take IDs and a client: str, and return records, statuses, or values. Client adds no semantics of its own. It does three things:

  • Reshaping: it accepts one, a list, or a dict, and returns the same shape.
  • Wrapping: it turns IDs into Record, Stage, and Accumulator objects.
  • Shortcuts: compute (submit, then wait), latest, and provenance.

So the protocol exists, but without a name:

  • client.py imports the concrete class: from .backend import Backend, Entry, Selection.
  • local() lives in client.py, which is why the client module imports the backend at all.
  • The rules are spread over:
    • the Client docstrings, which point to docs/developer/lifetimes.md and accumulators.md;
    • the root docs/developer/README.md;
    • ADRs 0002, 0003, 0004, and 0006;
    • the 139-line module docstring of backend.py, which mixes rules with implementation.

Proposal

A BackendProtocol in ess/apps/protocol.py, next to Entry, Selection, and the protocols FileSource and Store. Client.__init__ takes a BackendProtocol. local() moves next to the in-process Backend, and ess.apps keeps re-exporting it.

Group Methods
Sessions open_client, close_client
Specs spec, specs
Requests submit, cancel, status, wait, failure
History record, records, changes, plain_request
Values outputs, describe, readable, persist, release
Stages and accumulators open_stage, open_accumulator, push, freeze, accumulator_outputs
Files files, watch_files, file_metadata

Backend.close is the lifecycle of the server, not something a client calls, so it stays off the protocol.

The two sides differ as follows:

Client BackendProtocol
Serves one proposal, one session many clients and proposals, named by ID
Speaks in Python objects and shapes IDs, pydantic models, and values; nothing that could not cross a wire
Promises ergonomics: shapes and shortcuts the semantics: when a call returns, what is kept and readable, the order of the change feed, what each call may raise

Rule: anything that changes what happens belongs in the protocol. The client only adapts.

Docs move, they are not copied. The rules from lifetimes.md, accumulators.md, and the client docstrings move into the protocol's docstrings. The Client docstrings then describe shapes and shortcuts and link to the protocol. The backend.py module docstring keeps only how the in-process backend implements the rules. Copying the rules instead would add another place to keep in sync.

What it gives us

  • One home for the guarantees. App work needs only the client and the protocol, not the backend docs.
  • Contract changes are their own diff. changes and describe (#156) touched the client, the backend, views.py, ADR 0004, the README, and the app together. With a protocol, the new methods and their promises are a small diff to review, apart from the implementation behind them.
  • The seams of the backend are all named. Up to the client is the protocol. Down are FileSource and Store. Sideways is Binding from essspec.

What it does not give: it does not make backend.py smaller. That is a separate split. Test fakes of the backend would add little, since local() is cheap.

Toward an HTTP wrapper for a hosted backend

The protocol is the server's API surface. A hosted backend is then built as follows:

  • Server: a thin layer that maps each route to a protocol call on an in-process Backend.
  • Client side: a RemoteBackend that implements the protocol by making HTTP calls. connect(url, proposal) is Client(RemoteBackend(url), ...), and Client does not change.
  • Tests: the client and story tests run against both local() and Client(RemoteBackend(test_server(Backend()))), through one parametrized fixture. That shows the two deployments behave the same, which docs/developer/README.md promises ("the caller sees the same records and outputs either way").

Writing the protocol now makes every method's types say what crosses the wire. These methods would need a decision:

Method In-process today Over HTTP
open_client(proposal, submitter) the caller names the submitter the submitter comes from authentication, and access to the proposal is checked. The returned ID becomes a session with a lease (#27)
every method's client: str an argument from the session or token
spec, specs return WorkflowSpec with live pydantic models return SpecId and SerializedWorkflowSpec (#88). A Python client maps the SpecId to installed spec modules (ADR 0007)
submit(entries), push(rows), open_stage, open_accumulator Request.params and rows may hold pydantic model instances encoded through the params schema and request types (#88, #140)
wait(ids) blocks without a timeout blocking does not suit HTTP: make it a long poll with a timeout, with the client looping (#81: timeout= is undecided)
watch_files(selector) returns an iterator an iterator does not cross a wire: give it a cursor form like changes, and let the client build the iterator (new-file signal in #155)
outputs, accumulator_outputs return the value the workflow returned, not a copy a copy, through a codec per output type (#152); describe and select= (#149) limit what is sent
SubmitError (with FieldErrors), NotReadable, RecordFailed, ClientEnded Python exceptions part of the contract: a structured error body each
bindings, local(bind=...) code from the notebook not part of the protocol: a hosted backend finds bindings through entry points (ADR 0007)

This suggests one more rule: protocol methods are request and response; whatever blocks or iterates is built in Client on top. Today wait and watch_files break this rule.

Not settled by the protocol: authentication and access across proposals, leases and memory caps (#27), wire format and codecs (#152), and versioning of the HTTP API.

Steps

  1. Add protocol.py with BackendProtocol, and move Entry, Selection, FileSource, and Store there. Type Client against the protocol. Move local() next to Backend. Public imports do not change.
  2. Move the rules into the protocol's docstrings, and trim the client and backend docstrings and the docs pages that held them.
  3. Decide wait and watch_files (request/response, with the client looping).
  4. Later, with #27: RemoteBackend, the HTTP server, and the parametrized test fixture.

This goes with splitting ess.apps into subpackages (client/, backend/, files/, ui/, panel/, automatic/), with the layering checked by import-linter. That split is a separate issue.

Ngôn ngữ chính
Python
Star
0
Fork
1
Merge trung bình
9 giờ 40 phút
Pull request đã merge (30 ngày)
60

Chuẩn bị môi trường

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của scipp/essapps

Tất cả issue của scipp/essapps

Issue tương tự

Thêm issue về Python

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.