Roadmap step 5: HTTP transport
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 20/100
Research direction
Start with the task 5 files _checks.py, _host.py, and _declaration.py, then read ADR 0014 and the remote-provider sections of specs 02 and 05. Check how task 4 exposes the HTTP transport and how the existing settings and Resolve rows are built. Done means implementing task 5’s listed settings, checks, and resolution behavior, with tests for URL admission, unreachable providers, and the specified refusal cases.
Written by the indexing model from the issue text.
Description
Roadmap step 5: the HTTP transport. Provider endpoints served by the host, OpenAPI at the well-known path, the connector over HTTP, shared-secret tokens for the calling app, the principal header, and remote providers under single. Plan per CONTRIBUTING S4 and ADR 0018; it builds on step 4 (#11).
Implements
- spec 03: the whole document, from outline to requirements: addressing, call context, errors, projection (HTTP-2), calling app and principal, shared-secret tokens, health.
- spec 02: the network half of the deadline (TRN-4, TRN-38) and of sending at most once (TRN-17); choosing each contract version's one provider under
single. - spec 05: Resolve's "configured remote provider", and serving while a remote provider is down.
- spec 06: the contract suite over HTTP, and the conformance items of spec 03.
Depends on
- Accepted: ADR 0007, 0008, 0010 (narrowed by 0023 Decision 11), 0017, 0019, 0020, 0021, 0022, 0023, 0025. Step 4 (#11), with its PR 9 (#90) merged: task 3 builds on the principal backends.
- Proposed, each accepted by the first task that builds it: 0011 (#107), 0014 (#106), 0027 (#105), 0028 (#108). Until a record is Accepted, no task builds on it.
- Avoidable: 0016 (step 6, #13), since only
singleships; the repository's open question 7 (production authentication), since only the shared-secret backend ships, for development.
New records (proposed in #105 to #108; where this plan and a record differ, the record wins)
- Identity on an HTTP call, ADR 0027 (#105; tasks 3, 4, 6): caller backends under
NEOLITH["CALLER_BACKENDS"],SharedSecretBackendon the standard library, keys and rotation, the principal header read only after the allow-list, header outcomes, the 401 challenge, refusals before sending. Its Decision 11 narrows 0017 Decision 4 ontracestate. - Remote providers, ADR 0014 amended (#106; tasks 4, 5):
NEOLITH["REMOTE_PROVIDERS"]keyed by contract identity, the originurl, transport choice at set-up, no network check before serving (an exception to K2, stated in #106), the privateover_httptest seam. - Provider endpoints, ADR 0011 amended (#107; tasks 1, 6, 7): no web framework; the host owns each provided version's prefix; 0017's order on a contract path; one provider side for both transports; responses and their logging; ASGI; the OpenAPI document.
- HTTP client, ADR 0028 (#108; tasks 2, 4): the standard library's
http.client, one deadline for every blocking step, strict responses, a pool per origin, no resend, TLS on the system store.
How the four were reconciled (19 contradictions, each settled by the record that owns the subject) is summarised on this issue's comments.
Public surface
NEOLITH["CALLER_BACKENDS"]andNEOLITH["SIGNING_SECRETS"];neolith.callers.CallerBackendandSharedSecretBackend;neolith.CredentialRefused(0027).NEOLITH["REMOTE_PROVIDERS"];neolith.transports.HTTPTransport, a name for the setting only (0014).- Contract paths and
/.well-known/neolith/{openapi.json,health}, with nothing for an app author or deployer to write (0011). - Checks, working ids fixed by each PR:
neolith.W002plainhttp(0014),W003shadowed URL (0011),W004no caller backend,W005a principal backend declaring aneolith-header,W006(deploy)SharedSecretBackendlisted (0027). - No runtime dependency. One test-only dependency, trustme (task 2).
Design gate
- Admitted set: each record's admitted set: the four
NEOLITHkeys, every header and body form on a contract path, the request the client writes and the responses it accepts. Settings outside it stop the host before Serve with a message naming the input and never a secret or url; requests outside it get spec 03's kind, or no neolith error body off a contract path (APP-1, K2, K9). - Source of truth: Django 5.2.19 and 6.1.3 (
core/handlers,utils/log.py,views/debug.py,middleware/security.py,test/testcases.py), asgiref 3.13.0, Python 3.12 and 3.13http/client.py,ssl.pyandsocket, gunicorn 27.0.1, read as the records cite them. - Failure modes: WSGI and ASGI; repeated and underscored headers; a deadline passing in each phase, at a boundary, and with none; 1xx, ambiguous framing and an early refusal with a reset; fork during resolution; secrets and the principal in every output, Django's error mail included; an unreachable provider and one on an older contract version.
Tasks: one PR each
Each PR carries its own spec delta, tests and code (ADR 0018), targets the open release branch, and lists the files it owns. Tasks in one wave touch no common file, apart from the index rows of docs/adr/README.md and one new field each on the set-up table in _host.py's build; whichever lands second merges the release branch. New IDs are named HTTP-new-a and on until the PR numbers them from HTTP-3, TRN-64 and APP-51.
Wave 1, in parallel (after #105 to #108 and #90 merge)
- 1. One provider side (#122) (
refactor/provider-side;_connector.py,_context.py,_crossing.py). From 0017's step 4 on, the in-process transport's provider side becomes shared: it takes a deadline counted at arrival, an optional idempotency key, a principal already read, and a receiver kind that turns on 0007 Decision 3's network receive check (CON-39, CON-40, CON-42). In-process behaviour is unchanged. Covers the receive items left on this issue: exact integer bounds (2**60 + 1againstmaximum: 2**60), duplicate member names, lone surrogates,NaNand1e400. Adds no ID; moves CON-39, CON-40 and CON-42 to this step's network receiver intests/spec_trace.toml. Verify: the in-process suite unchanged; the network receiver refuses each listed text. Accepts 0011. Gate: provider endpoints. - 2. HTTP client (#124) (
feat/http-client; new_http.py,pyproject.tomltest group). Request writer, strict response reader, deadline-bounded socket wrapper, threaded resolution with partial deadlines, pool per origin with fork reset, TLS context and its test seam; size reported from the spike. Adds TRN-new-a (stop waiting at the deadline in every phase, resolution included), TRN-new-b (no resend; a reused connection that accepted no byte is not a send), HTTP-new-a (Content-LengthandAccept-Encoding: identity, never chunked), HTTP-new-b (no redirect, no content coding), HTTP-new-c (read past 1xx other than 101), HTTP-new-d (unavailableon an ambiguous status line, field line, framing or chunk size). Verify: 0028 Decision 9's socket-pair list, including a drip in headers and body cut at the deadline,103then200on a reused connection, CL with TE, fork with a resolution in flight, and trustme TLS with a CA lacking the AKI. Accepts 0028. Gate: HTTP client. - 3. Caller backends and the token (#123) (
feat/caller-backends; newcallers.py,_token.py;_checks.py,__init__.py,principals.py, one field in_host.py).CallerBackend,SharedSecretBackend(check and mint),CredentialRefusedon both sides (request side logged at WARNING, 0023 open question 4), theCALLER_BACKENDSandSIGNING_SECRETSrows, the secret form and key rules, W004 to W006. Adds HTTP-new-e (recognition and abstaining), HTTP-new-f to HTTP-new-m (one per validity item of spec 03's token list), HTTP-new-n (a recognized invalid token fails with no fallback). Verify: spec 03's example token verifies atiatand fails atexp + 60; Hypothesis mutations of one byte, member, type or time give the row they break; hex, padded and short secrets and the example secret in both spellings refused by name; no secret or token in any log line, refusal or error text. Accepts 0027, and addsNarrowed by 0027 (Decision 11)under 0017's status. Gate: identity on an HTTP call. - 7. OpenAPI projection (#121) (
feat/openapi; new_openapi.py). The 3.1.2 document fromContract.describe()at set-up, as fixed bytes: dialect, noservers, onepostper action,operationId,x-neolith-idempotent, keyed components, 422oneOf, the bearer scheme. Adds HTTP-new for HTTP-2's structure. Verify: the document validates against the OpenAPI 3.1 schema withjsonschema; the round-trip property (CON-16) runs on its components; equal hosts give equal bytes. Accepts nothing; merges after task 1. Gate: provider endpoints.
Wave 2
- 4. Connector over HTTP (#128) (
feat/http-transport; newtransports.py,_http_transport.py;_connector.py,_context.py). After tasks 1 to 3.Sentcarries the deadline with one budget helper; the transport writes 0027 Decision 9's headers in 0028's order with a fresh token per attempt, reads the budget once a connection is ready, maps responses (200 and 204 paired as 0011 Decision 4 says, the error body's kind,unavailable,internal) and runs the caller's receive check; the privateover_httpseam. Adds HTTP-new-o (budget read when the request is written), HTTP-new-p (a call with no calling app, or no signing key, refused before sending with the in-process outcome), HTTP-new-q (the kind from the body only; a non-success without a neolith error body isunavailable; a success that is not a valid reply isinternal); the spec delta settles CON-28 for an unknown kind whosedatais not an object or is absent (left on this issue). Verify: against a local server, each mapping row and the empty-reply pairing; a call from no app gives the in-process message unsent. Accepts 0014. Gate: remote providers, HTTP client.
Wave 3, in parallel
- 5. Remote providers (
feat/remote-providers;_checks.py,_host.py,_declaration.py). After task 4. TheREMOTE_PROVIDERSrows, the two Resolve rows, transport choice at set-up, W002, theoverride_settingsrefusal, and 0027 Decision 9's set-up refusal for an app with no signing key. Adds TRN-new-c (undersingle, the one provider: in the host when one exists, else the configured remote one), TRN-new-d (stop before serving on a remote entry no app requires, or one an app provides), APP-new-a (never refuse to serve because a remote provider cannot be reached). Verify: Hypothesis property that a drawn url is admitted exactly when it equals its origin's serialisation; a refused url with a password or a token absent from the message; W002 onfoo.localhostand not on loopback; an unreachable provider sets up, boots, serves and givesunavailable. Accepts nothing. Gate: remote providers. - 6. Provider endpoints (#129) (
feat/provider-endpoints; new_endpoint.py;middleware.py,_boot.py,_server.py, one field in_host.py). After tasks 1, 3 and 7.RootMiddlewareanswers every owned path; rows 1 to 13 of 0011's table in 0017's order; 0027's steps 1 to 3, header outcomes,tracestatecut and drop, the reporter filter; replies, errors,Allow/Accept-Post/Accept-Encoding; one log line per response marked logged; ASGI per request; the well-known paths with GET and HEAD; builds_openapi.document()at set-up into its_host.pyfield, where aValueErrorstops set-up as a package fault; W003;internal's cause ondjango.neolith;neolith runrefusing a repeatedContent-TypeorContent-Length. Adds about twenty HTTP IDs, one per rule: authenticate before parsing the body or reading context headers; theunauthenticatedcases; Host not served; owned path with no action; method; query;Content-Type;Content-Encoding;Content-Length; body limit (MAY); reply forms; unknown headers and the reservedNeolith-prefix; OWS;tracestate(three); arrival before authentication; the exact 401 challenge; the principal read only after the allow-list; well-known methods; health means ready. Verify: one test per row of 0011's and 0027 Decision 8's tables, under WSGI and ASGI; an unlisted app with a malformed principal getspermission_denied;log_responseskips a marked response on Django 5.2 and 6.1; no principal inmail_adminsfor aninternal. Accepts nothing. Gate: provider endpoints, identity on an HTTP call.
Wave 4
- 8. Over every transport, and the step's documents (
feat/http-conformance;tests/contract suite,tests/spec_trace.toml,docs/django-host.md,docs/roadmap.md,docs/adr/). After tasks 1 to 7. The shared contract suite throughover_httpagainst a live server (T2); in-process and HTTP parity for the same call, a call from no app included, on a host that signs and accepts under one key; spec 06's items for spec 03;03added toenforce. Documentation: owned paths, W003, decoded paths, the ASGI body bound, response logging, caller backends andWSGIPassAuthorization,REMOTE_PROVIDERSandALLOWED_HOSTS, calls with no deadline,sync_to_asyncserialisation, private CAs and their extensions, and the per-call cost of a new connection measured on the reference machine (K10). The roadmap's step 5 row and library paragraph drop django-ninja and httpx; 0011's file is renamed to its title, with every link. Verify:spec_trace.pyreports every step 5 ID covered. Accepts nothing. Gate: all four records.
Deferred
- To step 6 (#13):
local-first, failover and retries (TRN-8), more than one provider per identity, a per-try timeout, and the failover test. - To later records, each named as an open question in its record: production authentication (open question 7: mTLS, asymmetric tokens, signed delegation, per-provider keys); an async connector and transport; an outbound proxy; a host under a path prefix; a host maximum budget and Drain's bound for inbound calls; a size limit for replies and error bodies (0028 open question 1, which includes this issue's error-body bound); hooks for contract calls; a provisional IANA registration of the well-known name; an
unimplementedkind; headers a provider must understand.
- Dominant language
- Python
- Stars
- 1
- Forks
- 0
- Avg merge
- 5h 30m
- Merged PRs (30d)
- 85
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from stack314/django-neolith
-
help wanted
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
stack314/django-neolith#142 · 1 comment ·
Maintainers usually reply within 1 day
-
help wanted
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
stack314/django-neolith#141 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
stack314/django-neolith#140 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
stack314/django-neolith#126 · 2 comments ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
stack314/django-neolith#119 · 2 comments ·
Maintainers usually reply within 1 day
All issues in stack314/django-neolith
Similar issues
-
automated endpoint-dead needs-curation
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
almanac-data/climate-almanac#138 ·
Maintainers usually reply within 1 day
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 67/100
OceanDataTools/openrvdas#760 ·
Maintainers usually reply within 1 day
-
level/subtask type/enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
wazuh/wazuh-virtual-machines#1091 ·
Maintainers usually reply within 1 day
-
Python: Bug: ChatMessageContent.from_element drops AudioContent and BinaryContent into raw TextContentPossibly taken @halcyon-past claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
microsoft/semantic-kernel#14824 · 1 comment ·
Maintainers usually reply within 3 days
-
Difficulty 1/5 Under an hour Newbie friendliness 82/100
LedgerHQ/app-bitcoin-legacy#306 ·