Typed parsers and formatters for common HTTP header values
Maintainers usually reply within 1 day
Assessment
This issue has not been assessed yet.
Description
Problem
effect/http treats header values as plain strings. Each place that needs to understand a value parses it on the spot, and the copies disagree.
Retry-After is parsed in eight places with six different rules. RFC 9110 §10.2.3 allows whole seconds or an HTTP date. For the invalid value -5:
HttpClient(http/HttpClient.ts:1461) falls through toDate.parse("-5"), which V8 reads as a date in 2001, so it retries after 1 ms.- The OTLP exporter and the Anthropic, OpenAI, OpenRouter and OpenAI-compatible clients return a delay of −5 seconds. These four AI clients also ignore HTTP dates and accept
1e3andInfinity. - GraphQL and Cloudflare return
undefined, and the typesafe client returns 0.
Other examples on main:
- Media types are matched by substring (
HttpServerRequest.ts:274,GraphQLProtocol.ts:260,HttpStaticServer.ts:438). The static server treatsAccept: text/html;q=0, which means "never send HTML", as asking for HTML. - There are three separate
Accept-style parsers: compression,McpServerand HttpApi. - HTTP dates go through
Date.parse, which reads the asctime format in local time. WithTZ=America/New_York,Sun Nov 6 08:49:37 1994comes out five hours off. Etagcan format a tag but not parse or compare one. The static server splitsIf-None-Matchon raw commas (HttpStaticServer.ts:448), but a comma is valid inside a tag.- There is nothing for
Cache-Control,Range,Link,WWW-Authenticateor Structured Fields.
Two related bugs are filed separately: basicAuth encoding (#9678) and multipart filenames (#9679).
Proposal
Add one small module per header family to effect/http. Each one provides:
- an immutable value;
parse, which returns aResultand is forgiving where the RFC allows;format, which always produces the canonical form;- an
Equivalence; - a Schema codec from string, so the value can be used directly in HttpApi
headersschemas and inWithHeaders.
| Module | Standard |
|---|---|
MediaType |
RFC 9110 §8.3.1 |
Accept, AcceptEncoding, AcceptLanguage, with negotiation |
RFC 9110 §12.5, RFC 4647 |
ContentDisposition |
RFC 6266, RFC 8187 |
HttpDate, RetryAfter |
RFC 9110 §5.6.7, §10.2.3 |
Etag lists and comparison, conditional requests |
RFC 9110 §8.8.3, §13 |
Range, ContentRange |
RFC 9110 §14 |
CacheControl |
RFC 9111 §5.2 |
Link |
RFC 8288 |
Authorization, WWW-Authenticate (Basic, Bearer) |
RFC 9110 §11, RFC 7617, RFC 6750 |
StructuredField |
RFC 9651 |
// names illustrative
RetryAfter.parse("Wed, 21 Oct 2015 07:28:00 GMT") // Result<RetryAfter, HeaderValueError>
RetryAfter.delay(value, now) // Duration, never negative
Etag.parseList(`"a,b", W/"c"`) // two tags
HttpApiEndpoint.get("file", "/file", {
headers: { "if-none-match": Schema.optional(Etag.ListFromString) }
})
Then switch Effect's own call sites over to these modules.
Why in Effect
- Effect's own code needs these parsers, and today every bug has to be fixed in every copy.
- Header values are wire data. Schema codecs give HttpApi users typed headers, decode errors and OpenAPI output without writing their own parsers.
- Other ecosystems provide them too. Go has
mime.ParseMediaType,http.ParseTime, and precondition and range handling inhttp.ServeContent. Rust has theheadersandsfvcrates. Node has jshttp'scontent-type,negotiator,range-parserandfresh.
Scope
In scope:
- The modules above, and switching Effect's own call sites over to them.
Out of scope:
- An HTTP cache.
- Cookies and trace context, which already have modules.
Forwarded(#9681) and Problem Details (#9686), which have their own issues.- Accept-based response selection in HttpApi.
Open questions
- Should conditional-request evaluation (
If-Match,If-None-Match,If-Modified-Since,If-Range) be a public helper that any handler can call? The static server would then use it too. - One issue with a checklist, or one PR per module?
- Dominant language
- TypeScript
- Stars
- 16.7k
- Forks
- 808
- Avg merge
- 10h 7m
- Merged PRs (30d)
- 808
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from Effect-TS/effect
-
Support {self: this} for fnUntracedPossibly taken @ArjunCodess claimed this 32 days ago. Openenhancement
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
Effect-TS/effect#8101 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 3/5 1-2 days Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
v4 Cache.get: a caller that joins an abandoned lookup fails with an interruption it did not ask forOpen
Difficulty 4/5 3-5 days Newbie friendliness 42/100
Maintainers usually reply within 1 day
-
Typed unix socket and TLS options for `BunHttpClient` and `DenoHttpClient`Possibly taken @effect-bot claimed this today. Opendocker-made-me-do-it enhancement platform
Effect-TS/effect#9687 · 1 assignee ·
Maintainers usually reply within 1 day
-
RFC 9457 Problem Details for HttpApi errorsPossibly taken @effect-bot claimed this today. Opendocker-made-me-do-it enhancement
Difficulty 5/5 Over a week Newbie friendliness 20/100
Maintainers usually reply within 1 day
All issues in Effect-TS/effect
Similar issues
-
frontend
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 86/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
ranxianglei/billion-context#4172 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
awslabs/threat-composer#362 ·
Maintainers usually reply within 1 day
-
Area/Cloud Needs Triage Type/Bug
Difficulty 1/5 1-3 hours Newbie friendliness 86/100
wso2/product-integrator#2847 ·
Maintainers usually reply within 1 day