Provide a synchronous constructor for non-empty streams
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 38/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Quiet
- Tech stack
- wasm
- Domain
- backend-api-design
Research direction
Start with the WASI stream discussion linked from the issue, especially wit_stream::new() and descriptor.read-via-stream, then review the referenced work on stream slicing and splicing. The goal is to determine and implement a synchronous constructor that can populate a stream from a known list<u8> without a background task, and verify that it supports the WASI-Virt virtualization use case.
Written by the indexing model from the issue text.
Description
TL;DR: There is not currently a way to construct a non-empty stream in a synchronous context, making it impossible for a guest component to virtualize several WASI APIs. While planned features like cooperative threads will eventually make it possible to populate a stream by spawning background tasks, I believe that having a straightforward stream constructor that takes a list would be generally useful and hopefully (?) much simpler to implement.
(Apologies if this is not the best repo for this issue, please feel free to move it as appropriate.)
WASI-Virt currently supports WASI versions v0.2.1 and v0.2.3. Both of these are quite old relative the ecosystem; cargo build --target wasm32-wasip2 produces components that depend on WASI v0.2.9. The latest 2.x release is 0.2.12, and the first 0.3.x release landed last month. WASI-Virt also uses exact-match dependency resolution, so while a v0.2.12 export could satisfy a v0.2.3 import in theory, it gets rejected in practice. Functionally, this means that WASI-Virt cannot be used with recent toolchains.
I attempted to enable semver-compatible dependency resolution, but learned that it wouldn't be possible so long as WASI-Virt was using WASI p2. P2's use of resources to represent IO streams always lead to transitive import/export conflicts:
4: world exports
wasi:io/streams@0.2.3but it's also transitively used by an import which means that this is not valid
These conflicts should be obviated by P3's use of stream instead of resources. I set out to test this by making a simple component that would virtualize wasi:filesystem (and nothing else). However, I quickly ran into an issue when trying to implement descriptor.read-via-stream (see #wasi > Creating a `stream` in a sync context).
In short: read-via-stream is synchronous function. A guest implementation can construct a stream reader/writer pair via wit_stream::new(), but the guest cannot (meaningfully) spawn a background task to populate a stream via the writer. Cooperative threads should make this possible when they arrive early next year.
The WASI-Virt use case does not require a sophisticated writer implementation. The bytes that represent a file are known at virtualization time; the file descriptors are backed by a simple Vec<u8>. However, so far as I can tell, there is no way to construct a stream from a list<u8> in a synchronous context.
I know there's a larger design effort happening around slicing and splicing stream; I wonder if there's room for a simple synchronous constructor. (Regrettably, the lack of user generics does mean that you couldn't define a func from-list(list<T>) -> stream<T> in the near term.)
- Dominant language
- WebAssembly
- Stars
- 1.4k
- Forks
- 130
- Avg merge
- 2d 1h
- Merged PRs (30d)
- 15
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from WebAssembly/component-model
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
WebAssembly/component-model#609 · 2 comments · 1 reaction ·
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
WebAssembly/component-model#724 · 9 comments ·
-
Difficulty 5/5 Over a week Newbie friendliness 38/100
WebAssembly/component-model#718 · 1 comment ·
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
WebAssembly/component-model#695 · 1 comment ·
-
Difficulty 5/5 Over a week Newbie friendliness 45/100
WebAssembly/component-model#694 · 1 comment ·
All issues in WebAssembly/component-model
Similar issues
-
bug clawsweeper:linked-pr-open clawsweeper:needs-live-repro clawsweeper:no-new-fix-pr impact:message-loss issue-rating: 🐚 platinum hermit P2 regression
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
AXERA-TECH/ax-llm#77 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
games-on-whales/wolf#509 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100