Add KV Store Support

Open

#49 opened on Feb 20, 2026 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty: 5/5
Estimated time: Over a week
Newbie friendliness: 35/100

Issue type: Feature
Clarity: Mostly clear
Activity status: Stale
Tech stack: python
Domain: backend, databases

Research direction

Start with stubs/wit_world/imports/kv_store.py and compare the WIT interface with the requested Python API. Then inspect the Viceroy test configuration using test.toml and @on_viceroy for inline or file-based KV data. Done means CRUD, streaming entries, options, dict-like access, listing, and corresponding tests are implemented.

Written by the indexing model from the issue text.

Description

Overview

Add support for Fastly's KV Store, providing distributed key-value storage with read and write operations at the edge.

WIT Interface

interface kv-store {
  use types.{error, open-error};
  use http-body.{body};

  resource store {
    open: static func(name: string) -> result<store, open-error>;
    lookup: func(key: string) -> result<option<entry>, kv-error>;
    insert: func(key: string, body: body, options: insert-options) -> result<_, kv-error>;
    delete: func(key: string) -> result<bool, kv-error>;
    %list: func(options: list-options) -> result<body, kv-error>;
  }

  resource entry {
    take-body: func() -> option<body>;
    metadata: func(max-len: u64) -> result<option<string>, error>;
    generation: func() -> u64;
  }

  resource extra-kv-error;
  
  variant kv-error {
    bad-request,
    precondition-failed,
    payload-too-large,
    internal-error,
    too-many-requests,
    generic-error,
    extra(extra-kv-error),
  }

  enum insert-mode {
    overwrite,
    add,
    append,
    prepend,
  }

  resource extra-insert-options;

  record insert-options {
    background-fetch: bool,
    if-generation-match: option<u64>,
    metadata: option<string>,
    time-to-live-sec: option<u32>,
    mode: insert-mode,
    extra: option<borrow<extra-insert-options>>,
  }

  enum list-mode {
    strong,
    eventual,
  }

  resource extra-list-options;

  record list-options {
    mode: list-mode,
    cursor: option<string>,
    limit: option<u32>,
    prefix: option<string>,
    extra: option<borrow<extra-list-options>>,
  }
}

WIT bindings: stubs/wit_world/imports/kv_store.py

API Design

Implement KVStore resource wrapper
Implement KVStoreEntry as a file-like object inheriting from io.IOBase or io.RawIOBase:
- Implement read(size=-1) or readinto(b) for standard file-like interface
- Users get read_all() for free via .read() with no size argument
- Standard library functions work: shutil.copyfileobj(), io.BufferedReader, etc.
- metadata property, generation property
- text() convenience: entry.read().decode('utf-8')
- json() convenience: json.loads(entry.read())
Provide dict-like interface on Store: __getitem__, __setitem__, __delitem__, __contains__
Support InsertOptions with modes (overwrite, add, append, prepend), TTL, metadata, and generation matching
List operation returns iterator over keys with optional prefix filtering

Streaming Support: Entry values can be large (up to 25MB). By implementing standard io.IOBase:

Users can use familiar file-like API: entry.read(8192) for chunks, entry.read() for all
Works with stdlib: shutil.copyfileobj(entry, response_body) for zero-copy proxying
Can wrap in io.BufferedReader for additional buffering if needed
Document that .read() with no argument loads entire value into memory

Example:

entry = store.lookup("large-file")

# Streaming approach (memory-efficient) - standard file-like API
chunk = entry.read(8192)
while chunk:
    process_chunk(chunk)
    chunk = entry.read(8192)

# Or use with stdlib utilities
import shutil
shutil.copyfileobj(entry, output_file)

# Eager loading (beware large values!)
data = entry.read()  # reads all, standard Python pattern
text = entry.read().decode('utf-8')  # or entry.text()
obj = json.loads(entry.read())  # or entry.json()

Note on Future Async Support: Using io.IOBase for sync API is compatible with later adding async support. The WIT layer provides Pollable objects and select() for async operations. If/when async is added:

Sync API: entry.read() - returns immediately (blocking)
Async API: async def read() - returns coroutine, uses await with Pollable
These would be separate classes/methods, not the same io.IOBase instance
Similar to how aiofiles provides async wrappers over sync file operations

Cross-SDK Comparison:

Rust: StoreHandle::open() with methods lookup(), insert(), delete(). Returns LookupResponse with take_body() (streaming), take_body_bytes() (eager), metadata(), generation(). Has list() returning iterator over ListPage. Supports async with PendingLookupHandle, etc. Strongly typed InsertMode and ListMode enums.
Go: Open() returns *Store with Lookup(), Insert(), Delete(). Entry embeds io.Reader for streaming, plus String() helper for eager loading (with warning about memory). Meta(), Generation() accessors. No built-in async support (blocks). No list method yet.
JS: new KVStore(name) with async methods get(), put(), delete(), list(). Entry has body (ReadableStream) for streaming, text(), json(), arrayBuffer() for eager loading, metadata(), metadataText(). Put options include ttl, mode, gen. List returns {list: string[], cursor: string | undefined}.

Recommended Python approach:

Entry should inherit from io.IOBase to be a proper file-like object
Implement read(size=-1) method - standard file API that Python users know
No need for custom bytes(), read_all() - users just call .read() with no args
Works with stdlib: shutil.copyfileobj(), io.BufferedReader, etc.
Convenience methods (text(), json()) are thin wrappers over .read()
Dict-like API: store[key], store[key] = value, del store[key], key in store
list(prefix=None, limit=None, cursor=None) returning iterator/generator
InsertMode enum for put operations (overwrite, add, append, prepend)

Viceroy Testing

Viceroy supports KV Store with inline or file-based test data via test.toml:

[local_server]
# Inline data
kv_stores.my_store = [
  {key = "user:123", data = "John Doe"},
  {key = "config", file = "path/to/file.txt"},
  {key = "metadata_example", data = "value", metadata = "some metadata"}
]

# Or JSON file format
kv_stores.json_store = { file = "data/store.json", format = "json" }

Full CRUD operations (lookup, insert, delete) are supported. List operations work with test data. Tests can use @on_viceroy with inline TOML configuration.

Reference

Dominant language: Python
Stars: 5
Forks: 1
PR merge metrics: No merged PRs in 30d

Contributor guide

Open the contributing guide

First steps

Read the whole issue, then the project's contributing guide.
Comment on the issue to say you are picking it up — it saves two people doing the same work.
Fork the repository and make your change on a branch.
Open a pull request that references the issue number.

Similar issues

Use `charmlibs.interfaces.oauth` instead of deprecated `hydra.oauth` Open

enhancement

Difficulty 2/5 1-3 hours Newbie friendliness 70/100

canonical/paas-charm#368 · 1 comment · Opened Sep 23, 2026
Model API: distributable field returns false even when "Allow Distributed Inference Across Workers" is checked Open

Difficulty 2/5 1-3 hours Newbie friendliness 75/100

gpustack/gpustack#6275 · Opened Sep 23, 2026
attachment_urls parameter descriptions do not state that the value must be a reference Open

tech debt

Difficulty 2/5 1-3 hours Newbie friendliness 75/100

epam/ai-dial-quickapps-backend#579 · Opened Sep 23, 2026
addition to tracking list Open

Difficulty 1/5 Under an hour Newbie friendliness 90/100

StevenBlack/hosts#3256 · Opened Sep 23, 2026
Fix-WebUI-README-screenshot-links Open

Difficulty 1/5 Under an hour Newbie friendliness 90/100

qualcomm/qai-appbuilder#275 · Opened Sep 23, 2026

More Python issues