Catalog factory for the S3 bundle provider ("provider": "s3" in catalog.json)

Open
#239 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
22/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
aws, csharp
Domain
backend, cloud

Research direction

Start from the existing IBundleProviderFactory seam and how KnowledgeCatalogOptions.ProviderFactories registers factories, then read the catalog.json source handling in OKF4net.Catalog. The open questions are placement (shared with #236), endpoint trust, and client disposal, and they need a maintainer decision before code. Done means the S3 factory passes the three acceptance tests (matching filesystem results, diagnostics without bucket or signed-URL leakage, rejected settings) against RustFS under Category=S3Integration.

Written by the indexing model from the issue text.

Description

.NET enhancement

Area

Catalog integration for the S3 bundle provider (#142): let a catalog.json knowledge source name "provider": "s3" and be served from S3 / an S3-compatible store, through the existing IBundleProviderFactory seam (KnowledgeCatalogOptions.ProviderFactories).

Why it matters

#142 delivers the provider as a library only (out of its scope by decision). A host composing its knowledge through OKF4net.Catalog would otherwise build the handle by hand, outside the catalog, losing fused search, diagnostics and per-source policy (RefreshPolicy, OnAccess by default for catalog sources).

Scope

  • An IBundleProviderFactory (ProviderType = "s3") whose clients are handed to it by the host — never built from ambient credentials, never from catalog.json (spec §7, security framing on #139, 2026-10-06 §3).
  • AcceptedSettings: bucket, key prefix, region, endpoint, addressing style — nothing else; any credential-like setting is rejected (IllegalSetting).
  • Endpoint trust: catalog.json is operator configuration (trusted, #139 §1), but decide whether an endpoint written there is accepted as is or must match an endpoint (or client) the host registered — a manifest choosing the host its credentials are sent to is the riskiest part of this factory.
  • One client per endpoint/account, or one per source: decide, and state who disposes it (IBundleProviderFactory: the catalog never disposes a provider).
  • IBundleProvider.Id stable and non-blank so the fused resolvers deduplicate correctly.

Placement — decide once with #236

Where the factory lives is the same question as for the embedded provider (#236): in the provider project (which then references OKF4net.Catalog, so every host reading an S3 bundle inherits the catalog), or in a small leaf package so the provider stays OKF4net-only. Settle it for both factories together, so the two backends follow one rule. The filesystem provider is built into the catalog (filesystem cannot be registered), so no other existing provider needs a factory.

Acceptance

  • A catalog test with an S3 source (RustFS, Category=S3Integration, out of default CI like the provider's own integration tests) returns the same search results and diagnostics as the same bundle on the filesystem.
  • An unknown bucket or prefix, or an unreachable endpoint, is reported as a ProviderUnavailable / SourceUnavailable / SourceStale diagnostic naming an exception type only — never an exception out of the catalog, never a bucket name, request id or signed URL in a message.
  • A setting outside AcceptedSettings is rejected before anything is created.

Depends on

#142; coordinate with #236.

Effort

M

Dominant language
C#
Stars
4
Forks
1
Avg merge
1d 4h
Merged PRs (30d)
32

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from jchable/okf4net

All issues in jchable/okf4net

Similar issues

More C# issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.