feat(extensions): support custom model providers through the extension API

Start by reading the existing provider and extension integration points named in the issue: pkg/kit/providers.go, internal/extensions/api.go, internal/extensions/symbols.go, and the startup and model registry paths. Then trace how model events reach the agent loop in internal/agent/agent.go and internal/agent/stream_turn.go. Done means an interpreted fake provider runs through Yaegi tests, including streaming, tool-call round trips, cancellation, model selection, and instance-aware listing without network access.

Summary

Add an extension API that lets an interpreted Go extension register a custom model provider. The provider must use Kit's existing agent loop, tools, sessions, and UI. Users must not need to rebuild Kit to install a new inference backend.

An extension should be able to supply model metadata and callback-based inference backed by HTTP, a local process, an embedded service, or a test fixture.

All API names and code below are proposals, not existing interfaces. Final names can change during implementation. The behavior and acceptance criteria are the intended contract.

Motivation

Kit supports custom providers in compiled Go applications through kit.RegisterProvider, kit.WithProvider, and ProviderFactory. Interpreted extensions cannot register an equivalent backend.

This prevents an installed extension from adding a model that works through normal model selection, streaming, tool execution, completions, and subagents.

A subprocess tool is not an equivalent solution: it delegates work instead of supplying a model to Kit's agent loop.

Goals

• Register a provider from an extension's Init function.
• Select an extension model at startup with --model provider/model.
• Use the same provider through model switching, extension completions, and subagents.
• Support streamed text, reasoning, structured tool calls, usage, and errors.
• Keep tool execution and tool permission checks in Kit.
• Show extension models and their capabilities in the model picker and instance-level model queries.
• Support cancellation, resource cleanup, safe reload, and concurrent Kit instances.
• Keep provider definitions independent of internal model-library types.
• Supply a deterministic example provider and tests that require no network access or credentials.

Non-goals

• Letting providers execute Kit tools outside Kit's tool loop.
• Replacing the existing compiled Go provider API.
• Sandboxing extension code. Extensions remain trusted code.
• Requiring a particular HTTP protocol or external service.
• Requiring a full OAuth implementation in the first release.
• Automatically retrying requests after visible output or tool-call emission.

Current integration points and gaps

Area	Current code	Required change
Compiled provider API	pkg/kit/providers.go, internal/models/provider_registry.go	Adapt extension definitions into instance-local factories.
Extension registration	internal/extensions/api.go, loader and runner	Add provider definitions, registration validation, and ownership.
Interpreter exports	internal/extensions/symbols.go	Export all new concrete types and constants.
Startup	pkg/kit/new.go, internal/kitsetup/setup.go, pkg/kit/kit.go	Make definitions available before provider-dependent validation and initial model construction.
Model validation/catalog	internal/models/registry.go, internal/models/providers.go	Resolve instance providers and model metadata consistently.
Extension bridge	pkg/kit/extensions_bridge.go	Convert public extension DTOs to internal model types.
Streaming/tool loop	internal/agent/agent.go, internal/agent/stream_turn.go	Preserve existing event and execution behavior through the adapter.
Model listing	pkg/kit/models.go, cmd/models.go	Define instance-aware listing and CLI extension-loading behavior.

The existing Yaegi bindings do not provide wrappers for an interpreted implementation of the host model interface. Do not expose that interface directly. Use concrete data structures and function fields, with a compiled adapter implementing the internal interface.

Architecture

flowchart TD
    A[Extension Init] --> B[RegisterProvider definition]
    B --> C[Runner: validate and record extension owner]
    C --> D[Kit-local factories and model catalog overlay]
    D --> E[Compiled model adapter]
    E --> F[Extension inference callbacks]
    F --> G[HTTP service, local process, or fake backend]
    E --> H[Normal Kit model events]
    H --> I[Kit agent loop]
    I --> J[Kit tool wrappers and permission checks]
    I --> K[Sessions and UI]

No UI component should import extension implementation types. Continue to use the existing host conversion pattern.

Proposed API shape

Use a definition for registration and a separate runtime for resources created for a model instance.

// Sketch: supporting message/event types are described below.
type ProviderDef struct {
    ID                 string
    Name               string
    Models             []ProviderModel
    Auth               ProviderAuth
    AllowUnknownModels bool
    Create             func(ProviderCreateRequest) ProviderCreateResult
}

type ProviderRuntime struct {
    Stream func(ProviderRequest, func(ProviderEvent) bool) ProviderResult
    Close  func() error
}

type ProviderCreateResult struct {
    Runtime ProviderRuntime
    Error   *ProviderError
}

type ProviderResult struct {
    Error *ProviderError
}

type ProviderError struct {
    Code      string // authentication, rate_limit, invalid_request, transport, etc.
    Message   string
    Retryable bool
}

type ProviderRequest struct {
    Model       string
    Messages    []ProviderMessage
    Tools       []ProviderTool
    ToolChoice  ProviderToolChoice
    Parameters  ProviderParameters
    Done        <-chan struct{}
}

Requirements for the supporting types:

• Messages must preserve role, ordered content parts, tool-call IDs, tool results, and error results.
• Parts must represent text, reasoning, supported files/images, tool calls, and tool results.
• Preserve reasoning signatures and provider-specific replay data when needed. Define a versioned, provider-namespaced representation and persistence behavior. Do not silently convert it to visible text.
• Tool definitions must carry names, descriptions, and JSON schemas. Raw JSON strings are an option at the interpreter boundary; validate them before use.
• Tool choice must represent automatic selection, no tools, required tools, and a named tool where supported.
• Generation settings need explicit set/unset semantics. Zero must not mean both “unset” and a valid configured value.
• Requests must identify unsupported features before dispatch where possible. Do not silently discard images, reasoning data, schemas, or tool choices.
• Creation requests must include resolved model settings and a cancellation signal for blocking setup.
• Do not pass the full Kit runtime or a tool-execution callback to the provider.

A first implementation may derive non-streaming generation by collecting stream events. Structured-object methods must either be implemented or return an explicit unsupported-operation error.

Streaming contract

Define typed event constants and concrete payloads for:

• Text start, delta, and end.
• Reasoning start, delta, and end, including signature metadata.
• Tool-call start, argument delta, and end.
• Usage, including input/output and cache usage where available.
• Finish, including stop, length limit, and tool-call finish reasons.

Specify whether a completed tool-call payload is required or assembled by the host. Avoid two independent events that can each cause the same tool to execute.

Contract rules:

1. Stream blocks until completion, cancellation, or failure. It must not return while background code can still emit events.
2. emit is synchronous and provides backpressure. A false return means stop emitting and release request resources.
3. Done must close when the caller cancels, including while the provider waits on I/O and has no events to emit.
4. Events are ordered. Concurrent emission from one request is not permitted.
5. A successful request emits exactly one finish event. No event is accepted after finish.
6. Failures use one defined terminal error path. Do not report the same failure through both an event and the return value.
7. The host validates event order, IDs, tool names, completed JSON arguments, and supported finish reasons.
8. Incomplete or malformed tool calls never execute.
9. Cancellation is distinct from a retryable transport error.
10. Extension panics become contextual provider errors where recovery is safe; they must not crash the host.

Minimal fake provider example

The example must be executed by Yaegi in tests, not only compiled as ordinary Go. Helpers are declared before use to avoid interpreter forward-reference problems.

//go:build ignore

package main

import "kit/ext"

func fakeStream(req ext.ProviderRequest, emit func(ext.ProviderEvent) bool) ext.ProviderResult {
    select {
    case <-req.Done:
        return ext.ProviderResult{Error: &ext.ProviderError{
            Code: "cancelled", Message: "Request cancelled",
        }}
    default:
    }

    events := []ext.ProviderEvent{
        {Type: ext.ProviderTextStart, ID: "text-1"},
        {Type: ext.ProviderTextDelta, ID: "text-1", Delta: "Hello "},
        {Type: ext.ProviderTextDelta, ID: "text-1", Delta: "from a fake provider."},
        {Type: ext.ProviderTextEnd, ID: "text-1"},
        {Type: ext.ProviderFinish, FinishReason: ext.ProviderFinishStop},
    }
    for _, event := range events {
        if !emit(event) {
            return ext.ProviderResult{Error: &ext.ProviderError{
                Code: "cancelled", Message: "Stream stopped",
            }}
        }
    }
    return ext.ProviderResult{}
}

func createFake(req ext.ProviderCreateRequest) ext.ProviderCreateResult {
    return ext.ProviderCreateResult{
        Runtime: ext.ProviderRuntime{Stream: fakeStream},
    }
}

func Init(api ext.API) {
    api.RegisterProvider(ext.ProviderDef{
        ID: "fake", Name: "Fake Provider",
        Models: []ext.ProviderModel{
            {
                ID: "echo", Name: "Fake Echo",
                ContextWindow: 8192, MaxOutputTokens: 1024,
                Capabilities: ext.ProviderCapabilities{Streaming: true},
            },
        },
        Create: createFake,
    })
}

Expected use after implementation:

kit -e examples/extensions/fake-provider.go \
  --model fake/echo --no-session "Say hello"

Expected assistant text: Hello from a fake provider.

Registration errors must be reported with the extension path and provider ID. If registration methods do not return errors, the loader must collect these errors and fail validation before model construction.

Tool-call round trip

sequenceDiagram
    participant U as User
    participant K as Kit agent
    participant P as Extension provider
    participant T as Kit tool
    U->>K: What is 2 + 3?
    K->>P: History + add tool schema
    P-->>K: Tool call call-1: add({a:2,b:3})
    P-->>K: Finish: tool_calls
    K->>K: Run hooks and permission checks
    K->>T: Execute once
    T-->>K: 5
    K->>P: History + call-1 result
    P-->>K: Text: 5
    P-->>K: Finish: stop
    K-->>U: 5

A fake provider should script this sequence without using a real model. On its second request, it must assert that history contains the assistant tool call and matching tool result. It should choose the next response from request history, not a package-global counter, so concurrent tests and branches remain independent.

Startup and resolution

Required construction order:

Read options and locate extensions
    -> Load extensions and collect registration definitions
    -> Validate provider ownership and metadata
    -> Build instance-local provider/catalog view
    -> Resolve selected model and provider-dependent settings
    -> Create initial model and agent
    -> Attach runtime context and start lifecycle events

Extension Init already runs before initial agent creation inside setup, but earlier configuration resolution also exists. Audit both paths. Avoid a circular dependency where an extension needs a running model to register that model's provider.

Route startup, model switching, named completions, and subagent creation through the same provider resolver. Do not add separate global-registration workarounds for each path.

Scope and lifecycle

• Registrations belong to one Kit instance and retain the source extension identity.
• Do not implement extension registration by mutating the process-wide provider registry.
• Reject duplicate extension provider IDs with a deterministic error that names both owners.
• Reject built-in provider replacement by default. Any override support needs an explicit policy.
• Define precedence between SDK instance factories and extension factories; reject collisions initially rather than silently replacing them.
• Subagents inherit the appropriate provider definitions. Each model creation gets its own runtime unless sharing is explicitly supported.
• Close runs exactly once for each successfully created runtime, including temporary completions and model replacement.
• On failed creation, provider code releases partially created resources. Document that ownership boundary.
• Reload validates new definitions before atomically publishing them. A failed reload leaves the previous usable registry intact.
• Retain old callback/interpreter state while an active runtime uses it. Removing a registration stops future construction but must not invalidate an in-flight call.
• Final shutdown cancels active requests, waits according to the documented shutdown policy, and closes runtimes.
• Host cancellation cannot safely force-stop arbitrary interpreted code. Document provider cooperation requirements and bounded waits where applicable.

Define and test callback concurrency. A conservative initial contract can serialize calls per runtime while allowing separate runtimes to operate concurrently. Do not hold registry locks while invoking extension callbacks. Shared package-level extension state remains the extension author's synchronization responsibility.

Catalog, capabilities, and authentication

Each model definition should include:

• Stable model ID and display name.
• Context window and maximum output size.
• Supported input modalities.
• Tool, streaming, and reasoning capabilities.
• Optional pricing with clear units; missing prices mean unknown, not free.
• Optional provider-specific settings, with validation.

Use an instance-local catalog overlay for model picker entries, available-model queries, capability checks, context limits, and usage accounting. Do not mutate global catalog maps.

Define how models without metadata behave when AllowUnknownModels is true. Use explicit unknown capability/limit behavior rather than invented values.

Authentication should support no-auth, API-key/environment configuration, and externally managed authentication. An optional status callback can report configured/unconfigured state and a user-facing setup instruction. Status checks must be cancellable and must not cause model generation. Define refresh/invalidation behavior for cached status.

Secrets must not appear in model metadata, persisted sessions, logs, or error diagnostics. Keep authentication status separate from credentials.

Global SDK catalog functions cannot implicitly know which Kit instance owns an extension. Keep their scope explicit and provide or extend instance-aware queries. Decide and document whether kit models loads extensions; if it does, explain that this executes trusted extension initialization code.

Test plan

All required automated tests must run without API keys, external services, or network access. Use scripted callbacks, local fixture servers where needed, channels for synchronization, and bounded test timeouts. Avoid sleep-based ordering assertions.

Interpreter and API tests

• Load the fake provider from extension source through the real Yaegi loader.
• Resolve all exported types/constants and invoke create, stream, and close callbacks.
• Reject invalid/empty IDs, malformed metadata, duplicate models, missing callbacks, and provider collisions.
• Include a regression fixture with helpers declared before registration.
• Verify diagnostics contain extension/provider/model identity without secrets.

Deterministic stream tests

Case	Fake response	Required assertion
Text	Start, two deltas, end, finish	Output is exact; events remain ordered.
Usage	Known input/output/cache counts	Counts reach SDK/UI/session accounting correctly without double counting.
Reasoning	Reasoning deltas and signature	Visible reasoning is correct; signature survives supported replay.
Empty completion	Finish only	Valid empty result, no hang.
Invalid stream	Delta before start, duplicate finish, or event after finish	Defined protocol error; no invalid downstream events.
Backend error	Structured authentication/rate-limit/transport error	Error code and retry policy are preserved.
Panic	Callback panics	Host survives and closes owned resources.
Unsupported input	Image or feature not declared	Explicit error rather than silent content loss.
Consumer stop	Emitter returns false	Provider stops; no later event is accepted.

Tool-loop integration tests

1. Register a host add tool with an execution counter and a permission/hook counter.
2. First fake response emits stable ID call-1, name add, and JSON arguments in multiple chunks, then finishes with tool_calls.
3. Assert arguments assemble to {"a":2,"b":3} and the tool executes exactly once.
4. Assert Kit's hooks, execution-start/end events, and tool result events run through normal paths.
5. Assert the next provider request contains the tool call and result with the same ID.
6. Return final text 5; assert final output and stored conversation order.
7. Repeat with malformed JSON, unknown tool, duplicate ID, and cancellation during argument streaming. No incomplete/invalid call may execute.
8. Repeat with a denied tool. Verify existing denial semantics and ensure the actual tool body does not run.
9. Verify hidden tools are not exposed and tool wrappers are not bypassed, including code-mode behavior where enabled.

Lifecycle and isolation tests

• Startup with an extension-only provider succeeds without global registration.
• Model switching resolves the new provider and closes the replaced runtime once.
• Temporary named completion closes its runtime once, on success and failure.
• A subagent can use an inherited provider; parent and child runtime state do not mix.
• Two Kit instances can use the same provider ID with different fake responses without interference.
• Cancellation while blocked before the first event reaches Done, releases the fake worker, and returns promptly.
• Shutdown and cancellation do not leak goroutines, subprocesses, listeners, or temporary files owned by the adapter/fixture.
• Reload during a blocked stream keeps the old callback valid; new calls use the new definition according to the documented policy.
• Failed reload preserves the previous working provider.
• Removed providers cannot create new runtimes; existing runtimes follow the documented retirement policy.
• Concurrent completions and subagents pass race detection and do not deadlock.

Catalog and persistence tests

• Extension models appear once in the instance model list and picker data.
• Limits and capabilities affect normal Kit validation.
• Unknown prices remain distinguishable from zero prices.
• Auth status unavailable/configured transitions update availability without generation.
• Save and resume a conversation containing text, tool calls/results, reasoning signatures, and namespaced replay data; verify the reconstructed request.
• Branching or editing history sends the selected branch, not stale provider-local history.
• A missing extension on resume produces an actionable provider-unavailable error.

Suggested commands:

go test -race ./internal/extensions/... ./internal/models/... ./pkg/kit/...
go test -race ./...
go vet ./...
go build -o output/kit ./cmd/kit

Implementation plan

• Define versioned extension DTOs, event ordering, cancellation, errors, and runtime ownership.
• Add registration storage and validation to loader/runner; export symbols.
• Implement a compiled model adapter with protocol validation and cleanup.
• Connect registrations to startup before provider-dependent validation.
• Use one instance-aware resolver for startup, switching, completions, and subagents.
• Add instance catalog metadata and authentication status integration.
• Implement transactional reload and runtime retirement semantics.
• Add the no-network fake provider example and interpreter tests.
• Add stream, tool-loop, persistence, lifecycle, and race tests.
• Document extension provider creation, installation, limitations, and migration compatibility.

Acceptance criteria

• An installed interpreted extension can add a provider without rebuilding Kit.
• Its model can be selected at startup and through normal model switching.
• Text, reasoning, tool calls, usage, and errors use Kit's normal event paths.
• Kit remains the sole executor of its tools.
• Provider registrations and model metadata do not leak between Kit instances.
• Cancellation, cleanup, reload, completions, and subagents have tested behavior.
• The fake provider and tool round-trip tests pass without credentials or external network access.
• New public extension types do not expose internal dependency names or require interpreted implementations of host interfaces.
• Existing compiled providers and extensions continue to work unchanged.