Runtime Model Selection + Provider Fallback Policies

Summary

Add runtime model controls so users (or admins) can select which LLM profile to use per conversation, with backend fallback policies when a model is unavailable or rate-limited.

Problem

The assistant currently rotates Gemini models internally, but there is no explicit model/profile selection in product workflows. This makes behavior hard to tune, compare, and debug across conversations.

Proposed Solution

Introduce a model registry + conversation-level model profile selection, with safe fallback behavior.

Backend

• Add provider/model profile config (example: balanced, fast, high_quality).
• Add endpoints:
  • GET /api/models (available profiles/models and status)
  • PUT /api/conversations/:id/model-profile (set profile for conversation)
• Persist selected profile on conversation metadata.
• Implement deterministic fallback policy when selected model fails:
  • retries with bounded backoff
  • fallback to profile-compatible backup model
  • structured reason code in logs/telemetry
• Expose selected/actual model in response metadata for observability.

Frontend

• Add model-profile selector in chat UI (conversation scoped).
• Display active profile and fallback notice when backup model is used.
• Keep selector disabled for guest mode if desired by policy (or provide a default read-only profile).

Docs

• Update openapi.yaml with model/profile endpoints and schemas.
• Document environment variables and profile config in README.

Acceptance Criteria

• User can set model profile per conversation and value persists.
• Chat requests use selected profile by default.
• On model/provider failure, fallback policy executes without user-facing crash.
• Response payload includes effective model information.
• Metrics/logs record fallback count, failure reason, and selected profile.
• Tests cover profile selection, persistence, and fallback behavior.

Out of Scope

• Automatic benchmarking dashboard.
• Full multi-provider cost optimizer.

Suggested Tasks

• Add model profile config + validation layer.
• Implement /api/models and conversation profile update route.
• Wire profile selection into chat generation path.
• Add fallback policy + telemetry.
• Add chat UI profile selector and state handling.
• Add tests and docs updates.