In-App Knowledge Source Manager (CRUD, Re-Index, and Manifest Sync)

Summary

Add a secure, in-app Knowledge Source Manager so authorized users can create, edit, delete, and re-index RAG sources without using CLI commands.

Problem

Knowledge management is currently CLI-driven (knowledge:upsert, knowledge:list, knowledge:sync), which makes operational updates slower and limits maintainability for non-CLI workflows.

Proposed Solution

Implement a small admin surface in the existing web app plus backend APIs for knowledge source lifecycle management.

Backend

• Add protected endpoints under /api/knowledge:
  • GET /api/knowledge (list with filters/pagination)
  • POST /api/knowledge (create + embed)
  • PATCH /api/knowledge/:id (update + re-embed)
  • DELETE /api/knowledge/:id (delete DB row + vectors)
  • POST /api/knowledge/:id/reindex (force re-embed)
  • POST /api/knowledge/sync (manifest sync from JSON payload)
• Reuse existing ingestion/chunking utilities from knowledgeBase.ts.
• Add authorization guard for admin-only actions.

Frontend

• Add a Knowledge admin page:
  • Table of sources (title/type/tags/chunkCount/updatedAt)
  • Create/edit form with validation
  • Delete confirmation modal
  • Re-index action and status feedback
  • Optional manifest paste/upload panel for batch sync

Docs

• Update openapi.yaml with new schemas and endpoints.
• Add README section for in-app knowledge management and permissions model.

Acceptance Criteria

• Authorized user can perform full CRUD on knowledge sources from UI.
• Creating/updating sources updates Pinecone vectors and chunkCount.
• Deleting source removes both MongoDB record and associated vectors.
• Re-index endpoint can rebuild vectors without changing source metadata.
• API returns clear validation errors for missing/invalid fields.
• OpenAPI and README reflect the final API and UI flow.

Out of Scope

• Public/self-serve source editing for normal users.
• Multi-tenant role hierarchy beyond a minimal admin gate.

Suggested Tasks

• Add /api/knowledge routes + auth guard.
• Implement list/create/update/delete/reindex/sync handlers.
• Add frontend Knowledge admin page and forms.
• Add unit/integration tests for route handlers.
• Document endpoints in openapi.yaml and README.