feat(api+ui): streaming responses (SSE) for /query

Goal

Return assistant tokens as they are generated and render them live in the chat, ChatGPT-style.

Scope

• Backend: add POST /query/stream that streams text/event-stream with events:
  • data: {"token":"...","done":false} … data: {"done":true,"sources":[...]}
• Frontend: hook a ReadableStream/SSE client to incrementally append tokens.

Notes (FastAPI + LangChain + Ollama)

• LangChain ChatOllama supports streaming via callbacks.
• Use AsyncIteratorCallbackHandler to yield tokens.
• Wrap with `StreamingResponse(media_type="text/event-stream").
• Keep current /query for non-streaming.

Acceptance Criteria

• New endpoint /query/stream streams tokens.
• UI shows typing indicator and partial text updating.
• Cancelling the request (Esc or “Stop”) stops generation.
• Final sources appear identical to /query result.

Pseudocode (backend)

# app/main.py
from fastapi.responses import StreamingResponse
from langchain.callbacks import AsyncIteratorCallbackHandler
from langchain.schema.runnable import RunnableConfig

@app.post("/query/stream")
async def query_stream(req: QueryRequest):
    if store.vectorstore._collection.count() == 0:
        async def gen():
            yield 'data: {"token":"No documents ingested yet.","done":true,"sources": []}\n\n'
        return StreamingResponse(gen(), media_type="text/event-stream")

    cb = AsyncIteratorCallbackHandler()
    cfg: RunnableConfig = {"callbacks": [cb]}

    async def agen():
        # kick off generation in background
        import anyio
        async def run():
            RAG_CHAIN.stream({"input": req.question}, cfg)
        # stream tokens
        async with anyio.create_task_group() as tg:
            tg.start_soon(run)
            async for t in cb.aiter():
                yield f'data: {{"token":{t!r},"done":false}}\n\n'
        yield f'data: {{"done":true,"sources":[]}}\n\n'

    return StreamingResponse(agen(), media_type="text/event-stream")

Pseudocode (frontend)

// api/rag.ts
export async function streamAnswer(q: string, onToken: (t:string)=>void) {
  const res = await fetch(`${API}/query/stream`, {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ question: q }),
  });
  const reader = res.body!.getReader();
  const dec = new TextDecoder();
  let buf = "";
  for (;;) {
    const { value, done } = await reader.read();
    if (done) break;
    buf += dec.decode(value, { stream: true });
    for (const block of buf.split("\n\n")) {
      if (!block.startsWith("data:")) continue;
      const payload = JSON.parse(block.slice(5).trim());
      if (payload.token) onToken(payload.token);
      if (payload.done) return payload.sources ?? [];
    }
    buf = "";
  }
}

Test Plan

• Long prompt: tokens arrive in multiple chunks.
• Cancel midway → no network errors, UI stops typing.
• Fallback: old /query still works.

---

## 2) Source snippet preview (side panel with highlighted chunk)

**Title**

feat(ui): source preview with snippet & page link

**Labels**: `enhancement`, `good first issue`, `frontend`

**Body**
```md
### Goal
Make each source clickable to show a short snippet and (if PDF) the page number.

### Scope
- Use already-returned `sources[]` with `{source, page}`.
- Show a right-side Drawer/Panel with:
  - filename, page number
  - chunk text (first ~400 chars of the chunk used)
- No PDF rendering yet (that’s a separate issue).

### Implementation idea
- In `ChatTranscript`, wrap each source in a button.
- On click, open a Drawer (Chakra UI) showing snippet.
- For snippet text: include a small `context` echo in response:
  - On backend, attach `context[:400]` for each doc returned.

### Acceptance
- [ ] Each source opens a drawer with snippet and meta.
- [ ] Handles TXT/DOCX/PDF uniformly.
- [ ] Accessible focus trapping in Drawer.

---

3) Persist conversation history locally

Title

feat(ui): local conversation history (IndexedDB)

Labels: enhancement, good first issue, frontend

Body

### Goal
Keep chats after refresh; allow switching between recent conversations.

### Scope
- IndexedDB (use `idb-keyval` or `localforage`)
- Schema:
  ```ts
  type Chat = { id: string; createdAt: number; title: string; messages: Message[] };

• Save on every message; show a History list in Sidebar.

Acceptance

• New chats auto-title by first user message (truncated).
• Click history item to load messages into current transcript.
• Delete chat works.
• No backend changes required.

Notes

• Key by chat:ID.
• Keep max N (e.g., 20); drop oldest.

---

## 4) Multiple workspaces (collections)

**Title**

feat(workspaces): multiple collections with switcher

**Labels**: `enhancement`, `good first issue`, `backend`, `frontend`

**Body**
```md
### Goal
Support separate document sets (e.g., "Resume", "Research", "Manuals").

### Scope
- Backend: accept `workspace` string via header `X-Workspace` (or `?ws=` query).
- Use `Chroma(collection_name=f"cmd_{workspace}")`.
- Cache per-workspace Chroma instance (dict in `store.py`).
- Endpoints (`/ingest`, `/query`, `/stats`, `/reset`) read the workspace.

### Acceptance
- [ ] Workspace switcher UI (dropdown + “New…”).
- [ ] Each workspace has independent stats and chat.
- [ ] Reset affects only current workspace.

### Sketch (backend)
```py
# app/rag/store.py
_STORES: dict[str, Chroma] = {}

def get_store(ws: str) -> Chroma:
    if ws not in _STORES:
        _STORES[ws] = Chroma(collection_name=f"{settings.COLLECTION_NAME}_{ws}",
                             embedding_function=embeddings,
                             persist_directory=str(persist_dir))
    return _STORES[ws]

Tests

• Ingest in A does not change stats in B.
• Query in A returns only A’s sources.

---

## 5) PDF page thumbnails on hover (client-side with pdf.js)

**Title**

feat(ui): PDF page thumbnail preview on source hover

**Labels**: `enhancement`, `good first issue`, `frontend`

**Body**
```md
### Goal
When hovering a PDF source (with `page`), show a ~120px thumbnail of that page.

### Scope
- Use `pdfjs-dist` (client-side) to render a tiny canvas.
- Load the file via `<input type=file>` memory or via `/files/:name` if later added.
- Cache rendered thumbnails per `{filename,page}`.

### Acceptance
- [ ] Hover = popover with thumbnail + filename + page.
- [ ] Works in dark/light themes.
- [ ] Graceful fallback if file no longer available.

### Notes
- For first iteration, allow preview only for documents selected in this session (not persisted).

---

6) Keyboard UX: ↑ to edit last prompt, Enter to send, Shift+Enter for newline

Title

feat(ui): chat composer power keys (↑ edit, Enter send)

Labels: enhancement, good first issue, frontend, UX

Body

### Goal
Match chat UX standards.

### Behavior
- Enter: send
- Shift+Enter: newline
- ArrowUp at empty composer: load last user message into composer for quick edit

### Acceptance
- [ ] Works in light/dark.
- [ ] No conflicts with IME (composition events respected).
- [ ] Has tooltip describing shortcuts.

---

7) Ingestion progress & error surfacing

Title

feat(ui): ingestion progress bar + error details

Labels: enhancement, good first issue, frontend, backend

Body

### Goal
Make ingest status explicit (bytes saved → chunks split → vectors added).

### Scope
- Backend: send simple milestones in response JSON:
  `{ "saved_bytes": N, "chunks": M, "status": "splitting" | "embedding" | "done" }`
  (single response OK for first version)
- Frontend: stepper/progress bar in the Confirm panel.

### Acceptance
- [ ] Shows % based on milestones (33/66/100).
- [ ] Errors bubble up in a Toast with reason.
- [ ] Works for multi-file selection.

---

8) Voice input (browser speech-to-text)

Title

feat(ui): voice input for chat (Web Speech API)

Labels: enhancement, good first issue, frontend

Body

### Goal
Press mic icon → dictate a question → it fills the composer.

### Scope
- Web Speech API (if available), graceful fallback otherwise.
- Mic button toggles listening; waveform anim optional.

### Acceptance
- [ ] Start/stop works, text lands in composer.
- [ ] Shows “listening…” status.
- [ ] No backend changes.

---

Branch & commit guidance for contributors

• Branch name: feat/streaming-sse, feat/source-snippet, feat/workspaces, …

• Commit style:

  • feat(api): add /query/stream SSE endpoint
  • feat(ui): incremental token rendering with SSE
  • feat(ui): keyboard shortcuts (enter/shift+enter/up)
  • fix(api): guard no-docs case for stream)