Pluggable Tool-Calling & Action Framework (OAuth, Safe Sandboxes, Router)

Summary Add a first-class tool/plugin system so the chatbot can take actions (calendar, docs search, ticketing, DB lookups, etc.). Includes a secure tool registry, OAuth-permissioned connectors, JSON-Schema tool specs, a tool router, and a sandboxed execution layer with auditing.

---

Motivation

• Today the bot answers questions; many users want it to do things (create issues, search KBs, schedule meetings, query analytics).
• A pluggable framework lets the community ship integrations without touching core logic.
• Proper scopes + audit trails reduce security risks while enabling enterprise adoption.

---

Goals

1. Tool Registry & Specs

   • Each tool defines a JSON Schema (inputs/outputs), description, and safety hints.
   • Versioned tools with semantic versioning and deprecation paths.

2. Secure OAuth & Secrets

   • Per-user and/or org-level OAuth (e.g., Google, GitHub, Notion, Slack, Jira).
   • Encrypted secret storage; short-lived tokens; scope-aware consent UI.

3. Tool Router

   • LLM selects tools via function-calling (OpenAI/Anthropic/Fireworks) based on system prompt + tool metadata.
   • Deterministic fallbacks & retries; max parallel tool count per turn.

4. Sandboxed Execution

   • Server-side worker (Node runtime) with timeouts, memory caps, and egress allow-list.
   • Structured logs + redaction; no dynamic eval; fetch proxy with rate limits.

5. UX

   • “Apps & Tools” settings page: install, configure, revoke scopes.
   • In-chat action previews (what will run) + post-action citations of tool outputs.

6. Observability & Guardrails

   • Metrics: success/error/timeout rates per tool, latency histograms.
   • Policy checks (deny list domains, PII guard, confirmation prompts for destructive ops).

---

Non-Goals

• Building every integration now. Ship 3–4 reference tools only.
• Full marketplace UI (can come later).

---

Proposed Design

Tool Spec (example)

{
  "name": "create_github_issue",
  "version": "1.0.0",
  "description": "Create an issue in a GitHub repo",
  "schema": {
    "type": "object",
    "properties": {
      "owner": {"type":"string"},
      "repo": {"type":"string"},
      "title": {"type":"string"},
      "body": {"type":"string"}
    },
    "required": ["owner","repo","title"]
  },
  "scopes": ["repo:issues.write"],
  "timeoutMs": 8000,
  "egressAllow": ["api.github.com"]
}

Runtime

• Next.js API Route / Edge entry → Action Queue/Worker (Node) → Connector.
• Worker enforces timeout/memory caps; returns structured result (or tool_error).
• Router prompts LLM with available tools + few-shot examples; executes selected tools; streams results.

Reference Tools (MVP)

• web_search (Bing/SerpAPI-like), notion_search, github_create_issue, google_calendar_create_event.
• Each demonstrates OAuth, scopes, and result citations.

---

Acceptance Criteria

• Tool registry supports install/enable/disable, version pinning, and per-user/per-org config.
• OAuth flows for at least GitHub and Google (calendar) with token refresh.
• LLM tool-calling works across OpenAI / Anthropic / Fireworks providers.
• Sandboxed execution: timeout, mem cap, egress allow-list, retry w/ jitter.
• Chat UI shows pre-execution confirmation for destructive actions and post-action citations.
• Metrics dashboard: success rate, p95 latency per tool; audit log of invocations (who/when/what).
• Docs: How to build a community tool (template + example).

---

Tasks

• Core: tool registry, JSON Schema validation, Secrets service, token encrypt/decrypt.
• Router: provider-agnostic tool-calling + deterministic fallback.
• Worker: sandbox, fetch proxy, retries, redaction, structured logs.
• OAuth: GitHub + Google flows; revoke + reauth UX.
• UI: Tools settings page; in-chat action preview + result cards.
• Reference tools: web_search, notion_search, github_create_issue, google_calendar_create_event.
• Observability: counters, histograms, audit tables; feature flags.
• Security review & pentest checklist.
• Documentation & examples.

---