[FEATURE]: LLM Observability & Admin Audit Dashboard (LangSmith)

Aperta
#170 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub

@innovate-with-ravi ci sta già lavorando.

Dal 19/10/2026.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
18/100
Tipo di issue
Funzionalità
Chiarezza
Da chiarire
Stato di attività
Attiva
Stack tecnologico
fastapi, nextjs, python, typescript

Direzione di ricerca

The payload names no concrete files. It points to the Groq LLM calls added in PR #158, a FastAPI backend and a Next.js App Router route at /admin/audit. Before any code, the scope needs maintainer agreement: whether LangSmith is the right tool, how the route is protected, and what the telemetry endpoint returns. Done is not defined yet.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

enhancement
Feature and its Use Cases
Context

As Perspective scales its LLM pipelines (especially with the recent Groq integrations in PR #158 and upcoming dynamic features), we currently lack visibility into prompt execution, token usage, latency, and cost tracking.

I am proposing the integration of LangSmith for LLM Observability & Tracing, alongside a native Next.js Admin Dashboard so maintainers can monitor AI performance without leaving the application.

Proposed High-Level Architecture
  1. Backend Tracing Layer (FastAPI + LangSmith):
    • Integrate the LangSmith SDK into the Python backend.
    • Wrap the Groq LLM calls to automatically log prompt inputs, outputs, token counts, and execution latency.
  2. Frontend Admin Dashboard (Next.js 15):
    • Build a protected /admin/audit route in the Next.js App Router.
    • Design a clean UI to visualize token usage, average response times, and a table of recent AI generations.
  3. API Integration:
    • Create a dedicated FastAPI endpoint that fetches the telemetry data from LangSmith and serves it securely to the frontend dashboard.
Experience & Next Steps

I have worked extensively with LangSmith and recently implemented this exact observability dashboard architecture for an AI project during a hackathon, so I can port that setup over here efficiently.

This is a high-level architectural draft. Once assigned, I will flesh out the exact UI components, route protection strategy, and backend telemetry setup.

I would love to take ownership of this feature as part of my contributions for GSoC 2027. Let me know if you agree with this direction!

Additional Context

Here's a small proof of work:
Image

Code of Conduct
  • I have joined the Discord server and will post updates there
  • I have searched existing issues to avoid duplicates
Lingua principale
TypeScript
Stelle
58
Fork
90
Merge medio
11h 51m
PR unite (30g)
3

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di AOSSIE-Org/Perspective

Tutte le issue di AOSSIE-Org/Perspective

Issue simili

Altre issue su TypeScript

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.