[FEATURE]: LLM Observability & Admin Audit Dashboard (LangSmith)
@innovate-with-ravi ya está trabajando en esto.
Desde el 19/10/2026.
Evaluación
- Dificultad
- 5/5
- Tiempo estimado
- Más de una semana
- Aptitud para principiantes
- 18/100
- Tipo de issue
- Nueva funcionalidad
- Claridad
- Necesita aclaración
- Estado de actividad
- Activo
- Stack tecnológico
- fastapi, nextjs, python, typescript
- Área
- backend, frontend, observability
Línea de trabajo
The payload names no concrete files. It points to the Groq LLM calls added in PR #158, a FastAPI backend and a Next.js App Router route at /admin/audit. Before any code, the scope needs maintainer agreement: whether LangSmith is the right tool, how the route is protected, and what the telemetry endpoint returns. Done is not defined yet.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Feature and its Use Cases
Context
As Perspective scales its LLM pipelines (especially with the recent Groq integrations in PR #158 and upcoming dynamic features), we currently lack visibility into prompt execution, token usage, latency, and cost tracking.
I am proposing the integration of LangSmith for LLM Observability & Tracing, alongside a native Next.js Admin Dashboard so maintainers can monitor AI performance without leaving the application.
Proposed High-Level Architecture
- Backend Tracing Layer (FastAPI + LangSmith):
- Integrate the LangSmith SDK into the Python backend.
- Wrap the Groq LLM calls to automatically log prompt inputs, outputs, token counts, and execution latency.
- Frontend Admin Dashboard (Next.js 15):
- Build a protected
/admin/auditroute in the Next.js App Router. - Design a clean UI to visualize token usage, average response times, and a table of recent AI generations.
- Build a protected
- API Integration:
- Create a dedicated FastAPI endpoint that fetches the telemetry data from LangSmith and serves it securely to the frontend dashboard.
Experience & Next Steps
I have worked extensively with LangSmith and recently implemented this exact observability dashboard architecture for an AI project during a hackathon, so I can port that setup over here efficiently.
This is a high-level architectural draft. Once assigned, I will flesh out the exact UI components, route protection strategy, and backend telemetry setup.
I would love to take ownership of this feature as part of my contributions for GSoC 2027. Let me know if you agree with this direction!
Additional Context
Here's a small proof of work:
Code of Conduct
- I have joined the Discord server and will post updates there
- I have searched existing issues to avoid duplicates
- Lenguaje dominante
- TypeScript
- Estrellas
- 58
- Forks
- 90
- Merge medio
- 11 h 51 min
- PR fusionados (30 d)
- 3
Preparar el entorno
- Sin Dockerfile ni archivo de Docker Compose
- Sin plantilla de pull request
- Leer la guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de AOSSIE-Org/Perspective
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 83/100
AOSSIE-Org/Perspective#191 ·
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 85/100
AOSSIE-Org/Perspective#190 ·
-
[BUG]: Missing input validation in URlRequest and ChatQuery causes 500 crashes and wasteful scraper executionPosiblemente ocupada @agarwalpranav0711 la tomó hace 25 días. Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 76/100
AOSSIE-Org/Perspective#178 ·
-
[BUG]: chunk_rag_data crashes with ValueError when perspective is a dict, silently dropping RAG vector storagePosiblemente ocupada @agarwalpranav0711 la tomó hace 36 días. Abiertobug
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
AOSSIE-Org/Perspective#175 ·
-
[BUG]: Scraper passes headers as query params in requests.get (User-Agent never sent)Posiblemente ocupada @Th-Shivam la tomó hace 50 días. Abiertobug
Dificultad 1/5 Menos de una hora Aptitud para principiantes 92/100
AOSSIE-Org/Perspective#161 · 1 comentario ·
Todos los issues de AOSSIE-Org/Perspective
Issues similares
-
documentation lens:human needs-triage
Dificultad 2/5 1-3 horas Aptitud para principiantes 73/100
thebristolsound/birdbrain#2246 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 1/5 Menos de una hora Aptitud para principiantes 78/100
Los mantenedores suelen responder en 1 día
-
A voting window past about 69.9 million hours stores a deadline that reads back as no deadlineAbiertobug ready-for-human
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
BoardGamesEmpire/board-games-empire-backend#1452 · 1 comentario ·
Los mantenedores suelen responder en 1 día
-
good first issue stretch
Dificultad 2/5 1-3 horas Aptitud para principiantes 65/100
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 76/100
Los mantenedores suelen responder en 1 día