[FEATURE]: LLM Observability & Admin Audit Dashboard (LangSmith)

Abierto
#170 2 comentarios 0 reacciones 0 asignados Ver en GitHub

@innovate-with-ravi ya está trabajando en esto.

Desde el 19/10/2026.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
18/100
Tipo de issue
Nueva funcionalidad
Claridad
Necesita aclaración
Estado de actividad
Activo
Stack tecnológico
fastapi, nextjs, python, typescript

Línea de trabajo

The payload names no concrete files. It points to the Groq LLM calls added in PR #158, a FastAPI backend and a Next.js App Router route at /admin/audit. Before any code, the scope needs maintainer agreement: whether LangSmith is the right tool, how the route is protected, and what the telemetry endpoint returns. Done is not defined yet.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

enhancement
Feature and its Use Cases
Context

As Perspective scales its LLM pipelines (especially with the recent Groq integrations in PR #158 and upcoming dynamic features), we currently lack visibility into prompt execution, token usage, latency, and cost tracking.

I am proposing the integration of LangSmith for LLM Observability & Tracing, alongside a native Next.js Admin Dashboard so maintainers can monitor AI performance without leaving the application.

Proposed High-Level Architecture
  1. Backend Tracing Layer (FastAPI + LangSmith):
    • Integrate the LangSmith SDK into the Python backend.
    • Wrap the Groq LLM calls to automatically log prompt inputs, outputs, token counts, and execution latency.
  2. Frontend Admin Dashboard (Next.js 15):
    • Build a protected /admin/audit route in the Next.js App Router.
    • Design a clean UI to visualize token usage, average response times, and a table of recent AI generations.
  3. API Integration:
    • Create a dedicated FastAPI endpoint that fetches the telemetry data from LangSmith and serves it securely to the frontend dashboard.
Experience & Next Steps

I have worked extensively with LangSmith and recently implemented this exact observability dashboard architecture for an AI project during a hackathon, so I can port that setup over here efficiently.

This is a high-level architectural draft. Once assigned, I will flesh out the exact UI components, route protection strategy, and backend telemetry setup.

I would love to take ownership of this feature as part of my contributions for GSoC 2027. Let me know if you agree with this direction!

Additional Context

Here's a small proof of work:
Image

Code of Conduct
  • I have joined the Discord server and will post updates there
  • I have searched existing issues to avoid duplicates
Lenguaje dominante
TypeScript
Estrellas
58
Forks
90
Merge medio
11 h 51 min
PR fusionados (30 d)
3

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de AOSSIE-Org/Perspective

Todos los issues de AOSSIE-Org/Perspective

Issues similares

Más issues de TypeScript

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.