Hacktoberfest 2026: as issues que os mantenedores marcaram para outubro, abertas e boas para iniciantes. Ver issues do Hacktoberfest

[Feature] Content-level governance boundary — PII detection, prompt injection defense, and cost budgets for Bot tool arguments

Aberta
  • Algo errado?
#86 0 comentários 0 reações 0 responsáveis Ver no GitHub

Mantenedores costumam responder em até 1 dia

Ninguém assumiu esta issue ainda.

Avaliação

Dificuldade
5/5
Tempo estimado
Mais de uma semana
Facilidade para iniciantes
30/100
Tipo de issue
Funcionalidade
Clareza
Razoavelmente clara
Status de atividade
Ativa
Stack de tecnologia
typescript
Domínio
backend-api-design, security

Direção de pesquisa

Comece localizando o pipeline do gateway do servidor descrito na issue, especialmente o ponto após a avaliação de CEL e antes da chamada ao computador, e revise a interface existente do plugin de boundary. Compare os pontos de entrada propostos para o plugin de gateway, o preset de boundary e o middleware do agente; o trabalho estará concluído quando um caminho de integração definido e seu comportamento de governança de conteúdo estiverem especificados e cobertos pelos testes apropriados do projeto.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

Problem

OpenBot's gateway handles infrastructure-level governance brilliantly — CEL policies on browser URLs, file paths, shell commands, and MCP tool classification. But there's a gap at the content level:

  1. PII in tool arguments — A Bot can pass an SSN, credit card number, or email to a tool without the gateway noticing, because CEL evaluates tool.name and page.url, not the argument content itself.

  2. Prompt injection via tool inputs — If a retrieved document or user message contains "ignore previous instructions...", that flows through the AG-UI endpoint into the Bot's tool arguments. The gateway sees a valid tool call to a permitted target — the adversarial content is invisible to CEL.

  3. Per-session cost budgets — The gateway records actions but doesn't enforce cumulative spend limits. A Bot can loop tool calls indefinitely without hitting a cost ceiling.

  4. Secret leakage in arguments — A Bot might pass api_key=sk-abc123... as a tool argument. The gateway correctly redacts secrets from the audit trail, but doesn't prevent the call from executing with the secret in-flight.

Proposed Solution

Add a content-level governance evaluator that runs alongside CEL, inspecting the content of tool arguments (not just tool names and targets). This would fire between "policy evaluated" and "call the computer" in the gateway pipeline.

One implementation: TealTiger — deterministic, in-process, no LLM in the evaluation path, <2ms per check. Already provides:

Check What It Catches
PII detection SSN, credit card, email, phone in tool args
Secret detection API keys, tokens, credentials
Prompt injection Instruction override, DAN, delimiter injection, system prompt override (8 patterns)
Cost budget Cumulative session spend with hard-stop
Argument validation SQL injection, path traversal in tool inputs

It could integrate as:

Option A — Gateway plugin:

// In the server gateway, after CEL evaluation, before calling computer
const contentDecision = await tealtigerEvaluate({
  toolName: resolvedTarget.tool,
  arguments: resolvedTarget.input,
  botId: bot.id,
  sessionCost: auditRow.cumulativeCost,
});
if (contentDecision.action === "DENY") {
  return refuse(contentDecision.reason);
}

Option B — Boundary preset:
A preset at /admin/boundaries that enables content-level checks:

# Content governance preset
name: "Content Protection"
rules:
  - scope: tool_arguments
    check: pii
    action: deny
    categories: [ssn, credit_card, email]
  - scope: tool_arguments
    check: prompt_injection
    action: deny
  - scope: session
    check: cost_budget
    action: deny
    max_usd: 5.00

Option C — Agent-level middleware:
The AG-UI agent (LangGraph, etc.) wraps itself with TealTiger middleware before registering as a Bot endpoint. Content governance happens inside the agent, infrastructure governance happens at the gateway. Defense-in-depth.

Use Cases
  • Financial services Bot handling account data — must not leak PII in tool arguments to third-party MCP servers
  • Research Bot consuming external documents — needs injection defense before those documents influence tool calls
  • Multi-Bot deployment with cost allocation — each Bot needs a per-session budget that hard-stops regardless of CEL rules
  • Compliance audit — need evidence that content-level checks were evaluated (not just that the URL/tool was permitted)
Why not just extend CEL?

CEL is great for structural checks (tool.name == "browser_navigate" && page.host != "internal.company.com"). But content scanning requires:

  • Regex pattern matching across variable-length text (PII/injection)
  • Stateful tracking (cumulative cost)
  • Confidence thresholds (injection detection isn't binary)

These are better served by a purpose-built content evaluator running alongside CEL rather than shoehorning them into CEL expressions.

Additional Context
  • TealTiger: https://github.com/agentguard-ai/tealtiger (Apache 2.0)
  • Already integrated with AG2 (built-in), Haystack, LangChain, CrewAI, Strands
  • NVIDIA Inception member
  • No external server required — runs in-process
  • Happy to contribute a PR once the boundary plugin interface stabilizes

This is complementary to existing governance, not a replacement. CEL handles infrastructure boundaries. Content evaluation handles what's inside the calls that pass those boundaries.

Linguagem predominante
TypeScript
Estrelas
6.6k
Forks
914
Merge médio
19h 22min
PRs com merge (30d)
140

Preparar o ambiente

  • Inclui um Dockerfile ou arquivo Docker Compose
  • Tem um modelo de pull request
  • Sem guia de contribuição

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de CopilotKit/OpenBot

Todas as issues de CopilotKit/OpenBot

Issues semelhantes

Mais issues de TypeScript

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.