[External Plugin]: hermes-jailbench
Nadie ha tomado este issue todavía.
Evaluación
- Dificultad
- 3/5
- Tiempo estimado
- 1-2 días
- Aptitud para principiantes
- 48/100
Línea de trabajo
Start by reviewing the pinned commit ec933be5e4c9f8c07caa0c9f19b45a925067d655 in hermes-labs-ai/hermes-jailbench and compare its submission details with the repository's contribution, security, and responsible AI policies. Check the referenced .claude-plugin/marketplace.json entry and verify that the plugin is suitable for inclusion; done means the review is complete and any required marketplace update is identified.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Plugin name
hermes-jailbench
Short description
Jailbreak regression benchmark for LLM endpoints with repeatable known-pattern attacks and deterministic scoring.
GitHub repository
hermes-labs-ai/hermes-jailbench
Plugin path inside the repository
No response
Ref to review
No response
Commit SHA to review
ec933be5e4c9f8c07caa0c9f19b45a925067d655
Version
0.2.1
License identifier
MIT
Author name
Hermes Labs
Author URL
https://github.com/hermes-labs-ai
Homepage URL
https://github.com/hermes-labs-ai/hermes-jailbench
Keywords
security, red-teaming, llm-security, benchmark, evaluation, developer-tools
Additional notes for reviewers
This is a defensive regression-testing tool: it runs a fixed corpus of publicly known jailbreak patterns against an LLM endpoint the user already controls or is authorized to test, and produces deterministic pass/fail scoring so a team can track whether their own safety mitigations regress between releases. It does not discover novel attacks, target third-party services, or exfiltrate data. Listed in the Hermes Labs Copilot marketplace manifest (https://github.com/hermes-labs-ai/copilot-plugins/blob/main/.claude-plugin/marketplace.json) as a root url source, tracking the main branch. The commit SHA above is main's HEAD at submission time, pinned here for immutable review. Submitted by the plugin's own maintainer (Hermes Labs).
Submission checklist
- The plugin lives in a public GitHub repository.
- The ref and/or sha I provided is immutable (release tag and/or full 40-character commit SHA), not a branch.
- This submission follows this repository's contribution, security, and responsible AI policies.
- This plugin is not already listed in the Awesome Copilot marketplace.
- Lenguaje dominante
- JavaScript
- Estrellas
- 39.2k
- Forks
- 5k
- Merge medio
- 1 d 18 h
- PR fusionados (30 d)
- 114
Guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de github/awesome-copilot
-
[aw] Learning Hub Updater failed Abiertoagentic-workflows
Dificultad 2/5 1-3 horas Aptitud para principiantes 70/100
github/awesome-copilot#3720 ·
-
agentic-workflows
Dificultad 2/5 1-3 horas Aptitud para principiantes 75/100
github/awesome-copilot#3667 · 1 comentario ·
-
external-plugin ready-for-review
Dificultad 1/5 Menos de una hora Aptitud para principiantes 85/100
github/awesome-copilot#3662 · 1 comentario ·
-
external-plugin ready-for-review
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
github/awesome-copilot#3586 · 2 comentarios ·
-
external-plugin ready-for-review
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
github/awesome-copilot#3585 · 2 comentarios ·
Todos los issues de github/awesome-copilot
Issues similares
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
-
area-deployment area-integrations triage:bot-seen
Dificultad 2/5 Medio día Aptitud para principiantes 86/100
-
Issue-Bug
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
sugarlabs/musicblocks#8924 ·
-
Dificultad 1/5 Menos de una hora Aptitud para principiantes 90/100
ArduPilot/ardupilot_wiki#8088 ·
-
[BUG] createTool tools cannot be registered with Mastra when exactOptionalPropertyTypes is enabled Abiertocustomer-eng status: needs triage
Dificultad 2/5 1-3 horas Aptitud para principiantes 84/100