Hacktoberfest 2026 : les issues que les mainteneurs ont marquées pour octobre, ouvertes et accessibles aux débutants. Parcourir les issues Hacktoberfest

[Feature]: Keystroke telemetry capture and shortcut HUD visualization

Ouverte
  • Quelque chose ne va pas ?
#1,317 2 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Les mainteneurs répondent en général sous 1 jour

Personne n'a encore pris cette issue.

  • #1320 par @rainyflash — fermée sans fusion

Évaluation

Difficulté
5/5
Temps estimé
Plus d'une semaine
Accessibilité débutants
1/100
Type d'issue
Fonctionnalité
Clarté
Clairement spécifiée
Activité
À l'abandon
Stack technique
electron, typescript
Domaine
desktop, full-stack

Piste de recherche

The request spans recording telemetry, IPC and MCP tools, the AxcutDocument model, PreviewCanvas, SceneDescription, FloatingInspector, and V4Timeline; begin by reading the existing cursor telemetry pipeline at technical-documentation/architecture/cursor.md. Check the open pull request #1320 before taking this on. Done means privacy-filtered keystroke events can be captured, edited as timeline callouts, and rendered with configurable HUD settings.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

Search existing issues
  • I have searched the existing issues
Is your feature request related to a problem?

During software tutorials, coding demos, and shortcut-heavy workflows, viewers can see where the cursor clicks (via .cursor.json telemetry) but cannot see which keyboard shortcuts or key combinations the presenter pressed (e.g. Ctrl+Shift+P, Cmd+K, Enter, Esc). Manually adding text annotations for every shortcut is tedious and error-prone. (Related to #1286)

Describe the solution you'd like

Add end-to-end Keystroke Telemetry Capture & Shortcut HUD Visualization:

  1. Keystroke Telemetry Sidecar & Recording:
    • Capture keyboard shortcut events and special keys during recording into a <video>.keys.json sidecar (or alongside cursor telemetry) with privacy-safe filtering (capturing modifier combinations like Ctrl/Cmd/Alt/Shift + key and navigation/function keys by default, avoiding raw password/text typing).
    • Provide IPC and MCP/agent support (getKeystrokeTrack / setKeystrokeTrack) to read or inject recorded keystroke events.
  2. Document Model & Timeline (AxcutDocument):
    • Add clip-anchored keystrokeRanges (and editor-level keystroke HUD appearance settings: enabled, position, theme dark/light/glass, scale, holdDurationMs, modifiersOnly) so keystrokes can be automatically derived from telemetry or authored/edited/deleted as timeline pills.
  3. Editor UI (Preview HUD, Timeline & Inspector):
    • Render stylized keycap badges (KeystrokeOverlay) in PreviewCanvas and SceneDescription with smooth pop-in/fade-out transitions.
    • Allow toggling, styling, and editing keystroke callouts in FloatingInspector and V4Timeline.
  4. AI Agent & MCP Tools:
    • Expose getKeystrokeTrack, addKeystroke, setKeystroke, and keystroke HUD settings in setEditorSettings / getCurrentDocument so agents can inspect recorded shortcuts or place shortcut callout pills on the timeline.
Describe alternatives you've considered

Baking keystrokes directly into the recorded video pixels was rejected for the same reason cursor rendering uses .cursor.json telemetry: keeping keystrokes as structured data allows restyling, repositioning, filtering out mistakes, and staying crisp under zoom or aspect-ratio changes.

Additional context

Complements the existing cursor telemetry pipeline (technical-documentation/architecture/cursor.md).

Langage dominant
TypeScript
Étoiles
3.7k
Forks
231
Merge moyen
15 h 10 min
PR mergées (30 j)
324

Préparer son environnement

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de getopenscreen/openscreen

Toutes les issues de getopenscreen/openscreen

Issues similaires

Plus d'issues TypeScript

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.