Hacktoberfest 2026: die Issues, die Maintainer für den Oktober markiert haben – offen und einsteigerfreundlich. Hacktoberfest-Issues durchsuchen

[Feature]: Keystroke telemetry capture and shortcut HUD visualization

Offen
  • Stimmt etwas nicht?
#1,317 2 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

Dieses Issue hat noch niemand übernommen.

  • #1320 von @rainyflash — ohne Merge geschlossen

Bewertung

Schwierigkeit
5/5
Geschätzter Aufwand
Über eine Woche
Anfängerfreundlichkeit
1/100
Issue-Typ
Feature
Klarheit
Klar beschrieben
Aktivitätsstatus
Veraltet
Tech-Stack
electron, typescript
Bereich
desktop, full-stack

Rechercherichtung

The request spans recording telemetry, IPC and MCP tools, the AxcutDocument model, PreviewCanvas, SceneDescription, FloatingInspector, and V4Timeline; begin by reading the existing cursor telemetry pipeline at technical-documentation/architecture/cursor.md. Check the open pull request #1320 before taking this on. Done means privacy-filtered keystroke events can be captured, edited as timeline callouts, and rendered with configurable HUD settings.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

Search existing issues
  • I have searched the existing issues
Is your feature request related to a problem?

During software tutorials, coding demos, and shortcut-heavy workflows, viewers can see where the cursor clicks (via .cursor.json telemetry) but cannot see which keyboard shortcuts or key combinations the presenter pressed (e.g. Ctrl+Shift+P, Cmd+K, Enter, Esc). Manually adding text annotations for every shortcut is tedious and error-prone. (Related to #1286)

Describe the solution you'd like

Add end-to-end Keystroke Telemetry Capture & Shortcut HUD Visualization:

  1. Keystroke Telemetry Sidecar & Recording:
    • Capture keyboard shortcut events and special keys during recording into a <video>.keys.json sidecar (or alongside cursor telemetry) with privacy-safe filtering (capturing modifier combinations like Ctrl/Cmd/Alt/Shift + key and navigation/function keys by default, avoiding raw password/text typing).
    • Provide IPC and MCP/agent support (getKeystrokeTrack / setKeystrokeTrack) to read or inject recorded keystroke events.
  2. Document Model & Timeline (AxcutDocument):
    • Add clip-anchored keystrokeRanges (and editor-level keystroke HUD appearance settings: enabled, position, theme dark/light/glass, scale, holdDurationMs, modifiersOnly) so keystrokes can be automatically derived from telemetry or authored/edited/deleted as timeline pills.
  3. Editor UI (Preview HUD, Timeline & Inspector):
    • Render stylized keycap badges (KeystrokeOverlay) in PreviewCanvas and SceneDescription with smooth pop-in/fade-out transitions.
    • Allow toggling, styling, and editing keystroke callouts in FloatingInspector and V4Timeline.
  4. AI Agent & MCP Tools:
    • Expose getKeystrokeTrack, addKeystroke, setKeystroke, and keystroke HUD settings in setEditorSettings / getCurrentDocument so agents can inspect recorded shortcuts or place shortcut callout pills on the timeline.
Describe alternatives you've considered

Baking keystrokes directly into the recorded video pixels was rejected for the same reason cursor rendering uses .cursor.json telemetry: keeping keystrokes as structured data allows restyling, repositioning, filtering out mistakes, and staying crisp under zoom or aspect-ratio changes.

Additional context

Complements the existing cursor telemetry pipeline (technical-documentation/architecture/cursor.md).

Vorherrschende Sprache
TypeScript
Sterne
3.7k
Forks
231
Ø Merge
16 Std. 38 Min.
Gemergte PRs (30 T.)
327

Entwicklungsumgebung

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus getopenscreen/openscreen

Alle Issues in getopenscreen/openscreen

Ähnliche Issues

Weitere Issues zu TypeScript

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.