facebookresearch/labgraph

Real-time hand tracking using WebCam

Open

#89 opened on Aug 26, 2022

View on GitHubContributor guide
 (0 comments) (0 reactions) (0 assignees)Python (46 forks)auto 404
good first issue

Repository metrics

Stars
 (165 stars)
PR merge metrics
 (PR metrics pending)

Description

🚀 Feature

Hand tracking enables the use of hands as an input method for AR/VR applications. When using hands as input modality, it delivers a new sense of presence, enhances social engagement, and delivers more natural interactions with fully tracked hands and articulated fingers.

MediaPipe can be used for real-time hand tracking. LabGraph can be used to record the data captured from a webcam. It could also be extended to other downstream applications.

This task is to use MediaPipe for real-time hand tracking and labgraph for data logging (hdf5 format).

Additional context

  1. MediaPipe can be found [here] (https://google.github.io/mediapipe/solutions/hands)
  2. The code should be added at folder is https://github.com/facebookresearch/labgraph/tree/main/devices/webcam/
  3. Create setup.py and README.md, where example can be found at: https://github.com/facebookresearch/labgraph/tree/main/extensions/labgraph_viz
  4. Add github action support, reference: https://github.com/facebookresearch/labgraph/actions/workflows/main.yml
  5. Add proper license header.

Contributor guide

Research direction
Install MediaPipe and LabGraph. Follow the example in the webcam directory to create a new node that integrates MediaPipe hand tracking. Refer to the MediaPipe documentation for hand landmark detection and the LabGraph documentation for creating nodes and data logging with HDF5. Also, create setup.py and README.md as per the labgraph viz example, and add GitHub Actions workflow by adapting the existing main.yml.
Tech stack
python
Domain
computer visionbackend
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
2
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
PythonGitMediaPipe
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
70