ml5js/ml5-library

Add useful keypoint getters for Facemesh

Open

#1,028 opened on Jul 12, 2020

View on GitHub
 (11 comments) (0 reactions) (0 assignees)JavaScript (6,136 stars) (905 forks)batch import
facemeshfeature requestgood first issue

Description

See the keypoint map: https://github.com/tensorflow/tfjs-models/blob/master/facemesh/mesh_map.jpg

In the ml5.js Facemesh model, we can consider providing some functions that return an array of keypoints for useful facial landmark ranges like the left eye, right eye, mouth, etc. For example:

facemeshInstance.getLeftEye() // returns [<keypoint 22>, <keypoint 23>, <keypoint 24>, ...]

Contributor guide

Tech stack
javascript
Domain
machine learning
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
ml5 library fundamentalsFacemesh keypoint mapJavaScript array methods
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
65
Research direction
Start by examining the Facemesh model implementation in the ml5 library, particularly the keypoint array from the TensorFlow.js Facemesh model. Look at the referenced keypoint map (mesh map.jpg) to understand the indices for facial features. Then create new methods like getLeftEye() that filter the keypoints by predefined index ranges. Check the existing codebase for similar patterns (e.g., other getter methods) to follow the same style.