Métricas do repositório
- Stars
- (777 estrelas)
- Métricas de merge de PR
- (Mesclagem média 1m) (1 fundiu PR em 30d)
Description
Problem
Right now, the 3d avatar, and human realistic avatars have a specific appearance. We would like to support custom appearances.
Description
This feature will have an interface where the user can take a video of themselves (or a single picture?) and generate a SMPL texture map.
Here's an example of a texture map from MediaPipe face - https://github.com/apple2373/mediapipe-facemesh We could do something similar for hands and body using the "enable_segmentation" option.
MediPipe also supports hair segmentation, which may also be useful.
This can be done using a specific model, working on a single image, but probably best to compose on a video, prompting the user what we might be missing - hands, other side of face, etc.
Alternatives
Can use DensePose to generate SMPL maps, but that requires sending an image of the user to a server, and I don't think that's very good.
Or perhaps generate the full avatar, not only the texture map - https://github.com/sergeyprokudin/smplpix but that would even more likely require a server.
If we generate a SMPL/UV texture, it would be cross compatible. If we generate our own, whatever we can, it might be easier to create in the browser.
Additional context
It will support https://github.com/sign/translate/issues/33