Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Move CameraViewRenderer to a shared package (@webarkit/camera-view)

Open
  • Is something wrong?
#370 0 comments 0 reactions 1 assignee View on GitHub

Maintainers usually reply within 1 day

@kalwalt is already working on this.

Since Oct 25, 2026.

Assessment

This issue has not been assessed yet.

Description

:rocket: enhancement code-design documentation Typescript

Summary

Move CameraViewRenderer (and getProcessGeometry) out of ARnft into a small shared package, for example @webarkit/camera-view. ARnft, aframe-nft, webarkit-studio and future WebARKit trackers would then share one implementation of:

  • opening the camera: getUserMedia, picking the device by cameraLabel, the rear camera on phones, and the fallbacks;
  • fitting the video into the processing frame: the 320 px 4:3 letterbox, the opt-in rotatePortrait rotation, and recomputation on video resize;
  • producing frames: ImageData for the tracker, throttled to targetFrameRate.

Why

The same code exists in several copies today, each with its own fixes:

  • ARnft/src/renderers/CameraViewRenderer.ts and getProcessGeometry() in src/utils/ARnftUtils.ts;
  • ARnft/examples/js/cameraViewRenderer.js;
  • Aframe-nft/src/cameraViewRenderer.js;
  • webarkit-studio's own capture and letterbox code (src/runtime/nodes/jsartoolkit-nft.ts).

Each consumer also rescales the tracker's projection by hand from that geometry (pw / w, ph / h).

jsartoolkitNFT's data-oriented tracker facade (webarkit/jsartoolkitNFT#720, project 2) keeps camera handling out of the tracker (decision D17 in its design). process() takes frames already at the processing size. A FrameGeometry value, passed to projectionFor(viewport, geometry), describes where the video sits inside the frame:

interface FrameGeometry {
  frameWidth: number; frameHeight: number;             // pw x ph
  videoWidth: number; videoHeight: number;             // vw x vh
  x: number; y: number; width: number; height: number; // ox, oy, w, h
  rotated: boolean;                                    // rot
}

These fields are exactly what getProcessGeometry() already computes. A shared camera package that exposes this value removes every hand-written projection rescale from the consumers. It also keeps the camera code tracker-agnostic: it would work the same for jsartoolkitNFT and @webarkit/nft-tracker.

Proposed API (sketch)

import { CameraView } from "@webarkit/camera-view";

const view = new CameraView(videoElement);
await view.initialize({ facingMode: "environment", cameraLabel, width: { min, max }, targetFrameRate: 30, rotatePortrait: true });

view.geometry;          // FrameGeometry: plain data, structured-cloneable (can be posted to a worker)
view.getImage();        // ImageData at frameWidth x frameHeight (as today)
view.getImageBitmap();  // new: an ImageBitmap to transfer to a worker without copying
view.addEventListener("geometrychange", ...); // replaces the global "videoResize" event
view.destroy();

The class name is open (CameraView, ARCameraSource, WebARCamera, …).

Where the package lives: options

  1. A new repository, webarkit/camera-view, published as @webarkit/camera-view.
    • Pro: independent releases, a small dependency for every consumer, a clear owner.
    • Con: one more repository to maintain.
  2. A package in the webarkit monorepo, next to cv-backend-spec and nft-tracker.
    • Pro: shared tooling and CI; nft-tracker can depend on it directly.
    • Con: it ties ARnft and aframe-nft releases to the monorepo's process, and needs agreement with the monorepo maintainers.
  3. Keep it in ARnft and export it from the ARnft package.
    • Pro: the least work.
    • Con: aframe-nft and webarkit-studio would depend on all of ARnft (three.js glue, its workers) just for the camera. This does not solve the duplication cleanly.

Recommendation: option 1, or option 2 if the monorepo maintainers agree. Option 3 only as a stopgap.

Migration plan

  1. Extract.
    • Copy CameraViewRenderer and getProcessGeometry into the new package, keeping today's behaviour bit for bit. That covers the processing size, the letterbox, rotatePortrait, device selection and throttling.
    • Add the geometry: FrameGeometry getter and getImageBitmap().
    • Add unit tests for getProcessGeometry covering landscape, portrait, rotatePortrait, and a 4:3 / 16:9 / 9:16 table.
    • Publish 0.1.0.
  2. ARnft.
    • Depend on the package.
    • Keep CameraViewRenderer / ICameraViewRenderer as thin re-exports (deprecated) for one minor version, so existing apps keep working.
    • Keep the videoResize event, emitted from the new geometrychange.
    • Remove src/utils geometry code that becomes unused.
    • Update examples/js/cameraViewRenderer.js to the package.
  3. aframe-nft: replace src/cameraViewRenderer.js with the package.
  4. webarkit-studio: replace its capture and letterbox code with the package.
  5. With the jsartoolkitNFT facade (webarkit/jsartoolkitNFT#720, project 3, migrating the consumers): pass view.geometry to tracker.projectionFor(viewport, view.geometry) and drop the hand-written projection rescale in each consumer.

Steps 1–4 do not depend on the jsartoolkitNFT facade and can start any time. Step 5 needs the facade released.

Open questions

  • Which of the three locations above to use, and the final class name.
  • Whether the package should also provide a canvas or video display helper, for the CSS sizing and placement of the video behind the 3D scene that each consumer does, or stay capture-only.
  • The minimum browser target. ImageBitmap and OffscreenCanvas are fine for all current browsers.

Refs webarkit/jsartoolkitNFT#720.

Dominant language
TypeScript
Stars
235
Forks
50
Avg merge
11h 4m
Merged PRs (30d)
13

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from webarkit/ARnft

All issues in webarkit/ARnft

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.