kornia/kornia-rs

[Feature]: Fisheye camera model and undistortion

Open

#822 geöffnet am 16. März 2026

Auf GitHub ansehen
 (13 Kommentare) (0 Reaktionen) (1 zugewiesene Person)Rust (188 Forks)auto 404
enhancementhelp wantedtriage

Repository-Metriken

Stars
 (675 Stars)
PR-Merge-Metriken
 (PR-Metriken ausstehend)

Beschreibung

🚀 Feature Description

The current kornia-3d camera module only has PinholeCamera. We need a FisheyeCamera implementing the Kannala-Brandt equidistant projection model, with project/unproject so that downstream consumers can build fisheye → pinhole undistortion pipelines.

PRs for this issue should target the feat/slam-utils branch.

Proof of correctness

Use an actual Hilti fisheye image as your test sample. Images can be extracted from the challenge dataset using the https://github.com/hilti-research/hilti-trimble-slam-challenge-2026.

Include in your PR:

  • A before/after image showing the raw fisheye frame and the undistorted pinhole result

Acceptance criteria

  • FisheyeCamera with project and unproject using the Kannala-Brandt equidistant model
  • Round-trip test: project(point) → unproject(pixel) recovers the original ray direction
  • Before/after visualization on a real Hilti fisheye image
  • Unit tests with known 3D points → expected pixel coordinates

📂 Feature Category

Rust Core Library

💡 Motivation

kornia-slam needs to support fisheye cameras for datasets like https://github.com/Hilti-Research/hilti-trimble-slam-challenge-2026, which uses dual Kannala-Brandt equidistant fisheye cameras (Insta360, One-RS, 1472×1440, ~200° FoV per lens).

💭 Proposed Solution

What to implement

  Add `FisheyeCamera` to kornia-3d (new src/camera/ module),
  using the Kannala-Brandt equidistant projection model:

  /// Fisheye camera using the Kannala-Brandt equidistant projection model.
  ///
  /// Projects a 3D point (x, y, z) to pixel (u, v) using:
  ///   r = sqrt(x² + y²)
  ///   theta = atan2(r, z)
  ///   theta_d = theta + k1*θ³ + k2*θ⁵ + k3*θ⁷ + k4*θ⁹
  ///   u = fx * theta_d * (x/r) + cx
  ///   v = fy * theta_d * (y/r) + cy
  pub struct FisheyeCamera {
      pub fx: f64,
      pub fy: f64,
      pub cx: f64,
      pub cy: f64,
      pub k1: f64,
      pub k2: f64,
      pub k3: f64,
      pub k4: f64,
  }

  impl FisheyeCamera {
      /// Project a 3D point to pixel coordinates.
      /// Returns None if the point is behind the camera or at the optical center.
      pub fn project(&self, point: &Vec3F64) -> Option<[f64; 2]>;

      /// Unproject a pixel to a unit bearing ray in camera frame.
      /// Requires solving theta_d → theta via Newton's method.
      pub fn unproject(&self, pixel: [f64; 2]) -> Vec3F64;
  }

📚 Library Reference

🔄 Alternatives Considered

No response

🎯 Use Cases

No response

📝 Additional Context

No response

🤝 Contribution Intent

  • I plan to submit a PR to implement this feature
  • I'm requesting this feature but not planning to implement it

Contributor Guide