Measure Localization Accuracy and Pipeline Performance

Ouverte
#753 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Les mainteneurs répondent en général sous 2 jours

Personne n'a encore pris cette issue.

Évaluation

Difficulté
5/5
Temps estimé
Plus d'une semaine
Accessibilité débutants
22/100
Type d'issue
Fonctionnalité
Clarté
Plutôt claire
Activité
Active
Stack technique
rust

Piste de recherche

The issue names no files, so start by finding the preprocessing-to-NDT pipeline in the repository and noting its stages: crop box filtering, voxel grid downsampling and NDT scan matching. Check whether any timing or memory hooks already exist. Done means a documented metric and timing-boundary definition, plus a machine-readable export with sample counts and a reproducible command. Confirm the scope with maintainers first, since the work is large and design-heavy.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

Establish a reproducible evaluation workflow for localization accuracy, execution time, end-to-end latency, and memory usage of the Awkernel preprocessing-to-NDT pipeline.

Scope:

  • Evaluate position and orientation errors against timestamp-aligned ground-truth poses where available
  • Measure execution time for crop box filtering, voxel grid downsampling, and NDT scan matching
  • Measure end-to-end latency from scan availability at the pipeline input to publication of the corresponding localization result
  • Distinguish processing time from queueing, scheduling, and data-transfer delays
  • Measure peak memory usage, separating map preparation, persistent map storage, and per-scan processing
  • Record workload characteristics, including point counts, NDT iteration counts, and convergence outcomes

Acceptance criteria:

  • Metric definitions, timing boundaries, coordinate frames, and measurement methods are documented.
  • Accuracy results include position and orientation error distributions, with ground-truth sources and alignment procedures specified.
  • Datasets without ground truth are explicitly identified; reference-output differences are not reported as ground-truth accuracy.
  • Timing results include median, tail percentiles, and maximum observed values, together with sample counts.
  • Failed, non-converged, and dropped scans are reported rather than silently excluded.
  • Peak memory measurements identify which allocations and buffers are included.
  • Hardware, software versions, parameters, replay rate, warm-up procedure, and instrumentation overhead are recorded.
  • Results are exported in a machine-readable format with reproducible evaluation commands.

Out of scope:

Formal worst-case execution-time guarantees, EKF integration, vehicle-control evaluation, and broad scheduler comparisons.

Langage dominant
Rust
Étoiles
282
Forks
12
Merge moyen
2 j 9 h
PR mergées (30 j)
28

Préparer son environnement

Ce projet ne fournit ni conteneur de développement, ni Dockerfile, ni guide de contribution : l'installation est à votre charge. Commencez par son README, et consultez notre guide de la première contribution pour les étapes générales.

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de tier4/awkernel

Toutes les issues de tier4/awkernel

Issues similaires

Plus d'issues Rust

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.