Measure Localization Accuracy and Pipeline Performance

Aberta
#753 0 comentários 0 reações 0 responsáveis Ver no GitHub

Mantenedores costumam responder em até 2 dias

Ninguém assumiu esta issue ainda.

Avaliação

Dificuldade
5/5
Tempo estimado
Mais de uma semana
Facilidade para iniciantes
22/100
Tipo de issue
Funcionalidade
Clareza
Razoavelmente clara
Status de atividade
Ativa
Stack de tecnologia
rust
Domínio
performance, robotics

Direção de pesquisa

The issue names no files, so start by finding the preprocessing-to-NDT pipeline in the repository and noting its stages: crop box filtering, voxel grid downsampling and NDT scan matching. Check whether any timing or memory hooks already exist. Done means a documented metric and timing-boundary definition, plus a machine-readable export with sample counts and a reproducible command. Confirm the scope with maintainers first, since the work is large and design-heavy.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

Establish a reproducible evaluation workflow for localization accuracy, execution time, end-to-end latency, and memory usage of the Awkernel preprocessing-to-NDT pipeline.

Scope:

  • Evaluate position and orientation errors against timestamp-aligned ground-truth poses where available
  • Measure execution time for crop box filtering, voxel grid downsampling, and NDT scan matching
  • Measure end-to-end latency from scan availability at the pipeline input to publication of the corresponding localization result
  • Distinguish processing time from queueing, scheduling, and data-transfer delays
  • Measure peak memory usage, separating map preparation, persistent map storage, and per-scan processing
  • Record workload characteristics, including point counts, NDT iteration counts, and convergence outcomes

Acceptance criteria:

  • Metric definitions, timing boundaries, coordinate frames, and measurement methods are documented.
  • Accuracy results include position and orientation error distributions, with ground-truth sources and alignment procedures specified.
  • Datasets without ground truth are explicitly identified; reference-output differences are not reported as ground-truth accuracy.
  • Timing results include median, tail percentiles, and maximum observed values, together with sample counts.
  • Failed, non-converged, and dropped scans are reported rather than silently excluded.
  • Peak memory measurements identify which allocations and buffers are included.
  • Hardware, software versions, parameters, replay rate, warm-up procedure, and instrumentation overhead are recorded.
  • Results are exported in a machine-readable format with reproducible evaluation commands.

Out of scope:

Formal worst-case execution-time guarantees, EKF integration, vehicle-control evaluation, and broad scheduler comparisons.

Linguagem predominante
Rust
Estrelas
282
Forks
12
Merge médio
2d 9h
PRs com merge (30d)
28

Preparar o ambiente

Este projeto não oferece contêiner de desenvolvimento, Dockerfile nem guia de contribuição, então a configuração fica por sua conta: comece pelo README e veja nosso guia da primeira contribuição para os passos gerais.

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de tier4/awkernel

Todas as issues de tier4/awkernel

Issues semelhantes

Mais issues de Rust

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.