[tracking] Jaeger v2 based on OpenTelemetry collector

Proposal: https://docs.google.com/document/d/1s4_6VgAS7qAVp6iEm5KYvpiGw3h2Ja5T5HpKo29iv00/view

This replaces the previous attempt: #3500.

Work tracked in the project https://github.com/orgs/jaegertracing/projects/3

Roadmap

Proof of concept

• A single binary with memory storage and all-in-one functionality
  • #4766
• Validate that the current storage configuration works for primary / archive storage.
• Default configuration solution (do not require config file for all-in-one)
  • #4842
• Demonstrate Kafka exporter / receiver (ideally from OTEL), compatible with existing Jaeger Kafka writer
  • #4868
• Define v2 storage API that operates on OTLP batches, see if internal & remote APIs can be identical
  • #5052
  • #5079
• Benchmarks #4869

Feature parity

• Extend configuration of jaeger_storage_exporter to support all existing backends -- project in https://github.com/jaegertracing/jaeger/issues/5084
• Metrics & Traces -- project in https://github.com/jaegertracing/jaeger/issues/5084
• What to do with admin port / services -- https://github.com/jaegertracing/documentation/issues/750
• Adaptive sampling processor -- project in https://github.com/jaegertracing/jaeger/issues/5084

Prepare for Beta

• Add to integration tests
• Add to release pipeline
• Documentation

Prepare for GA

• Helm chart https://github.com/jaegertracing/helm-charts/issues/534
• Operator https://github.com/jaegertracing/jaeger-operator/issues/2411
• Clarify https://github.com/jaegertracing/jaeger#version-compatibility-guarantees

Enhancements

• SPM built-in
• Integrate with OTEL collector builder ocb
• Print active configuration (unresolved: Print OTEL configuration #5223), https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/27003