Feature request: support prometheus 3.0+

No source files or tests for the custom OpenTelemetry Collector are identified, and the issue says that component is not publicly available. Start by reproducing the scrape with Prometheus 3.4.0 and 2.55.1 and compare the emitted metric names; done means AST metrics are parsed and displayed correctly on Prometheus 3.0+.

Describe the bug

The custom OpenTelemetry Collector integrated with the Application Study Tool (AST) is observed to write metrics in an incorrect format (dots instead of underscores) when scraped by Prometheus versions newer than 2.x (specifically, versions 3.0 and above). This leads to metrics not being properly parsed and displayed in Prometheus's "Metrics Explorer."

To Reproduce

1. Deploy the Application Study Tool stack, ensuring Prometheus version 3.0 or higher is used.
2. Navigate to the Prometheus UI.
3. Access the 'Metrics Explorer'.
4. Observe that metrics originating from the AST's custom OpenTelemetry Collector are either missing or malformed, as depicted in the screenshot below.

Expected behavior

Metrics pushed by the custom OpenTelemetry Collector should be in a format compatible with Prometheus versions 3.0+, allowing for correct scraping, parsing, and display within the Prometheus UI and downstream monitoring tools.

Screenshots

Other informations

• Server OS: RHEL 8.10
• Container Orchestrator: Podman
• Prometheus Version: Tested with 3.4.0 and 2.55.1

Prometheus (v3.4.0) command flags:

spec:
  containers:
  - args:
    - --config.file=/etc/prometheus/prometheus.yml
    - --config.auto-reload-interval=30s
    - --web.listen-address=0.0.0.0:9090
    - --web.config.file=/etc/prometheus/web-config.yml
    - --web.external-url=/prometheus
    - --web.route-prefix=/prometheus
    - --web.enable-lifecycle
    - --web.enable-remote-write-receiver
    - --web.remote-write-receiver.accepted-protobuf-messages=prometheus.WriteRequest
    - --web.enable-otlp-receiver
    - --web.console.libraries=/etc/prometheus/console_libraries
    - --web.console.templates=/etc/prometheus/consoles
    - --storage.tsdb.path=/prometheus/data/
    - --storage.tsdb.retention.time=30d
    - --enable-feature=auto-reload-config
    - --enable-feature=otlp-native-delta-ingestion
    - --log.level=debug
    - --log.format=logfmt

• OTEL Custom collector: Tested with 0.9.3

Workaround
Currently, the issue can be mitigated by downgrading Prometheus to the latest 2.x stable release (2.55.1)

Additional Questions/Considerations

• The latest supported Prometheus version (v2.55.1) is approximately 7 months old. Are there any plans to update the Application Study Tool's OpenTelemetry Collector to support newer Prometheus versions (3.0+)?
• We would be interested in contributing to a fix for this issue. However, the source code for the custom OpenTelemetry Collector is not publicly available. Are there any plans to open-source this component?

Thank you very much.