[Feature Request] Support access to metric meter in multi-processed workers

Bắt đầu bằng cách đọc _MetricMeter trong temporalio/runtime.py và _Context trong temporalio/activity.py, sau đó so sánh việc giao tiếp của heartbeat manager hiện có với process pool. Được xem là hoàn tất khi các metric do các process con phát ra đến worker cha thông qua một multiprocessing queue và được áp dụng thông qua metric meter thực tế.

Is your feature request related to a problem? Please describe.

Currently, the SDK doesn't extend custom metric support via the Runtime's MetricMeter to activities running within a multi-processed worker. This means users have to figure out how to support this themselves, which adds complexity to the project and may be infeasible for some.

Describe the solution you'd like

Ideally, I'd hope to see metrics wired up across the process pool out of the box. So following a similar approach to the existing heartbeat manager, metrics emitted by the child process would be sent via a multiprocessing queue to the parent worker process.

Possible high-level approach:

• Provide temporalio.common.MetricMeter implementation alternate to _MetricMeter that is set in _Context automatically in process pool workers.
• Instead of being backed by temporalio.bridge.metric.MetricMeter, they would push messages onto a queue consumed by the parent process, which would then rely those actions against the real temporalio.bridge.metric.MetricMeter.

This might be easier Python side, than trying to wire up the Rust code under the hood.

Worth pointing out there are on-going proposals to do this natively in OTEL: https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/4968.

I'd be happy to support this work, if you think the plan sounds reasonable.

Additional context

Background

The previous work to add custom metric support, https://github.com/temporalio/sdk-python/pull/384, at the time didn't include support for multi-processed worker:

NOTE: This does not return a metric meter for sync activities that are non-threaded (i.e. multiprocess) because a meter cannot be shared across processes and it's a large effort to support metric shipping across processes for this rarely used activity type

Case to support this feature

The multi-processed worker provides a lot of value, at least on my project which uses Temporal for its typical system integration use cases but also for managing some reasonably heavy CPU bound workloads (e.g. processing PDFs, NLP, small batch jobs, etc.) all within one tool.

It might be the least frequently used worker type, but it provides several key benefits of its own:

• It highlights a clear design principle to split up CPU bound workloads at the workflow level, i.e. fanning out, and then keep each activity single threaded/single processed to optimise performance, avoid under/over utilisation of the underlying CPU cores, and support better auto-scaling decisions. Its easier to understand when size and scheduling of tasks on the process pool is managed by the worker itself, rather than a free-for-all that could occur if async/threaded activities were to manage their own/shared processes or pools.
• It provides out of the box support for heartbeating in activities running in the process pool, which users would need to wire up themselves if managing processes in the asyncio/threaded workers.
• Python is a common data science language, so a process pool based worker would be the go to over async/threaded

Existing solution

Besides having to set up and configure a second OTEL metrics toolchain (MeterProvider, processors, exporters, and using different meter and metric classes), users would need to:

• ensure the toolchain is initialised correctly to properly configure the process pool
• be concerned with different default mp_context behaviours on different OSs, e.g. maybe they develop on MacOS which defaults to spawn, but deploy in a linux based image, which defaults to fork (the default will change to forkserver Python 3.14)
• ensure with the global metric provider isn't accidentally inherited by the child process, which can cause deadlocks, or understand fork pre init hooks.

Once the process pool is configured, we'd then need to get metrics out of the process correctly, e.g.:

• Tagging metrics with the process ID pid and using OTLPMetricExporter provides a simple solution, since each process would just push its own metrics to a collector. However, this would lead to high-cardinality, increased costs, and further complexity down the line.

  Note: the metric payloads scraped/pushed to the backend need to be uniquely labeled, otherwise you get out-of-order writes, conflicts, and are otherwise left with meaningless metric values interleaved from multiple ambiguous sources. (It can also cause you to exceed the platform's data points per minute (DPM), which in Grafana Clouds case DPM=1, i.e. scrape/emit every 60s.

• The initial simplicity would then come at the cost of having to set up some external metric aggregation. E.g. within an OTEL collector's transformprocessor using aggregate_on_attributes to aggregate out the PID label. This also requires sensitive handling using interval, batch, metricstarttime, etc. to ensure you capture all the metrics within one interval.