Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

Alternative concurrency approaches for Lambda Managed Instances

Abierto
  • ¿Algo no está bien?
#1,120 0 comentarios 2 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 11 días

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
25/100
Tipo de issue
Nueva funcionalidad
Claridad
Necesita aclaración
Estado de actividad
Estancado
Stack tecnológico
rust
Área
backend, cloud

Línea de trabajo

Comienza con la implementación existente de concurrency-tokio y las APIs de run_concurrent, y revisa después el contexto de seguimiento en #1067. Compara los enfoques propuestos de worker, semaphore, rayon y compio, y establece un diseño acotado con criterios de aceptación; actualmente, el issue da seguimiento a una discusión en lugar de a una implementación definida.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Alternative concurrency approaches for Lambda Managed Instances

Follows up on #1067.

Background

With the upcoming release of Lambda Managed Instances, we are shipping the concurrency-tokio feature flag and run_concurrent APIs. The initial implementation takes a deliberately conservative approach, optimizing for footgun resistance over raw throughput. It uses bounded concurrency with separate worker tasks, each independently long-polling the Runtime API. This design helps reduce the impact of long-polling or slow handler futures locking up the tokio runtime, which an cause catastrophic tail latency if they delay the tasks polling the lambda runtime API.

This might be a good default for most users, but it leaves performance on the table for well-behaved handlers that don't risk starving the runtime. We intentionally released under a tokio-namespaced feature flag to leave room for alternative concurrency strategies in the future.

This issue tracks discussion of those alternatives. Glad to help scope work for anybody that picks it up, though I
don't have immediate plans to myself.

How the Lambda Runtime API manages concurrency

To my understanding, the Lambda Runtime API data plane itself manages concurrency. The /next long-poll endpoint only returns an invocation when the runtime has capacity to handle it, so the data plane acts as the ultimate backpressure mechanism. A single worker polling /next in a loop will only receive work as fast as the data plane is willing to hand it out.

It's unclear to me whether a single /next connection can be saturated to the point where multiple pollers improve throughput (likely a Lambda team maintainer knows more). Either way, multiple worker loop futures are compatible with the alternative designs below: you can have multiple pollers while still spawning handler invocations as separate stealable tasks.

A Few Options

1. Unbounded tokio worker (no concurrency limits)

Offer an alternative tokio implementation that runs a single worker talking to the Runtime API, spawning tasks directly onto a tokio runtime with no concurrency bounds. This relies on the Lambda runtime data plane to manage concurrency and processes incoming requests at maximum possible speed.

We could also self-manage concurrency via a semaphore instead of sequential worker processing loops.

Tradeoffs:

  • For well-behaved handlers, this is likely more efficient (less scheduling overhead, no artificial backpressure).
  • The main downside is footgun resistance: it is easier to lock up worker futures when long-polling tasks cause tokio scheduling contention. This can be catastrophic for tail latency.

This could be exposed as a tunable strategy on a builder within the existing concurrency-tokio feature, rather than requiring a new feature flag. Strawman API:

use lambda_runtime::ConcurrentConfig;

// Current default: N independent workers, each with their own /next loop.
// Conservative, footgun-resistant. These two are equivalent:
lambda_runtime::run_concurrent(handler).await?;
let config = ConcurrentConfig::builder()
    .strategy(Strategy::WorkerPerSlot)
    .build();
lambda_runtime::run_concurrent_with_config(handler, config).await?;

// Option A: single /next poller, unbounded task spawning.
// Relies on the Lambda data plane for backpressure.
let config = ConcurrentConfig::builder()
    .strategy(Strategy::Unbounded)
    .build();
lambda_runtime::run_concurrent_with_config(handler, config).await?;

// Option B: N workers, but concurrency bounded by a semaphore
// instead of one-request-per-worker.
let config = ConcurrentConfig::builder()
    .strategy(Strategy::Semaphore { permits: 64 })
    .build();
lambda_runtime::run_concurrent_with_config(handler, config).await?;
2. Rayon support for CPU-dominated work

For handlers that are CPU-bound, integrating with rayon's work-stealing thread pool would allow offloading compute-heavy tasks without blocking the async runtime.

This would require either a broader executor abstraction or a dedicated implementation behind a feature flag (e.g. rayon-concurrency). Strawman API:

// feature = "rayon-concurrency"
use lambda_runtime::rayon::run_concurrent;

// Handler runs on a rayon thread pool. The runtime manages
// async I/O (polling /next, posting responses) on a small
// tokio runtime, and dispatches handler invocations to rayon.
run_concurrent(|event: LambdaEvent<Request>| {
    // CPU-heavy sync work here, no async required.
    // rayon's work-stealing distributes across cores.
    Ok(Response { /* ... */ })
}).await?;
3. Compio / thread-per-core async

Support compio or a similar thread-per-core async model. This is relevant for workloads where per-core isolation matters and shared-nothing architectures are preferred.

More generally, this could take the form of an async executor abstraction that allows plugging in alternative runtimes beyond tokio. Strawman API:

// feature = "compio-concurrency"
use lambda_runtime::compio::run_concurrent;

// Each core runs its own async runtime with a dedicated /next poller.
// No cross-thread work stealing; shared-nothing by default.
run_concurrent(handler).await?;

Alternatively, a more general executor abstraction:

use lambda_runtime::{run_concurrent_with_executor, Executor};

struct MyExecutor;
impl Executor for MyExecutor {
    // Spawn, block_on, etc.
}

run_concurrent_with_executor(handler, MyExecutor).await?;

Design considerations

  • Options 1 could easily fit within the existing concurrency-tokio feature as a builder configuration on the concurrency strategy.
  • Options 2 and 3 are larger efforts that need either a general executor abstraction or separate feature-flagged implementations.
  • Footgun resistance vs. performance is the central tension: stricter concurrency bounds protect against tail latency blowups, while looser bounds improve throughput for well-behaved handlers.
Lenguaje dominante
Rust
Estrellas
3.6k
Forks
398
Merge medio
7 d 4 h
PR fusionados (30 d)
3

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de aws/aws-lambda-rust-runtime

Todos los issues de aws/aws-lambda-rust-runtime

Issues similares

Más issues de Rust

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.