nextflow-io/nextflow

Support container streaming for Google Batch

Open

#4,719 创建于 2024年2月7日

在 GitHub 查看
 (3 评论) (0 反应) (0 负责人)Groovy (784 fork)batch import
executor/google-batchgood first issue

仓库指标

Star
 (3,382 star)
PR 合并指标
 (平均合并 3天 18小时) (30 天内合并 38 个 PR)

描述

Google Batch now supports container streaming (i.e. lazy loading container images) which improves startup time and reduces overhead, especially for fine-grained workloads.

https://cloud.google.com/batch/docs/use-image-streaming

For us to support it in Nextflow we just need to add a config option for enableImageStreaming.

Note that container streaming is currently only supported for container images in Artifact Registry. This is likely because they need to modify the container image format to support the lazy loading, so this requirement is unlikely to change.

贡献者指南