nextflow-io/nextflow

Support container streaming for Google Batch

开放

#4,719 创建于 2024年2月7日

 (3 条评论) (0 个反应) (0 位负责人)Groovy (784 个派生)batch import
executor/google-batchgood first issue

仓库指标

星标
 (3,382 个星标)
PR 合并指标
 (平均合并 16天 9小时) (30 天内合并 54 个 PR)

描述

Google Batch now supports container streaming (i.e. lazy loading container images) which improves startup time and reduces overhead, especially for fine-grained workloads.

https://cloud.google.com/batch/docs/use-image-streaming

For us to support it in Nextflow we just need to add a config option for enableImageStreaming.

Note that container streaming is currently only supported for container images in Artifact Registry. This is likely because they need to modify the container image format to support the lazy loading, so this requirement is unlikely to change.

贡献者指南