nextflow-io/nextflow

Support container streaming for Google Batch

Open

#4,719 opened on 2024年2月7日

GitHub で見る
 (3 comments) (0 reactions) (0 assignees)Groovy (3,382 stars) (784 forks)batch import
executor/google-batchgood first issue

説明

Google Batch now supports container streaming (i.e. lazy loading container images) which improves startup time and reduces overhead, especially for fine-grained workloads.

https://cloud.google.com/batch/docs/use-image-streaming

For us to support it in Nextflow we just need to add a config option for enableImageStreaming.

Note that container streaming is currently only supported for container images in Artifact Registry. This is likely because they need to modify the container image format to support the lazy loading, so this requirement is unlikely to change.

コントリビューターガイド

技術スタック
なし
領域
cloudinfrastructure
Issue 種別
feature
難度新しい貢献者にとっての実装難度です。1 は非常に小さな変更、5 は専門的な作業です。
3
推定時間経験ある貢献者が調査、実装、テスト、pull request 準備にかけるおおよその時間範囲です。
1-3 hours
活動状況Issue が今どれだけ取り組みやすいかを示します。新しい、活発、古い、ブロック中、またはメンテナー入力待ちなどです。
fresh
明確さ期待される変更、受け入れ条件、次の手順がどれだけ明確かを示します。
clear
前提条件
Understanding of Nextflow configurationBasic knowledge of Google Batch
初心者向け度初回貢献者にどれだけ取り組みやすいかを 1-100 で推定したスコアです。
60
調査方針
The issue requests adding a configuration option `enableImageStreaming` for Google Batch. Start by examining Nextflow's configuration loading mechanism, likely in files related to Google Batch executor or general configuration. Review the linked Google Cloud documentation on image streaming. Check comments on the issue for any additional context or implementation ideas. There are no linked PRs yet, so the implementation is open. The change involves adding a boolean config key and passing it to the Google Batch API when creating a job.