meilisearch/MeiliSearch

Improve Docker CI speed with cache

Open

#3,782 opened on May 25, 2023

View on GitHub
聽(2 comments)聽(0 reactions)聽(0 assignees)Rust聽(20,887 stars)聽(733 forks)batch import
contribution acceptedgithub actionshelp wantedmaintenanceOLD

Description

Currently, our Docker CI is really long to compute: around 1h30

A contributor implemented Docker cache on this PR to improve the speed. However, it did not work.

As I described here, if I run the CI twice on the same commit, the cache is taken into account. But if I run the CI on two different commits (1 line of code changed) the CI lasts 1h30 again, no cache used at all.

The contributor suggested some improvements we should do here, but the team is not a Docker expert and cannot prioritize it at the moment.

So any help or contributions are welcomed! 鉂わ笍

Also, if you have any other ideas to improve the speed of the CI that is not a cache solution, you are also more than welcome to open a PR 馃榿

Contributor guide

Tech stack
dockerrustshell
Domain
devopsci cdperformance
Issue type
performance
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
fresh
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
mostly clear
Prerequisites
DockerCI/CD concepts
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
35
Research direction
Investigate the current Docker CI workflow at .github/workflows/publish docker images.yml and the failed cache attempt in PR #3566. Focus on the suggestions in the PR comments (e.g., using Docker buildx caching or layer caching). Experiment with caching strategies across different commits to ensure cache reuse. Also consider non cache alternatives like reducing build steps or using lighter base images.
Improve Docker CI speed with cache 路 meilisearch/MeiliSearch#3782 | Good First Issue