Hacktoberfest 2026 : les issues que les mainteneurs ont marquées pour octobre, ouvertes et accessibles aux débutants. Parcourir les issues Hacktoberfest

Evaluate replacing MinIO in the test fixture: the vendor has withdrawn public images

Ouverte
  • Quelque chose ne va pas ?
#1,560 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Personne n'a encore pris cette issue.

Évaluation

Difficulté
4/5
Temps estimé
3-5 jours
Accessibilité débutants
45/100
Type d'issue
Refactorisation
Clarté
Clairement spécifiée
Activité
Active
Stack technique
docker, python
Domaine
data-engineering, testing-qa

Piste de recherche

Start with tests/conftest.py and the DJ_TEST_MINIO_IMAGE fixture, then compare the three listed candidate images and the existing Chainguard image from cold pulls. Run the five object-storage suites, including all 44 test_gc tests, and verify directory-shaped objects, public credential-free access, active maintenance, and changes limited to the fixture. Done means the evidence identifies a suitable image or supports retaining MinIO.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

enhancement

tests/conftest.py starts MinIO to exercise the object-storage paths. MinIO has withdrawn from public distribution, so that dependency now rests on a third party continuing to publish builds of an archived codebase. #1559 restored CI by switching to Chainguard's build; this issue is about whether to keep depending on MinIO at all.

Filing it separately on purpose: #1559 was the release-eve fix and needed to change no test code. This one needs measurement, and picking wrong here produces the worst kind of green — a suite that passes while the behavior it claims to cover is untested.

Why the current position is not stable
Oct 2025 MinIO stopped publishing free community builds
Feb 2026 open-source repository archived
2026-09-11 minio/minio deleted from Docker Hub — the whole repository, every tag
2026-09-24 quay.io/minio/minio began requiring authentication

Chainguard's build is a good mitigation and the right call for now — it is public, credential-free, drop-in for MinioContainer, and rebuilt continuously, where the last community image is a year stale.

But continuous rebuilds address the base image and dependencies. A defect in MinIO's own Go code has no upstream fix, because upstream is archived — worth confirming against Chainguard's advisory data rather than taking from me. And the arrangement depends on a third party's free tier continuing, for software whose vendor has already shown its direction. That is a reasonable place to stand for a release, and a poor place to stand indefinitely.

What a replacement has to satisfy

This is the part that decides it, and it rules out the obvious shortcut.

DataJoint reaches object storage through fsspec/s3fs, not boto3. The storage layer calls:

isdir · walk · size · exists · rm · open · pipe_file · get_file · put · put_file · cat_file

S3 has no directories. isdir and walk are emulated by s3fs over list_objects_v2 using common prefixes and zero-byte directory markers, and the emulation is sensitive to how a server reports prefixes, delimiters, and empty keys.

Garbage collection is built directly on that emulation — gc.py:205 and gc.py:255 both walk a store recursively to decide what is orphaned. A server that diverges on listing will not throw; it will return a different set of keys, and GC will classify live files as orphans or miss real ones. test_gc is 44 tests and exists precisely to catch that.

So a lightweight S3 mock is the tempting choice and the risky one. Fidelity on listing matters more here than startup time.

Candidates

All three pull publicly today, verified:

Image Notes
chrislusf/seaweedfs real object store with an S3 gateway, Apache-2.0, actively maintained
localstack/localstack full AWS emulator, very widely used, heavier
adobe/s3mock purpose-built test mock, starts in 0.12s against MinIO's 1.0s — and the one most likely to diverge on listing

Staying on cgr.dev/chainguard/minio is also a legitimate outcome if the alternatives cost more fidelity than the dependency risk is worth. The point of the exercise is to answer that with evidence rather than assume it.

Acceptance

  • The five object-storage suites pass against the candidate from a cold pull, with any local image cache deleted — 118 tests: test_gc (44), test_object (46), test_npy_codec (22), test_codecs (4), test_attach (2).
  • test_gc passes specifically, including the prefix-scoping and orphan-detection cases, since that is where listing semantics bite.
  • Directory-shaped objects round-trip: isdir and walk agree with what MinIO reported for the same tree.
  • The image is public, needs no credentials, and is actively maintained.
  • Nothing changes outside tests/conftest.py and the fixture; DJ_TEST_MINIO_IMAGE already makes the source swappable.

Out of scope

What DataJoint recommends to customers for production object storage. That is a product question, it touches an existing partnership conversation, and it should not be decided as a side effect of a test-fixture choice.

Langage dominant
Python
Étoiles
197
Forks
98
Merge moyen
23 h 9 min
PR mergées (30 j)
6

Préparer son environnement

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de datajoint/datajoint-python

Toutes les issues de datajoint/datajoint-python

Issues similaires

Plus d'issues Python

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.