Hacktoberfest 2026: die Issues, die Maintainer für den Oktober markiert haben – offen und einsteigerfreundlich. Hacktoberfest-Issues durchsuchen

Evaluate replacing MinIO in the test fixture: the vendor has withdrawn public images

Offen
  • Stimmt etwas nicht?
#1,560 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Dieses Issue hat noch niemand übernommen.

Bewertung

Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Anfängerfreundlichkeit
45/100
Issue-Typ
Refactoring
Klarheit
Klar beschrieben
Aktivitätsstatus
Aktiv
Tech-Stack
docker, python
Bereich
data-engineering, testing-qa

Rechercherichtung

Start with tests/conftest.py and the DJ_TEST_MINIO_IMAGE fixture, then compare the three listed candidate images and the existing Chainguard image from cold pulls. Run the five object-storage suites, including all 44 test_gc tests, and verify directory-shaped objects, public credential-free access, active maintenance, and changes limited to the fixture. Done means the evidence identifies a suitable image or supports retaining MinIO.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

enhancement

tests/conftest.py starts MinIO to exercise the object-storage paths. MinIO has withdrawn from public distribution, so that dependency now rests on a third party continuing to publish builds of an archived codebase. #1559 restored CI by switching to Chainguard's build; this issue is about whether to keep depending on MinIO at all.

Filing it separately on purpose: #1559 was the release-eve fix and needed to change no test code. This one needs measurement, and picking wrong here produces the worst kind of green — a suite that passes while the behavior it claims to cover is untested.

Why the current position is not stable
Oct 2025 MinIO stopped publishing free community builds
Feb 2026 open-source repository archived
2026-09-11 minio/minio deleted from Docker Hub — the whole repository, every tag
2026-09-24 quay.io/minio/minio began requiring authentication

Chainguard's build is a good mitigation and the right call for now — it is public, credential-free, drop-in for MinioContainer, and rebuilt continuously, where the last community image is a year stale.

But continuous rebuilds address the base image and dependencies. A defect in MinIO's own Go code has no upstream fix, because upstream is archived — worth confirming against Chainguard's advisory data rather than taking from me. And the arrangement depends on a third party's free tier continuing, for software whose vendor has already shown its direction. That is a reasonable place to stand for a release, and a poor place to stand indefinitely.

What a replacement has to satisfy

This is the part that decides it, and it rules out the obvious shortcut.

DataJoint reaches object storage through fsspec/s3fs, not boto3. The storage layer calls:

isdir · walk · size · exists · rm · open · pipe_file · get_file · put · put_file · cat_file

S3 has no directories. isdir and walk are emulated by s3fs over list_objects_v2 using common prefixes and zero-byte directory markers, and the emulation is sensitive to how a server reports prefixes, delimiters, and empty keys.

Garbage collection is built directly on that emulation — gc.py:205 and gc.py:255 both walk a store recursively to decide what is orphaned. A server that diverges on listing will not throw; it will return a different set of keys, and GC will classify live files as orphans or miss real ones. test_gc is 44 tests and exists precisely to catch that.

So a lightweight S3 mock is the tempting choice and the risky one. Fidelity on listing matters more here than startup time.

Candidates

All three pull publicly today, verified:

Image Notes
chrislusf/seaweedfs real object store with an S3 gateway, Apache-2.0, actively maintained
localstack/localstack full AWS emulator, very widely used, heavier
adobe/s3mock purpose-built test mock, starts in 0.12s against MinIO's 1.0s — and the one most likely to diverge on listing

Staying on cgr.dev/chainguard/minio is also a legitimate outcome if the alternatives cost more fidelity than the dependency risk is worth. The point of the exercise is to answer that with evidence rather than assume it.

Acceptance

  • The five object-storage suites pass against the candidate from a cold pull, with any local image cache deleted — 118 tests: test_gc (44), test_object (46), test_npy_codec (22), test_codecs (4), test_attach (2).
  • test_gc passes specifically, including the prefix-scoping and orphan-detection cases, since that is where listing semantics bite.
  • Directory-shaped objects round-trip: isdir and walk agree with what MinIO reported for the same tree.
  • The image is public, needs no credentials, and is actively maintained.
  • Nothing changes outside tests/conftest.py and the fixture; DJ_TEST_MINIO_IMAGE already makes the source swappable.

Out of scope

What DataJoint recommends to customers for production object storage. That is a product question, it touches an existing partnership conversation, and it should not be decided as a side effect of a test-fixture choice.

Vorherrschende Sprache
Python
Sterne
197
Forks
98
Ø Merge
23 Std. 9 Min.
Gemergte PRs (30 T.)
6

Entwicklungsumgebung

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus datajoint/datajoint-python

Alle Issues in datajoint/datajoint-python

Ähnliche Issues

Weitere Issues zu Python

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.