milvus-io/milvus

[Feature]: running local milvus in singularity container or containers

Open

#33,964 opened on Jun 18, 2024

View on GitHub
 (10 comments) (0 reactions) (1 assignee)Go (44,298 stars) (4,000 forks)batch import
help wantedkind/feature

Description

Is there an existing issue for this?

  • I have searched the existing issues

Is your feature request related to a problem? Please describe.

Several of the environments I would like to use Milvus in are HPC-derived, and for various reasons require singularity/apptainer rather than docker. There is not a straightforward way to convert docker-compose.yml to singularity, and since I haven't compiled it, the contents of the docker containers (etcd, minio, milvus 2.2.16) are currently opaque to me so I can't easily convert it myself.

Describe the solution you'd like.

It would be wonderful if someone who's expert in milvus internals would make (convert, build) the existing docker solution to work with singularity. I think it would be straightforward, so am asking here before I scope out the feasibility of doing it myself. (I'm not lazy, just not a milvus expert and not formally tasked by employer with vector database stuff -- I'd like to be just a python-level user of it).

Describe an alternate solution.

I could build executables from source, but would have to do so for each of the HPC-style clusters I'd like to run on. Having singularity containers would enable more people (e.g. not just me) to use milvus for prototyping and potential solution development.

Anything else? (Additional Context)

See https://docs.sylabs.io/guides/latest/user-guide/ : Singularity and Apptainer have a commercial/open-source relationship much like Zilliz and Milvus, but are close enough that either solution would be helpful for my need.

https://github.com/milvus-io/milvus/issues/17165 : There was a similar feature request (now closed) here a couple years ago, but I didn't see anything that's happened in the interim.

Contributor guide

[Feature]: running local milvus in singularity container or containers · milvus-io/milvus#33964 | Good First Issue