milvus-io/milvus

Milvus can not use oracle cloud object storage

Open

#25,713 opened on Jul 18, 2023

View on GitHub
 (2 comments) (0 reactions) (1 assignee)Go (44,298 stars) (4,000 forks)batch import
good first issuekind/feature

Description

Is there an existing issue for this?

  • I have searched the existing issues

Is your feature request related to a problem? Please describe.

Our company is using Oracle cloud (OCI) object storage (OCI OSS is based on S3 protocol). So far we have tried two ways: first, we set minio: enabled value to false, and then turn on externalS3 configuration, then the service can't start normally. Secondly, if we turn off the externalS3 configuration and then turn on the minio configuration, the following error occurs: [2023/07/18 07:01:27.381 +00:00] [WARN] [storage/minio_chunk_manager.go:118] ["failed to check blob bucket exist"] [bucket=oci-useast-midpub-public] [error="The secret key required to complete authentication could not be found. The region must be specified if this is not the home region for the tenancy."] [2023/07/18 07:01:27.381 +00:00] [WARN] [retry/retry.go:44] ["retry func failed"] ["retry time"=0] [error="The secret key required to complete authentication could not be found. The region must be specified if this is not the home region for the tenancy."] [2023/07/18 07:01:27.590 +00:00] [WARN] [storage/minio_chunk_manager.go:118] ["failed to check blob bucket exist"] [bucket=oci-useast-midpub-public] [error="The secret key required to complete authentication could not be found. The region must be specified if this is not the home region for the tenancy."] image 05046e7dfb7c47514901876f146a3d6

Describe the solution you'd like.

Expect to provide support for normal access to Oracle cloud (OCI) object storage（OSS）

Describe an alternate solution.

No response

Anything else? (Additional Context)

No response

Contributor guide

Tech stack
go
Domain
backenddatabasecloud
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
Go programmingCloud object storage conceptsMilvus storage configuration
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
20
Research direction
The issue involves adding support for Oracle Cloud object storage (OCI OSS) which uses S3 protocol. The error occurs in storage/minio chunk manager.go. Investigation should start by understanding the current S3 configuration handling in Milvus, particularly the externalS3 settings and minio chunk manager. Look for how authentication is handled, possibly needing to support OCI specific authentication keys. Check if there are any existing discussions or PRs addressing OCI support. The assignee may have progress; consider reaching out to them.