Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

[Feature Request]: Add Chroma vector database ingestion connector for RAG pipelines

Abierto
  • ¿Algo no está bien?
#40,047 1 comentario 0 reacciones 1 asignado Ver en GitHub

Los mantenedores suelen responder en 1 día

@nitinware ya está trabajando en esto.

Desde el 7/9/2026.

Evaluación

Este issue todavía no se ha evaluado.

Descripción

io new feature P2 python
What would you like to happen?

Motivation

Beam's Python ml/rag ingestion module currently supports Milvus and Qdrant (sdks/python/apache_beam/ml/rag/ingestion/). Chroma is one of the most widely-adopted open-source vector databases, especially for local and embedded RAG workloads, and is the missing piece in Beam's OSS vector-DB coverage.

Proposal

Add sdks/python/apache_beam/ml/rag/ingestion/chroma.py implementing a WriteToChroma PTransform matching the shape of the existing Milvus and Qdrant connectors:

  • Accepts Chunk records from the existing RAG chunking transforms
  • Supports both chromadb.HttpClient (remote) and chromadb.PersistentClient
    (embedded / local) client modes
  • Configurable collection name, embedding + metadata mapping, batch size
  • Standard retry/backoff

Benefits

  • Completes Beam's coverage of the OSS local-first vector-DB tier
  • Chroma is Apache-2.0 licensed — no dependency-license concerns
  • PersistentClient runs in-process, so integration tests do not need
    Docker or a remote service — a meaningful advantage over Milvus/Qdrant
    which require running servers in CI

Testing plan

  • Unit tests mirroring qdrant_test.py
  • Integration test using chromadb.PersistentClient against a tmpdir
    (no external service required)
  • Docs + example consistent with existing Milvus / Qdrant docs
Issue Priority

Priority: 2 (default / most feature requests should be filed as P2)

Issue Components
  • Component: Python SDK
  • Component: Java SDK
  • Component: Go SDK
  • Component: Typescript SDK
  • Component: IO connector
  • Component: Beam YAML
  • Component: Beam examples
  • Component: Beam playground
  • Component: Beam katas
  • Component: Website
  • Component: Infrastructure
  • Component: Spark Runner
  • Component: Flink Runner
  • Component: Prism Runner
  • Component: Twister2 Runner
  • Component: Hazelcast Jet Runner
  • Component: Google Cloud Dataflow Runner
Lenguaje dominante
Java
Estrellas
8.7k
Forks
4.7k
Merge medio
2 d 4 h
PR fusionados (30 d)
278

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de apache/beam

Todos los issues de apache/beam

Issues similares

Más issues de Java

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.