[BUG]: HuggingFace embedder remote mode fails to initialize (endpoint_url rejected as model and timeout dropped)

Chiusa
  • Qualcosa non va?
#2,541 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Una pull request collegata è già stata integrata.

  • #2542 di @Bhumit267 — integrata

Valutazione

Difficoltà
2/5
Tempo stimato
1-3 ore
Idoneità per principianti
25/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
huggingface, python
Ambito
ai, backend

Direzione di ricerca

Start in packages/nvidia_nat_langchain/src/nat/plugins/langchain/embedder.py at the huggingface_langchain branch that builds HuggingFaceEndpointEmbeddings when endpoint_url is set, and check the timeout field in packages/nvidia_nat_core/src/nat/embedder/huggingface_embedder.py. Done means the remote branch passes timeout through and does not feed the URL into model. The issue already has an open linked pull request (#2542), so coordinate with it before starting.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

bug
Version

develop at commit 15fa7b6e0de923a6f535351720d4ab6353d78ef3

Which installation method(s) does this occur on?
  • Source
  • PyPi
Describe the bug.

In packages/nvidia_nat_langchain/src/nat/plugins/langchain/embedder.py, huggingface_langchain attempts to initialize HuggingFaceEndpointEmbeddings when embedder_config.endpoint_url is set:

    if embedder_config.endpoint_url:
        from langchain_huggingface import HuggingFaceEndpointEmbeddings

        client = HuggingFaceEndpointEmbeddings(
            model=embedder_config.endpoint_url,
            huggingfacehub_api_token=get_secret_value(embedder_config.api_key),
        )

This causes two bugs during remote embedder initialization:

  1. endpoint_url rejected as model: HuggingFaceEndpointEmbeddings (from langchain-huggingface) expects model to be a Hugging Face Hub repository ID (e.g., BAAI/bge-small-en-v1.5), not an HTTP/HTTPS URL. Passing model=embedder_config.endpoint_url raises a validation error from huggingface_hub (HFValidationError: Repo id must use alphanumeric chars... ':' is forbidden: 'http...'). As a result, remote mode fails at startup for any valid TEI or Hugging Face Inference Endpoint URL.
  2. timeout is dropped: HuggingFaceEmbedderConfig in packages/nvidia_nat_core/src/nat/embedder/huggingface_embedder.py defines timeout: float = Field(default=120.0, ge=1.0), but timeout is never forwarded to HuggingFaceEndpointEmbeddings(timeout=...), ignoring user-configured network timeouts.
  3. model_name is discarded in remote mode: If a user supplies both a model identifier and an endpoint_url (e.g., when targeting a multi-model TEI gateway or HuggingFace inference endpoint), model_name is completely ignored.
Minimum reproducible example
import asyncio
from pydantic import SecretStr
from nat.embedder.huggingface_embedder import HuggingFaceEmbedderConfig
from nat.plugins.langchain.embedder import huggingface_langchain

async def reproduce():
    config = HuggingFaceEmbedderConfig(
        endpoint_url="https://api-inference.huggingface.co/models/BAAI/bge-small-en-v1.5",
        api_key=SecretStr("hf_dummy_token"),
        timeout=60.0,
    )
    # Attempting to yield client
    async for client in huggingface_langchain(config, None):
        print(client)

asyncio.run(reproduce())

Equivalent workflow YAML:

embedders:
  remote_tei:
    _type: huggingface
    endpoint_url: "https://api-inference.huggingface.co/models/BAAI/bge-small-en-v1.5"
    api_key: "${HF_API_KEY}"
    timeout: 60.0
Relevant log output
Click here to see error details
huggingface_hub.utils.validators.HFValidationError: Repo id must use alphanumeric chars or '-', '', '.', '--' and '..' are forbidden, ':' is forbidden: 'https://api-inference.huggingface.co/models/BAAI/bge-small-en-v1.5'. Use repo_type argument if needed.

Other/Misc.
  • Previously noted in discussion on #2528 / PR #2529, where local HuggingFace embeddings were fixed, but remote endpoint_url support was excluded from scope.
  • Suggested direction:
    • If using HuggingFaceEndpointEmbeddings, provide the model repo ID via embedder_config.model_name (or deduce repo ID if endpoint follows Hub URL pattern), and forward timeout=embedder_config.timeout.
    • For custom self-hosted TEI servers running with an OpenAI-compatible endpoint, document or route through the OpenAI embedder client pointing to TEI's /v1 endpoint.
Code of Conduct
  • I agree to follow the NeMo Agent Toolkit Code of Conduct
  • I have searched the open bugs and have found no duplicates for this bug report
Lingua principale
Python
Stelle
2.7k
Fork
823
Merge medio
1g 5h
PR unite (30g)
139

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di NVIDIA/NeMo-Agent-Toolkit

Tutte le issue di NVIDIA/NeMo-Agent-Toolkit

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.