[BUG]: HuggingFace embedder remote mode fails to initialize (endpoint_url rejected as model and timeout dropped)

Open
  • Is something wrong?
#2,541 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@Bhumit267 is already working on this.

Since Nov 2, 2026.

  • #2542 by @Bhumit267 — open

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
25/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
huggingface, python
Domain
ai, backend

Research direction

Start in packages/nvidia_nat_langchain/src/nat/plugins/langchain/embedder.py at the huggingface_langchain branch that builds HuggingFaceEndpointEmbeddings when endpoint_url is set, and check the timeout field in packages/nvidia_nat_core/src/nat/embedder/huggingface_embedder.py. Done means the remote branch passes timeout through and does not feed the URL into model. The issue already has an open linked pull request (#2542), so coordinate with it before starting.

Written by the indexing model from the issue text.

Description

bug
Version

develop at commit 15fa7b6e0de923a6f535351720d4ab6353d78ef3

Which installation method(s) does this occur on?
  • Source
  • PyPi
Describe the bug.

In packages/nvidia_nat_langchain/src/nat/plugins/langchain/embedder.py, huggingface_langchain attempts to initialize HuggingFaceEndpointEmbeddings when embedder_config.endpoint_url is set:

    if embedder_config.endpoint_url:
        from langchain_huggingface import HuggingFaceEndpointEmbeddings

        client = HuggingFaceEndpointEmbeddings(
            model=embedder_config.endpoint_url,
            huggingfacehub_api_token=get_secret_value(embedder_config.api_key),
        )

This causes two bugs during remote embedder initialization:

  1. endpoint_url rejected as model: HuggingFaceEndpointEmbeddings (from langchain-huggingface) expects model to be a Hugging Face Hub repository ID (e.g., BAAI/bge-small-en-v1.5), not an HTTP/HTTPS URL. Passing model=embedder_config.endpoint_url raises a validation error from huggingface_hub (HFValidationError: Repo id must use alphanumeric chars... ':' is forbidden: 'http...'). As a result, remote mode fails at startup for any valid TEI or Hugging Face Inference Endpoint URL.
  2. timeout is dropped: HuggingFaceEmbedderConfig in packages/nvidia_nat_core/src/nat/embedder/huggingface_embedder.py defines timeout: float = Field(default=120.0, ge=1.0), but timeout is never forwarded to HuggingFaceEndpointEmbeddings(timeout=...), ignoring user-configured network timeouts.
  3. model_name is discarded in remote mode: If a user supplies both a model identifier and an endpoint_url (e.g., when targeting a multi-model TEI gateway or HuggingFace inference endpoint), model_name is completely ignored.
Minimum reproducible example
import asyncio
from pydantic import SecretStr
from nat.embedder.huggingface_embedder import HuggingFaceEmbedderConfig
from nat.plugins.langchain.embedder import huggingface_langchain

async def reproduce():
    config = HuggingFaceEmbedderConfig(
        endpoint_url="https://api-inference.huggingface.co/models/BAAI/bge-small-en-v1.5",
        api_key=SecretStr("hf_dummy_token"),
        timeout=60.0,
    )
    # Attempting to yield client
    async for client in huggingface_langchain(config, None):
        print(client)

asyncio.run(reproduce())

Equivalent workflow YAML:

embedders:
  remote_tei:
    _type: huggingface
    endpoint_url: "https://api-inference.huggingface.co/models/BAAI/bge-small-en-v1.5"
    api_key: "${HF_API_KEY}"
    timeout: 60.0
Relevant log output
Click here to see error details
huggingface_hub.utils.validators.HFValidationError: Repo id must use alphanumeric chars or '-', '', '.', '--' and '..' are forbidden, ':' is forbidden: 'https://api-inference.huggingface.co/models/BAAI/bge-small-en-v1.5'. Use repo_type argument if needed.

Other/Misc.
  • Previously noted in discussion on #2528 / PR #2529, where local HuggingFace embeddings were fixed, but remote endpoint_url support was excluded from scope.
  • Suggested direction:
    • If using HuggingFaceEndpointEmbeddings, provide the model repo ID via embedder_config.model_name (or deduce repo ID if endpoint follows Hub URL pattern), and forward timeout=embedder_config.timeout.
    • For custom self-hosted TEI servers running with an OpenAI-compatible endpoint, document or route through the OpenAI embedder client pointing to TEI's /v1 endpoint.
Code of Conduct
  • I agree to follow the NeMo Agent Toolkit Code of Conduct
  • I have searched the open bugs and have found no duplicates for this bug report
Dominant language
Python
Stars
2.7k
Forks
814
Avg merge
1d 7h
Merged PRs (30d)
131

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from NVIDIA/NeMo-Agent-Toolkit

All issues in NVIDIA/NeMo-Agent-Toolkit

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.