Document passing custom embedModel or llm per request to avoid global Settings usage
#2,016 opened on Jun 10, 2025
Repository metrics
- Stars
- (3,078 stars)
- PR merge metrics
- (PR metrics pending)
Description
I'm currently integrating LlamaIndexTS into a multi-tenant backend API, where each client has their own OpenAI API key and model configuration.
Right now, it seems like the only way to set a custom embedding model or llm is by using the global Settings.embedModel or Settings.llm, like this:
import { Settings } from 'llamaindex';
import { OpenAIEmbedding } from '@llamaindex/embeddings-openai';
Settings.embedModel = new OpenAIEmbedding({ apiKey: 'CLIENT_API_KEY' });
Settings.llm = openai({ apiKey: key, model: 'gpt-4o' })
For example, to use here:
const reader = new SimpleDirectoryReader()
const documents = await reader.loadData({
directoryPath: `${process.env.STORAGE_PATH}/llama`
})
const index = await VectorStoreIndex.fromDocuments(documents)
If global setting embedModel is not set, it cause error: 'Error: Cannot find Embedding, please set Settings.embedModel = ... on the top of your code'
However, this creates issues when handling concurrent requests from multiple clients, because Settings is global and mutable. In a typical API server (like with NestJS or Express), this can lead to race conditions and incorrect behavior.
Is there any alternative to this that I might have missed?