FlowiseAI/Flowise

[FEATURE] SageMaker Endpoint Model

Open

#1,050 opened on Oct 13, 2023

View on GitHub
 (3 comments) (1 reaction) (0 assignees)TypeScript (52,823 stars) (24,335 forks)batch import
enhancementhelp wanted

Description

Hi, my team and I are trying to implement Flowise for our GenAI projects. We are wondering if there is a way to invoke ChatModels/LLMs/Embeddings through AWS SageMaker Endpoints similar to how HuggingFaceInference Endpoints operate.

For example, we have subscribed to a Cohere Embedding Model via AWS Marketplace and wish to use it as our embedding model as deployed as a SageMaker Endpoint.

Thanks!

Contributor guide

Tech stack
typescriptaws
Domain
backendcloud
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
3-5 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
fresh
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
mostly clear
Prerequisites
AWS SageMaker knowledgeFlowise development setup
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
35
Research direction
This issue requests support for AWS SageMaker endpoints similar to existing HuggingFace Inference integration. Implementation would require creating a new credential type and node for ChatModels/LLMs/Embeddings using AWS SDK v3. Relevant files to examine are in packages/components/nodes/chatmodels and embeddings. The maintainer has not yet provided specific guidance; reviewing existing HuggingFace node patterns will help. The user did not specify model details, so flexibility is needed.
[FEATURE] SageMaker Endpoint Model · FlowiseAI/Flowise#1050 | Good First Issue