examples(spin-llm): updates needed?

Open
#135 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
45/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Quiet
Tech stack
python
Domain
machine-learning

Research direction

Review examples/spin-llm and the request handler in app.py, then reproduce the failure with the shown .spin/ai-models structure. Trace the inference path through spin_sdk/llm.py and check whether the example or model layout needs updating; done means the required change is identified and the example handles a supported model without Error_ModelNotSupported.

Written by the indexing model from the issue text.

Description

As of https://github.com/spinframework/spin-python-sdk/pull/132, it is unclear if the spin-llm example requires updates. An attempt was made to test the updated example but the following was encountered:

With the following ai-model structure:

         .spin/ai-models/
         └── sentence-transformers
             └── all-minilm-l6-v2
                 ├── model.safetensors
                 └── tokenizer.json

When handling a request, the app produces this error:

Traceback (most recent call last):
  File "/1/spin_sdk/http/__init__.py", line 86, in handle
  File "/0/app.py", line 6, in handle_request
  File "/1/spin_sdk/llm.py", line 55, in infer
  File "/1/spin_sdk/llm.py", line 45, in infer_with_options
  File "/1/spin_sdk/wit/imports/fermyon_spin_llm_2_0_0.py", line 93, in infer
componentize_py_types.Err: Error_ModelNotSupported()
Dominant language
Python
Stars
48
Forks
15
Avg merge
2d 23h
Merged PRs (30d)
1

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from spinframework/spin-python-sdk

All issues in spinframework/spin-python-sdk

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.