I would like to test out laya of system1 models

Open
#2,405 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
25/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
huggingface, python

Research direction

Start by reviewing the Python wrapper's model loading and inference entry points, then compare them with the Laya-GGUF model linked in the issue and the llama.cpp support it requires. The issue does not name files or tests, and the requested scope is broad. Done means callers can use System1 models through the Python wrapper; confirm the model loads and produces output.

Written by the indexing model from the issue text.

Description

Is your feature request related to a problem? Please describe.
I would like to test out laya - e.g. the model at https://huggingface.co/ggml-org/Laya-GGUF

Describe the solution you'd like
To be able to call system1 models

Describe alternatives you've considered
Directly use llama.cpp instead of the python wrapper

Additional context

Dominant language
Python
Stars
10.7k
Forks
1.5k
Avg merge
1h 40m
Merged PRs (30d)
5

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from abetlen/llama-cpp-python

All issues in abetlen/llama-cpp-python

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.