ggml-org/llama.cpp

Feature Request: Mapping model name to LoRA config

已關閉

#11,031 建立於 2025年1月1日

 (6 則留言) (2 個反應) (0 位負責人)C++ (21,874 個分叉)batch import
enhancementgood first issueserver

倉庫指標

星標
 (124,541 顆星)
PR 合併指標
 (平均合併 6天 8小時) (30 天內合併 389 個 PR)

描述

Prerequisites

  • I am running the latest code. Mention the version if possible as well.
  • I carefully followed the README.md.
  • I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
  • I reviewed the Discussions, and have a new and useful enhancement to share.

Feature Description

I came across this idea while working on #10994

The idea is that we can maintain a list of model name mapped to LoRA config, for example:

{
    "llama-base":               [{"id": 0, "scale": 0.0}, {"id": 1, "scale": 0.0}],
    "llama-story":              [{"id": 0, "scale": 1.0}, {"id": 1, "scale": 0.0}],
    "llama-abliteration":       [{"id": 0, "scale": 0.0}, {"id": 1, "scale": 1.0}],
    "llama-story-abliteration": [{"id": 0, "scale": 0.5}, {"id": 1, "scale": 0.5}]
}

Then, user can switch the model by specifying model in the request, for example:

# first user:
{
    "model": "llama-story-abliteration",
    "messages": [
        {"role": "user", "content": "Write a NSFW story"}
    ]
}

# second user:
{
    "model": "llama-base",
    "messages": [
        {"role": "user", "content": "Is this NSFW?"}
    ]
}

Motivation

N/A

Possible Implementation

No response

貢獻者指南