mozilla-ai/llamafile

How do I add the ChatML prompt template to the .args file when creating a llamafile?

Open

#227 opened on Jan 24, 2024

View on GitHub
 (3 comments) (1 reaction) (0 assignees)C++ (1,358 forks)batch import
enhancementhelp wanted

Repository metrics

Stars
 (24,439 stars)
PR merge metrics
 (Avg merge 2d 20h) (14 merged PRs in 30d)

Description

Simply something like this:

-template "<|im_start|>system {system_message}<|im_end|> <|im_start|>user {prompt}<|im_end|> <|im_start|>assistant"

or something else entirely?

Contributor guide

Research direction
Check the llamafile documentation for the .args file format and the template flag. You need to include the full prompt template as a single line or properly escaped string in the .args file. For ChatML, use template "<|im start|>system\n{system message}<|im end|>\n<|im start|>user\n{prompt}<|im end|>\n<|im start|>assistant"
Tech stack
None
Domain
climachine learning
Issue type
Documentation
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
1
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
Under 1 hour
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
basic command line
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
85