Support input prompt like llama.cpp

Open
#23 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
25/100
Issue type
Feature
Clarity
Needs clarification
Activity status
Stale
Tech stack
cpp
Domain
ai, cli

Research direction

Start by locating the main program and reviewing how it currently accepts model input. Compare its behavior with the four prompt modes listed in the issue, then determine the expected command-line and interactive behavior. Done means the requested llama.cpp-style input prompt support is implemented and verified, but the issue names no files or tests.

Written by the indexing model from the issue text.

Description

Input Prompts

The main program provides several ways to interact with the LLaMA models using input prompts:

  • --prompt PROMPT: Provide a prompt directly as a command-line option.
  • --file FNAME: Provide a file containing a prompt or multiple prompts.
  • --interactive-first: Run the program in interactive mode and wait for input right away. (More on this below.)
  • --random-prompt: Start with a randomized prompt.
Dominant language
C++
Stars
752
Forks
94
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from MegEngine/InferLLM

All issues in MegEngine/InferLLM

Similar issues

More C++ issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.