Lightning-AI/lit-llama

`generate.py` and `generate/full.py` could be merged

Open

#313 opened on May 22, 2023

View on GitHub
 (4 comments) (0 reactions) (0 assignees)Python (5,533 stars) (473 forks)batch import
good first issueinference

Description

Proposed in https://github.com/Lightning-AI/lit-llama/pull/255

The only difference in logic is the instruction tuning. We could add a flag for it as in https://github.com/Lightning-AI/lit-llama/pull/278

Contributor guide

Tech stack
python
Domain
machine learning
Issue type
refactor
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
Pythonbasic understanding of lit llama generation scripts
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
55
Research direction
Review the linked PRs #255 and #278 to understand the proposed flag approach. Examine generate.py and generate/full.py to identify common logic. Implement a flag to toggle instruction tuning and consolidate the two files.