lm-sys/FastChat

Can you provide instructions to use the fastchat/data folder?

Open

#428 opened on Apr 14, 2023

View on GitHub
 (4 comments) (0 reactions) (0 assignees)Python (38,959 stars) (4,736 forks)batch import
documentationgood first issuehelp wanted

Description

such as, how to generate training sample step by step from original file? in addtion, what are the content of the sharegpt.json, as following: python3 -m fastchat.data.sample --in sharegpt.json --out sampled.json

Contributor guide

Tech stack
python
Domain
documentation
Issue type
documentation
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
2
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
under 1 hour
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
needs maintainer response
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
Basic knowledge of PythonFamiliarity with JSON data
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
60
Research direction
Examine the fastchat/data module to understand the data processing pipeline. Document the format of sharegpt.json based on the script that generates it. Provide a step by step guide from original file to training sample, including example usage of the sample command.