help wanted
倉庫指標
- Star
- (64,271 star)
- PR 合併指標
- (平均合併 3天 15小時) (30 天內合併 525 個 PR)
描述
Hi I am trying ot run the kaggle version but I have a list of .md files which I wan to feed the model to train and then export the model. but is .md file enough to feed model I can see you have this ,
from datasets import Dataset
dataset = Dataset.from_list([{"prompt" : [{"role": "user", "content": prompt.strip()}], "answer" : 0, "reasoning_effort": "low"}]*1000)
maximum_length = len(tokenizer(prompt.strip())["input_ids"])
print(maximum_length)
dataset[0]
given that I wan this plus all my md files , how best to merge these any tips