unslothai/unsloth

Add spectrum finetuning support

Open

#1360 aperta il 30 nov 2024

Vedi su GitHub
 (3 commenti) (0 reazioni) (0 assegnatari)Python (5658 fork)batch import
feature requesthelp wanted

Metriche repository

Star
 (64.271 star)
Metriche merge PR
 (Merge medio 3g 15h) (525 PR mergiate in 30 g)

Descrizione

Not sure if this allign with unsloth but spectrum is actually a very great way to finetune models it selects layers with higher Signal to Noise Ratio while adhering great performance A great model which use spectrum is Llama-3.1-Storm-8B

paper github

Performance

Screenshot (76)

Guida contributor