CarperAI/trlx

Collapse reference+learner hydra heads when using LoRa

Open

#320 建立於 2023年2月21日

在 GitHub 查看
 (6 留言) (2 反應) (0 負責人)Python (450 fork)batch import
contributions welcomefeature requestgood first issue

倉庫指標

Star
 (4,184 star)
PR 合併指標
 (30 天內沒有已合併 PR)

描述

🚀 The feature, motivation, and pitch

With additive (delta-style) parameter-efficient tuning methods such as LoRa, we should be able to make a slightly more mem-efficient hydra architecture by using a single block that does ~frozen_head + tunable_weights for the learner/policy head's fwd-pass and simply frozen_head for the reference, instead of maintaining 2x heads.

CC @LouisCastricato and @cat-state for pointing this out

Alternatives

No response

Additional context

No response

貢獻者指南