containers/ramalama

Allow passing additional config files (e.g. chat templates) into the model container

Aperta

#1783 aperta il 1 ago 2025

 (8 commenti) (3 reazioni) (0 assegnatari)Python (355 fork)github user discovery
enhancementgood first issuestale-issue

Metriche repository

Star
 (3000 stelle)
Metriche merge PR
 (Merge medio 6g 7h) (32 PR mergiate in 30 g)

Descrizione

Feature request description

I was poking at ramalama today, and wound up trying to run the unsloth version of the new Qwen3 coder model. There's a bug in it, though. They sent out a tentative fix - an updated chat template file - with instructions to just run with --chat-template-file <path/to/thatfile>, no need to redownload the whole model!

So, OK, I try with --runtime-args="--jinja --chat-template-file <path/to/thatfile>"...and it complains about not being able to read the file. I assume this is because the model container doesn't have my home directory mounted into it or anything like that.

Is there any way to handle this kind of situation? I poked about in the docs but couldn't find anything. If I'm understanding it all right, it seems like it would be useful to have an option to mount some arbitrary file or directory into the model container for cases like this. If I'm not understanding it all right, please let me know. :D

Suggest potential solution

No response

Have you considered any alternatives?

No response

Additional context

No response

Guida contributor