Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

[Bug] Unable to create Qwen3 MoE model

Open
  • Is something wrong?
#444 4 comments 1 reaction 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
python, pytorch
Domain
machine-learning

Research direction

Reproduce the registration failure from the supplied TrainableModel configuration, then inspect art/unsloth/state.py and art/unsloth/service.py around ModelState and start_openai_server. Compare the current FastLanguageModel entry point with Unsloth's FastModel requirement for Qwen3 MoE; done means the model registers without the gate_up_proj AttributeError.

Written by the indexing model from the issue text.

Description

bug

How do I get ART to instantiate the model with FastModel instead of FastLanguageModel in Unsloth (Unsloth docs says: If you're fine-tuning the MOE models, please use FastModel and not FastLanguageModel)? I seem to be running into a model loading issue as seen from the error.

model = art.TrainableModel(
    name="001-script",
    project="testing",
    base_model="Qwen/Qwen3-30B-A3B-Thinking-2507",
    _internal_config=art.dev.InternalModelConfig(  
        init_args=art.dev.InitArgs(
            load_in_4bit=False,
            max_seq_length=65536,
        ),  
        engine_args=art.dev.EngineArgs(  
            max_model_len=65536,
            tensor_parallel_size=8,
            gpu_memory_utilization=0.75,
        ),  
    ),  
)
await model.register(backend)

Error:

Traceback (most recent call last):
  File "/root/openpipe/train.py", line 337, in <module>
    asyncio.run(train())
  File "/root/openpipe/.venv/lib/python3.11/site-packages/nest_asyncio.py", line 30, in run
    return loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/openpipe/.venv/lib/python3.11/site-packages/nest_asyncio.py", line 98, in run_until_complete
    return f.result()
           ^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/futures.py", line 203, in result
    raise self._exception.with_traceback(self._exception_tb)
  File "/usr/lib/python3.11/asyncio/tasks.py", line 277, in __step
    result = coro.send(None)
             ^^^^^^^^^^^^^^^
  File "/root/openpipe/train.py", line 296, in train
    await model.register(backend)
  File "/root/openpipe/.venv/lib/python3.11/site-packages/art/model.py", line 335, in register
    base_url, api_key = await backend._prepare_backend_for_training(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/openpipe/.venv/lib/python3.11/site-packages/art/local/backend.py", line 282, in _prepare_backend_for_training
    await service.start_openai_server(config=config)
  File "/root/openpipe/.venv/lib/python3.11/site-packages/mp_actors/traceback.py", line 26, in async_wrapper
    raise e.with_traceback(streamlined_traceback())
  File "/root/openpipe/.venv/lib/python3.11/site-packages/art/unsloth/service.py", line 60, in start_openai_server
    self.state.trainer.save_model(lora_path)
^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/functools.py", line 1001, in __get__
    val = self.func(instance)
^^^^^^^
  File "/root/openpipe/.venv/lib/python3.11/site-packages/art/unsloth/service.py", line 45, in state
    return ModelState(self.config)
  ^^^^^^^^^^^^^^^^^
  File "/root/openpipe/.venv/lib/python3.11/site-packages/art/unsloth/state.py", line 82, in __init__
    unsloth.FastLanguageModel.from_pretrained(**config.get("init_args", {})),
^^^^^^^^^^^^^^^
  File "/root/openpipe/.venv/lib/python3.11/site-packages/unsloth/models/loader.py", line 397, in from_pretrained
    return FastModel.from_pretrained(
^^^^^^^^^^^^^^^
  File "/root/openpipe/.venv/lib/python3.11/site-packages/unsloth/models/loader.py", line 930, in from_pretrained
    model, tokenizer = FastBaseModel.from_pretrained(
  ^^^^^^^^^^^^^^^^^
  File "/root/openpipe/.venv/lib/python3.11/site-packages/unsloth/models/vision.py", line 621, in from_pretrained
    _, quant_state_dict = get_vllm_state_dict(
^^^^^^^^^^^^^^^
  File "/root/openpipe/.venv/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
^^^^^^^^^^^^^^^
  File "/root/openpipe/.venv/lib/python3.11/site-packages/unsloth_zoo/vllm_utils.py", line 960, in get_vllm_state_dict
    proj = layer.mlp.gate_up_proj
  ^^^^^^^^^^^^^^^^^
  File "/root/openpipe/.venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1940, in __getattr__
    raise AttributeError(
  ^^^^^^^^^^^^^^^^^
AttributeError: 'Qwen3MoeSparseMoeBlock' object has no attribute 'gate_up_proj'
Dominant language
Python
Stars
10.8k
Forks
997
Avg merge
10h 1m
Merged PRs (30d)
117

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from OpenPipe/ART

All issues in OpenPipe/ART

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.