DeepLabCut-live fails to run exported superanimal_humanbody model from DeepLabCut

Open
#137 1 comment 0 reactions 0 assignees View on GitHub

@deruyter92 is already working on this.

Since Feb 18, 2026.

  • #164 by @deruyter92 — open

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
25/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
python, pytorch

Research direction

Start in dlclive/pose_estimation_pytorch/runner.py around the detector handling near line 192 and PyTorchRunner.get_pose(). Reproduce a frame where no detector prediction exceeds cutoff_confidence, then trace the model input shape before the call shown in the traceback. Done means such frames no longer cause the reported tensor-shape error; issue 164 is already an open linked pull request.

Written by the indexing model from the issue text.

Description

bug DLC 3.0 🔥

Concerns the DLC3-pytorch branch maxim/dlclive3

When trying to run rtmpose models or other models with a detector on top, i saw that DLCLive crashes at certain frames.
Looking more into the issue: this happens whenever the detector doesnt have any predictions > cutoff_confidence. here then it will try to pass the raw image to the model which will results in array mismatch.

`---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
Cell In[11], line 1
----> 1 dlc.runner.get_pose(frame)

File c:\Users\User.conda\envs\dlclive3\Lib\site-packages\torch\utils_contextlib.py:120, in context_decorator..decorate_context(*args, **kwargs)
117 @functools.wraps(func)
118 def decorate_context(*args, **kwargs):
119 with ctx_factory():
--> 120 return func(*args, **kwargs)

File ~\Repos\dlc_live3\DeepLabCut-live\dlclive\pose_estimation_pytorch\runner.py:211, in PyTorchRunner.get_pose(self, frame)
208 if self.precision == "FP16":
209 model_input = model_input.half()
--> 211 outputs = self.model(model_input)
212 batch_pose = self.model.get_predictions(outputs)["bodypart"]["poses"]
214 if self.dynamic is not None:

File c:\Users\User.conda\envs\dlclive3\Lib\site-packages\torch\nn\modules\module.py:1775, in Module._wrapped_call_impl(self, *args, **kwargs)
1773 return self._compiled_call_impl(*args, **kwargs) # type: ignore[misc]
1774 else:
-> 1775 return self._call_impl(*args, **kwargs)

File c:\Users\User.conda\envs\dlclive3\Lib\site-packages\torch\nn\modules\module.py:1786, in Module._call_impl(self, *args, **kwargs)
1781 # If we don't have any hooks, we want to skip the rest of the logic in
1782 # this function, and just call forward.
1783 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
1784 or _global_backward_pre_hooks or _global_backward_hooks
1785 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1786 return forward_call(*args, **kwargs)
1788 result = None
1789 called_always_called_hooks = set()

File ~\Repos\dlc_live3\DeepLabCut-live\dlclive\pose_estimation_pytorch\models\model.py:74, in PoseModel.forward(self, x)
72 outputs = {}
73 for head_name, head in self.heads.items():
---> 74 outputs[head_name] = head(features)
75 return outputs

File c:\Users\User.conda\envs\dlclive3\Lib\site-packages\torch\nn\modules\module.py:1775, in Module._wrapped_call_impl(self, *args, **kwargs)
1773 return self._compiled_call_impl(*args, **kwargs) # type: ignore[misc]
1774 else:
-> 1775 return self._call_impl(*args, **kwargs)

File c:\Users\User.conda\envs\dlclive3\Lib\site-packages\torch\nn\modules\module.py:1786, in Module._call_impl(self, *args, **kwargs)
1781 # If we don't have any hooks, we want to skip the rest of the logic in
1782 # this function, and just call forward.
1783 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
1784 or _global_backward_pre_hooks or _global_backward_hooks
1785 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1786 return forward_call(*args, **kwargs)
1788 result = None
1789 called_always_called_hooks = set()

File ~\Repos\dlc_live3\DeepLabCut-live\dlclive\pose_estimation_pytorch\models\heads\rtmcc_head.py:110, in RTMCCHead.forward(self, x)
108 feats = self.final_layer(x) # -> B, K, H, W
109 feats = torch.flatten(feats, start_dim=2) # -> B, K, hidden=HxW
--> 110 feats = self.mlp(feats) # -> B, K, hidden
111 feats = self.gau(feats)
112 x, y = self.cls_x(feats), self.cls_y(feats)

File c:\Users\User.conda\envs\dlclive3\Lib\site-packages\torch\nn\modules\module.py:1775, in Module._wrapped_call_impl(self, *args, **kwargs)
1773 return self._compiled_call_impl(*args, **kwargs) # type: ignore[misc]
1774 else:
-> 1775 return self._call_impl(*args, **kwargs)

File c:\Users\User.conda\envs\dlclive3\Lib\site-packages\torch\nn\modules\module.py:1786, in Module._call_impl(self, *args, **kwargs)
1781 # If we don't have any hooks, we want to skip the rest of the logic in
1782 # this function, and just call forward.
1783 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
1784 or _global_backward_pre_hooks or _global_backward_hooks
1785 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1786 return forward_call(*args, **kwargs)
1788 result = None
1789 called_always_called_hooks = set()

File c:\Users\User.conda\envs\dlclive3\Lib\site-packages\torch\nn\modules\container.py:250, in Sequential.forward(self, input)
246 """
247 Runs the forward pass.
248 """
249 for module in self:
--> 250 input = module(input)
251 return input

File c:\Users\User.conda\envs\dlclive3\Lib\site-packages\torch\nn\modules\module.py:1775, in Module._wrapped_call_impl(self, *args, **kwargs)
1773 return self._compiled_call_impl(*args, **kwargs) # type: ignore[misc]
1774 else:
-> 1775 return self._call_impl(*args, **kwargs)

File c:\Users\User.conda\envs\dlclive3\Lib\site-packages\torch\nn\modules\module.py:1786, in Module._call_impl(self, *args, **kwargs)
1781 # If we don't have any hooks, we want to skip the rest of the logic in
1782 # this function, and just call forward.
1783 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
1784 or _global_backward_pre_hooks or _global_backward_hooks
1785 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1786 return forward_call(*args, **kwargs)
1788 result = None
1789 called_always_called_hooks = set()

File c:\Users\User.conda\envs\dlclive3\Lib\site-packages\torch\nn\modules\linear.py:134, in Linear.forward(self, input)
130 def forward(self, input: Tensor) -> Tensor:
131 """
132 Runs the forward pass.
133 """
--> 134 return F.linear(input, self.weight, self.bias)

RuntimeError: mat1 and mat2 shapes cannot be multiplied (12x391 and 144x256)`

Dominant language
Python
Stars
240
Forks
57
PR merge metrics
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from DeepLabCut/DeepLabCut-live

All issues in DeepLabCut/DeepLabCut-live

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.