`SocketcanBus` does not support socket file descriptors over 1023
Ninguém assumiu esta issue ainda.
Avaliação
- Dificuldade
- 3/5
- Tempo estimado
- 1-2 dias
- Facilidade para iniciantes
- 72/100
- Tipo de issue
- Bug
- Clareza
- Claramente especificada
- Status de atividade
- Pouca atividade
- Stack de tecnologia
- python
- Domínio
- networking, operating-systems
Direção de pesquisa
Comece em can/interfaces/socketcan/socketcan.py, em SocketcanBus.send() e _recv_internal(), onde select.select() é usado para verificar a prontidão. Revise os testes fornecidos para descritores de arquivo altos quanto ao comportamento do socket simulado e, em seguida, execute a suíte de testes do SocketCAN. Está concluído quando send() e recv() funcionarem com um descritor de arquivo de socket acima de 1023 sem o ValueError de select().
Escrita pelo modelo de indexação a partir do texto da issue.
Descrição
Describe the bug
Python-can’s SocketcanBus uses Python’s select.select() to wait for socket read and write availability. This function is an interface to the Unix select system call (ref). There is a known issue where the file descriptor limit FD_SETSIZE is hardcoded to 1024 in the glibc implementation (ref).
This means that even on systems where the file descriptor limit is > 1023, when the socket file descriptor is > 1023, SocketcanBus propagates a ValueError raised by select.select() with the message, filedescriptor out of range in select().
The recommended fix is to use poll() or epoll() instead.
To Reproduce
- Set the system file descriptor limit above 1024 (e.g.,
ulimit -n 8192) - Open > 1024 files to simulate a high number of open files (e.g.,
for i in {1..2000}; do exec {fd}>/dev/null; done - Use a
SocketcanBusinstance to send or receive a CAN message
Expected behavior
I'd expect SocketcanBus to just work when socket fd > 1023
Additional context
OS and version: Ubuntu 22.04.5 LTS
Python version: 3.10
python-can version: first seen on 4.5.0, verified also an issue on 4.6.1 with unit tests below
python-can interface/s (if applicable): SocketcanBus
Traceback and logs
Traceback from repro on python-can 4.5.0:
self = <can.interfaces.socketcan.socketcan.SocketcanBus object at 0x758060538730>
msg = can.Message(timestamp=0.0, arbitration_id=0x18ebff17, is_extended_id=True, dlc=8, data=[0x1, 0x31, 0x58, 0x50, 0x42, 0x44, 0x34, 0x39])
timeout = 0
def send(self, msg: Message, timeout: Optional[float] = None) -> None:
"""Transmit a message to the CAN bus.
:param msg: A message object.
:param timeout:
Wait up to this many seconds for the transmit queue to be ready.
If not given, the call may fail immediately.
:raises ~can.exceptions.CanError:
if the message could not be written.
"""
log.debug("We've been asked to write a message to the bus")
logger_tx = log.getChild("tx")
logger_tx.debug("sending: %s", msg)
started = time.time()
# If no timeout is given, poll for availability
if timeout is None:
timeout = 0
time_left = timeout
data = build_can_frame(msg)
while time_left >= 0:
# Wait for write availability
> ready = select.select([], [self.socket], [], time_left)[1]
E ValueError: filedescriptor out of range in select()
../python_can+/can/interfaces/socketcan/socketcan.py:788: ValueError
Unit tests that also reproduce the error:
#!/usr/bin/env python
"""
Test that SocketcanBus.send() and recv() work with file descriptors > 1023.
"""
import select
import unittest
from unittest.mock import MagicMock, patch
import can
from can import Message
from can.interfaces.socketcan.socketcan import build_can_frame
from .config import IS_LINUX
HIGH_FD = 1024
@unittest.skipUnless(IS_LINUX, "socketcan is only available on Linux")
class TestSocketcanHighFd(unittest.TestCase):
"""Verify SocketcanBus works when the underlying socket fd exceeds 1023."""
def setUp(self):
patcher_create = patch("can.interfaces.socketcan.socketcan.create_socket")
patcher_bind = patch("can.interfaces.socketcan.socketcan.bind_socket")
self.mock_create_socket = patcher_create.start()
self.mock_bind_socket = patcher_bind.start()
self.mock_socket = MagicMock()
self.mock_socket.fileno.return_value = HIGH_FD
self.mock_create_socket.return_value = self.mock_socket
self.bus = can.Bus(interface="socketcan", channel="can0")
self.addCleanup(patcher_create.stop)
self.addCleanup(patcher_bind.stop)
def tearDown(self):
self.bus.shutdown()
def test_send_high_fd(self):
"""send() succeeds when the socket fd > 1023."""
msg = Message(arbitration_id=0x123, data=[1, 2, 3, 4, 5, 6, 7, 8])
frame_data = build_can_frame(msg)
self.mock_socket.send.return_value = len(frame_data)
self.bus.send(msg)
self.mock_socket.send.assert_called_once_with(frame_data)
@patch("can.interfaces.socketcan.socketcan.capture_message")
def test_recv_high_fd(self, mock_capture):
"""recv() succeeds when the socket fd > 1023."""
expected_msg = Message(
arbitration_id=0x123,
data=[1, 2, 3, 4, 5, 6, 7, 8],
channel="can0",
timestamp=1000.0,
)
mock_capture.return_value = expected_msg
msg = self.bus.recv(timeout=1.0)
self.assertIsNotNone(msg)
self.assertEqual(msg.arbitration_id, 0x123)
self.assertEqual(msg.data, bytearray([1, 2, 3, 4, 5, 6, 7, 8]))
mock_capture.assert_called_once_with(self.mock_socket, False)
if __name__ == "__main__":
unittest.main()
Unit test failure output on 4.6.1:
FAIL [ 9.983ms] test/test_high_fd.py::TestSocketcanHighFd::test_recv_high_fd
stdout ───
stderr ───
test/test_high_fd.py:66 in test_recv_high_fd
self = <test.test_high_fd.TestSocketcanHighFd testMethod=test_recv_high_fd>, mock_capture = <MagicMock name='capture_message' id='140435170767152'>
63 │ │ )
64 │ │ mock_capture.return_value = expected_msg
65 │ │
❱ 66 │ │ msg = self.bus.recv(timeout=1.0)
67 │ │
68 │ │ self.assertIsNotNone(msg)
69 │ │ self.assertEqual(msg.arbitration_id, 0x123)
can/bus.py:121 in recv
118 │ │
119 │ │ while True:
120 │ │ │ # try to get a message
❱ 121 │ │ │ msg, already_filtered = self._recv_internal(timeout=time_left)
122 │ │ │
123 │ │ │ # return it, if it matches
124 │ │ │ if msg and (already_filtered or self._matches_filters(msg)):
121: in recv
❱
can/interfaces/socketcan/socketcan.py:827 in _recv_internal
self = <can.interfaces.socketcan.socketcan.SocketcanBus object at 0x7fb99c7661d0>, timeout = 1.0
824 │ │ try:
825 │ │ │ # get all sockets that are ready (can be a list with a single value
826 │ │ │ # being self.socket or an empty list if self.socket is not ready)
❱ 827 │ │ │ ready_receive_sockets, _, _ = select.select([self.socket], [], [], timeout)
828 │ │ except OSError as error:
829 │ │ │ # something bad happened (e.g. the interface went down)
830 │ │ │ raise can.CanOperationError(
827: ValueError
❱ ready_receive_sockets, _, _ = select.select([self.socket], [], [], timeout)
E ValueError: filedescriptor out of range in select()
FAIL [ 4.712ms] test/test_high_fd.py::TestSocketcanHighFd::test_send_high_fd
stdout ───
stderr ───
test/test_high_fd.py:50 in test_send_high_fd
self = <test.test_high_fd.TestSocketcanHighFd testMethod=test_send_high_fd>
47 │ │ frame_data = build_can_frame(msg)
48 │ │ self.mock_socket.send.return_value = len(frame_data)
49 │ │
❱ 50 │ │ self.bus.send(msg)
51 │ │
52 │ │ self.mock_socket.send.assert_called_once_with(frame_data)
53
can/interfaces/socketcan/socketcan.py:869 in send
self = <can.interfaces.socketcan.socketcan.SocketcanBus object at 0x7fb99c71bf70>, msg = can.Message(timestamp=0.0, arbitration_id=0x123,
is_extended_id=True, dlc=8, data=[0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7, 0x8]), timeout = 0
866 │ │
867 │ │ while time_left >= 0:
868 │ │ │ # Wait for write availability
❱ 869 │ │ │ ready = select.select([], [self.socket], [], time_left)[1]
870 │ │ │ if not ready:
871 │ │ │ │ # Timeout
872 │ │ │ │ break
869: ValueError
❱ ready = select.select([], [self.socket], [], time_left)[1]
E ValueError: filedescriptor out of range in select()
- Linguagem predominante
- Python
- Estrelas
- 1.6k
- Forks
- 697
- Métricas de merge de PRs
- Nenhum PR com merge em 30d
Guia de contribuição
Primeiros passos
- Leia a issue inteira e depois o guia de contribuição do projeto.
- Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
- Faça um fork do repositório e trabalhe em uma branch.
- Abra um pull request que referencie o número da issue.
Mais de hardbyte/python-can
-
Dificuldade 2/5 1-3 horas Facilidade para iniciantes 88/100
hardbyte/python-can#2103 ·
-
bug
Dificuldade 1/5 Menos de uma hora Facilidade para iniciantes 78/100
hardbyte/python-can#2077 · 1 comentário · 1 reação ·
-
bug
Dificuldade 1/5 Menos de uma hora Facilidade para iniciantes 68/100
hardbyte/python-can#1922 · 1 reação ·
-
enhancement
Dificuldade 5/5 Mais de uma semana Facilidade para iniciantes 30/100
hardbyte/python-can#2102 ·
-
bug
Dificuldade 3/5 1-2 dias Facilidade para iniciantes 68/100
hardbyte/python-can#2092 ·
Todas as issues de hardbyte/python-can
Issues semelhantes
-
bug confirmed issue
Dificuldade 2/5 1-3 horas Facilidade para iniciantes 75/100
open-webui/open-webui#30750 · 1 comentário ·
-
Dificuldade 2/5 1-3 horas Facilidade para iniciantes 75/100
-
enhancement
Dificuldade 2/5 1-3 horas Facilidade para iniciantes 75/100
OpenwaterHealth/openmotion-bloodflow-app#604 · 1 comentário ·
-
Dificuldade 2/5 1-3 horas Facilidade para iniciantes 70/100
-
good first issue
Dificuldade 1/5 Menos de uma hora Facilidade para iniciantes 90/100