vaexio/vaex
View on GitHub[BUG-REPORT] Vaex dataframe access crashes asyncio event loop tasks
Open
#1,701 opened on Nov 14, 2021
bughelp wanted
Repository metrics
- Stars
- (8,117 stars)
- PR merge metrics
- (No merged PRs in 30d)
Description
Description I'm experiencing issues when using vaex in the context of an asyncio event loop. The repeated interaction with the data frame seems to kill auxiliary tasks in the same event loop. The following code reproduces my issue:
import asyncio
import vaex
import numpy as np
async def vaex_task(df):
while True:
await asyncio.sleep(0.1)
print(df.count())
async def aux_task():
while True:
await asyncio.sleep(0.3)
print("Aux task alive")
df = vaex.from_dict(dict(x=np.linspace(0, 1., 100)))
loop = asyncio.get_event_loop()
loop.create_task(vaex_task(df))
loop.create_task(aux_task())
loop.run_forever()
Example output is:
100
100
Aux task alive
100
100
100
Aux task alive
100
100
Aux task alive
100
100
100
Aux task alive
100
100
Exception in callback <TaskWakeupMethWrapper object at 0x00000156BC40C4F8>(<Future finished result=None>)
handle: <Handle <TaskWakeupMethWrapper object at 0x00000156BC40C4F8>(<Future finished result=None>)>
Traceback (most recent call last):
File "C:\Python37\lib\asyncio\events.py", line 88, in _run
self._context.run(self._callback, *self._args)
RuntimeError: Cannot enter into task <Task pending coro=<aux_task() running at D:\minimal_vaex_problem.py:20> wait_for=<Future finished result=None>> while another task <Task pending coro=<vaex_task() running at D:\minimal_vaex_problem.py:16>> is being executed.
Task was destroyed but it is pending!
task: <Task pending coro=<aux_task() running at D:\minimal_vaex_problem.py:20> wait_for=<Future finished result=None>>
100
100
100
100
100
100
100
100
100
100
The time after which the exception is thrown varies from run to run, but usually happens within the first 10 iterations of aux_task
Does that look like a bug in vaex or am I doing something wrong here?
Thanks in advance for looking into this!
Software information
- Vaex version (
import vaex; vaex.__version__): {'vaex': '4.5.0', 'vaex-core': '4.5.1', 'vaex-viz': '0.5.0', 'vaex-hdf5': '0.10.0', 'vaex-server': '0.6.1', 'vaex-astro': '0.9.0', 'vaex-jupyter': '0.6.0', 'vaex-ml': '0.14.0'} - Vaex was installed via: pip
- OS: Win10
- Python: 3.7.7