vaexio/vaex
Auf GitHub ansehen[BUG-REPORT] Vaex dataframe access crashes asyncio event loop tasks
Open
#1.701 geöffnet am 14. Nov. 2021
bughelp wanted
Repository-Metriken
- Stars
- (8.117 Stars)
- PR-Merge-Metriken
- (Keine gemergten PRs in 30 T)
Beschreibung
Description I'm experiencing issues when using vaex in the context of an asyncio event loop. The repeated interaction with the data frame seems to kill auxiliary tasks in the same event loop. The following code reproduces my issue:
import asyncio
import vaex
import numpy as np
async def vaex_task(df):
while True:
await asyncio.sleep(0.1)
print(df.count())
async def aux_task():
while True:
await asyncio.sleep(0.3)
print("Aux task alive")
df = vaex.from_dict(dict(x=np.linspace(0, 1., 100)))
loop = asyncio.get_event_loop()
loop.create_task(vaex_task(df))
loop.create_task(aux_task())
loop.run_forever()
Example output is:
100
100
Aux task alive
100
100
100
Aux task alive
100
100
Aux task alive
100
100
100
Aux task alive
100
100
Exception in callback <TaskWakeupMethWrapper object at 0x00000156BC40C4F8>(<Future finished result=None>)
handle: <Handle <TaskWakeupMethWrapper object at 0x00000156BC40C4F8>(<Future finished result=None>)>
Traceback (most recent call last):
File "C:\Python37\lib\asyncio\events.py", line 88, in _run
self._context.run(self._callback, *self._args)
RuntimeError: Cannot enter into task <Task pending coro=<aux_task() running at D:\minimal_vaex_problem.py:20> wait_for=<Future finished result=None>> while another task <Task pending coro=<vaex_task() running at D:\minimal_vaex_problem.py:16>> is being executed.
Task was destroyed but it is pending!
task: <Task pending coro=<aux_task() running at D:\minimal_vaex_problem.py:20> wait_for=<Future finished result=None>>
100
100
100
100
100
100
100
100
100
100
The time after which the exception is thrown varies from run to run, but usually happens within the first 10 iterations of aux_task
Does that look like a bug in vaex or am I doing something wrong here?
Thanks in advance for looking into this!
Software information
- Vaex version (
import vaex; vaex.__version__): {'vaex': '4.5.0', 'vaex-core': '4.5.1', 'vaex-viz': '0.5.0', 'vaex-hdf5': '0.10.0', 'vaex-server': '0.6.1', 'vaex-astro': '0.9.0', 'vaex-jupyter': '0.6.0', 'vaex-ml': '0.14.0'} - Vaex was installed via: pip
- OS: Win10
- Python: 3.7.7