bug/coroutine-await

Open
#149 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Stale
Tech stack
python
Domain
api

Research direction

Start with unstructured_client/_hooks/custom/split_pdf_hook.py around line 202 and review how call_api_partial coroutines are stored and awaited. Reproduce the intermittent failure under the reported high-volume workload, then verify that parsing completes without “coroutine is being awaited already” or “was never awaited” warnings.

Written by the indexing model from the issue text.

Description

bug

Describe the bug
i'm currently using the open source version of unstructured, and getting an exception about coroutine is being awaited already. This causes the parse to fail, and not work.

To Reproduce
It just seems to happen every so often, I'm not sure how to reproduce it. It's not on the same files, if I retry them, they'll succeed. I am putting a lot of files + data through the system, but nothing that seems to be something it shouldnt be able to handle.

Expected behavior
Not get this error.

Screenshots
N/A

Environment Info
I'm pullling downloads.unstructured.io/unstructured-io/unstructured-api:latest as my container and just running this.
Also running unstructured-client = "^0.23.8" on my client side.

Additional context
There's also a few warning that are showing up related to coroutines too:

[2024-08-08 17:10:50,239: WARNING/MainProcess] /home/appuser/.cache/pypoetry/virtualenvs/app-9TtSrW0h-py3.11/lib/python3.11/site-packages/unstructured_client/_hooks/custom/split_pdf_hook.py:202: RuntimeWarning: coroutine 'SplitPdfHook.before_request.<locals>.call_api_partial' was never awaited
  self.coroutines_to_execute[operation_id] = []
[2024-08-08 17:47:48,365: WARNING/MainProcess] Traceback (most recent call last):
[2024-08-08 17:47:48,365: WARNING/MainProcess]   File "<string>", line 1, in <lambda>
[2024-08-08 17:47:48,365: WARNING/MainProcess] KeyError
[2024-08-08 17:47:48,365: WARNING/MainProcess] :
[2024-08-08 17:47:48,365: WARNING/MainProcess] '__import__'
[2024-08-08 17:47:48,366: WARNING/MainProcess] Exception ignored in:
[2024-08-08 17:47:48,366: WARNING/MainProcess] <coroutine object SplitPdfHook.before_request.<locals>.call_api_partial at 0x7f4753372480> 
Dominant language
Python
Stars
119
Forks
22
Avg merge
1d 21h
Merged PRs (30d)
1

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Unstructured-IO/unstructured-python-client

All issues in Unstructured-IO/unstructured-python-client

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.