MissingHeaderBodySeparatorDefect when uploading items to OSS

報告された put_object 呼び出しと、oci/_vendor/urllib3/connectionpool.py および oci/_vendor/urllib3/util/response.py の traceback のエントリーポイントから調査を開始します。OSS のアップロードを繰り返した際に失敗を再現し、なぜレスポンスが MissingHeaderBodySeparatorDefect を引き起こし、示されている try/except の外でハングするのかを特定し、そのうえで成功するアップロードを妨げずに失敗が確実に表面化することを検証します。

When uploading files to OSS, I randomly (~1000 items in) experience this error after I run

with open(file_path, 'rb') as f:
                obj_data = f.read()
                try:
                      response = oci_client.put_object(
                                    OSS_NAMESPACE, 
                                    OSS_BUCKET_NAME,
                                    oss_object_name,  # object name 
                                    obj_data, # object body (bytes)
                                )
                       if response.status != 200:
                              msg = f'WARNING: {oss_object_name} upload resulted in status {response.status}. {response.data}'
                              logging.log(logging.WARN, msg)
                              print(msg)
                        else:
                              msg = f'SUCCESS! {oss_object_name} uploaded'
                              logging.log(logging.INFO, msg)
                              print(msg)
                except oci.exceptions.ServiceError as e:
                    # Handling OCI specific exceptions
                    msg = f'ERROR: {oss_object_name} upload failed with ServiceError. {e.status} {e.message}'
                    logging.error(msg)
                    print(msg)
                except Exception as e:
                    msg = f'ERROR: {oss_object_name} upload failed. {e}'
                    logging.log(logging.ERROR, msg)
                    print(msg)

Here's the traceback:

2024-02-19 20:09:39,991:WARNING:Failed to parse headers (url=https://objectstorage.ap-osaka-1.oraclecloud.com:443/n/ax2tjlcefnht/b/laion2B-en-aesthetic/o/000000401/000000401.txt): [MissingHeaderBodySeparatorDefect()], unparsed data: 'HTTP/1.1 200 OK\r\netag: 07f8f92b-fd96-49ec-b9db-1b927e144116\r\nlast-modified: Mon, 19 Feb 2024 20:09:39 GMT\r\nopc-content-md5: tMGI50PK84P9//D2pQ8evQ==\r\nversion-id: 04303300-a3ec-4fdb-a2eb-58cd65effb06\r\nContent-Length: 0\r\ndate: Mon, 19 Feb 2024 20:09:39 GMT\r\nopc-request-id: kix-1:BaX2PWrWLWStJYPKMWmC_TRLmcJaINqwJQey95jbJrG80yzz4rVgEBGGUeet-gvk\r\nx-api-id: native\r\nx-content-type-options: nosniff\r\nstrict-transport-security: max-age=31536000; includeSubDomains\r\naccess-control-allow-origin: *\r\naccess-control-allow-methods: POST,PUT,GET,HEAD,DELETE,OPTIONS\r\naccess-control-allow-credentials: true\r\naccess-control-expose-headers: access-control-allow-credentials,access-control-allow-methods,access-control-allow-origin,content-length,date,etag,last-modified,opc-client-info,opc-content-md5,opc-request-id,strict-transport-security,version-id,x-api-id,x-content-type-options\r\n\r\n'
Traceback (most recent call last):
  File "/home/Mindy/.cache/pypoetry/virtualenvs/t2i-adapter-3QVI5i9t-py3.10/lib/python3.10/site-packages/oci/_vendor/urllib3/connectionpool.py", line 474, in _make_request
    assert_header_parsing(httplib_response.msg)
  File "/home/Mindy/.cache/pypoetry/virtualenvs/t2i-adapter-3QVI5i9t-py3.10/lib/python3.10/site-packages/oci/_vendor/urllib3/util/response.py", line 96, in assert_header_parsing
    raise HeaderParsingError(defects=defects, unparsed_data=unparsed_data)
oci._vendor.urllib3.exceptions.HeaderParsingError: [MissingHeaderBodySeparatorDefect()], unparsed data: 'HTTP/1.1 200 OK\r\netag: 07f8f92b-fd96-49ec-b9db-1b927e144116\r\nlast-modified: Mon, 19 Feb 2024 20:09:39 GMT\r\nopc-content-md5: tMGI50PK84P9//D2pQ8evQ==\r\nversion-id: 04303300-a3ec-4fdb-a2eb-58cd65effb06\r\nContent-Length: 0\r\ndate: Mon, 19 Feb 2024 20:09:39 GMT\r\nopc-request-id: kix-1:BaX2PWrWLWStJYPKMWmC_TRLmcJaINqwJQey95jbJrG80yzz4rVgEBGGUeet-gvk\r\nx-api-id: native\r\nx-content-type-options: nosniff\r\nstrict-transport-security: max-age=31536000; includeSubDomains\r\naccess-control-allow-origin: *\r\naccess-control-allow-methods: POST,PUT,GET,HEAD,DELETE,OPTIONS\r\naccess-control-allow-credentials: true\r\naccess-control-expose-headers: access-control-allow-credentials,access-control-allow-methods,access-control-allow-origin,content-length,date,etag,last-modified,opc-client-info,opc-content-md5,opc-request-id,strict-transport-security,version-id,x-api-id,x-content-type-options\r\n\r\n'

Even after adding try...except, I can't catch the exception and my code hangs. put_object worked fine for me two weeks ago when I uploaded hundreds of thousands of items.