"UnicodeEncodeError: 'latin-1' codec can't encode character, ordinal not in range" on GCS upload_object_via_stream

Abierto
  • ¿Algo no está bien?
#1,802 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
32/100
Tipo de issue
Error
Claridad
Necesita aclaración
Estado de actividad
Estancado
Stack tecnológico
google-cloud, python
Área
cloud

Línea de trabajo

Comienza con upload_object_via_stream en libcloud/storage/base.py y la ruta del controlador S3 en libcloud/storage/drivers/s3.py; después, sigue la preparación de la solicitud a través de libcloud/common/base.py y libcloud/http.py. Reproduce el fallo con un CSV que contenga caracteres no pertenecientes a Latin-1 en el entorno de Python y contenedor indicado. Se considera terminado cuando la carga gestiona ese contenido CSV sin el UnicodeEncodeError e incluye cobertura de regresión para la ruta que falla.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Summary

I'm trying to upload some CSV files to GCS using libcloud, today I this error:

UnicodeEncodeError: 'latin-1' codec can't encode character '\u201c' in position 628: ordinal not in range(256)

Detailed Information

The encode character and position change with each run, even though the files don't change. I'm not sure what's causing that, CSV files are basically plaintext.

Here's the full stack (some internal code ommitted):

Traceback (most recent call last):
  [...]
    self.__container.upload_object_via_stream(stream, name)  # type: ignore
  File "/usr/local/lib/python3.8/site-packages/libcloud/storage/base.py", line 227, in upload_object_via_stream
    return self.driver.upload_object_via_stream(
  File "/usr/local/lib/python3.8/site-packages/libcloud/storage/drivers/s3.py", line 843, in upload_object_via_stream
    return self._put_object(
  File "/usr/local/lib/python3.8/site-packages/libcloud/storage/drivers/s3.py", line 1007, in _put_object
    result_dict = self._upload_object(
  File "/usr/local/lib/python3.8/site-packages/libcloud/storage/base.py", line 874, in _upload_object
    response = self.connection.request(
  File "/usr/local/lib/python3.8/site-packages/libcloud/common/base.py", line 659, in request
    return request_to_be_executed(
  File "/usr/local/lib/python3.8/site-packages/libcloud/common/base.py", line 678, in _retryable_request
    self.connection.prepared_request(
  File "/usr/local/lib/python3.8/site-packages/libcloud/http.py", line 260, in prepared_request
    self.response = self.session.send(
  File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 645, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 440, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 398, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 239, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/usr/local/lib/python3.8/http/client.py", line 1256, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/lib/python3.8/http/client.py", line 1302, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.8/http/client.py", line 1251, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.8/http/client.py", line 1040, in _send_output
    for chunk in chunks:
  File "/usr/local/lib/python3.8/http/client.py", line 999, in _read_readable
    datablock = datablock.encode("iso-8859-1")
UnicodeEncodeError: 'latin-1' codec can't encode character '\u2019' in position 5251: ordinal not in range(256)

I'm running this from inside a k8s pod in a GKE cluster, trying to write to a GCS bucket.

apache-libcloud version: 3.5.1
python version: 3.8.15
pod image: python:3.8-slim

I'm not sure what are the steps to reproduce it. It's a CSV file generated by pulling some BigQuery schema metadata. It works locally, but when I ran it in the pod it raised this error.

Lenguaje dominante
Python
Estrellas
2.1k
Forks
935
Merge medio
5 d 3 h
PR fusionados (30 d)
15

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de apache/libcloud

Todos los issues de apache/libcloud

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.