Unicode patch email rejected

Bắt đầu với lệnh quản lý parsemail và lời gọi parse_mail tại patchwork/parser.py:1191, sau đó theo dõi Patch.save trong models.py. Tái hiện lỗi Unicode email đã được báo cáo và kiểm tra cách nội dung patch được lưu trữ trong cơ sở dữ liệu sử dụng MySQL. Được xem là hoàn tất khi patch được phân tích và lưu trữ mà không xảy ra OperationalError đã được báo cáo.

This patch on libc-alpha did not appear in patchwork as it crashed during parsing:

https://sourceware.org/pipermail/libc-alpha/2022-October/142506.html

Here's the backtrace at the point of the crash:

Error when parsing incoming email: OperationalError(1366, "Incorrect string value: '\\xF0\\x91\\x82\\x80 0...' for column `patchwork`.`patchwork_patch`.`content` at row 1")
Traceback (most recent call last):
  File "/path/to/patchwork/lib/python3/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/path/to/patchwork/lib/python3/lib/python3.6/site-packages/django/db/backends/mysql/base.py", line 73, in execute
    return self.cursor.execute(query, args)
  File "/path/to/patchwork/lib/python3/lib64/python3.6/site-packages/MySQLdb/cursors.py", line 206, in execute
    res = self._query(query)
  File "/path/to/patchwork/lib/python3/lib64/python3.6/site-packages/MySQLdb/cursors.py", line 319, in _query
    db.query(q)
  File "/path/to/patchwork/lib/python3/lib64/python3.6/site-packages/MySQLdb/connections.py", line 254, in query
    _mysql.connection.query(self, query)
MySQLdb._exceptions.OperationalError: (1366, "Incorrect string value: '\\xF0\\x91\\x82\\x80 0...' for column `patchwork`.`patchwork_patch`.`content` at row 1")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/path/to/patchwork/patchwork.sourceware.org/patchwork/management/commands/parsemail.py", line 60, in handle
    result = parse_mail(mail, options['list_id'])
  File "/path/to/patchwork/patchwork.sourceware.org/patchwork/parser.py", line 1191, in parse_mail
    state=find_state(mail),
  File "/path/to/patchwork/lib/python3/lib/python3.6/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/path/to/patchwork/lib/python3/lib/python3.6/site-packages/django/db/models/query.py", line 447, in create
    obj.save(force_insert=True, using=self.db)
  File "/path/to/patchwork/patchwork.sourceware.org/patchwork/models.py", line 554, in save
    super(Patch, self).save(**kwargs)
  File "/path/to/patchwork/patchwork.sourceware.org/patchwork/models.py", line 385, in save
    super(EmailMixin, self).save(*args, **kwargs)
  File "/path/to/patchwork/lib/python3/lib/python3.6/site-packages/django/db/models/base.py", line 754, in save
    force_update=force_update, update_fields=update_fields)
  File "/path/to/patchwork/lib/python3/lib/python3.6/site-packages/django/db/models/base.py", line 792, in save_base
    force_update, using, update_fields,
  File "/path/to/patchwork/lib/python3/lib/python3.6/site-packages/django/db/models/base.py", line 895, in _save_table
    results = self._do_insert(cls._base_manager, using, fields, returning_fields, raw)
  File "/path/to/patchwork/lib/python3/lib/python3.6/site-packages/django/db/models/base.py", line 935, in _do_insert
    using=using, raw=raw,
  File "/path/to/patchwork/lib/python3/lib/python3.6/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/path/to/patchwork/lib/python3/lib/python3.6/site-packages/django/db/models/query.py", line 1254, in _insert
    return query.get_compiler(using=using).execute_sql(returning_fields)
  File "/path/to/patchwork/lib/python3/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1397, in execute_sql
    cursor.execute(sql, params)
  File "/path/to/patchwork/lib/python3/lib/python3.6/site-packages/django/db/backends/utils.py", line 66, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/path/to/patchwork/lib/python3/lib/python3.6/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/path/to/patchwork/lib/python3/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/path/to/patchwork/lib/python3/lib/python3.6/site-packages/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/path/to/patchwork/lib/python3/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/path/to/patchwork/lib/python3/lib/python3.6/site-packages/django/db/backends/mysql/base.py", line 73, in execute
    return self.cursor.execute(query, args)
  File "/path/to/patchwork/lib/python3/lib64/python3.6/site-packages/MySQLdb/cursors.py", line 206, in execute
    res = self._query(query)
  File "/path/to/patchwork/lib/python3/lib64/python3.6/site-packages/MySQLdb/cursors.py", line 319, in _query
    db.query(q)
  File "/path/to/patchwork/lib/python3/lib64/python3.6/site-packages/MySQLdb/connections.py", line 254, in query
    _mysql.connection.query(self, query)
django.db.utils.OperationalError: (1366, "Incorrect string value: '\\xF0\\x91\\x82\\x80 0...' for column `patchwork`.`patchwork_patch`.`content` at row 1")

The backtrace was attained using the following change to the sourceware instance, maybe something like this could be useful for general purpose logging too:

 
 PYTHONPATH="${PATCHWORK_BASE}:${PATCHWORK_BASE}/lib/python:$PYTHONPATH" \
     DJANGO_SETTINGS_MODULE="$DJANGO_SETTINGS_MODULE" \
-    "$PW_PYTHON" "$PATCHWORK_BASE/manage.py" parsemail "$@"
+    "$PW_PYTHON" "$PATCHWORK_BASE/manage.py" parsemail \
+    "$@" >> $PATCHWORK_BASE/patchwork-mailstat.log 2>&1
 
 # NOTE(stephenfin): We must return 0 here. When parsemail is used as a
 # delivery command from a mail server like postfix (as it is intended