Have the possibility to force disable async archiving in case async archiving is not fully working and we detect it falsely

I've been experiencing an issue with Matomo core:archive scheduled cron jobs throwing "Got invalid response from API request: The response was empty" errors ever since I upgraded to Matomo 4.4.1. Not all of the API requests fail, only a few, and there doesn't seem to be any pattern to which ones fail.

My scheduled cron task sends me an email with errors every single time the cron job runs (hourly), until I have ~800 emails from the past few weeks. More details. Strangely, if I run the exact same command manually via SSH, it will always complete successfully without any errors.

The Error Message

Your scheduled task "matomo" on site _____:

php /home/public/stats/matomo/console core:archive
--url=https://www.matomo.matomo/stats/matomo/ >/dev/null

completed at 2021-08-23 18:03:39 UTC after 3 minutes, 6 seconds and
produced the following output:

ERROR [2021-08-23 18:01:03] 29023  Got invalid response from API request:
?module=API&method=CoreAdminHome.archiveReports&idSite=3&period=day&date=2021-08-23&format=json&trigger=archivephp.
The response was empty. This usually means a server error. A solution to
this error is generally to increase the value of 'memory_limit' in your
php.ini file.  For more information and the error message please check in
your PHP CLI error log file. As this core:archive command triggers PHP
processes over the CLI, you can find where PHP CLI logs are stored by
running this command: php -i | grep error_log
ERROR [2021-08-23 18:01:03] 29023  Error unserializing the following
response from
?module=API&method=CoreAdminHome.archiveReports&idSite=3&period=day&date=2021-08-23&format=json&trigger=archivephp:
''

Obviously, I've tried raising the memory limit, but I'm quite sure that's not the real issue, and it didn't make the errors go away. I've tried debugging further, but neither PHP nor Matomo will log any meaningful errors to shed any more light on the issue. A full summary of what I've tried already and additional info can be found at the Matomo forums: (https://forum.matomo.org/t/uncaught-exception-matomo-core-cronarchive-php-599-got-invalid-response-from-api-request/42849) Downgrading to Matomo 4.3.1 temporarily solved the problem for me, so I'm stuck using an old version of Matomo until this issue can be resolved. I will be happy to provide any additional details/logs if they would be helpful.

Your Environment

Matomo version: 4.4.1 MySQL version: 10.5.8-MariaDB-1:10.5.8+maria~bionic PHP version: 7.4.21 Server Operating System: FreeBSD