[Enhancement] Implement a fail-fast observable version of the export_to_s3 method
#6 ouverte le 20 sept. 2025
Métriques du dépôt
- Stars
- (1 étoile)
- Métriques de merge PR
- (Métriques PR en attente)
Description
Description:
Currently, the method responsible for exporting data to AWS S3 fails silently when errors occur, making it difficult to detect issues or take corrective action. This silent failure mode can lead to data loss or inconsistencies, and hinders debugging and reliability.
Expected behavior The backend should use an AWS sdk method that provides a proper json response message with details on the outcomes of the upload request issued by the app's export service.
The export_to_s3 method should provide a clear response message and logs on its status, which can be collected by the frontend service and conveyed to the user.
The user gets a clear status on the outcome of their export request.
Actual Behaviour
The current implemenation of the export_to_s3 method uses the s3client.upload_file method without collecting any feedback from AWS server side nor implementing any tests to validate the export. Additionally, Its also not clear if the AWS implementation of s3client.upload_file provides any feedback in the first place.
this implementation leaves the user in confusion about the status of their request
Request: Please implement an alternative export method that adopts the fail-fast principle. The new method should:
- Immediately raise or log errors when failures occur during the S3 export process.
- Provide clear and actionable feedback (e.g., error messages, status codes) to the user or calling process.
Proposed solution
One hint that might be useful towards fulfilling the goal is to explore using the put_object method in the boto3 aws sdk which appears to provide a detailed json response object to put requests.
Benefits:
- Improved reliability and debuggability of data export workflows.
- Early detection of issues, reducing risk of silent data loss.
- Better integration with monitoring and alerting systems.
Context: This enhancement builds on the current export functionality and aims to make export failures explicit and observable.