Lots of S3 files still breaks stuff
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 30/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Stale
- Domain
- backend, performance
Research direction
Start by reproducing the hang on staging after clearing the cache and stache, then opening an asset container in the control panel. Read the related issues #7317 and #8626 and the linked eloquent-driver pull request #218 for context. Done means the asset container loads without the site hanging or requiring a server restart.
Written by the indexing model from the issue text.
Description
Bug description
I've been wrestling with this for the past few months unable to identify what is actually going on. For some context, I maintain 10+ production Statamic sites and I've only experienced this on one, and the only thing that stands out about that one as far as I can tell is it has a large number of assets in s3. Roughly 7.5GB of images across ~16,500 files (spread across hundreds of top level directories).
My working theory is this is related to some "background task" that is running and initializing something (stache? metadata files? no idea really) - I tried asking about this in discord but didn't get any replies.
The best example of what happens here is when I deploy to our staging server, which wipes the cache/stache etc and I navigate to an asset container in the control panel, the whole site hangs. Sometimes it resolves after a few minutes, sometimes it leads to what I assume is thrashing and I have to force stop the EC2 (even that takes about 5 minutes so whatever is happening is so intense an EC2 "force stop" can't even execute).
If it does resolve on our staging server, the site appears to be pretty snappy from that point on so it really seems like some sort of secret initialization process. Another interesting data point is that this is far less of an issue on our production server is which significantly higher resourced (but also like 10x higher resourced than any of our other Statamic sites).
This may be related to these:
- https://github.com/statamic/cms/issues/7317
- https://github.com/statamic/eloquent-driver/pull/218
- https://github.com/statamic/cms/issues/8626
How to reproduce
See above.
Logs
Environment
Environment
Application Name: <redacted>
Laravel Version: 11.45.1
PHP Version: 8.3.10
Composer Version: 2.7.7
Environment: production
Debug Mode: OFF
URL: <redacted>
Maintenance Mode: OFF
Timezone: UTC
Locale: en
Cache
Config: NOT CACHED
Events: NOT CACHED
Routes: NOT CACHED
Views: CACHED
Drivers
Broadcasting: log
Cache: file
Database: mysql
Logs: stack / single
Mail: ses
Queue: sync
Session: file
Statamic
Addons: 4
Sites: 1
Stache Watcher: Disabled
Static Caching: half
Version: 5.58.1 PRO
Statamic Addons
alt-design/alt-redirect: 1.3.3
aryehraber/statamic-logbook: 3.3.0
jacksleight/statamic-bard-texs
Installation
Fresh statamic/statamic site via CLI
Additional details
No response
- Dominant language
- PHP
- Stars
- 4.9k
- Forks
- 647
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 102
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from statamic/cms
-
Difficulty 1/5 Under an hour Newbie friendliness 82/100
Maintainers usually reply within 1 day
-
assets
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
Maintainers usually reply within 1 day
-
accessibility
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
Maintainers usually reply within 1 day
-
antlers
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
Maintainers usually reply within 1 day
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Maintainers usually reply within 2 days
-
bug
Difficulty 1/5 Under an hour Newbie friendliness 85/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 63/100
smarty-php/smarty#1215 ·
-
sync-en
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
Maintainers usually reply within 1 day
-
sync-en
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 4 days