Access denied or empty body
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 25/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- javascript, node.js, redis
- Domain
- backend
Research direction
Start by reproducing the 403 or empty-body response with the shown prerender-node middleware, comparing requests with and without the Googlebot user agent. Inspect the beforeRender and afterRender hooks, Redis cache behavior, and the /render?url=https://www.google.com/ path. Done means identifying the failure and restoring a non-empty response with useful error logging.
Written by the indexing model from the issue text.
Description
Issue: HTTP ERROR 403 - Access denied.
Issue appears during last 3 days, before everything was working fine.
Setup:
- Server is Linux Ubuntu
- Local prerender under Node, which is working fine, can access page: /render?url=https://www.google.com/
- "prerender-node" setup with Redias cache.
- Assessing website by browsing with googlebot in User agent string.
Deep dive into testing:
- Website is accessible without googlebot in User agent string.
- In afterRender event getting empty body for any request url
- In afterRender error object is null
- Redis cache is working, even when removing cache mechanism fully, issue persist
- Files permission is correct and was not changed.
- When removing
app.use(prerender);website again accessible with googlebot in User agent string. - Version of packages including "prerender-node" was not released, so nothing to update
- Version of "google-chrome-stable" checked and updated to latest one - 92.0.4515
- Reason of this all points is not applicable: [https://docs.prerender.io/article/25-empty-or-partially-rendered-pages](Empty or partially rendered pages) as it is working with url /render?url=https://www.google.com/
- pageDoneCheckInterval increased up to 20sec to make sure it is not a reason
- Getting empty body immediately, some some error is happening there, but no any error log, as I have pageDoneCheckInterval: 5000, I should get response in 5 sec, which is not happening.
Ideas what can be a reason:
- Something got broken in process of Prerender in some dependencies.
- There is some limit of requested pages per month, but as I use Prerender locally, as I understood it is not limited somehow.
- Some another crazy reason))
Code prerender section for any case:
// Setup prerender
prerender = require('prerender-node');
prerender.set('protocol', 'https');
prerender.crawlerUserAgents.push('Validator.nu/LV');
prerender.crawlerUserAgents.push('W3C-checklink');
app.use(prerender);
// Setup redis cache
var redis = require("redis"),
client = redis.createClient();
client.flushdb( function (err, succeeded) {
});
prerender.set('beforeRender', function(req, done) {
client.get(req.url, done);
}).set('afterRender', function(err, req, prerender_res) {
if ( prerender_res ) {
client.set(req.url, prerender_res.body)
}
});
Anyway this was working previously very long and very stable.
Any idea will appreciated, thanks in advance.
- Dominant language
- JavaScript
- Stars
- 921
- Forks
- 185
- PR merge metrics
- No merged PRs in 30d
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from prerender/prerender-node
-
Release new version? Open
Difficulty 3/5 1-2 days Newbie friendliness 35/100
prerender/prerender-node#274 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 48/100
prerender/prerender-node#272 ·
-
Fastify support? Open
Difficulty 5/5 Over a week Newbie friendliness 15/100
prerender/prerender-node#224 · 1 reaction ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 45/100
prerender/prerender-node#223 · 1 comment ·
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
prerender/prerender-node#220 · 2 comments · 2 reactions ·
All issues in prerender/prerender-node
Similar issues
-
bug customer-eng Durable Agents Inngest status: needs triage
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
optimization optimization:agents-md-curator
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
githubnext/gh-aw-cao#13475 ·
-
[BUG]: "Clear All" in Settings doesn't clear the saved analysis, old data comes back after reload Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
AOSSIE-Org/OrgExplorer#253 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
oxc-project/oxc#26944 ·
-
ai-observability bug team/ai-observability
Difficulty 2/5 1-3 hours Newbie friendliness 78/100