Can't catch an error thrown when there's an invalid status code 522 · matthewmueller/x-ray#90

(1 comment) (1 reaction) (0 assignees)JavaScript (5,804 stars) (415 forks)batch import

bughelp wanted

説明

Hi,

I'm trying to scrape the url below but after some time it shows an error: http://www.lazygamer.net/boost/the-top-10-gaming-youtubers-in-2014-so-far/

assert.js:86
  throw new assert.AssertionError({
        ^
AssertionError: invalid status code: 522
    at Object.status (/Users/jonathan/git/youtube_urls_from_url/node_modules/x-ray/node_modules/http-context/lib/response.js:81:5)
    at context.status (/Users/jonathan/git/youtube_urls_from_url/node_modules/x-ray/node_modules/http-context/node_modules/delegates/index.js:92:31)
    at /Users/jonathan/git/youtube_urls_from_url/node_modules/x-ray/node_modules/x-ray-crawler/lib/http-driver.js:30:20
    at Request.callback (/Users/jonathan/git/youtube_urls_from_url/node_modules/x-ray/node_modules/x-ray-crawler/node_modules/superagent/lib/node/index.js:797:3)
    at IncomingMessage.<anonymous> (/Users/jonathan/git/youtube_urls_from_url/node_modules/x-ray/node_modules/x-ray-crawler/node_modules/superagent/lib/node/index.js:990:12)
    at IncomingMessage.emit (events.js:129:20)
    at _stream_readable.js:908:16
    at process._tickCallback (node.js:355:11)

I also attached a screenshot of the webpage loaded from the browser if it's helpful.

コントリビューターガイド

技術スタック: javascriptnodejs
領域: backend
Issue 種別: bug
難度: 2
推定時間: under 1 hour
活動状況: stale
明確さ: clear
前提条件: Node.js basicserror handling
初心者向け度: 70
調査方針: The issue occurs when an HTTP status code 522 is encountered, causing an unhandled AssertionError in `http context/lib/response.js` (line 81). The error propagates from x ray crawler's http driver.js via superagent. To fix, consider catching the assertion error or validating status codes before they are passed to the response object. No linked PRs or maintainer comments exist, so investigate whether the assertion should be wrapped in a try catch or the status code should be allowed.