matthewmueller/x-ray

Can't catch an error thrown when there's an invalid status code 522

Open

#90 创建于 2015年9月2日

在 GitHub 查看
 (1 评论) (1 反应) (0 负责人)JavaScript (5,804 star) (415 fork)batch import
bughelp wanted

描述

Hi,

I'm trying to scrape the url below but after some time it shows an error: http://www.lazygamer.net/boost/the-top-10-gaming-youtubers-in-2014-so-far/

assert.js:86
  throw new assert.AssertionError({
        ^
AssertionError: invalid status code: 522
    at Object.status (/Users/jonathan/git/youtube_urls_from_url/node_modules/x-ray/node_modules/http-context/lib/response.js:81:5)
    at context.status (/Users/jonathan/git/youtube_urls_from_url/node_modules/x-ray/node_modules/http-context/node_modules/delegates/index.js:92:31)
    at /Users/jonathan/git/youtube_urls_from_url/node_modules/x-ray/node_modules/x-ray-crawler/lib/http-driver.js:30:20
    at Request.callback (/Users/jonathan/git/youtube_urls_from_url/node_modules/x-ray/node_modules/x-ray-crawler/node_modules/superagent/lib/node/index.js:797:3)
    at IncomingMessage.<anonymous> (/Users/jonathan/git/youtube_urls_from_url/node_modules/x-ray/node_modules/x-ray-crawler/node_modules/superagent/lib/node/index.js:990:12)
    at IncomingMessage.emit (events.js:129:20)
    at _stream_readable.js:908:16
    at process._tickCallback (node.js:355:11)

I also attached a screenshot of the webpage loaded from the browser if it's helpful.

贡献者指南

技术栈
javascriptnodejs
领域
backend
议题类型
bug
难度面向新贡献者的预计实现难度，1 表示很小改动，5 表示专家级工作。
2
预计时间有经验贡献者完成调查、实现、测试并准备 pull request 的粗略时间范围。
under 1 hour
活动状态议题当前的可参与程度：新鲜、活跃、陈旧、阻塞或等待维护者输入。
stale
清晰度议题是否清楚说明期望改动、验收标准和下一步。
clear
前置要求
Node.js basicserror handling
新手友好度1-100 的估计分数，表示该议题对首次贡献者的友好程度。
70
研究方向
The issue occurs when an HTTP status code 522 is encountered, causing an unhandled AssertionError in `http context/lib/response.js` (line 81). The error propagates from x ray crawler's http driver.js via superagent. To fix, consider catching the assertion error or validating status codes before they are passed to the response object. No linked PRs or maintainer comments exist, so investigate whether the assertion should be wrapped in a try catch or the status code should be allowed.