dask/dask-examples

Occasional failure in HTTP bytes

Open

#85 opened on Jun 24, 2019

 (7 comments) (0 reactions) (1 assignee)Jupyter Notebook (225 forks)auto 404
help wanted

Repository metrics

Stars
 (386 stars)
PR merge metrics
 (PR metrics pending)

Description

When running CI in this project I sometimes run across the following error:

~/miniconda/envs/test/lib/python3.7/site-packages/dask/bag/core.py in reify()
   1603 def reify(seq):
   1604     if isinstance(seq, Iterator):
-> 1605         seq = list(seq)
   1606     if seq and isinstance(seq[0], Iterator):
   1607         seq = list(map(list, seq))
~/miniconda/envs/test/lib/python3.7/site-packages/dask/bag/core.py in map_chunk()
   1769                 yield f(**k)
   1770     else:
-> 1771         for a in zip(*args):
   1772             yield f(*a)
   1773 
~/miniconda/envs/test/lib/python3.7/site-packages/dask/bag/text.py in file_to_blocks()
    103 def file_to_blocks(lazy_file):
    104     with lazy_file as f:
--> 105         for line in f:
    106             yield line
    107 
~/miniconda/envs/test/lib/python3.7/site-packages/dask/bytes/http.py in read()
    247             # EOF (python files don't error, just return no data)
    248             return b''
--> 249         self. _fetch(self.loc, end)
    250         data = self.cache[self.loc - self.start:end - self.start]
    251         self.loc = end
~/miniconda/envs/test/lib/python3.7/site-packages/dask/bytes/http.py in _fetch()
    258             self.start = start
    259             self.end = end + self.blocksize
--> 260             self.cache = self._fetch_range(start, self.end)
    261         elif start < self.start:
    262             if self.end - end > self.blocksize:
~/miniconda/envs/test/lib/python3.7/site-packages/dask/bytes/http.py in _fetch_range()
    320             if cl <= end - start:
    321                 # data size OK
--> 322                 return r.content
    323             else:
    324                 raise ValueError('Got more bytes (%i) than requested (%i)' % (
~/miniconda/envs/test/lib/python3.7/site-packages/requests/models.py in content()
    826                 self._content = None
    827             else:
--> 828                 self._content = b''.join(self.iter_content(CONTENT_CHUNK_SIZE)) or b''
    829 
    830         self._content_consumed = True
~/miniconda/envs/test/lib/python3.7/site-packages/requests/models.py in generate()
    751                         yield chunk
    752                 except ProtocolError as e:
--> 753                     raise ChunkedEncodingError(e)
    754                 except DecodeError as e:
    755                     raise ContentDecodingError(e)
ChunkedEncodingError: ('Connection broken: OSError("(104, \'ECONNRESET\')")', OSError("(104, 'ECONNRESET')"))
ChunkedEncodingError: ('Connection broken: OSError("(104, \'ECONNRESET\')")', OSError("(104, 'ECONNRESET')"))
You can ignore this error by setting the following in conf.py:
    nbsphinx_allow_errors = True
Notebook error:
CellExecutionError in applications/json-data-on-the-web.ipynb:
------------------
df.spec.value_counts().nlargest(20).to_frame().compute()
------------------

@martindurant , this seems to be in your general domain. Do you have any suggestions on what might be happening here?

Contributor guide