RuntimeError: unexpected EOF, expected 2051578 more bytes. The file might be corrupted.

Start with robosat/tools/predict.py at line 60, where torch.load reads the supplied checkpoint, and run the reported Docker prediction command. Check the checkpoint file and compare the failure on GPU and CPU; done means identifying whether the checkpoint is incomplete or the loading path is at fault and documenting a reproducible resolution.

Hello! @daniel-j-h Thank you for your project.

I am having an error during prediction:

sudo docker run --runtime=nvidia -it --rm -v $PWD:/data --ipc=host mapbox/robosat:latest-gpu predict --tile_size 512 --model /data/robosat/config/model-unet.toml --dataset /data/robosat/config/dataset-building.toml --checkpoint /data/robosat/checkpoint/checkpoint-00001-of-00010.pth /data/robosat/datasets /data/

Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/src/app/robosat/tools/__main__.py", line 58, in <module>
    args.func(args)
  File "/usr/src/app/robosat/tools/predict.py", line 60, in main
    chkpt = torch.load(args.checkpoint, map_location=map_location)
  File "/opt/venv/lib/python3.6/site-packages/torch/serialization.py", line 387, in load
    return _load(f, map_location, pickle_module, **pickle_load_args)
  File "/opt/venv/lib/python3.6/site-packages/torch/serialization.py", line 581, in _load
    deserialized_objects[key]._set_from_file(f, offset, f_should_read_directly)
RuntimeError: unexpected EOF, expected 2051578 more bytes. The file might be corrupted.
terminate called after throwing an instance of 'c10::Error'
  what():  owning_ptr == NullType::singleton() || owning_ptr->refcount_.load() > 0 ASSERT FAILED at /pytorch/c10/util/intrusive_ptr.h:350, please report a bug to PyTorch. intrusive_ptr: Can only intrusive_ptr::reclaim() owning pointers that were created using intrusive_ptr::release(). (reclaim at /pytorch/c10/util/intrusive_ptr.h:350)
frame #0: std::function<std::string ()>::operator()() const + 0x11 (0x7fb856a7d441 in /opt/venv/lib/python3.6/site-packages/torch/lib/libc10.so)
frame #1: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x2a (0x7fb856a7cd7a in /opt/venv/lib/python3.6/site-packages/torch/lib/libc10.so)
frame #2: THStorage_free + 0xca (0x7fb857a1729a in /opt/venv/lib/python3.6/site-packages/torch/lib/libcaffe2.so)
frame #3: <unknown function> + 0x53a157 (0x7fb89f421157 in /opt/venv/lib/python3.6/site-packages/torch/lib/libtorch_python.so)
frame #4: python3() [0x54f1c6]
frame #5: python3() [0x573420]
frame #6: python3() [0x4b1a48]
frame #7: python3() [0x5891e8]
frame #8: python3() [0x5add18]
frame #9: python3() [0x5add2e]
frame #10: python3() [0x5add2e]
frame #11: python3() [0x5add2e]
frame #12: python3() [0x5add2e]
frame #13: python3() [0x5add2e]
frame #14: python3() [0x56b906]
<omitting python frames>
frame #20: __libc_start_main + 0xe7 (0x7fb8c8cbbb97 in /lib/x86_64-linux-gnu/libc.so.6)

Aborted (core dumped)

I am running on my laptop, with GTX 1660TI. Do you think this is the reason for that? Furthermore, getting same error while trying with only CPU.
Can you please comment on that?