Retry functionality in the library only covers the initial response. While this is the most common point of failure, some errors, especially Connection Reset by Peer (which usually boils down to requests.ChunkedEncodingError), escape the retry clause because they only appear during the actual streaming/writing to file period. We will need to adjust our control flow so that retry covers both the initial response (receiving/evaluating http headers) and the streaming portion of the download.

With a 16-core GCE VM and multiple processes, I observed this issue about 10 times in 1,000,000 downloads of 0.5-1 GB files.