Skip to content
This repository was archived by the owner on May 6, 2026. It is now read-only.

fix: add retry coverage to the streaming portion of a download#241

Merged
andrewsg merged 7 commits into
googleapis:masterfrom
cojenco:retry-download-block
Jul 27, 2021
Merged

fix: add retry coverage to the streaming portion of a download#241
andrewsg merged 7 commits into
googleapis:masterfrom
cojenco:retry-download-block

Conversation

@cojenco

@cojenco cojenco commented Jul 20, 2021

Copy link
Copy Markdown
Contributor

This broadens the retry coverage in downloads to cover both the initial response (receiving/evaluating http headers) and the streaming/writing to file portion of the download.

  • Modifies the original consume() method to an internal method _consume() and replace the retry-able http request to a raw http request
  • Adds an internal method consume_with_retries() within classes Download and ChunkedDownload that will attempt to retry a callable _consume() until success
  • Updates tests and docstrings

Fixes #240

@product-auto-label product-auto-label Bot added the api: storage Issues related to the googleapis/google-resumable-media-python API. label Jul 20, 2021
@google-cla google-cla Bot added the cla: yes This human has signed the Contributor License Agreement. label Jul 20, 2021
@cojenco cojenco requested a review from andrewsg July 20, 2021 19:36
@cojenco cojenco marked this pull request as ready for review July 27, 2021 23:12
@cojenco cojenco requested a review from a team July 27, 2021 23:12

@andrewsg andrewsg left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this awesome work!

@andrewsg andrewsg merged commit cc1f07c into googleapis:master Jul 27, 2021
cojenco added a commit to cojenco/google-resumable-media-python that referenced this pull request Jul 30, 2021
cojenco added a commit to cojenco/google-resumable-media-python that referenced this pull request Jul 30, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

api: storage Issues related to the googleapis/google-resumable-media-python API. cla: yes This human has signed the Contributor License Agreement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Exception retries do not cover mid-stream errors which occur during downloads.

2 participants