Eliminate inefficiency in form of some extra data downloaded in sync for large files

オープン
#380 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
4/5
見積もり時間
3〜5日
初心者へのやさしさ
25/100
issue の種類
機能追加
明瞭さ
説明が足りない
活発さ
停滞
技術スタック
python
領域
performance

調査の方向性

No files or tests are named in the payload. Start by locating the Synchronizer class and the downloader it drives in the b2-sdk-python codebase, then trace where a sync download sets the Range header for the first part. The linked B2 integration checklist explains multithreaded downloads. Done means the sync downloader receives the file size and opens the first part with an exact range, avoiding the extra bytes.

索引モデルが issue の本文から書いたものです。

説明

help wanted

Large files are downloaded in parts in parallel by all compliant clients, as specified by B2 integration checklist.

The extra data cost is there because when we start streaming the file, we don't (usually) know the size, so we are not requesting an exact range for the download of the first part, but we are sort of requesting the entire file. As soon as the headers come we realize that it's actually a big file, so we start a download session for the further parts of the file. When the data stream for the first part will emit enough data for us to satisfy the first part of the download range, we close the socket HOWEVER because there is some latency on the network, the server has already managed to send us some extra data which is being transmitted from the server to the client while the connection close request is transmitted from the client to the server. These few extra KB are lost.

When in context of sync we do know the file size. Synchronizer could communicate the file size to a specialized implementation of downloader which would then set a clean download part size using the Range header even for the first part.

主要言語
Python
スター
214
フォーク
68
平均マージ
51分
マージ済み PR(30日)
6

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

Backblaze/b2-sdk-python のほかの issue

Backblaze/b2-sdk-python の issue をすべて見る

似ている issue

Python の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。