How to calculate metrics? Bucket size, last updated, etc

オープン
  • 何か問題がありますか？
#137 コメント 10 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
5/5
見積もり時間
1週間以上
初心者へのやさしさ
12/100
issue の種類
機能追加
明瞭さ
説明が足りない
活発さ
停滞
技術スタック
python
領域
api, cloud

調査の方向性

The payload names no file or test in this repository. The request depends on whether the B2 service exposes per-bucket size, file count and last-upload time, which is a server-side question outside this SDK. Start from the linked B2_Command_Line_Tool issue 404 and the --showSize recursive sum it describes. Done would need a maintainer decision on an API endpoint, so do not start a patch before that is settled.

索引モデルが issue の本文から書いたものです。

説明

enhancement

We are in the process of setting up monitoring and alerts for our backblaze backups, so we are notified if one of our backup processes stops working.

Some metrics I'd like to track per bucket are:

  • time since last updated (timestamp of most recently uploaded file)
    • this one is the most critical -- is our backup process running at all?
  • total size and number of files
    • we are adding data daily, so in general these numbers should slowly increase over time
  • number of versions
    • to make sure config didn't get accidentally messed up for buckets that should have versions

For total size and number of files -- I found https://github.com/Backblaze/B2_Command_Line_Tool/issues/404 which adds --showSize to the CLI. But I looked at the code that calculates this, and it recursively looks at every file in the bucket and adds it up. That's simply not going to perform well. (It worked fine for a small bucket, but when I tried it for one of our larger buckets I didn't get a result after 15 minutes of waiting and killed the process) What's strange is, I can see this info on the Backblaze website, so it seems like you guys know these stats per bucket? Is there a chance it could be exposed somehow?

For time since last updated, we could add a "canary file" to the root of each bucket, make sure it gets updated regularly, and check that... but it would be far less brittle if backblaze provided this info. Do you store this info? If so, is there any way to access it?

For number of versions, I could parse out lifecycle_rules from the bucket info, so that's fine.

Any guidance here? We use prometheus for metrics, so the plan is to use this python sdk and write a simple client to export the above metrics. It would be generic enough that we could open-source the project so others could use it. But as it stands right now, I can't figure out a way to get enough information to create useful metrics in a generic way.

Thanks!

主要言語
Python
スター
214
フォーク
68
平均マージ
51分
マージ済み PR（30日）
6

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

Backblaze/b2-sdk-python のほかの issue

Backblaze/b2-sdk-python の issue をすべて見る

似ている issue

Python の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。