Clarify scope for integration tests without api keys

Aperta
  • Qualcosa non va?
#2,871 3 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
38/100
Tipo di issue
Funzionalità
Chiarezza
Da chiarire
Stato di attività
Attiva
Stack tecnologico
python
Ambito
api, testing

Direzione di ricerca

Inizia da CONTRIBUTING.md e dalla suite di test di integrazione, in particolare test_workflow_with_google_vision_ocr.py e test_workflow_with_google_ocr_without_api_key_via_proxy. Leggi come i test scaricano i modelli tramite la Roboflow API e come viene utilizzato il model manager per l’inferenza. Chiarisci i vincoli previsti e definisci quali test possono essere eseguiti senza chiavi API prima di decidere cosa significhi che il lavoro è completato.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

question
Search before asking
  • I have searched the Inference issues and found no similar feature requests.
Question

Hi roboflow team. I'm interested in contributing to this project. I saw the CONTRIBUTING.md had a contribution idea of making integration tests not require api keys so I thought that would be a good place to start.

I've been digging into the integration test suite and my current understanding is that almost every integration test that requires a roboflow api key needs it to download a model the test needs from the roboflow api.

I only found one suite, test_workflow_with_google_vision_ocr.py , that has an exception to this, test_workflow_with_google_ocr_without_api_key_via_proxy, where the roboflow api key is used because the test itself is checking if the roboflow api can be used as a proxy in this workflow when no google api key is set. I assume that test would need to be addressed differently than the majority that need to download a model.

Before I start working on an implementation I was hoping you could clarify

  1. Would making the integration test suite no longer require api keys still be useful?
  2. I'm assuming that the request to make the tests work without an api keys, means the tests themselves don't exist to test that the inference api is able to download models from roboflow as they would need to bypass calling the roboflow api to not require a key. Is that assumption correct?
  3. I noticed that most of the tests are asserting just that the response shape is correct and not really asserting that the content/result of the endpoint call has a specific result. For those tests did you have in mind that the api call is bypassed through some sort of mock of the model loading into the modelmanager that can be used for inferencing or whatever the endpoint being tested needs or were you thinking more of actually loading the true model artifact through a different means than the roboflow api?
  4. Are there any other constraints I should account for?

Thanks for any guidance. I'm happy to take this on if it'd be useful, or if there's something that would be more helpful I'd appreciate it if you can point me that direction.

Additional

No response

Lingua principale
Python
Stelle
2.5k
Fork
340
Merge medio
2g 11h
PR unite (30g)
92

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di roboflow/inference

Tutte le issue di roboflow/inference

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.