`contributor_repo` table does not relate to `repo`

Aperta
#551 9 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
20/100
Tipo di issue
Refactoring
Chiarezza
Da chiarire
Stato di attività
Attiva
Stack tecnologico
python, sql
Ambito
databases

Direzione di ricerca

The contributor_repo table is filled by the contributor breadth worker; inspect that worker and the database schema to see how repo_git and gh_repo_id are written. First read the worker code and any migration scripts, then discuss with maintainers whether to link to repo via repo_src_id. Done means an agreed plan to reclaim space and add the foreign key before multi-forge support.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

database discussion needs-plan tech debt

the contributor_repo table does not have a foreign key to the repo table.

It has its own repo_git column and repo_name column which i dont believe is getting updated when repos move/get renamed. it appears to store events from the github events api (such as PushEvent and CreateEvent). Do we want to keep storing the historical names as they were when the data was collected? the urls seem likely redundant.

We do also have a gh_repo_id column so we can maybe associate this table with repo using the repo_src_id - this would be best to do before we move over to supporting other forges, otherwise this migration may be more difficult

In our 42k repo instance, this table is ~120GB

This table seems to be filled in by the contributor breadth worker.

The unique key on this table seems to be up to ~1.4 billion (im pretty sure its an autoincrement integer)

I suspect this table also has not been the target of any deletion queries we have ever run on this database

This table does not appear to be used by 8Knot but may be used by DSWG notebooks (and some internal stuff)

What should we do to make this table a more sensible use of space?

Lingua principale
Python
Stelle
17
Fork
30
Merge medio
6h 16m
PR unite (30g)
9

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di chaoss/CollectOSS

Tutte le issue di chaoss/CollectOSS

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.