GoogleCloudPlatform/cloud-opensource-java

Make LinkageChecker faster

Offen

#489 geöffnet am 05.03.2019

 (1 Kommentar) (0 Reaktionen) (1 zugewiesene Person)Java (80 Forks)auto 404
Tech Debtenhancementgood first issuehelp wantedp3

Repository-Metriken

Stars
 (162 Sterne)
PR-Merge-Metriken
 (PR-Metriken ausstehend)

Beschreibung

It seems trying to fetch Maven artifacts from remote repository unnecessarily.

It would take 40000 seconds to fill 167 x 167 table for pair-wise comparison.

Running io.grpc:grpc-auth:jar:1.18.0 com.google.cloud:google-cloud-monitoring:jar:1.63.0 (923/27889). ETA: 408741 seconds
Running io.grpc:grpc-auth:jar:1.18.0 com.google.api.grpc:proto-google-cloud-monitoring-v3:jar:1.45.0 (924/27889). ETA: 408741 seconds
Running io.grpc:grpc-auth:jar:1.18.0 com.google.api.grpc:grpc-google-cloud-monitoring-v3:jar:1.45.0 (925/27889). ETA: 408769 seconds
Running io.grpc:grpc-auth:jar:1.18.0 com.google.cloud:google-cloud-os-login:jar:0.81.0-alpha (926/27889). ETA: 408796 seconds

There was a bug where each cell of the pair-wise comparison fetching all artifacts in the BOM. Fixed. Now it looks like this:

Running com.google.guava:guava:jar:26.0-android com.google.cloud:google-cloud-errorreporting:jar:0.81.0-beta (69/27889). ETA: 40857 seconds
Running com.google.guava:guava:jar:26.0-android com.google.api.grpc:proto-google-cloud-error-reporting-v1beta1:jar:0.46.0 (70/27889). ETA: 41164 seconds
Running com.google.guava:guava:jar:26.0-android com.google.api.grpc:grpc-google-cloud-error-reporting-v1beta1:jar:0.46.0 (71/27889). ETA: 40885 seconds
Running com.google.guava:guava:jar:26.0-android com.google.cloud:google-cloud-firestore:jar:0.81.0-beta (72/27889). ETA: 40717 seconds

DashboardMain

Wed Mar 06 16:17:34 EST 2019: reading bom
Wed Mar 06 16:17:38 EST 2019: resolving BOM dependencies
Wed Mar 06 16:17:41 EST 2019: creating linkage checker
Wed Mar 06 16:17:50 EST 2019: finding linkage errors
Wed Mar 06 16:17:57 EST 2019: generating HTML
Wed Mar 06 16:18:02 EST 2019: done.

image

Even with session.setOffline(true), it remains the same. So the slowness is not from network IO.

Wed Mar 06 16:39:52 EST 2019: reading bom
Wed Mar 06 16:39:52 EST 2019: loading artifact info
Wed Mar 06 16:39:56 EST 2019: resolving BOM dependencies
Wed Mar 06 16:39:59 EST 2019: creating linkage checker
Wed Mar 06 16:40:08 EST 2019: finding linkage errors
Wed Mar 06 16:40:15 EST 2019: generating HTML
Wed Mar 06 16:40:20 EST 2019: done.

Pairwise comparison table for 167x167 (skipping same version) took 9 hours.

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 09:29 h
[INFO] Finished at: 2019-03-08T05:41:16-05:00
[INFO] Final Memory: 89M/15503M
[INFO] ------------------------------------------------------------------------

Contributor Guide