taskflow/taskflow

dumping/profiling tf::DataPipeline example

Offen

#599 geöffnet am 26.06.2024

 (1 Kommentar) (0 Reaktionen) (0 zugewiesene Personen)C++ (1.101 Forks)batch import
help wanted

Repository-Metriken

Stars
 (9.162 Sterne)
PR-Merge-Metriken
 (Durchschn. Merge 35T 4h) (1 gemergte PR in 30 T)

Beschreibung

I am trying to understand how DataPipeline works exactly and was hoping that dumping the graph and/or profiling would help.

I am using the example in https://taskflow.github.io/taskflow/DataParallelPipeline.html

When I dump the graph, I get the picture also shown on that page with tasks "rt-0" to "rt-3", which is not helpful to understand what is actually executed. If I generate profile data I get:

[
{}
,{"executor":"1","data":[{"worker":13,"level":0,"data":[{"span":[304,3000457],"name":"rt-2","type":"static"},{"span":[6001163,10001414],"name":"rt-3","type":"static"},{"span":[10001859,14002142],"name":"rt-3","type":"static"},{"span":[14002801,18003053],"name":"rt-3","type":"static"},{"span":[18003797,22004197],"name":"rt-3","type":"static"}]},{"worker":15,"level":0,"data":[{"span":[328,3000492],"name":"rt-3","type":"static"},{"span":[5000936,9001253],"name":"rt-2","type":"static"},{"span":[9001738,13002007],"name":"rt-2","type":"static"},{"span":[13002642,17002958],"name":"rt-2","type":"static"},{"span":[17003666,21003893],"name":"rt-2","type":"static"}]},{"worker":16,"level":0,"data":[{"span":[240,246],"name":"cond","type":"condition"},{"span":[251,7000794],"name":"rt-0","type":"static"},{"span":[7001322,11001640],"name":"rt-0","type":"static"},{"span":[11001990,15002472],"name":"rt-0","type":"static"},{"span":[15002940,19003278],"name":"rt-0","type":"static"},{"span":[19004113,20004227],"name":"rt-0","type":"static"},{"span":[20004234,23004558],"name":"rt-1","type":"static"}]},{"worker":17,"level":0,"data":[{"span":[226,23004570],"name":"pipeline","type":"module"}]},{"worker":17,"level":1,"data":[{"span":[277,3000437],"name":"rt-1","type":"static"},{"span":[4000597,8001050],"name":"rt-1","type":"static"},{"span":[8001499,12001996],"name":"rt-1","type":"static"},{"span":[12002241,16002878],"name":"rt-1","type":"static"},{"span":[16003139,20003673],"name":"rt-1","type":"static"}]}]}
]

which is an invalid format according to https://taskflow.github.io/tfprof/ You can fix this by deleting {}, at the beginning. I assume this is a bug?

Finally, that is also not very helpful to understand what is happening exactly in this example. Is there any other way to inspect how the jobs in the pipeline are scheduled?

Thank you.

Contributor Guide