Tutorial Design Ideas

The issue names no implementation files or tests; begin by reviewing the proposed story units, skill-tree maps, and the Benchpark docs “Using Benchpark” page. Define the tutorial structure, online access, and adaptive path selection before implementation, since the issue does not yet specify a concrete deliverable or completion criteria.

Overall Goal: Introduce structure of shorter lessons alternating with hands-on exercises

Core Concept: "Story" Units

• Build the tutorial out of small, self-contained "stories", each pairing a short slide segment (concept, why it matters, what it enables) with a hands-on exercise that puts it into practice
• Create far more stories than any single half-day tutorial needs, forming a reusable content library rather than a fixed curriculum

Choose-Your-Own-Adventure Delivery

• The tutorial starts with intros, a high-level overview of the tools, and a super basic intro. Then poll attendees (Slido) to learn which topics they most want to learn
• Use poll results to select which stories to actually teach, so the session adapts to attendee interest instead of hoping a fixed agenda matches their needs
• We wouldn't necessarily get to teach everyone exactly what they want, but this way the tutorials can change to fit what the people want instead of us having to hope our content covers what they want.

Skill Tree / Map Structure

• Reframe stories as nodes in a "skill tree" (like a video game): white boxes = mini-tutorials/stories, colored boxes at the bottom = high-level end-goal tasks, colored arrows = the path through mini-tutorials needed to reach each goal (see figure)
• Example goals: "add your own benchmark" vs. "run existing benchmarks on your system". Both might share early steps ("Intro to the tools" → "Intro to workspaces") but diverge later ("Writing an application" vs. "writing a system")
• Build similar maps for other skill sets beyond benchmarking, e.g., profiling applications
• Poll attendees using the end-goal boxes, then pick whichever paths cover the maximum number of desired skills across the group

First goal (yellow), existing benchpark tutorial
Second goal (pink), write system.py for nate's container. Maybe teach ramble's tutorial if we are consolidating system object

PACT: teach systems that are open access (frontier, perlmutter, aurora-not yet)
AWS, perlmutter, frontier covers cpu-only systems, nvidia gpus, and amd gpus
how to write a system is useful, but not sure which one people would be most interested in, so show them a variety
Showing you how to do this with spack-based system, other build systems exist

Green - spack package, application py, and experiment py (not enough time to show all 3)

(to add box (1 or more) at the bottom) Run with a different software stack on existing system: init 2 variants of system in benchpark and can compare them with benchpark -- we could pick a system that has a couple different variants. go into system class, find sw they want to modify to setup a variant. Could be a more natural way to show how to write ht eystem.py class in the first place. Start with cpu only, then add things for gpu system as an example. Instead of compilers, run different versions of kripke. this would be a variant of an application. experiment pointing at different source, branch, etc. pick different target of spack package -- two independent workspaces. Editing experiment.py systme.py to edit software. Changing system software or application software. 2 separate questions. Different system software vs different versions of the application. MPI or compilesr or runtime version. From ramble's perspective, these are the same tutorial since it ends up in the yaml.

Pink - generalize to parameter study, but currently is scaling study
ramble - how you do it, what do you get out of it
benchpark - focuses on performance, how much time is comp vs comm
parameter -- how to look at parameters that are available
for scaling study -- we know you are changing the number of nodes

Documentation & Online Access

• Publish the stories online as cached mini-presentations so people can access them asynchronously/offline
• Document the skill tree online so people can map their own end goal to the sequence of tutorials/stories needed to get there
• Tie this into existing docs structure — e.g., on the Benchpark docs "Using Benchpark" page, distinguish mini-story content ("Set up a workspace") from end-goal content ("Analyze a scaling study")

End Goals

1. I want to repeat an experiment someone gave me on the same system
   How to query what is in ramble (Ramble)
   How to create/configure a workspace (one for Benchpark, one for Ramble)

2. I want to replicate an experiment using a different compiler on the same system

3. I want to run (reproduce) an existing experiment on my new systemHow to write a system object (Benchpark)

4. I want to run my new workload on an existing system
   How to write an applicaiton object (Ramble)

5. I want to perform a scaling study of an existing workload on an existing system
   How to interact with a workspace (Benchpark)
   How to work with vectors and matrices (Ramble)

6. I want to analyze the performance of an existing workload on an existing system@pearce8
   How to apply modifiers (Ramble)

At the beginning of the tutorial, have a slido for attendees to vote on end goals that we have tutorial content for. But also an option for them to list things we haven't thought about. We can use this for next tutorial, which skills we did not convey in the tutorial. We would order the cases such that there's not a lot of overlap in the material we cover.

Same example throughout

• must be instrumented with caliper
• difference in performance with different compilers

canonical example: aws (doesn't cover gpu systems though)
canonical experiment that demonstrates what we want

want to use fluxtainer for pact at minimum
yellow box
(add more for sc)

@douglasjacobsen