apple/foundationdb

Add documentation on simulation testing at scale

Open

#4,663 opened on Apr 14, 2021

View on GitHub
 (1 comment) (0 reactions) (0 assignees)C++ (13,912 stars) (1,266 forks)batch import
documentationgood first issue

Description

Now we have open sourced the Joshua testing framework. It's time to add documentation on how to run simulation testings at scale, probably improve both documentation here and at Joshua project. This will make it easier for contributors other than Apple and Snowflake.

To start, we need to describe the steps for setting up the testing infrastructure:

  • Have a FoundationDB cluster running.
  • Build and run the docker image from this script, make sure the docker image uses the same FDB version as the cluster in the first step. Go inside the docker image and make sure it can talk to the cluster.
  • Create a correctness package (i.e., as simple as ninja package_tests).
  • Submit correctness package to the cluster.

We also need descriptions on how to check simulation results, e.g., how many failed, how to reproduce, inactive jobs, etc.

Contributor guide

Tech stack
cppdockershell
Domain
documentationtesting
Issue type
documentation
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
Basic understanding of FoundationDBFamiliarity with DockerAbility to run shell commands
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
60
Research direction
Review the Joshua testing framework repository and the build.sh script. Understand the requirements for setting up a FoundationDB cluster and the Docker image creation process. Check any existing comments on this issue for additional context. Draft documentation covering the steps listed in the issue body, ensuring clarity for external contributors.
Add documentation on simulation testing at scale · apple/foundationdb#4663 | Good First Issue