swiftlang/swift

[SR-1691] Run longer tests first

Open

#44,300 opened on 2016年6月6日

GitHub で見る
 (6 comments) (0 reactions) (0 assignees)Swift (69,989 stars) (10,719 forks)batch import
good first issuepythontest suite

説明
Previous ID SR-1691
Radar None
Original Reporter @gribozavr
Type New Feature
Votes 0
Component/s Project Infrastructure
Labels New Feature, Python, StarterBug
Assignee None
Priority Medium

md5: d2501feeca870b45e5e159e000fd4619

Issue Description:

In the Swift testsuite there many short tests and a few long tests. Sometimes the longer tests happen to be scheduled last, creating a "long tail" and keeping just a few cores busy. It would be better to start the longer tests first, and then run the short tests. This can shave off 1..2 minutes from testing time.

The suggested solution is to add a new feature to lit, that would allow to attach arbitrary labels to tests. lit could embed definition of special label type, or test suite could have extra code to take specific label names and then attach more semantic attributes to test.

// RUN: some-long-test
// LABEL: run_first

We could also migrate the limit_to_features feature to use labels, it would be a much better fit.

コントリビューターガイド

技術スタック
pythonswift
領域
testingperformance
Issue 種別
feature
難度新しい貢献者にとっての実装難度です。1 は非常に小さな変更、5 は専門的な作業です。
3
推定時間経験ある貢献者が調査、実装、テスト、pull request 準備にかけるおおよその時間範囲です。
half day
活動状況Issue が今どれだけ取り組みやすいかを示します。新しい、活発、古い、ブロック中、またはメンテナー入力待ちなどです。
stale
明確さ期待される変更、受け入れ条件、次の手順がどれだけ明確かを示します。
clear
前提条件
Pythonlit test frameworktest infrastructure concepts
初心者向け度初回貢献者にどれだけ取り組みやすいかを 1-100 で推定したスコアです。
55
調査方針
The issue suggests adding a label mechanism to lit to prioritize longer tests. Start by familiarizing with the lit test framework used in the Swift project. Look at the existing test suite organization and scheduling logic. Explore how tests are currently defined and run. Consider implementing a simple prototype that adds a `LABEL` directive to lit test files, allowing tests to be tagged with priority. Check the lit source code for any existing prioritization features. Since the issue is from 2016, verify if any similar changes have been made or if the approach is still relevant.