Smoke testing core features
維護者通常 2 天內回覆
還沒有人認領這個 Issue。
評估
研究方向
從現有的用戶端輔助介面開始，包括 client.CreateRoom，以及使用它的 Complement 測試。將提議的已登入使用者/權杖、即時更新和房間介面與 matrix-authentication-service、sliding-sync 和 MSC4014 的情境進行比較；完成的標準是定義一套 smoke testing 方法，並就其失敗模式和範圍達成共識。
由索引模型根據 Issue 內容生成。
描述
Matrix has many moving parts. Tests typically only care about one single moving part at a time e.g send a message, create a room, upload OTKs. The setup code that happens to get to the point where we can test that moving part is for the most part, boilerplate. Complement has to date provided helper functions in client to do this sort of thing, failing the test if the boilerplate fails e.g client.CreateRoom. However, this then fixes the test into a certain way to setup the test. This is "okay" in that it is clear what is happening, but it limits the ability to do smoke tests:
smoke testing is preliminary testing or sanity testing to reveal simple failures severe enough to, for example, reject a prospective software release. Smoke tests are a subset of test cases that cover the most important functionality of a component or system, used to aid assessment of whether main functions of the software appear to work correctly.
There are several projects which would benefit from smoke testing:
- matrix-authentication-service - all tests could start by logging into MAS and getting MAS tokens before checking basic functionality (the tests) work.
- sliding-sync - all tests could use sliding sync APIs to consume live data rather than mandating sync v2.
- MSC4014 pseudo IDs - all tests could be made as pseudo ID rooms which would otherwise then function the same as a normal room.
These could "reasonably" be split into 3 interfaces:
- give me a logged in user and access token.
- give me live updates.
- give me a room.
There may be more. Smoke testing in this way is error-prone. Some tests will fail due to this setup in flakey ways, e.g:
- The MAS token expires.
- The sliding sync proxy didn't get the event, or the API doesn't expose the data you want to see in the test.
- The room version behaves in a subtley different way.
However, there is real value in being able to run a sanity check like this, even if it isn't fully reliable (read: likely not in CI).
- 主要語言
- Go
- 星號
- 102
- 分支
- 73
- 平均合併
- 3 天 10 小時
- 30 天內合併 PR
- 6
環境準備
- 沒有 Dockerfile 或 Docker Compose 檔案
- 有 Pull Request 範本
- 閱讀貢獻指南
從這裡開始
- 先讀完整個 Issue，再讀專案的貢獻指南。
- 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
- Fork 儲存庫，在一個分支上完成修改。
- 送出 Pull Request，並在描述裡引用這個 Issue 編號。
matrix-org/complement 的其他 Issue
-
難度 2/5 1-3 小時 新手友好度 65/100
matrix-org/complement#957 ·
維護者通常 2 天內回覆
-
`POST /register downcases capitals in usernames` enforces behaviour that spec defines as optional未關閉
難度 2/5 1-3 小時 新手友好度 78/100
matrix-org/complement#956 ·
維護者通常 2 天內回覆
-
難度 2/5 1-3 小時 新手友好度 72/100
matrix-org/complement#955 · 1 則留言 ·
維護者通常 2 天內回覆
-
難度 2/5 1-3 小時 新手友好度 67/100
matrix-org/complement#954 · 1 則留言 ·
維護者通常 2 天內回覆
-
難度 1/5 1 小時以內 新手友好度 88/100
matrix-org/complement#953 ·
維護者通常 2 天內回覆
查看 matrix-org/complement 的全部 Issue
相似的 Issue
-
難度 2/5 1-3 小時 新手友好度 65/100
shukiv/jabali-panel#2423 ·
維護者通常 1 天內回覆
-
難度 2/5 1-3 小時 新手友好度 78/100
-
area/docs good first issue help wanted kind/docs priority/p2 triage/accepted
難度 2/5 1-3 小時 新手友好度 88/100
維護者通常 1 天內回覆
-
難度 2/5 1-3 小時 新手友好度 78/100
mattabate/life-distributable#158 ·
維護者通常 1 天內回覆
-
難度 2/5 1-3 小時 新手友好度 78/100
router-for-me/CLIProxyAPI#7330 ·
維護者通常 1 天內回覆