trailofbits/polytracker
在 GitHub 查看Polytracker CI should run all possible commands for regression purposes
Open
#6,532 创建于 2023年1月3日
CI/CDgood first issuemaintenanceusability
仓库指标
- Star
- (592 star)
- PR 合并指标
- (PR 指标待抓取)
描述
We currently run unit tests as well as a couple of the example Dockerfiles in CI, but this existing coverage does not seem to cover all Polytracker commands. Some features such as grammar generation and now info (see #6530, https://github.com/trailofbits/polytracker/tree/fix_polytracker_info) have bitrotted over time.
While I'm not sure if functional testing comes from "agile" full stack development, that's where I'm familiar with it from!
If the existing test suite and example Dockerfiles running in CI are not sufficient to ensure all the Polytracker features continue to work (preventing regressions), some ideas to broaden test coverage:
- use a "Does It All" Dockerfile which runs all the commands, proving each works on each CI run, or maybe:
- add functional-esque tests to the Polytracker test suite (we currently primarily have unit tests, which to me are more about proving a single unit or function works correctly than proving WYSIWYG from the customer POV / feature level) to prove each command works on each CI run