trufflesecurity/trufflehog

Fix broken & inaccurate detector unit tests

开放

#3,817 创建于 2024年12月26日

 (7 条评论) (0 个反应) (1 位负责人)Go (2,397 个派生)batch import
bugcontributions welcomedgood first issuepkg/detectors

仓库指标

星标
 (26,285 个星标)
PR 合并指标
 (平均合并 28天 2小时) (30 天内合并 41 个 PR)

描述

The detector unit tests were created by "reverse engineering" the patterns, thus they do not provide any real value or confirm how accurate the detectors are.

Additionally, the structure of the tests are (in my opinion) not maintainable as they are difficult to understand and do not reflect any realistic scenarios.

https://github.com/trufflesecurity/trufflehog/blob/def734a783b2d4542f2b353d326130d618f3c421/pkg/detectors/netsuite/netsuite_test.go#L14-L44

Examples

There are dozens, if not hundreds, of problematic test files. These are illustrative.

BombBomb

The "valid" tests for BombBomb do not match the detector's pattern.

https://github.com/trufflesecurity/trufflehog/blob/def734a783b2d4542f2b353d326130d618f3c421/pkg/detectors/bombbomb/bombbomb.go#L24

https://github.com/trufflesecurity/trufflehog/blob/def734a783b2d4542f2b353d326130d618f3c421/pkg/detectors/bombbomb/bombbomb_test.go#L15

Kraken

The "valid" pattern is nonsensical and not correct base64 encoding. The detector should not match this, that is a defect.

https://github.com/trufflesecurity/trufflehog/blob/def734a783b2d4542f2b353d326130d618f3c421/pkg/detectors/kraken/kraken_test.go#L16

image https://support.kraken.com/hc/en-us/articles/360000919966-How-to-create-an-API-key

viewneo

A few hundred detectors contain tests tightly coupled to the current implementation of PrefixRegex. Any changes to the prefix pattern will break the detector tests, which seems inadvisable.

https://github.com/trufflesecurity/trufflehog/blob/def734a783b2d4542f2b353d326130d618f3c421/pkg/detectors/viewneo/viewneo_test.go#L38-L42

贡献者指南