Build tracing filtering: path-based inclusion & content-based exclusion
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 38/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Quiet
- Domain
- build-system, tooling
Research direction
Start with the CodeQL CLI database trace-command flow and the existing extractor exclude option, then trace how Java and Kotlin compiler inputs are passed to extraction. The request has two separate goals: path-based inclusion and content-based exclusion. Done means defining and implementing both filtering behaviors, with their command-line configuration and supported matching semantics documented.
Written by the indexing model from the issue text.
Description
I'm using the CodeQL CLI with a large Java/Kotlin project that makes heavy use of autogenerated source during the build process. Because of this, I am using build tracing (codeql database trace-command) to extract CodeQL databases.
Is there any way to filter the files processed by the extractor in the following ways? If not, please consider this a feature request :)
Include-filtering by path
I'd like to be able to extract databases for certain components of the project. Due to the complexity of the project's build system, it's not easy to just build/trace only those components of the project; instead it would be much easier if I could just restrict extraction to those components' files by providing path globs during build tracing.
I'm aware that files can be excluded using the exclude extractor option, but this only allows certain path globs to be excluded, not included.
Exclude-filtering by file contents
As mentioned earlier, the project has a lot of autogenerated code, some (but only some) of which needs to be ignored. This code is often generated with paths that are not easily excludable using the existing path-glob-based exclusions. It would be great if I can specify regexes to search for in the content of files that indicate that the file should not be extracted (e.g. to look for things like "// Autogenerated file, do not edit", etc.)
If there isn't a way to do these, I'm happy to hear about hacky workarounds. For example, the commands I trace are already wrapped in shell scripts that wrap the underlying compilers (javac etc.) in order to provide for more filtering criteria than mentioned here. However, it's hard to do the above properly using such a wrapper because of i.e. the need to properly parse the original compiler command line arguments, handle things like files in JAR files, indirect command line options, etc. The extractor is much better placed to handle such filtering.
- Dominant language
- CodeQL
- Stars
- 10.1k
- Forks
- 2.1k
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 143
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from github/codeql
-
agentic-workflows
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
-
false-positive javascript
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
-
C#: cs/simplifiable-boolean-expression false positive on Nullable<bool> compared with a literal Open
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
false-positive
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
Similar issues
-
Update to NCCL 2.32 Open
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
conda-forge/nccl-feedstock#166 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
dsx-ai-factory/infra-controller#6694 ·
-
bug build
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
facebookincubator/velox#19143 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
pydantic/pydantic-ai#8636 ·
-
0.kind: enhancement 9.needs: package (update)
Difficulty 2/5 1-3 hours Newbie friendliness 74/100