pySCG: Add a testing framework for Python code examples and documentation

The issue points to docs/Secure-Coding-Guide-for-Python/tests/, pyproject.toml, tox.ini, and .github/workflows/python-tests.yml; begin with tests/README.md and the existing test results in KNOWN_ISSUES.md. The described framework spans code examples, Markdown checks, link validation, and CI across multiple Python versions. Done means those checks and CI behavior match the listed requirements; the issue notes 44 existing problem files for follow-up.

Summary

Add a pytest-based testing framework to validate Python code examples and Markdown documentation structure across the Secure Coding Guide for Python.

[!NOTE]
We moved repo since creating this source code for this solution is on a closed PR in source
https://github.com/ossf/wg-best-practices-os-developers/pull/1011

Motivation

Currently, there is no automated validation of:

• Python code syntax and deprecation warnings in examples
• Markdown documentation structure conformance to the template
• Internal links and code file references
• Consistency across 100+ CWE articles

This leads to potential issues with broken examples, missing documentation sections, and broken links that could confuse contributors and readers.

Solution

Implement a comprehensive testing framework that validates:

1. Python Code Quality

• Syntax validation using ast.parse()
• Deprecation warning detection via subprocess execution
• Expected output validation against documented examples
• Support for expected failure markers (intentional issues)
• Automatic cleanup of test artifacts

2. Markdown Documentation Structure

• Required sections validation (title, introduction, Non-Compliant, Compliant, Automated Detection, Related Guidelines, Bibliography)
• Code file reference validation (compliant01.py, noncompliant01.py exist)
• Table structure validation (Automated Detection, Related Guidelines, Bibliography)
• Section order validation

3. Link Validation

• Internal link validation in README.md files
• Index table link validation
• Integration with lychee for comprehensive link checking (internal only in CI, external available locally)

4. CI/CD Integration

• GitHub Actions workflow for automated testing on PRs
• Smart testing: Only tests changed CWE directories on PRs (fast, ~1 min)
• Full suite: Runs all tests on pushes to main (~3 min)
• Multi-version testing (Python 3.9-3.14)
• Coverage reporting to Codecov
• Fast execution with uv package manager

5. Local Development Support

• Easy setup with uv sync --group test
• Multi-version testing with tox
• Parallel execution support with pytest-xdist
• Clean issue reporting with generate_issue_report.py
• Comprehensive documentation in tests/README.md

Implementation Details

• Uses pytest with parametrized tests for all Python and Markdown files
• Organized test utilities in tests/utils/ for maintainability
• PEP 735 dependency groups for clean dependency management
• Automatic cleanup of test artifacts (database files, temp files, etc.)
• Cross-platform Python-based issue report generator

Benefits

• Ensures all code examples are syntactically valid and run without warnings
• Validates documentation consistency across all CWE articles
• Catches broken links and missing files before merge
• Provides clear, actionable error messages for contributors
• Reduces review burden by automating quality checks
• Supports Python 3.9-3.14 compatibility testing
• Fast CI/CD with smart testing (only changed files on PRs)

Current State

The framework identifies issues in 44 files (documented in KNOWN_ISSUES.md):

• 31 Documentation files with issues:
  • Missing required sections (Bibliography, Automated Detection, etc.)
  • Missing required tables
  • Section order issues
  • Missing code file references
• 13 Python code files with issues:
  • 11 files with execution timeouts (intentional infinite loops/blocking operations)
  • 1 file with DeprecationWarning
  • 1 file with output mismatch

These pre-existing issues can be addressed in follow-up PRs. The testing framework will prevent new issues from being introduced.

Files Added/Modified

New Files:

• .github/workflows/python-tests.yml - CI/CD workflow with smart testing
• docs/Secure-Coding-Guide-for-Python/tests/ - Complete test framework
• docs/Secure-Coding-Guide-for-Python/pyproject.toml - Project configuration
• docs/Secure-Coding-Guide-for-Python/tox.ini - Multi-version testing config
• docs/Secure-Coding-Guide-for-Python/.gitignore - Ignore test artifacts
• docs/Secure-Coding-Guide-for-Python/generate_test_summary.sh - Summary script
• docs/Secure-Coding-Guide-for-Python/KNOWN_ISSUES.md - Reference of existing issues

Modified Files:

• docs/Secure-Coding-Guide-for-Python/CONTRIBUTING.md - Added testing instructions
• Fixed typo: Renamed compliants02.py → compliant02.py

Testing

All 765 tests collected, 531 passed, 80 failures (pre-existing issues), 154 skipped.
Tests run successfully across Python 3.9-3.14 in CI/CD.