Unescaped File Path Injected into Python String Constant in test_build_unflags

What kind of issue is this?

• PlatformIO Core. If you’ve found a bug, please provide an information below.

---

Configuration

Operating system: Microsoft Windows 11 Pro 10.0.22000 Build 22000

PlatformIO Version (platformio --version): PlatformIO Core, version 6.1.5a1

• Dell Inc., XPS 15 9500, x64
• Git 2.37.1
• Python 3.7.9
• tox 3.25.1
• PowerShell 7.2.5

Description of problem

In the unit test test_build_unflags:

https://github.com/platformio/platformio-core/blob/8814f4e92d37822abbcbdc973fb2f9f2ac40b7a4/tests/commands/test_run.py#L123

A python script is generated by using the % Python operator to format a string:

https://github.com/platformio/platformio-core/blob/8814f4e92d37822abbcbdc973fb2f9f2ac40b7a4/tests/commands/test_run.py#L143-L153

The argument that is injected is a string representation of an absolute file path:

https://github.com/platformio/platformio-core/blob/8814f4e92d37822abbcbdc973fb2f9f2ac40b7a4/tests/commands/test_run.py#L153

The path is injected between double quotes to act as a string constant in the resulting Python file:

https://github.com/platformio/platformio-core/blob/8814f4e92d37822abbcbdc973fb2f9f2ac40b7a4/tests/commands/test_run.py#L145

On Windows, the file path separator is the backslash, \.

This can create a SyntaxError: (unicode error) in the generated Python file if the backslash separator precedes a character that creates an escape sequence, such as \t.

Steps to Reproduce

1. git clone https://github.com/platformio/platformio-core.git
2. cd platformio-core
3. git submodule init
4. git submodule update
5. tox -e py37
6. .\.tox\py37\Scripts\activate.ps1
7. py.test -n 1 tests --ignore tests/test_examples.py -k test_build_unflags

Actual Results

$ (py37) > py.test -n 1 tests --ignore tests/test_examples.py -k test_build_unflags
============================================================================================= test session starts ==============================================================================================
platform win32 -- Python 3.7.9, pytest-7.1.2, pluggy-1.0.0
rootdir: C:\*********\platformio-core, configfile: tox.ini
plugins: anyio-3.6.1, forked-1.4.0, xdist-2.5.0
gw0 [1]
F                                                                                                                                                                                                         [100%]
=================================================================================================== FAILURES ===================================================================================================
______________________________________________________________________________________________ test_build_unflags ______________________________________________________________________________________________
[gw0] win32 -- Python 3.7.9 C:\*********\platformio-core\.tox\py37\Scripts\python.EXE

clirunner = <click.testing.CliRunner object at 0x00000204EF958C08>, validate_cliresult = <function validate_cliresult.<locals>.decorator at 0x00000204EF9559D8>
tmpdir = local('C:\\Users\\*********\\AppData\\Local\\Temp\\pytest-of-*********\\pytest-17\\popen-gw0\\test_build_unflags0')

    def test_build_unflags(clirunner, validate_cliresult, tmpdir):
        tmpdir.join("platformio.ini").write(
            """
    [env:native]
    platform = native
    build_unflags =
        -DTMP_MACRO_1=45
        -DTMP_MACRO_3=13
        -DTMP_MACRO_4
        -DNON_EXISTING_MACRO
        -I.
        -lunknownLib
        -Os
    build_flags =
        -DTMP_MACRO_3=10
    extra_scripts = pre:extra.py
    """
        )

        tmpdir.join("extra.py").write(
            """
    Import("env")
    env.Append(CPPPATH="%s")
    env.Append(CPPDEFINES="TMP_MACRO_1")
    env.Append(CPPDEFINES=["TMP_MACRO_2"])
    env.Append(CPPDEFINES=[("TMP_MACRO_3", 13)])
    env.Append(CPPDEFINES=[("TMP_MACRO_4", 4)])
    env.Append(CCFLAGS=["-Os"])
    env.Append(LIBS=["unknownLib"])
        """
            % str(tmpdir)
        )

        tmpdir.mkdir("src").join("main.c").write(
            """
    #ifndef TMP_MACRO_1
    #error "TMP_MACRO_1 should be defined"
    #endif

    #ifndef TMP_MACRO_2
    #error "TMP_MACRO_2 should be defined"
    #endif

    #if TMP_MACRO_3 != 10
    #error "TMP_MACRO_3 should be 10"
    #endif

    #ifdef TMP_MACRO_4
    #error "TMP_MACRO_4 should not be defined"
    #endif

    int main() {
    }
    """
        )

        result = clirunner.invoke(cmd_run, ["--project-dir", str(tmpdir), "--verbose"])
>       validate_cliresult(result)

tests\commands\test_run.py:180:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

result = <Result ReturnErrorCode(1)>

    def decorator(result):
>       assert result.exit_code == 0, "{} => {}".format(result.exception, result.output)
E       AssertionError: 1 => Processing native (platform: native; build_unflags: -DTMP_MACRO_1=45, -DTMP_MACRO_3=13, -DTMP_MACRO_4, -DNON_EXISTING_MACRO, -I., -lunknownLib, -Os; build_flags: -DTMP_MACRO_3=10; extra_scripts: pre:extra.py)
E         --------------------------------------------------------------------------------
E           File "C:\Users\*********\AppData\Local\Temp\pytest-of-*********\pytest-17\popen-gw0\test_build_unflags0\extra.py", line 3
E
E             env.Append(CPPPATH="C:\Users\*********\AppData\Local\Temp\pytest-of-*********\pytest-17\popen-gw0\test_build_unflags0")
E
E                               ^
E
E         SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape
E
E         ========================== [FAILED] Took 0.51 seconds ==========================
E
E       assert 1 == 0
E        +  where 1 = <Result ReturnErrorCode(1)>.exit_code

tests\conftest.py:36: AssertionError
=========================================================================================== short test summary info ============================================================================================
FAILED tests/commands/test_run.py::test_build_unflags - AssertionError: 1 => Processing native (platform: native; build_unflags: -DTMP_MACRO_1=45, -DTMP_MACRO_3=13, -DTMP_MACRO_4, -DNON_EXISTING_MACRO, -I.,...
============================================================================================== 1 failed in 1.27s ===============================================================================================

(with ********* replacing some personal info.)

Expected Results

(py37) > py.test -n 1 tests --ignore tests/test_examples.py -k test_build_unflags
============================================================================================= test session starts ==============================================================================================
platform win32 -- Python 3.7.9, pytest-7.1.2, pluggy-1.0.0
rootdir: C:\*********\platformio-core, configfile: tox.ini
plugins: anyio-3.6.1, forked-1.4.0, xdist-2.5.0
gw0 [1]
.                                                                                                                                                                                                         [100%]
============================================================================================== 1 passed in 1.47s ===============================================================================================

(with ********* replacing some personal info.)

Additional info

The fix:

https://github.com/COTSWORKS-MackeyK/platformio-core/commit/49ede3e57c99ad37b9614f7f96dcc3191b926bdb

makes the error go away.