Python builder can give a hint to build on linux system, whenever a dependency fails because of its C (native) dependency

aws_lambda_builders/workflows/python_pip/actions.py と packager.py にある PythonPipBuilder の ResolveDependencies フローから始め、MissingDependencyError も含めて確認し、その後 workflow.py がどのように失敗を表面化するかを追跡します。C に依存する失敗に、--use-container を含む Linux ビルドに関する明確なヒントが示され、無関係な依存関係エラーが変更されなければ完了です。

Steps to reproduce the issue:
Whenever sam build command fails for certain dependencies which requires to be build on linux environment (because of C dependencies of that package), output logs can contain a hint for user to build their functions on a linux environment (probably using --use-container flag)

Steps to reproduce the issue:

1. Init a project (ex: HelloWorld) which uses python3.8
2. Update requirements.txt file inside the function, and add aws-xray-sdk as a dependency
3. Try to build function on Mac or Windows by running sam build

Observed result:
Build fails with a dependency error;

C:\agent2_work\142\s\Assets\events\Post>sam build --template "C:\agent2_work\142\s\Assets\events\Post/Assets_Events_POST-build.yaml" --region us-west-2 --debug
Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
'build' command is called
No Parameters detected in the template
2 resources found in the template
Found Serverless function with name='AssetsEventsPOST' and CodeUri='.'
No Parameters detected in the template
Building function 'AssetsEventsPOST'
Loading workflow module 'aws_lambda_builders.workflows'
Registering workflow 'PythonPipBuilder' with capability 'Capability(language='python', dependency_manager='pip', application_framework=None)'
Registering workflow 'NodejsNpmBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm', application_framework=None)'
Registering workflow 'RubyBundlerBuilder' with capability 'Capability(language='ruby', dependency_manager='bundler', application_framework=None)'
Registering workflow 'GoDepBuilder' with capability 'Capability(language='go', dependency_manager='dep', application_framework=None)'
Registering workflow 'GoModulesBuilder' with capability 'Capability(language='go', dependency_manager='modules', application_framework=None)'
Registering workflow 'JavaGradleWorkflow' with capability 'Capability(language='java', dependency_manager='gradle', application_framework=None)'
Registering workflow 'JavaMavenWorkflow' with capability 'Capability(language='java', dependency_manager='maven', application_framework=None)'
Registering workflow 'DotnetCliPackageBuilder' with capability 'Capability(language='dotnet', dependency_manager='cli-package', application_framework=None)'
Registering workflow 'CustomMakeBuilder' with capability 'Capability(language='provided', dependency_manager=None, application_framework=None)'
Found workflow 'PythonPipBuilder' to support capabilities 'Capability(language='python', dependency_manager='pip', application_framework=None)'
Running workflow 'PythonPipBuilder'
Running PythonPipBuilder:ResolveDependencies
calling pip download -r C:\agent2_work\142\s\Assets\events\Post\requirements.txt --dest C:\Users\dummyUser\AppData\Local\Temp\tmp85csqu6i
Full dependency closure: {python-dateutil==2.8.1(wheel), jsonpickle==1.4.1(wheel), six==1.15.0(wheel), zipp==3.3.0(wheel), wrapt==1.12.1(sdist), botocore==1.18.15(wheel), aws-xray-sdk==2.6.0(wheel), jmespath==0.10.0(wheel), urllib3==1.25.10(wheel), future==0.18.2(sdist), importlib-metadata==2.0.0(wheel)}
initial compatible: {python-dateutil==2.8.1(wheel), jsonpickle==1.4.1(wheel), six==1.15.0(wheel), zipp==3.3.0(wheel), botocore==1.18.15(wheel), aws-xray-sdk==2.6.0(wheel), jmespath==0.10.0(wheel), urllib3==1.25.10(wheel), importlib-metadata==2.0.0(wheel)}
initial incompatible: {future==0.18.2(sdist), wrapt==1.12.1(sdist)}
Downloading missing wheels: {future==0.18.2(sdist), wrapt==1.12.1(sdist)}
calling pip download --only-binary=:all: --no-deps --platform manylinux1_x86_64 --implementation cp --abi cp37m --dest C:\Users\dummyUser\AppData\Local\Temp\tmp85csqu6i future==0.18.2
calling pip download --only-binary=:all: --no-deps --platform manylinux1_x86_64 --implementation cp --abi cp37m --dest C:\Users\dummyUser\AppData\Local\Temp\tmp85csqu6i wrapt==1.12.1
compatible wheels after second download pass: {python-dateutil==2.8.1(wheel), jsonpickle==1.4.1(wheel), six==1.15.0(wheel), zipp==3.3.0(wheel), botocore==1.18.15(wheel), aws-xray-sdk==2.6.0(wheel), jmespath==0.10.0(wheel), urllib3==1.25.10(wheel), importlib-metadata==2.0.0(wheel)}
Build missing wheels from sdists (C compiling True): {future==0.18.2(sdist), wrapt==1.12.1(sdist)}
calling pip wheel --no-deps --wheel-dir C:\Users\dummyUser\AppData\Local\Temp\tmp85csqu6i C:\Users\dummyUser\AppData\Local\Temp\tmp85csqu6i\future-0.18.2.tar.gz
calling pip wheel --no-deps --wheel-dir C:\Users\dummyUser\AppData\Local\Temp\tmp85csqu6i C:\Users\dummyUser\AppData\Local\Temp\tmp85csqu6i\wrapt-1.12.1.tar.gz
compatible after building wheels (no C compiling): {python-dateutil==2.8.1(wheel), jsonpickle==1.4.1(wheel), six==1.15.0(wheel), zipp==3.3.0(wheel), botocore==1.18.15(wheel), aws-xray-sdk==2.6.0(wheel), jmespath==0.10.0(wheel), urllib3==1.25.10(wheel), future==0.18.2(wheel), importlib-metadata==2.0.0(wheel)}
Build missing wheels from sdists (C compiling False): {wrapt==1.12.1(sdist)}
calling pip wheel --no-deps --wheel-dir C:\Users\dummyUser\AppData\Local\Temp\tmp85csqu6i C:\Users\dummyUser\AppData\Local\Temp\tmp85csqu6i\wrapt-1.12.1.tar.gz
compatible after building wheels (C compiling): {python-dateutil==2.8.1(wheel), jsonpickle==1.4.1(wheel), six==1.15.0(wheel), zipp==3.3.0(wheel), botocore==1.18.15(wheel), aws-xray-sdk==2.6.0(wheel), jmespath==0.10.0(wheel), urllib3==1.25.10(wheel), future==0.18.2(wheel), importlib-metadata==2.0.0(wheel)}
Final compatible: {python-dateutil==2.8.1(wheel), jsonpickle==1.4.1(wheel), six==1.15.0(wheel), zipp==3.3.0(wheel), botocore==1.18.15(wheel), aws-xray-sdk==2.6.0(wheel), jmespath==0.10.0(wheel), urllib3==1.25.10(wheel), future==0.18.2(wheel), importlib-metadata==2.0.0(wheel)}
Final incompatible: {wrapt==1.12.1(wheel)}
Final missing wheels: {wrapt==1.12.1(sdist)}
PythonPipBuilder:ResolveDependencies failed
Traceback (most recent call last):
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\aws_lambda_builders\workflows\python_pip\actions.py", line 42, in execute
requirements_path=self.manifest_path,
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\aws_lambda_builders\workflows\python_pip\packager.py", line 137, in build_dependencies
self._dependency_builder.build_site_packages(requirements_path, artifacts_dir_path, scratch_dir_path)
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\aws_lambda_builders\workflows\python_pip\packager.py", line 201, in build_site_packages
raise MissingDependencyError(packages_without_wheels)
aws_lambda_builders.workflows.python_pip.packager.MissingDependencyError: {wrapt==1.12.1(sdist)}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\aws_lambda_builders\workflow.py", line 269, in run
action.execute()
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\aws_lambda_builders\workflows\python_pip\actions.py", line 45, in execute
raise ActionFailedError(str(ex))
aws_lambda_builders.actions.ActionFailedError: {wrapt==1.12.1(sdist)}

Build Failed
Sending Telemetry: {'metrics': [{'commandRun': {'awsProfileProvided': False, 'debugFlagProvided': True, 'region': 'us-west-2', 'commandName': 'sam build', 'duration': 20257, 'exitReason': 'WorkflowFailedError', 'exitCode': 1, 'requestId': 'c5551ae9-caea-4b93-9c48-d07fa2cbc8e3', 'installationId': 'b7024037-b875-42d8-8e62-2abac9896f08', 'sessionId': 'd38fda56-1b47-4cf4-8840-65a6966432b4', 'executionEnvironment': 'CLI', 'pyversion': '3.7.6', 'samcliVersion': '0.53.0'}}]}
Telemetry response: 200
Error: PythonPipBuilder:ResolveDependencies - {wrapt==1.12.1(sdist)}

Expected result:
If the build failure is because of C dependency, it may contain more information for user which will suggest them to run build on a linux environment, probably using --use-container

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)
This only happens on non linux environments (Mac and Windows).

Related issues

• https://github.com/aws/aws-sam-cli/issues/2291
• https://github.com/aws/aws-lambda-builders/issues/201