swiftlang/swift-package-manager

[SR-8671] ARG_MAX limit exceeded when compiles huge project

Open

#5,341 创建于 2018年8月31日

在 GitHub 查看
 (9 评论) (0 反应) (0 负责人)Swift (10,145 star) (1,462 fork)batch import
buggood first issue

描述
Previous ID SR-8671
Radar rdar://problem/43940293
Original Reporter Pogonets (JIRA User)
Type Bug

Attachment: Download
Votes 3
Component/s Package Manager
Labels Bug, StarterBug
Assignee None
Priority Medium

md5: 7fd3bd1478f560273a7837164b9d76a9

Issue Description:

macOS have a relative small ARG_MAX limit (256kb)

When you compile huge project with SwiftPM they pass all sources one by one as arguments. Each file is passed as absolute path. And with count of files ~1000 (800 in our case) command line args become too long to even start command.

Error message is meaningless by the way

unable to spawn process (File exists)

This issue even easier to reach when linking multiple swift modules to single dynamic library (all object files from all modules passed to compiler as single line).

This is pretty common issue for build tools and usually solved with response files. Response files supported by clang, ld.gold and many other tools.

贡献者指南

技术栈
swift
领域
build systemperformance
议题类型
bug
难度面向新贡献者的预计实现难度，1 表示很小改动，5 表示专家级工作。
3
预计时间有经验贡献者完成调查、实现、测试并准备 pull request 的粗略时间范围。
half day
活动状态议题当前的可参与程度：新鲜、活跃、陈旧、阻塞或等待维护者输入。
stale
清晰度议题是否清楚说明期望改动、验收标准和下一步。
clear
前置要求
SwiftSwiftPM internalsARG MAX limitclang response files
新手友好度1-100 的估计分数，表示该议题对首次贡献者的友好程度。
70
研究方向
Examine the SwiftPM source code, particularly the `SwiftCompilerTool` or `BuildOperation` files, to understand how command lines are constructed for clang. Implement a check for command line length against ARG MAX, and if exceeded, write a temporary response file containing the input files and pass `@file` to clang. Ensure the solution also applies to the linking step. Test with a large project to verify the fix.