swiftlang/swift-package-manager

[SR-8671] ARG_MAX limit exceeded when compiles huge project

Open

#5,341 建立於 2018年8月31日

在 GitHub 查看
 (9 留言) (0 反應) (0 負責人)Swift (10,145 star) (1,462 fork)batch import
buggood first issue

描述
Previous ID SR-8671
Radar rdar://problem/43940293
Original Reporter Pogonets (JIRA User)
Type Bug

Attachment: Download
Votes 3
Component/s Package Manager
Labels Bug, StarterBug
Assignee None
Priority Medium

md5: 7fd3bd1478f560273a7837164b9d76a9

Issue Description:

macOS have a relative small ARG_MAX limit (256kb)

When you compile huge project with SwiftPM they pass all sources one by one as arguments. Each file is passed as absolute path. And with count of files ~1000 (800 in our case) command line args become too long to even start command.

Error message is meaningless by the way

unable to spawn process (File exists)

This issue even easier to reach when linking multiple swift modules to single dynamic library (all object files from all modules passed to compiler as single line).

This is pretty common issue for build tools and usually solved with response files. Response files supported by clang, ld.gold and many other tools.

貢獻者指南

技術棧
swift
領域
build systemperformance
議題類型
bug
難度面向新貢獻者的預計實作難度，1 表示很小改動，5 表示專家級工作。
3
預計時間有經驗貢獻者完成調查、實作、測試並準備 pull request 的粗略時間範圍。
half day
活動狀態議題目前的可參與程度：新鮮、活躍、陳舊、阻塞或等待維護者輸入。
stale
清晰度議題是否清楚說明預期改動、驗收標準和下一步。
clear
前置要求
SwiftSwiftPM internalsARG MAX limitclang response files
新手友善度1-100 的估計分數，表示該議題對首次貢獻者的友善程度。
70
研究方向
Examine the SwiftPM source code, particularly the `SwiftCompilerTool` or `BuildOperation` files, to understand how command lines are constructed for clang. Implement a check for command line length against ARG MAX, and if exceeded, write a temporary response file containing the input files and pass `@file` to clang. Ensure the solution also applies to the linking step. Test with a large project to verify the fix.