Transparent Compiler: check a single file using only its dependency subgraph
I maintainer di solito rispondono entro 3 giorni
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 5/5
- Tempo stimato
- Più di una settimana
- Idoneità per principianti
- 35/100
- Tipo di issue
- Funzionalità
- Chiarezza
- Abbastanza chiara
- Stato di attività
- Attiva
- Stack tecnologico
- fsharp
- Ambito
- compilers
Direzione di ricerca
Start with ComputeDependencyGraphForFile in src/Compiler/Service/TransparentCompiler.fs and the Transparent Compiler tests in tests/FSharp.Compiler.ComponentTests/FSharpChecker/TransparentCompiler.fs; the issue links the relevant code and test cases. First investigate why dependency-subgraph checking and the cache tests were disabled, including the TODO about complete versus minimal checking. Done means an opt-in single-file check uses its dependency subgraph while the full check remains the default, with correctness and cache behavior covered by tests.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Is your feature request related to a problem? Please describe.
When the Transparent Compiler checks a single file, it type checks every file before it. It does not limit itself to the files that file depends on in the graph.
ComputeDependencyGraphForFile builds the graph for all prior files. The line that would trim it to the target file is commented out:
let ComputeDependencyGraphForFile (tcConfig: TcConfig) (priorSnapshot: ProjectSnapshotBase<FSharpParsedFile>) =
let key = priorSnapshot.SourceFiles.Key(DependencyGraphType.File)
//let lastFileIndex = (parsedInputs |> Array.length) - 1
//caches.DependencyGraph.Get(key, computeDependencyGraph parsedInputs (Graph.subGraphFor lastFileIndex))
caches.DependencyGraph.Get(
key,
computeDependencyGraph
tcConfig
(priorSnapshot.SourceFiles |> Seq.map (fun f -> f.ParsedInput))
(removeImplFilesThatHaveSignatures true priorSnapshot)
)
processGraphAsync then runs every node. So the graph only decides what can run in parallel, not what gets checked.
Our use case
Telplin generates signature files from the typed tree. A common request is to generate the signature of a single file: telplin MyProject.fsproj --files Api.fs. For that we only need the FSharpCheckFileResults of Api.fs with correct types. The types of a file depend only on the files it references, so the dependency subgraph would be enough. Today, generating the signature of a file near the end of a large project costs nearly as much as checking the whole project.
Telplin is switching to the Transparent Compiler, partly to help dogfood it. Without the subgraph, a single file gains only from the parallelism.
Measurements
FCS 43.12.401, FSharpChecker.ParseAndCheckFileInProject (options overload), a new checker for each check. The synthetic project has 30 independent files (F01.fs to F30.fs, about 750 declarations each) and a Last.fs that only uses F01:
module Lib.Last
let value = Lib.F01.f1 1
| File checked | BackgroundCompiler | TransparentCompiler |
|---|---|---|
F01.fs (first file) |
732 ms | 720 ms |
F02.fs |
1429 ms | 953 ms |
Last.fs (depends on F01 only) |
20145 ms | 12186 ms |
With the subgraph, Last.fs would cost about as much as F02.fs, roughly 1 s instead of 12 s.
History
As far as I could trace it:
- In the experiment branches of #15179 the subgraph was active. It was replaced by
removeImplFilesThatHaveSignaturesExceptLastOnein 0101/fsharp@0a35243068ff179a6d8445aa1951b4f7840b1ffd (2023-07-24, commit message "stuff"). That commit gives no reason. - 0101/fsharp@0b2897d6582648e7fe3bfae9994a7fbad35b3ccf (2023-11-15, "test todos") disabled
We don't check files that are not depended onand the twoFiles that are not depended on don't invalidate cachetests withTODO: differentiate complete and minimal checking requests. - #17966 re-enabled the first test, but its key assertion is still commented out:
// check "FileSecond.fs" [] // TODO: assert does not hold.. The two cache tests are still skipped.
Questions
- Why was the subgraph commented out? I'm guessing that "complete vs minimal" refers to check results that need the full
TcEnvof every prior file. For example, completion in the target file should list modules from files it does not reference yet. Is that the reason, or was there a correctness problem (for example one found by the Giraffe fuzzing test, which the same July commit re-enabled with a fixed seed)? - Would you accept an opt-in "minimal" check of a single file, where only the target's dependency subgraph is type checked? For example a parameter on
ParseAndCheckFileInProjector on the snapshot API, or a checker option. Tools like Telplin, analyzers and code generators only need the target file's own typed tree.
Describe the solution you'd like
A way to ask the Transparent Compiler to check one file using only its dependency subgraph, with the full check staying the default.
Describe alternatives you've considered
- Building a trimmed
FSharpProjectOptionsourselves that lists only the target's dependencies. That means reimplementing the dependency resolution of graph-based checking, which is internal. - Checking the whole project, which is what we do now.
- Lingua principale
- F#
- Stelle
- 4.3k
- Fork
- 882
- Merge medio
- 4g 13h
- PR unite (30g)
- 138
Preparare l'ambiente
Avvia il container di sviluppo del progetto nel browser, con il tuo account GitHub.
- Nessun Dockerfile né file Docker Compose
- Ha un modello di pull request
- Leggi la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di dotnet/fsharp
-
Needs-Triage
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 95/100
I maintainer di solito rispondono entro 3 giorni
-
Semantic classification cache for opened documents is never populated (written to the unopened-documents cache)Forse già presa @xperiandri l’ha presa 50 giorni fa. ApertaNeeds-Triage
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 90/100
I maintainer di solito rispondono entro 3 giorni
-
Needs-Triage
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
dotnet/fsharp#20265 · 1 commento ·
I maintainer di solito rispondono entro 3 giorni
-
Bug Needs-Triage
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
I maintainer di solito rispondono entro 3 giorni
-
agentic-workflows
Difficoltà 4/5 3-5 giorni Idoneità per principianti 38/100
I maintainer di solito rispondono entro 3 giorni
Tutte le issue di dotnet/fsharp
Issue simili
-
Slang input: int8_t4_packed and uint8_t4_packed are copied into CrossGLForse già presa @ninglinLiu l’ha presa oggi. Apertaarea:frontend backend:slang bug direction:input
Difficoltà 2/5 1-3 ore Idoneità per principianti 70/100
I maintainer di solito rispondono entro 1 giorno
-
Decompiler: convertCharRef() in xml.y overflows its int4 accumulator on long character referencesForse già presa @kaayzouee l’ha presa oggi. Aperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 76/100
NationalSecurityAgency/ghidra#10009 ·
I maintainer di solito rispondono entro 2 giorni
-
[Bug] tree-sitter-bash parses `$_` / `${_}` as variable_name instead of special_variable_nameAperta
Difficoltà 2/5 Meno di un'ora Idoneità per principianti 85/100
MoonshotAI/kimi-code#4367 ·
I maintainer di solito rispondono entro 1 giorno
-
clang:frontend code-cleanup HLSL
Difficoltà 2/5 1-3 ore Idoneità per principianti 74/100
llvm/llvm-project#235100 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 76/100
paritytech/polkavm#442 ·