[CAS] gmodule support for caching build #11026
Conversation
cachemeifyoucan
requested review from
akyrtzi,
adrian-prantl,
benlangmuir and
jansvoboda11
cachemeifyoucan
force-pushed
the
casify-module
branch
2 times, most recently
from
2bbc405 to
075be6d
Compare
cachemeifyoucan
force-pushed
the
casify-module
branch
from
075be6d to
2a1ffca
Compare
cachemeifyoucan
force-pushed
the
casify-module
branch
from
2a1ffca to
d5d9a15
Compare
There was a problem hiding this comment.
This would potentially fit better into llvm-project/lldb/packages/Python/lldbsuite/
There was a problem hiding this comment.
This is a compiler launcher type of script and requires a just built clang (because it needs clang-scan-deps next to it). I don't what is a good way to hack into lldbsuite or driven from makefile. Let me know if you have any good idea.
cachemeifyoucan
force-pushed
the
casify-module
branch
from
d5d9a15 to
0173e49
Compare
cachemeifyoucan
force-pushed
the
casify-module
branch
2 times, most recently
from
8063aee to
512a886
Compare
cachemeifyoucan
force-pushed
the
casify-module
branch
from
512a886 to
cb036bf
Compare
|
@adrian-prantl @benlangmuir Ping for review |
| @@ -6150,7 +6168,7 @@ bool ASTReader::readASTFileControlBlock( | |||
| auto Filename = ResolveImportedPath(PathBuf, FilenameStr, ModuleDir); | |||
| StringRef CacheKey = ReadStringBlob(Record, Idx, Blob); | |||
| if (!CacheKey.empty()) | |||
| Listener.readModuleCacheKey(ModuleName, *Filename, CacheKey); | |||
| Listener.readModuleCASID(ModuleName, *Filename, CacheKey); | |||
There was a problem hiding this comment.
How do we know here that this is not a key?
There was a problem hiding this comment.
This code path is never used tor read scanning module. I will rename the variable. Let me think if there is a way to guard it against using it to read scanning module (other than this callback will fail).
There was a problem hiding this comment.
Would it make sense to store a bit in the pcm for whether it's a key or just an id? Even if it's not strictly necessary it might make it less likely we screw it up in the future
cachemeifyoucan
force-pushed
the
casify-module
branch
from
cb036bf to
4ae6ba2
Compare
|
@swift-ci please test llvm |
cachemeifyoucan
force-pushed
the
casify-module
branch
from
4ae6ba2 to
5e96cdf
Compare
|
@swift-ci please test llvm |
cachemeifyoucan
force-pushed
the
casify-module
branch
3 times, most recently
from
88f25d5 to
479c33e
Compare
Teach clang to encode CASID as splitDwarfFilename for gmodule when clang caching is enabled. This allows the outputs from compiler do not contain paths to the clang module files, thus allows distributed caching without the need of a unified clang module cache directory path.
Teach dsymutil how to use CAS and how to load clang modules from CAS when building dSYM when gmodule is used.
Teach lldb to load clang modules when gmodule + clang caching is used.
Teach SwiftASTContext to load clang module dependencies from CAS instead of FileSystem.
cachemeifyoucan
force-pushed
the
casify-module
branch
from
479c33e to
5c6f640
Compare
|
@swift-ci please test llvm |
| } | ||
| llvm::sys::path::append(PCM, Mod.getASTFile()); | ||
|
|
||
| // FIXME: Prefer CASID if exists. |
There was a problem hiding this comment.
I'm not clear what this FIXME means
| HelpText<"Specify a directory that contain dSYM files to search for.">, | ||
| Group<grp_general>; | ||
|
|
||
| def cas : Separate<["-", "--"], "cas">, |
There was a problem hiding this comment.
Should we provide the additional options needed to configure plugin path and plugin options?
Prototype for gmodule support by switching
splitDwarfreferences to CASIDs of the module/PCHAlso teach dsymutil to support reading from CAS as a prototype for lldb support.