-
Notifications
You must be signed in to change notification settings - Fork 15.2k
[tools] LLVM Compiler Visualization Tool for Offloading - GSoC 2025 #147451
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
✅ With the latest revision this PR passed the Python code formatter. |
0fba87c
to
a4ab9b9
Compare
@llvm/pr-subscribers-offload Author: Miguel Cárdenas (miguelcsx) ChangesSummaryThis PR introduces LLVM Advisor, a new tool that analyzes compilation commands and collects generated artifacts (Remarks, IR, AST dumps, etc.) for optimization analysis. What this adds: Core Infrastructure:
Analysis Engine:
Key Features:
> This tool addresses the need for compilation analysis and artifact collection, providing a foundation for optimization guidance in modern development workflows. Future PRs will add web-based visualization and advanced analysis capabilities. Patch is 71.61 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/147451.diff 25 Files Affected:
diff --git a/llvm/tools/llvm-advisor/CMakeLists.txt b/llvm/tools/llvm-advisor/CMakeLists.txt
new file mode 100644
index 0000000000000..d2389bdd1e0fa
--- /dev/null
+++ b/llvm/tools/llvm-advisor/CMakeLists.txt
@@ -0,0 +1,15 @@
+cmake_minimum_required(VERSION 3.18)
+
+set(LLVM_TOOL_LLVM_ADVISOR_BUILD_DEFAULT ON)
+set(LLVM_REQUIRE_EXE_NAMES llvm-advisor)
+
+add_subdirectory(src)
+
+# Set the executable name
+set_target_properties(llvm-advisor PROPERTIES
+ OUTPUT_NAME llvm-advisor)
+
+# Install the binary
+install(TARGETS llvm-advisor
+ RUNTIME DESTINATION bin
+ COMPONENT llvm-advisor)
diff --git a/llvm/tools/llvm-advisor/config/config.json b/llvm/tools/llvm-advisor/config/config.json
new file mode 100644
index 0000000000000..9e94a41ff46c4
--- /dev/null
+++ b/llvm/tools/llvm-advisor/config/config.json
@@ -0,0 +1,7 @@
+{
+ "outputDir": ".llvm-advisor",
+ "verbose": false,
+ "keepTemps": false,
+ "runProfiler": true,
+ "timeout": 60
+}
diff --git a/llvm/tools/llvm-advisor/src/CMakeLists.txt b/llvm/tools/llvm-advisor/src/CMakeLists.txt
new file mode 100644
index 0000000000000..81088f8231625
--- /dev/null
+++ b/llvm/tools/llvm-advisor/src/CMakeLists.txt
@@ -0,0 +1,35 @@
+# Gather all .cpp sources in this directory tree
+file(GLOB_RECURSE LLVM_ADVISOR_SOURCES CONFIGURE_DEPENDS
+ ${CMAKE_CURRENT_SOURCE_DIR}/*.cpp
+)
+
+# Define the executable target
+add_llvm_tool(llvm-advisor
+ ${LLVM_ADVISOR_SOURCES}
+)
+
+# Link required LLVM libraries
+target_link_libraries(llvm-advisor PRIVATE
+ LLVMSupport
+ LLVMCore
+ LLVMIRReader
+ LLVMBitWriter
+ LLVMRemarks
+ LLVMProfileData
+)
+
+# Set include directories
+target_include_directories(llvm-advisor PRIVATE
+ ${CMAKE_CURRENT_SOURCE_DIR}
+)
+
+# Install the Python view module alongside the binary
+install(DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/../view/
+ DESTINATION ${CMAKE_INSTALL_BINDIR}/view
+ FILES_MATCHING
+ PATTERN "*.py"
+ PATTERN "*.html"
+ PATTERN "*.css"
+ PATTERN "*.js"
+ PATTERN "*.md"
+)
diff --git a/llvm/tools/llvm-advisor/src/Config/AdvisorConfig.cpp b/llvm/tools/llvm-advisor/src/Config/AdvisorConfig.cpp
new file mode 100644
index 0000000000000..69f1e3d52702e
--- /dev/null
+++ b/llvm/tools/llvm-advisor/src/Config/AdvisorConfig.cpp
@@ -0,0 +1,64 @@
+#include "AdvisorConfig.h"
+#include "llvm/Support/FileSystem.h"
+#include "llvm/Support/JSON.h"
+#include "llvm/Support/MemoryBuffer.h"
+#include "llvm/Support/Path.h"
+
+namespace llvm {
+namespace advisor {
+
+AdvisorConfig::AdvisorConfig() {
+ // Use relative path as default, will be resolved by CompilationManager
+ OutputDir_ = ".llvm-advisor";
+}
+
+Expected<bool> AdvisorConfig::loadFromFile(const std::string &path) {
+ auto BufferOrError = MemoryBuffer::getFile(path);
+ if (!BufferOrError) {
+ return createStringError(BufferOrError.getError(),
+ "Cannot read config file");
+ }
+
+ auto Buffer = std::move(*BufferOrError);
+ Expected<json::Value> JsonOrError = json::parse(Buffer->getBuffer());
+ if (!JsonOrError) {
+ return JsonOrError.takeError();
+ }
+
+ auto &Json = *JsonOrError;
+ auto *Obj = Json.getAsObject();
+ if (!Obj) {
+ return createStringError(std::make_error_code(std::errc::invalid_argument),
+ "Config file must contain JSON object");
+ }
+
+ if (auto outputDirOpt = Obj->getString("outputDir"); outputDirOpt) {
+ OutputDir_ = outputDirOpt->str();
+ }
+
+ if (auto verboseOpt = Obj->getBoolean("verbose"); verboseOpt) {
+ Verbose_ = *verboseOpt;
+ }
+
+ if (auto keepTempsOpt = Obj->getBoolean("keepTemps"); keepTempsOpt) {
+ KeepTemps_ = *keepTempsOpt;
+ }
+
+ if (auto runProfileOpt = Obj->getBoolean("runProfiler"); runProfileOpt) {
+ RunProfiler_ = *runProfileOpt;
+ }
+
+ if (auto timeoutOpt = Obj->getInteger("timeout"); timeoutOpt) {
+ TimeoutSeconds_ = static_cast<int>(*timeoutOpt);
+ }
+
+ return true;
+}
+
+std::string AdvisorConfig::getToolPath(const std::string &tool) const {
+ // For now, just return the tool name and rely on PATH
+ return tool;
+}
+
+} // namespace advisor
+} // namespace llvm
diff --git a/llvm/tools/llvm-advisor/src/Config/AdvisorConfig.h b/llvm/tools/llvm-advisor/src/Config/AdvisorConfig.h
new file mode 100644
index 0000000000000..b7f553fddbb23
--- /dev/null
+++ b/llvm/tools/llvm-advisor/src/Config/AdvisorConfig.h
@@ -0,0 +1,41 @@
+#ifndef LLVM_ADVISOR_CONFIG_H
+#define LLVM_ADVISOR_CONFIG_H
+
+#include "llvm/Support/Error.h"
+#include <string>
+
+namespace llvm {
+namespace advisor {
+
+class AdvisorConfig {
+public:
+ AdvisorConfig();
+
+ Expected<bool> loadFromFile(const std::string &path);
+
+ void setOutputDir(const std::string &dir) { OutputDir_ = dir; }
+ void setVerbose(bool verbose) { Verbose_ = verbose; }
+ void setKeepTemps(bool keep) { KeepTemps_ = keep; }
+ void setRunProfiler(bool run) { RunProfiler_ = run; }
+ void setTimeout(int seconds) { TimeoutSeconds_ = seconds; }
+
+ const std::string &getOutputDir() const { return OutputDir_; }
+ bool getVerbose() const { return Verbose_; }
+ bool getKeepTemps() const { return KeepTemps_; }
+ bool getRunProfiler() const { return RunProfiler_; }
+ int getTimeout() const { return TimeoutSeconds_; }
+
+ std::string getToolPath(const std::string &tool) const;
+
+private:
+ std::string OutputDir_;
+ bool Verbose_ = false;
+ bool KeepTemps_ = false;
+ bool RunProfiler_ = true;
+ int TimeoutSeconds_ = 60;
+};
+
+} // namespace advisor
+} // namespace llvm
+
+#endif
diff --git a/llvm/tools/llvm-advisor/src/Core/BuildContext.h b/llvm/tools/llvm-advisor/src/Core/BuildContext.h
new file mode 100644
index 0000000000000..4f40c37ca8706
--- /dev/null
+++ b/llvm/tools/llvm-advisor/src/Core/BuildContext.h
@@ -0,0 +1,52 @@
+#ifndef LLVM_ADVISOR_BUILD_CONTEXT_H
+#define LLVM_ADVISOR_BUILD_CONTEXT_H
+
+#include <map>
+#include <string>
+#include <vector>
+
+namespace llvm {
+namespace advisor {
+
+enum class BuildPhase {
+ Unknown,
+ Preprocessing,
+ Compilation,
+ Assembly,
+ Linking,
+ Archiving,
+ CMakeConfigure,
+ CMakeBuild,
+ MakefileBuild
+};
+
+enum class BuildTool {
+ Unknown,
+ Clang,
+ GCC,
+ LLVM_Tools,
+ CMake,
+ Make,
+ Ninja,
+ Linker,
+ Archiver
+};
+
+struct BuildContext {
+ BuildPhase phase;
+ BuildTool tool;
+ std::string workingDirectory;
+ std::string outputDirectory;
+ std::vector<std::string> inputFiles;
+ std::vector<std::string> outputFiles;
+ std::vector<std::string> expectedGeneratedFiles;
+ std::map<std::string, std::string> metadata;
+ bool hasOffloading = false;
+ bool hasDebugInfo = false;
+ bool hasOptimization = false;
+};
+
+} // namespace advisor
+} // namespace llvm
+
+#endif
diff --git a/llvm/tools/llvm-advisor/src/Core/BuildExecutor.cpp b/llvm/tools/llvm-advisor/src/Core/BuildExecutor.cpp
new file mode 100644
index 0000000000000..a4af5a660c80e
--- /dev/null
+++ b/llvm/tools/llvm-advisor/src/Core/BuildExecutor.cpp
@@ -0,0 +1,109 @@
+#include "BuildExecutor.h"
+#include "llvm/Support/FileSystem.h"
+#include "llvm/Support/Path.h"
+#include "llvm/Support/Program.h"
+#include "llvm/Support/raw_ostream.h"
+
+namespace llvm {
+namespace advisor {
+
+BuildExecutor::BuildExecutor(const AdvisorConfig &config) : config_(config) {}
+
+Expected<int> BuildExecutor::execute(const std::string &compiler,
+ const std::vector<std::string> &args,
+ BuildContext &buildContext,
+ const std::string &tempDir) {
+ auto instrumentedArgs = instrumentCompilerArgs(args, buildContext, tempDir);
+
+ auto compilerPath = sys::findProgramByName(compiler);
+ if (!compilerPath) {
+ return createStringError(
+ std::make_error_code(std::errc::no_such_file_or_directory),
+ "Compiler not found: " + compiler);
+ }
+
+ std::vector<StringRef> execArgs;
+ execArgs.push_back(compiler);
+ for (const auto &arg : instrumentedArgs) {
+ execArgs.push_back(arg);
+ }
+
+ if (config_.getVerbose()) {
+ outs() << "Executing: " << compiler;
+ for (const auto &arg : instrumentedArgs) {
+ outs() << " " << arg;
+ }
+ outs() << "\n";
+ }
+
+ return sys::ExecuteAndWait(*compilerPath, execArgs);
+}
+
+std::vector<std::string>
+BuildExecutor::instrumentCompilerArgs(const std::vector<std::string> &args,
+ BuildContext &buildContext,
+ const std::string &tempDir) {
+
+ std::vector<std::string> result = args;
+ std::set<std::string> existingFlags;
+
+ // Scan existing flags to avoid duplication
+ for (const auto &arg : args) {
+ if (arg.find("-g") == 0)
+ existingFlags.insert("debug");
+ if (arg.find("-fsave-optimization-record") != std::string::npos)
+ existingFlags.insert("remarks");
+ if (arg.find("-fprofile-instr-generate") != std::string::npos)
+ existingFlags.insert("profile");
+ }
+
+ // Add debug info if not present
+ if (existingFlags.find("debug") == existingFlags.end()) {
+ result.push_back("-g");
+ }
+
+ // Add optimization remarks with proper redirection
+ if (existingFlags.find("remarks") == existingFlags.end()) {
+ result.push_back("-fsave-optimization-record");
+ result.push_back("-foptimization-record-file=" + tempDir +
+ "/remarks.opt.yaml");
+ buildContext.expectedGeneratedFiles.push_back(tempDir +
+ "/remarks.opt.yaml");
+ } else {
+ // If user already specified remarks, find and redirect the file
+ bool foundFileFlag = false;
+ for (auto &arg : result) {
+ if (arg.find("-foptimization-record-file=") != std::string::npos) {
+ // Extract filename and redirect to temp
+ StringRef existingPath = StringRef(arg).substr(26);
+ StringRef filename = sys::path::filename(existingPath);
+ arg = "-foptimization-record-file=" + tempDir + "/" + filename.str();
+ buildContext.expectedGeneratedFiles.push_back(tempDir + "/" +
+ filename.str());
+ foundFileFlag = true;
+ break;
+ }
+ }
+ // If no explicit file specified, add our own
+ if (!foundFileFlag) {
+ result.push_back("-foptimization-record-file=" + tempDir +
+ "/remarks.opt.yaml");
+ buildContext.expectedGeneratedFiles.push_back(tempDir +
+ "/remarks.opt.yaml");
+ }
+ }
+
+ // Add profiling if enabled and not present, redirect to temp directory
+ if (config_.getRunProfiler() &&
+ existingFlags.find("profile") == existingFlags.end()) {
+ result.push_back("-fprofile-instr-generate=" + tempDir +
+ "/profile.profraw");
+ result.push_back("-fcoverage-mapping");
+ buildContext.expectedGeneratedFiles.push_back(tempDir + "/profile.profraw");
+ }
+
+ return result;
+}
+
+} // namespace advisor
+} // namespace llvm
diff --git a/llvm/tools/llvm-advisor/src/Core/BuildExecutor.h b/llvm/tools/llvm-advisor/src/Core/BuildExecutor.h
new file mode 100644
index 0000000000000..a77ffd70c9b57
--- /dev/null
+++ b/llvm/tools/llvm-advisor/src/Core/BuildExecutor.h
@@ -0,0 +1,34 @@
+#ifndef LLVM_ADVISOR_BUILD_EXECUTOR_H
+#define LLVM_ADVISOR_BUILD_EXECUTOR_H
+
+#include "../Config/AdvisorConfig.h"
+#include "BuildContext.h"
+#include "llvm/Support/Error.h"
+#include <set>
+#include <string>
+#include <vector>
+
+namespace llvm {
+namespace advisor {
+
+class BuildExecutor {
+public:
+ BuildExecutor(const AdvisorConfig &config);
+
+ Expected<int> execute(const std::string &compiler,
+ const std::vector<std::string> &args,
+ BuildContext &buildContext, const std::string &tempDir);
+
+private:
+ std::vector<std::string>
+ instrumentCompilerArgs(const std::vector<std::string> &args,
+ BuildContext &buildContext,
+ const std::string &tempDir);
+
+ const AdvisorConfig &config_;
+};
+
+} // namespace advisor
+} // namespace llvm
+
+#endif
diff --git a/llvm/tools/llvm-advisor/src/Core/CommandAnalyzer.cpp b/llvm/tools/llvm-advisor/src/Core/CommandAnalyzer.cpp
new file mode 100644
index 0000000000000..3192c42669e65
--- /dev/null
+++ b/llvm/tools/llvm-advisor/src/Core/CommandAnalyzer.cpp
@@ -0,0 +1,167 @@
+#include "CommandAnalyzer.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/ADT/StringSwitch.h"
+#include "llvm/Support/FileSystem.h"
+#include "llvm/Support/Path.h"
+
+namespace llvm {
+namespace advisor {
+
+CommandAnalyzer::CommandAnalyzer(const std::string &command,
+ const std::vector<std::string> &args)
+ : command_(command), args_(args) {}
+
+BuildContext CommandAnalyzer::analyze() const {
+ BuildContext context;
+ SmallString<256> cwd;
+ sys::fs::current_path(cwd);
+ context.workingDirectory = cwd.str().str();
+
+ context.tool = detectBuildTool();
+ context.phase = detectBuildPhase(context.tool);
+ context.inputFiles = extractInputFiles();
+ context.outputFiles = extractOutputFiles();
+ detectBuildFeatures(context);
+
+ return context;
+}
+
+BuildTool CommandAnalyzer::detectBuildTool() const {
+ return StringSwitch<BuildTool>(sys::path::filename(command_))
+ .StartsWith("clang", BuildTool::Clang)
+ .StartsWith("gcc", BuildTool::GCC)
+ .StartsWith("g++", BuildTool::GCC)
+ .Case("cmake", BuildTool::CMake)
+ .Case("make", BuildTool::Make)
+ .Case("ninja", BuildTool::Ninja)
+ .EndsWith("-ld", BuildTool::Linker)
+ .Case("ld", BuildTool::Linker)
+ .Case("ar", BuildTool::Archiver)
+ .Case("llvm-ar", BuildTool::Archiver)
+ .StartsWith("llvm-", BuildTool::LLVM_Tools)
+ .Default(BuildTool::Unknown);
+}
+
+BuildPhase CommandAnalyzer::detectBuildPhase(BuildTool tool) const {
+ if (tool == BuildTool::CMake) {
+ for (const auto &arg : args_) {
+ if (arg == "--build")
+ return BuildPhase::CMakeBuild;
+ }
+ return BuildPhase::CMakeConfigure;
+ }
+
+ if (tool == BuildTool::Make || tool == BuildTool::Ninja) {
+ return BuildPhase::MakefileBuild;
+ }
+
+ if (tool == BuildTool::Linker) {
+ return BuildPhase::Linking;
+ }
+
+ if (tool == BuildTool::Archiver) {
+ return BuildPhase::Archiving;
+ }
+
+ if (tool == BuildTool::Clang || tool == BuildTool::GCC) {
+ for (const auto &arg : args_) {
+ if (arg == "-E")
+ return BuildPhase::Preprocessing;
+ if (arg == "-S")
+ return BuildPhase::Assembly;
+ if (arg == "-c")
+ return BuildPhase::Compilation;
+ }
+
+ bool hasObjectFile = false;
+ for (const auto &Arg : args_) {
+ StringRef argRef(Arg);
+ if (argRef.ends_with(".o") || argRef.ends_with(".O") ||
+ argRef.ends_with(".obj") || argRef.ends_with(".OBJ")) {
+ hasObjectFile = true;
+ break;
+ }
+ }
+ if (hasObjectFile) {
+ return BuildPhase::Linking;
+ }
+
+ bool hasSourceFile = false;
+ for (const auto &Arg : args_) {
+ StringRef argRef(Arg);
+ if (argRef.ends_with(".c") || argRef.ends_with(".C") ||
+ argRef.ends_with(".cpp") || argRef.ends_with(".CPP") ||
+ argRef.ends_with(".cc") || argRef.ends_with(".CC") ||
+ argRef.ends_with(".cxx") || argRef.ends_with(".CXX")) {
+ hasSourceFile = true;
+ break;
+ }
+ }
+ if (hasSourceFile) {
+ return BuildPhase::Compilation; // Default for source files
+ }
+ }
+
+ return BuildPhase::Unknown;
+}
+
+void CommandAnalyzer::detectBuildFeatures(BuildContext &context) const {
+ for (const auto &arg : args_) {
+ if (arg == "-g" || StringRef(arg).starts_with("-g")) {
+ context.hasDebugInfo = true;
+ }
+
+ if (StringRef(arg).starts_with("-O") && arg.length() > 2) {
+ context.hasOptimization = true;
+ }
+
+ if (arg.find("openmp") != std::string::npos ||
+ arg.find("openacc") != std::string::npos ||
+ arg.find("cuda") != std::string::npos ||
+ arg.find("offload") != std::string::npos) {
+ context.hasOffloading = true;
+ }
+
+ if (StringRef(arg).starts_with("-march=")) {
+ context.metadata["target_arch"] = arg.substr(7);
+ }
+ if (StringRef(arg).starts_with("-mtune=")) {
+ context.metadata["tune"] = arg.substr(7);
+ }
+ if (StringRef(arg).starts_with("--offload-arch=")) {
+ context.metadata["offload_arch"] = arg.substr(15);
+ }
+ }
+}
+
+std::vector<std::string> CommandAnalyzer::extractInputFiles() const {
+ std::vector<std::string> inputs;
+ for (size_t i = 0; i < args_.size(); ++i) {
+ const auto &arg = args_[i];
+ if (StringRef(arg).starts_with("-")) {
+ if (arg == "-o" || arg == "-I" || arg == "-L" || arg == "-D") {
+ i++;
+ }
+ continue;
+ }
+ if (sys::fs::exists(arg)) {
+ inputs.push_back(arg);
+ }
+ }
+ return inputs;
+}
+
+std::vector<std::string> CommandAnalyzer::extractOutputFiles() const {
+ std::vector<std::string> outputs;
+ for (size_t i = 0; i < args_.size(); ++i) {
+ const auto &arg = args_[i];
+ if (arg == "-o" && i + 1 < args_.size()) {
+ outputs.push_back(args_[i + 1]);
+ i++;
+ }
+ }
+ return outputs;
+}
+
+} // namespace advisor
+} // namespace llvm
diff --git a/llvm/tools/llvm-advisor/src/Core/CommandAnalyzer.h b/llvm/tools/llvm-advisor/src/Core/CommandAnalyzer.h
new file mode 100644
index 0000000000000..c3efdff147e5f
--- /dev/null
+++ b/llvm/tools/llvm-advisor/src/Core/CommandAnalyzer.h
@@ -0,0 +1,32 @@
+#ifndef LLVM_ADVISOR_COMMAND_ANALYZER_H
+#define LLVM_ADVISOR_COMMAND_ANALYZER_H
+
+#include "BuildContext.h"
+#include <string>
+#include <vector>
+
+namespace llvm {
+namespace advisor {
+
+class CommandAnalyzer {
+public:
+ CommandAnalyzer(const std::string &command,
+ const std::vector<std::string> &args);
+
+ BuildContext analyze() const;
+
+private:
+ BuildTool detectBuildTool() const;
+ BuildPhase detectBuildPhase(BuildTool tool) const;
+ void detectBuildFeatures(BuildContext &context) const;
+ std::vector<std::string> extractInputFiles() const;
+ std::vector<std::string> extractOutputFiles() const;
+
+ std::string command_;
+ std::vector<std::string> args_;
+};
+
+} // namespace advisor
+} // namespace llvm
+
+#endif
diff --git a/llvm/tools/llvm-advisor/src/Core/CompilationManager.cpp b/llvm/tools/llvm-advisor/src/Core/CompilationManager.cpp
new file mode 100644
index 0000000000000..e07db9d365009
--- /dev/null
+++ b/llvm/tools/llvm-advisor/src/Core/CompilationManager.cpp
@@ -0,0 +1,257 @@
+#include "CompilationManager.h"
+#include "../Detection/UnitDetector.h"
+#include "../Utils/FileManager.h"
+#include "CommandAnalyzer.h"
+#include "DataExtractor.h"
+#include "llvm/Support/FileSystem.h"
+#include "llvm/Support/Path.h"
+#include <chrono>
+#include <cstdlib>
+#include <set>
+
+namespace llvm {
+namespace advisor {
+
+CompilationManager::CompilationManager(const AdvisorConfig &config)
+ : config_(config), buildExecutor_(config) {
+
+ // Get current working directory first
+ SmallString<256> currentDir;
+ sys::fs::current_path(currentDir);
+ initialWorkingDir_ = currentDir.str().str();
+
+ // Create temp directory with proper error handling
+ SmallString<128> tempDirPath;
+ if (auto EC = sys::fs::createUniqueDirectory("llvm-advisor", tempDirPath)) {
+ // Use timestamp for temp folder naming
+ auto now = std::chrono::system_clock::now();
+ auto timestamp =
+ std::chrono::duration_cast<std::chrono::seconds>(now.time_since_epoch())
+ .count();
+ tempDir_ = "/tmp/llvm-advisor-" + std::to_string(timestamp);
+ sys::fs::create_directories(tempDir_);
+ } else {
+ tempDir_ = tempDirPath.str().str();
+ }
+
+ // Ensure the directory actually exists
+ if (!sys::fs::exists(tempDir_)) {
+ sys::fs::create_directories(tempDir_);
+ }
+
+ if (config_.getVerbose()) {
+ outs() << "Using temporary directory: " << tempDir_ << "\n";
+ }
+}
+
+CompilationManager::~CompilationManager() {
+ if (!config_.getKeepTemps() && sys::fs::exists(tempDir_)) {
+ sys::fs::remove_directories(tempDir_);
+ }
+}
+
+Expected<int> CompilationManager::executeWithDataCollection(
+ const std::string &compiler, const std::vector<std::string> &args) {
+
+ // Analyze the build command
+ BuildContext buildContext = CommandAnalyzer(compiler, args).analyze();
+
+ if (config_.getVerbose()) {
+ outs() << "Build ...
[truncated]
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here and other places:
LLVM Coding standard
https://llvm.org/docs/CodingStandards.html#don-t-use-braces-on-simple-single-statement-bodies-of-if-else-loop-statements
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In general, get familiar with:
https://llvm.org/docs/CodingStandards.html
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For some reason git clang-format
it's changing me that, normal behaviour?
944ae3a
to
b57c71c
Compare
The AdvisorConfig class provides JSON based configuration loading with file classification patterns and output directory management.
- Add FileManager for file operations. - Add FileClassifier for compilation artifact categorization. - Add ProcessRunner for subprocess execution.
Introduce data structures that represent a single build phase and compilation unit.
CommandAnalyzer inspects an incoming compiler or build-system invocation and classifies the tool in use, the build phase, input/output files and notable flags.
This change adds logic to run compiler processes and automatically add options to collect optimization remarks, profiling data, and debug information when needed.
This change adds logic to manage builds end to end. It runs the build process, calls the detector, extracts data, and moves generated files to the output directory.
Adds helpers to run the compilation with extra flags to collect IR, assembly, AST dumps, include trees, debug info, and other data.
Adds logic to scan the origin compiler arguments, find source files, figure out compile flags and output paths, and create a description of each compilation unit. This is the entry point for the analysis pipeline.
Adds the command-line front-end that handles advisor options, finds the compiler commands, create configuration, and starts the build data collection.
- use cpp code standard based on llvm guidelines
Add RST documentation for the compilation analysis tool - complete usage guide with synopsis, options and examples
- Implement the main llvm advisor tool with - command-line interface supporting compile and view modes - configuration system support - support for compiler wrapping with data collection - web viewer launch capabilities with port configuration
- add fundamental infrastructure components - Json-based configuration management system - file system operations and path handling
- enhance build execution and error handling - included more command analysis capabilities - add compilation unit processing
-implement data extraction capabilities - add unit metadata management and tracking - support multiple artifact types
- implement compilation manager - add viewer launcher for web server - improve compilation pipeline
- implement HTTP server with RESTful API endpoints - add base API classes
- implement units API for compilation unit lising and details - add file content API with multiple format support - add artifacts API for build output analysis
- implement optimization remarks analysis API - add diagnostics patterns and compilation phases - add performance profiling -implement binary size analysis and optimization
-implement responsive HTML5 dashboard - add interactive javascript modules for data visualization - implement code explorer - add performance analysis views
- implement data models for compilation artifacts - add artifact collector for discovering and organizing build outputs - support for multiple file types and compilation units with metadata
- implement base parser infrastructure for file processing - add diagnostics parser for compiler diagnostic output - add optimization remarks parser with YAML support - support for large file handling and error recovery
- implement time-trace parser - add runtime trace parser for offloading analysis - add compilation phases parser for build pipeline analysis - add ftime-report parser for detailed timing breakdowns
- implement AST JSON parser for syntax tree analysis - add LLVM IR parser for intermediate representation analysis - add assembly parser for machine code inspection - add binary size parser for executable size optimization - add symbols parser for debugging and analysis support
- implement preprocessed source parser for macro analysis - add macro expansion parser for preprocessing inspection - add include tree parser for dependency visualization - add build dependencies parser for project structure analysis - add debug information parser for DWARF data processing
- implement static analyzer parser for code quality analysis - add SARIF parser for standardized analysis result format - add objdump parser for object file inspection - add PGO profile parser for profile-guided optimization data - add XRay parser for runtime tracing and profiling - add version info parser for toolchain metadata
Remove orphaned code block marker at end of file - Fix 'Inline literal start-string without end-string' error -m "- Update SEE ALSO section to follow LLVM CommandGuide standards"
- refactor on tab that was deprecated
b57c71c
to
87badd8
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Quick check. I'd prefer we don't use the _
member variable naming convention at all. Update all these single line blocks to omit braces. We're doing a lot of work here that should be using LLVM's ArgList parsing / rendering support instead. There's also work that's duplicating Clang's phases but I'm assuming it'd be too much of a pain to include clang from here.
if (auto outputDirOpt = Obj->getString("outputDir"); outputDirOpt) { | ||
OutputDir_ = outputDirOpt->str(); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LLVM style, omit all the braces in single blocks.
std::string AdvisorConfig::getToolPath(llvm::StringRef tool) const { | ||
return tool.str(); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we have a function that wraps StringRef::str()
without doing anything?
namespace llvm { | ||
namespace advisor { | ||
|
||
BuildExecutor::BuildExecutor(const AdvisorConfig &config) : config_(config) {} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I dislike the member_
syntax and I don't think it's used in LLVM.
BuildExecutor::BuildExecutor(const AdvisorConfig &config) : config_(config) {} | ||
|
||
llvm::Expected<int> | ||
BuildExecutor::execute(llvm::StringRef compiler, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should try to use LLVM variable names, but we're not fully consistent on that in the tools directory so I won't require it.
BuildContext &buildContext, llvm::StringRef tempDir) { | ||
auto instrumentedArgs = instrumentCompilerArgs(args, buildContext, tempDir); | ||
|
||
auto compilerPath = llvm::sys::findProgramByName(compiler); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm assuming this is trying to find clang
? This just searches PATH, it should search the same directory this executable lives in if we're assuming a coherent toolchain.
// If user already specified remarks, find and redirect the file | ||
bool foundFileFlag = false; | ||
for (auto &arg : result) { | ||
if (llvm::StringRef(arg).contains("-foptimization-record-file=")) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is throwing me for a loop, how are we handling these arguments? I would expect we'd use the LLVM ArgList API and just define the ones we need to handle like in https://github.com/llvm/llvm-project/blob/main/clang/tools/clang-linker-wrapper/LinkerWrapperOpts.td. I'm just doing a brief scan though so maybe the expected interface for parsing + rendering arguments is different.
a REST API for programmatic access to the collected data. | ||
|
||
All collected data is stored in a timestamped, structured format within the output | ||
directory (default: ``.llvm-advisor``) and can be analyzed using the built-in web |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure using a hidden output folder is the best option.
Configuration File Usage | ||
~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
|
||
Use a custom configuration file: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add reference to the Configuration section.
|
||
Specify a configuration file to customize :program:`llvm-advisor` behavior. | ||
The configuration file uses JSON format and can override default settings | ||
for output directory, verbosity, and other options. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add reference to the Configuration section too.
View Mode | ||
~~~~~~~~~ | ||
|
||
The **view** subcommand combines data collection with automatic web viewer |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Isn't there a command to just open the web viewer from data that has been already collected?
├── assembly/main.s # Assembly output | ||
├── ast/main.ast # AST dump | ||
├── diagnostics/ # Compiler warnings/errors | ||
└── ... # Additional analysis data |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missing one space
std::string path; | ||
std::string language; | ||
bool isHeader = false; | ||
llvm::SmallVector<std::string, 8> dependencies; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the llvm::
needed?
struct CompilationUnitInfo { | ||
std::string name; | ||
llvm::SmallVector<SourceFile, 4> sources; | ||
llvm::SmallVector<std::string, 8> compileFlags; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The code is full of SmallVector types with hard-coded size. Is this actually necessary? If so, you could use a using
or typedef
.
sys::fs::create_directories(tempDir + "/pgo"); | ||
sys::fs::create_directories(tempDir + "/ftime-report"); | ||
sys::fs::create_directories(tempDir + "/version-info"); | ||
sys::fs::create_directories(tempDir + "/sources"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The creation of directories could be done directly inside each of the function calls below.
sys::fs::create_directories(tempDir + "/version-info"); | ||
sys::fs::create_directories(tempDir + "/sources"); | ||
|
||
if (auto Err = extractIR(unit, tempDir)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You may pass the sub-directory from this place to have them centralized at this function, e.g.:
if (auto Err = extractIR(unit, tmpDir, "ir")
return Err;
...
return Err; | ||
|
||
// Run additional extractors | ||
for (size_t i = 0; i < numExtractors_; ++i) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why some extractors work in this way and some others are called manually? I'd prefer to use this way for all.
This pull request serves as the final work submission for Google Summer of Code 2025, developed by @miguelcsx . It provides a report on the project's goals, accomplishments, current status, and future work.
Project Goals
LLVM generates a wealth of data during compilation, including optimization remarks, profiling information, and timing data. However, this information is often scattered across different flags, output formats, and locations, making it overwhelming for developers (especially those new to LLVM) to analyze.
The goal of this project was to create a unified infrastructure to organize, visualize, and analyze compilation data for the LLVM toolchain. By making this data accessible and easy to understand, the tool aims to empower developers to make data-driven optimization decisions, reduce the frustration of debugging, and make offloading development more approachable for everyone.
Work Accomplished
Summary of Deliverables
Over the program, we successfully developed a working prototype that fulfills the core objectives outlined in the proposal. The key deliverables include:
A compiler wrapper tool that automatically collects compilation artifacts.
A storage layer that supports project-wide linking and differential views.
Initial analysis capabilities, including cross-unit analysis, performance tracking, and search functionality.
Weekly Breakdown of Commits and Tasks
This is a week-by-week summary of the work completed during the program:
Community Bonding Period:
Weeks 1-2
Weeks 3-4
Week 5-6
/tmp/
, but later restructured because system cleanup was deleting them).Weeks 7-8
Weeks 9-10
Weeks 11-12
Expanded parsing support to more file types for the source viewer.
Implemented the performance timeline, including sandwich and trace viewers.
Updated the PR in preparation for final evaluation.
Final Week: Final Touches
Current State
The project is in a functional and usable state. The core features: data collection, storage, visualization, and basic analysis, are implemented. The application is stable for intended use cases, and the documentation provides a clear guide for setup and usage.
The final commit for the GSoC 2025 period is final commit. All work completed during the program is included in this pull request.
What's Left to Do (Future Work)
This project lays a strong foundation for a powerful tool, and there are many exciting opportunities for future development:
Project Code
All work for this project is contained within this single pull request to provide a unified view of the contributions.
Challenges and Learnings
This project was a tremendous learning experience.
Challenges: One of the biggest challenges was designing a flexible data pipeline capable of correlating scattered data from various compiler outputs. Creating an intuitive UI that could present this complex, multi-layered data without overwhelming the user was also a significant but rewarding effort.
Learnings: Through GSoC, I have gained deep technical knowledge in LLVM's remark system, full-stack application development, and the intricacies of build system integration. This project confirmed my passion for compilers and open-source development. My goal is to make compilers more accessible for everyone, and I believe this tool is a step in that direction. I look forward to continuing as a maintainer and helping grow the LLVM community.
Some screenshots
Dashboard
Source Viewer
Performance