Skip to content

Commit 70a9347

Browse files
authored
feat: automatic generation of compile_commands.json
1 parent 395f565 commit 70a9347

File tree

5 files changed

+361
-26
lines changed

5 files changed

+361
-26
lines changed

.github/workflows/ci.yml

Lines changed: 2 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -317,17 +317,6 @@ jobs:
317317
modules-exclude-paths: ''
318318
trace-commands: true
319319

320-
- name: Configure Boost.URL
321-
working-directory: boost/libs/url
322-
run: |
323-
set -x
324-
if [ -d "__build__" ]; then
325-
rm -rf __build__
326-
fi
327-
mkdir __build__
328-
cd __build__
329-
cmake -D BOOST_URL_BUILD_TESTS=OFF -D BOOST_URL_BUILD_EXAMPLES=OFF -D CMAKE_EXPORT_COMPILE_COMMANDS=ON -D CMAKE_CXX_STANDARD_INCLUDE_DIRECTORIES="$default_includes" -D CMAKE_CXX_COMPILER=${{ steps.setup-cpp.outputs.cxx }} -D CMAKE_C_COMPILER=${{ steps.setup-cpp.outputs.cc }} ..
330-
331320
- name: Generate demos
332321
run: |
333322
config_template=$(printf '%s\n' \
@@ -338,6 +327,7 @@ jobs:
338327
"multipage: %s" \
339328
"inaccessible-members: never" \
340329
"inaccessible-bases: never" \
330+
"cmake: -D BOOST_URL_BUILD_TESTS=OFF -D BOOST_URL_BUILD_EXAMPLES=OFF -D CMAKE_EXPORT_COMPILE_COMMANDS=ON -D CMAKE_CXX_STANDARD_INCLUDE_DIRECTORIES='$default_includes' -D CMAKE_CXX_COMPILER=${{ steps.setup-cpp.outputs.cxx }} -D CMAKE_C_COMPILER=${{ steps.setup-cpp.outputs.cc }}" \
341331
"filters:" \
342332
" symbols:" \
343333
" exclude:" \
@@ -350,7 +340,7 @@ jobs:
350340
[[ $variant = multi ]] && multiline="true" || multiline="false"
351341
printf "$config_template\n" $format $multiline > $(pwd)/boost/libs/url/mrdocs.yml
352342
mkdir -p "demos/boost-url/$variant/$format"
353-
mrdocs --config="$(pwd)/boost/libs/url/mrdocs.yml" "$(pwd)/boost/libs/url/__build__/compile_commands.json" --output="$(pwd)/demos/boost-url/$variant/$format"
343+
mrdocs --config="$(pwd)/boost/libs/url/mrdocs.yml" "$(pwd)/boost/libs/url/" --output="$(pwd)/demos/boost-url/$variant/$format"
354344
done
355345
asciidoctor -d book -R "$(pwd)/demos/boost-url/$variant/adoc" -D "$(pwd)/demos/boost-url/$variant/adoc-asciidoc" "$(pwd)/demos/boost-url/$variant/adoc/**/*.adoc"
356346
done

docs/modules/ROOT/pages/usage.adoc

Lines changed: 24 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -16,32 +16,26 @@ Where `MRDOCS_ROOT` is the path of the mrdocs executable, and `MRDOCS_CONFIG` is
1616
We also assume `PROJECT_SOURCE_DIR` is the path to the root of your project's source code, where its main `CMakeLists.txt` file is located, and `PROJECT_BUILD_DIR` is the path to the directory where you want to generate the documentation.
1717
Feel free to change these variables to suit your needs.
1818

19-
The first step to generate your documentation is to generate the `compile_commands.json` file by running `cmake` with the `-DCMAKE_EXPORT_COMPILE_COMMANDS=ON` flag.
19+
MrDocs simplifies the documentation generation process. Generating the `compile_commands.json` file by running `cmake` is optional. If the path to `compile_commands.json` is not provided when calling MrDocs, the tool will automatically run `cmake` for you, provided that you have CMake version >=3.13.5 installed. Parameters for cmake, such as `-D BOOST_URL_BUILD_TESTS=OFF`, should be specified in the `cmake:` key of the `mrdocs.yml` configuration file.
2020

21-
[source,bash]
22-
----
23-
cd $PROJECT_SOURCE_DIR
24-
mkdir $PROJECT_BUILD_DIR
25-
cd $PROJECT_BUILD_DIR
26-
cmake .. -DCMAKE_EXPORT_COMPILE_COMMANDS=ON
27-
----
21+
It is still possible, but optional, to manually generate the `compile_commands.json` file. For instructions on manually generating this file, see <<manual-compile-commands, this section>>.
2822

29-
At this step, you can also add any other flags you want to pass to `cmake`, such as `-DCMAKE_BUILD_TYPE=Release` or `-DCMAKE_CXX_COMPILER=clang++`.
30-
By running CMake with the `CMAKE_EXPORT_COMPILE_COMMANDS` flag, you will generate a `compile_commands.json` file in your build directory.
31-
This file contains all the information mrdocs needs to generate the documentation.
3223

33-
Now let's generate the reference files.
3424
The following command will generate the documentation with the most common options:
3525

3626
[source,bash]
3727
----
3828
cd $PROJECT_BUILD_DIR
3929
MRDOCS_OUTPUT=$PROJECT_BUILD_DIR/docs/reference/adoc
40-
$MRDOCS_ROOT/mrdocs $PROJECT_BUILD_DIR/compile_commands.json --config=$MRDOCS_CONFIG --addons=$MRDOCS_ROOT/addons --output=$MRDOCS_OUTPUT
30+
$MRDOCS_ROOT/mrdocs <ProjectPath> --config=$MRDOCS_CONFIG --addons=$MRDOCS_ROOT/addons --output=$MRDOCS_OUTPUT
4131
----
4232

4333
Here's a description of these options:
4434

35+
* `<ProjectPath>`: the path to the project to document. This can be a path to a `compile_commands.json` file, a directory, or a `CMakeLists.txt` file.
36+
If a path to a `compile_commands.json` file is provided, MrDocs will use this file and will not call CMake. It is assumed the user has already manually run CMake.
37+
If a directory (not a file) is provided, it is assumed that this directory is the `ProjectPath` and contains a `CMakeLists.txt` describing the project. MrDocs will automatically run CMake to generate the `compile_commands.json`, using the parameters specified in the `mrdocs.yml` configuration file.
38+
If a `CMakeLists.txt` file is provided, it is assumed that the directory containing this file is the `ProjectPath`. MrDocs will automatically run CMake to generate the `compile_commands.json`, using the parameters specified in the `mrdocs.yml` configuration file.
4539
* `--config=$MRDOCS_CONFIG`: the path to the `mrdocs.yml` configuration file.
4640
This file configures which generator is used, which directory to process,
4741
and what symbols should be extracted.
@@ -54,6 +48,23 @@ This is where the generated documentation will be placed.
5448

5549
MrDocs ignores non-c++ source files, so nothing more needs to be done to generate the documentation for your project.
5650

51+
[[manual-compile-commands]]
52+
=== Generating the compile_commands.json Manually
53+
54+
To generate the `compile_commands.json` file by running `cmake` with the `-DCMAKE_EXPORT_COMPILE_COMMANDS=ON` flag.
55+
56+
[source,bash]
57+
----
58+
cd $PROJECT_SOURCE_DIR
59+
mkdir $PROJECT_BUILD_DIR
60+
cd $PROJECT_BUILD_DIR
61+
cmake .. -DCMAKE_EXPORT_COMPILE_COMMANDS=ON
62+
----
63+
64+
At this step, you can also add any other flags you want to pass to `cmake`, such as `-DCMAKE_BUILD_TYPE=Release` or `-DCMAKE_CXX_COMPILER=clang++`.
65+
By running CMake with the `CMAKE_EXPORT_COMPILE_COMMANDS` flag, you will generate a `compile_commands.json` file in your build directory.
66+
This file contains all the information mrdocs needs to generate the documentation.
67+
5768
== Demos
5869

5970
A few examples of reference documentation generated with MrDocs are available in https://mrdocs.com/demos/.

src/lib/Lib/CMakeExecution.cpp

Lines changed: 236 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,236 @@
1+
//
2+
// Licensed under the Apache License v2.0 with LLVM Exceptions.
3+
// See https://llvm.org/LICENSE.txt for license information.
4+
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
5+
//
6+
// Copyright (c) 2024 Fernando Pelliccioni ([email protected])
7+
//
8+
// Official repository: https://github.com/cppalliance/mrdocs
9+
//
10+
11+
#include "lib/Lib/CMakeExecution.hpp"
12+
13+
#include <llvm/Support/FileSystem.h>
14+
#include <llvm/Support/MemoryBuffer.h>
15+
#include <llvm/Support/Path.h>
16+
#include <llvm/Support/Program.h>
17+
18+
namespace clang {
19+
namespace mrdocs {
20+
21+
namespace {
22+
23+
Expected<std::string>
24+
getCmakePath() {
25+
auto const path = llvm::sys::findProgramByName("cmake");
26+
MRDOCS_CHECK(path, "CMake executable not found");
27+
std::optional<llvm::StringRef> const redirects[] = {llvm::StringRef(), llvm::StringRef(), llvm::StringRef()};
28+
std::vector<llvm::StringRef> const args = {*path, "--version"};
29+
int const result = llvm::sys::ExecuteAndWait(*path, args, std::nullopt, redirects);
30+
MRDOCS_CHECK(result == 0, "CMake execution failed when checking version");
31+
return *path;
32+
}
33+
34+
Expected<std::string>
35+
executeCmakeHelp(llvm::StringRef cmakePath)
36+
{
37+
llvm::SmallString<128> outputPath;
38+
MRDOCS_CHECK(!llvm::sys::fs::createTemporaryFile("cmake-help", "txt", outputPath),
39+
"Failed to create temporary file");
40+
std::optional<llvm::StringRef> const redirects[] = {llvm::StringRef(), outputPath.str(), llvm::StringRef()};
41+
std::vector<llvm::StringRef> const args = {cmakePath, "--help"};
42+
llvm::ArrayRef<llvm::StringRef> emptyEnv;
43+
int const result = llvm::sys::ExecuteAndWait(cmakePath, args, emptyEnv, redirects);
44+
MRDOCS_CHECK(result == 0, "CMake execution failed when trying to get help");
45+
46+
auto const bufferOrError = llvm::MemoryBuffer::getFile(outputPath);
47+
MRDOCS_CHECK(bufferOrError, "Failed to read CMake help output");
48+
49+
return bufferOrError.get()->getBuffer().str();
50+
}
51+
52+
Expected<std::string>
53+
getCmakeDefaultGenerator(llvm::StringRef cmakePath)
54+
{
55+
MRDOCS_TRY(auto const cmakeHelp, executeCmakeHelp(cmakePath));
56+
57+
std::istringstream stream(cmakeHelp);
58+
std::string line;
59+
std::string defaultGenerator;
60+
61+
while (std::getline(stream, line)) {
62+
if (line[0] == '*' && line[1] == ' ') {
63+
size_t const start = 2;
64+
size_t const end = line.find("=", start);
65+
if (end == std::string::npos) {
66+
continue;
67+
}
68+
return line.substr(start, end - start);
69+
}
70+
}
71+
return Unexpected(Error("Default CMake generator not found"));
72+
}
73+
74+
Expected<bool>
75+
cmakeDefaultGeneratorIsVisualStudio(llvm::StringRef cmakePath)
76+
{
77+
MRDOCS_TRY(auto const defaultGenerator, getCmakeDefaultGenerator(cmakePath));
78+
return defaultGenerator.starts_with("Visual Studio");
79+
}
80+
81+
std::vector<std::string>
82+
parseCmakeArgs(std::string const& cmakeArgsStr) {
83+
std::vector<std::string> args;
84+
std::string currentArg;
85+
char quoteChar = '\0';
86+
bool escapeNextChar = false;
87+
88+
for (char ch : cmakeArgsStr)
89+
{
90+
if (escapeNextChar)
91+
{
92+
currentArg += ch;
93+
escapeNextChar = false;
94+
}
95+
else if (ch == '\\')
96+
{
97+
escapeNextChar = true;
98+
}
99+
else if ((ch == '"' || ch == '\''))
100+
{
101+
if (quoteChar == '\0')
102+
{
103+
quoteChar = ch;
104+
}
105+
else if (ch == quoteChar)
106+
{
107+
quoteChar = '\0';
108+
}
109+
else
110+
{
111+
currentArg.push_back(ch);
112+
}
113+
} else if (std::isspace(ch))
114+
{
115+
if (quoteChar != '\0')
116+
{
117+
currentArg.push_back(ch);
118+
}
119+
else
120+
{
121+
if ( ! currentArg.empty())
122+
{
123+
args.push_back(currentArg);
124+
currentArg.clear();
125+
}
126+
}
127+
} else
128+
{
129+
currentArg += ch;
130+
}
131+
}
132+
133+
if ( ! currentArg.empty())
134+
{
135+
args.push_back(currentArg);
136+
}
137+
138+
return args;
139+
}
140+
141+
} // anonymous namespace
142+
143+
Expected<std::string>
144+
executeCmakeExportCompileCommands(llvm::StringRef projectPath, llvm::StringRef cmakeArgs)
145+
{
146+
MRDOCS_CHECK(llvm::sys::fs::exists(projectPath), "Project path does not exist");
147+
MRDOCS_TRY(auto const cmakePath, getCmakePath());
148+
149+
llvm::SmallString<128> tempDir;
150+
MRDOCS_CHECK(!llvm::sys::fs::createUniqueDirectory("compile_commands", tempDir), "Failed to create temporary directory");
151+
152+
llvm::SmallString<128> errorPath;
153+
MRDOCS_CHECK(!llvm::sys::fs::createTemporaryFile("cmake-error", "txt", errorPath),
154+
"Failed to create temporary file");
155+
156+
std::optional<llvm::StringRef> const redirects[] = {llvm::StringRef(), llvm::StringRef(), errorPath.str()};
157+
std::vector<llvm::StringRef> args = {cmakePath, "-S", projectPath, "-B", tempDir.str(), "-DCMAKE_EXPORT_COMPILE_COMMANDS=ON"};
158+
159+
auto const additionalArgs = parseCmakeArgs(cmakeArgs.str());
160+
161+
bool visualStudioFound = false;
162+
for (size_t i = 0; i < additionalArgs.size(); ++i)
163+
{
164+
auto const& arg = additionalArgs[i];
165+
if (arg.starts_with("-G"))
166+
{
167+
if (arg.size() == 2)
168+
{
169+
if (i + 1 < additionalArgs.size())
170+
{
171+
auto const& generatorName = additionalArgs[i + 1];
172+
if (generatorName.starts_with("Visual Studio"))
173+
{
174+
args.push_back("-GNinja");
175+
visualStudioFound = true;
176+
++i;
177+
continue;
178+
}
179+
}
180+
} else {
181+
if (arg.find("Visual Studio", 2) != std::string::npos)
182+
{
183+
args.push_back("-GNinja");
184+
visualStudioFound = true;
185+
continue;
186+
}
187+
}
188+
}
189+
190+
if (arg.starts_with("-D"))
191+
{
192+
if (arg.size() == 2)
193+
{
194+
if (i + 1 < additionalArgs.size())
195+
{
196+
auto const& optionName = additionalArgs[i + 1];
197+
if (optionName.starts_with("CMAKE_EXPORT_COMPILE_COMMANDS"))
198+
{
199+
++i;
200+
continue;
201+
}
202+
}
203+
} else {
204+
if (arg.find("CMAKE_EXPORT_COMPILE_COMMANDS", 2) != std::string::npos)
205+
{
206+
continue;
207+
}
208+
}
209+
}
210+
args.push_back(arg);
211+
}
212+
213+
if ( ! visualStudioFound)
214+
{
215+
MRDOCS_TRY(auto const cmakeDefaultGeneratorIsVisualStudio, cmakeDefaultGeneratorIsVisualStudio(cmakePath));
216+
if (cmakeDefaultGeneratorIsVisualStudio)
217+
{
218+
args.push_back("-GNinja");
219+
}
220+
}
221+
222+
int const result = llvm::sys::ExecuteAndWait(cmakePath, args, std::nullopt, redirects);
223+
if (result != 0) {
224+
auto bufferOrError = llvm::MemoryBuffer::getFile(errorPath);
225+
MRDOCS_CHECK(bufferOrError, "CMake execution failed (no error output available)");
226+
return Unexpected(Error("CMake execution failed: \n" + bufferOrError.get()->getBuffer().str()));
227+
}
228+
229+
llvm::SmallString<128> compileCommandsPath(tempDir);
230+
llvm::sys::path::append(compileCommandsPath, "compile_commands.json");
231+
232+
return compileCommandsPath.str().str();
233+
}
234+
235+
} // mrdocs
236+
} // clang

src/lib/Lib/CMakeExecution.hpp

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
//
2+
// Licensed under the Apache License v2.0 with LLVM Exceptions.
3+
// See https://llvm.org/LICENSE.txt for license information.
4+
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
5+
//
6+
// Copyright (c) 2024 Fernando Pelliccioni ([email protected])
7+
//
8+
// Official repository: https://github.com/cppalliance/mrdocs
9+
//
10+
11+
#ifndef MRDOCS_LIB_TOOL_CMAKE_EXECUTION_HPP
12+
#define MRDOCS_LIB_TOOL_CMAKE_EXECUTION_HPP
13+
14+
#include <string>
15+
16+
#include <llvm/ADT/StringRef.h>
17+
#include <mrdocs/Support/Error.hpp>
18+
19+
namespace clang {
20+
namespace mrdocs {
21+
22+
/**
23+
* Executes CMake to generate the `compile_commands.json` file for a project.
24+
*
25+
* This function runs CMake in a temporary directory for the given project path
26+
* to create a `compile_commands.json` file.
27+
*
28+
* @param projectPath The path to the project directory.
29+
* @param cmakeArgs The arguments to pass to CMake when generating the compilation database.
30+
* @return An `Expected` object containing the path to the generated `compile_commands.json` file if successful.
31+
* Returns `Unexpected` if the project path is not found or if CMake execution fails.
32+
*/
33+
Expected<std::string>
34+
executeCmakeExportCompileCommands(llvm::StringRef projectPath, llvm::StringRef cmakeArgs);
35+
36+
} // mrdocs
37+
} // clang
38+
39+
#endif // MRDOCS_LIB_TOOL_CMAKE_EXECUTION_HPP
40+

0 commit comments

Comments
 (0)