27 November 2014

Speeding up CLion build times with Ninja & CCache

CLion has quickly become my favorite C++ IDE, with refactoring abilities far beyond any other free or commercial offering. However, the app is still in EAP (read: unstable/beta) and has many performance issues and feature limitations. But EAP build has significantly improved each of these areas, and for that reason I rarely need to open other "backup" IDEs.

However, one feature that I greatly miss from other IDEs is built-in Ninja and CCache integrations.

For the uninitiated, Ninja is a build system designed to replace GNU Make. If you are working on a large C++ codebase built with CMake, then it's a lifesaver because target dependency calculation is almost instantaneous. A practical example: the codebase I am actively developing at the moment is approximately 1,000,000 LoC. A full rebuild takes several minutes, but in the case of changing a single *.cpp file, GNU Make takes about 1.5 minutes just to parse the project and see that this file must be recompiled and the target relinked. Ninja does the same work just a few seconds.

CCache is a compiler cache which basically keeps cached copies of object files and uses them if it determines that they can be used instead of recompiling the file from scratch. The key advantage here is that when switching or doing a full rebuild, much time is saved compared to recompling everything from scratch. Again, with this big project, a full rebuild without CCache takes about 10-20 minutes. With everything cached (best case scenario), the build time reduces to just a few minutes (linking cannot be cached).

So, let's get to it! Since CLion has no native support for Ninja & CCache, you'll need to do it manually. First, make sure you have both packages installed for your OS (this guide is written for Linux but should work almost the same on Mac. Not sure about Windows, sorry).

Next, go to the CMake tab of CLion at the bottom of the screen. In this panel, select the Cache tab. Find the following CMake variables and replace them as such:
  • CMAKE_MAKE_PROGRAM: Change to /usr/bin/ninja (or whenever it is installed on your system)
  • MAKECOMMAND: Change to /usr/bin/ninja -k 100
Now enable CCache support by finding and replacing the following variables:
  • CMAKE_CXX_COMPILER: Change to /usr/lib/ccache/c++ (or wherever CCache is installed on your system)
  • CMAKE_C_COMPILER: Change to /usr/lib/ccache/cc
Be sure to press the little "save" button (the floppy disk icon) in the upper left-hand corner of the CMake panel! Otherwise CLion will not save the CMakeCache.txt file.

Now, trigger a build. It will probably fail because CLion defaults to GNU Make for the CMake generator. That's ok, but you will need to manually run CMake from the build directory. You can find the build directory in the first line of the build output (in the "Messages" tab at bottom). For example, on my system it prints:

/opt/clion-140.569.17/bin/cmake/bin/cmake --build /home/nik/.clion10/system/cmake/generated/c9614a53/c9614a53/Debug --target my_project_test -- -j 8

So, pop open a terminal and go to the build directory, and run rm -rf * there, then cmake -GNinja /path/to/your/project.

Now run "build", and watch Ninja & CCache do their magic! Sadly, CLion seems to hardcode the "-j" option to Make, which means that the "Rebuild" and "Clean" options in CLion are broken. Sorry! You'll have to work around this by just manually cd'ing to the build dir and running ninja clean from there.

Finally, for bonus points, installing gold-ld can improve linking times drastically, making even giant C++ programs manageable without too many compiling breaks.

Also, JetBrains has told me that integrated Ninja & CCache support are coming in the future. So this tutorial might be obsolete in the future; be sure to keep an eye on CLion's preferences to see if options for these build tools appear there.