The source code comes with a small py2exe script which makes it easy to compile the Python code into an executable. You can generate an executable on your system by running

This will put the resulting binary and all the dependencies into a 'dist\\' subdirectory. You could then tell your build system to use 'clcache.exe' as the compiler binary, or you could rename 'clcache.exe' to 'cl.exe' and put it into a directory which comes first in the PATH. In my case, I went for the latter solution; I put the 'cl.exe' binary into '%HOME%\bin' and prepended that directory to the PATH. The 'cl.exe' wrapper binary will notice that it was renamed and forward all the arguments to the real compiler executable.

This way, simply running 'cl' will invoke the script instead of the real compiler.

--help

Print usage information

-s

Print some statistics about the cache (cache hits, cache misses, cache size etc.)

-z

Reset the cache statistics, i.e. number of cache hits, cache misses etc.. Doesn’t actually clear the cache, so the number of cached objects and the cache size will remain unchanged.

-M <size>

Sets the maximum size of the cache in bytes. The default is 1048576000 Bytes.

CLCACHE_DIR

If set, points to the directory within which all the cached object files should be stored. This defaults to %HOME%\clcache

CLCACHE_CL

Can be set to the actual 'cl.exe' executable to use. If this variable is not set, the 'clcache.py' script will scan the directories listed in the PATH environment variable for 'cl.exe'.

CLCACHE_LOG

If this variable is set, a bit of diagnostic information is printed which can help with debugging cache problems.

CLCACHE_DISABLE

Setting this variable will disable 'clcache.py' completely. The script will relay all calls to the real compiler.

CLCACHE_HARDLINK

If this variable is set, cached object files won’t be copied to their final location. Instead, hard links pointing to the cached object files will be created. This is more efficient (faster, and uses less disk space) but doesn’t work if the cache directory is on a different drive than the build directory.

CLCACHE_NODIRECT

Disable direct mode. If this variable is set, clcache will always run preprocessor on source file and will hash preprocessor output to get cache key. Use this if you experience problems with direct mode or if you need built-in macroses like _TIME_ to work correctly.

CLCACHE_BASEDIR

Has effect only when direct mode is on. Set this to path to root directory of your project. This allows clcache to cache relative paths, so if you move your project to different directory, clcache will produce cache hits as before.

clcache.py was designed to intercept calls to the actual cl.exe compiler binary. Once an invocationw as intercepted, the command line is analyzed for whether its a command line which just compiles a single source file into an object file. This means that all of the following requirements on the command line must be true:

• The /link switch must not be present

• The /c switch must be present

• The /Zi switch must not be present (/Z7 is okay though)

• There must be exactly one source file present on the command line.

If all the above requirements are met, clcache forwards the call to the preprocessor by replacing /c with /EP in the command line and then invoking it. This will cause the complete preprocessed source code to be printed. clcache then generates a hash sum out of

• The complete preprocessed source code

• The `normalized' command line

• The file size of the compiler binary

• The modification time of the compiler binary

The `normalized' command line is the given command line minus all switches which either don’t influence the generated object file (such as /Fo) or which have already been covered otherwise. For instance, all switches which merely influence the preprocessor can be skipped since their effect is already implicitely contained in the preprocessed source code.

Once the hash sum was computed, it is used as a key (actually, a directory name) in the cache (which is a directory itself). If the cache entry exists already, it is supposed to contain a file with the stdout output of the compiler as well as the previously generated object file. clcache will copy the previously generated object file to the designated output path and then print the contents of the stdout text file. That way, the script behaves as if the actual compiler was invoked.

If the hash sum was not yet used in the cache, clcache will forward the invocation to the actual compiler. Once the real compiler successfully finished its work, the generated object file (as well as the output printed by the compiler) is copied to the cache.

1. Create clcache.exe.

   1. py2exe http://www.py2exe.org/ can create executables from Python code.

      1. Open a command window, cd to the clcache directory (containing 'clcache.py') and type 'python setup.py py2exe'.

2. Visual Studio knows and uses the full path to 'cl.exe', so we need to replace the real 'cl.exe'.

   1. Determine which 'cl.exe' your build process uses. One approach is to start the task manager, switch to the Processes tab, then start a build in Visual Studio and watch for Image Names of 'cl.exe'. Right click on one and select 'Open File Location' to open the directory containing your real compiler. On my system, that is c:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\bin\x86_amd64\

   2. Rename 'cl.exe' in that folder to something else, e.g., 'cl-real.exe'. Also rename 'cl.exe.config' similarly, e.g., 'cl-real.exe.config'.

   3. Then copy 'clcache.exe' to that folder and rename it to 'cl.exe'.

3. clcache needs to know how to call the real compiler. Create an environment variable called CLCACHE_CL and set it to the full path to the real compiler, e.g., c:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\bin\x86_amd64\cl-real.exe

   1. Since this approach uses an environment variable, it can only work for one real compiler at any point in time. So if you switch between different real compilers, you will need to switch the environment variable appropriately.

4. VS will normally track file access during builds, storing that information in '.tlog' files, and using it to know when to rebuild objects and which files should be deleted on 'rebuild' or 'clean' actions. Since 'clcache.exe' will access the stats file, and the cached objects, but not the actual dependencies (which are accessed by the real compiler) we need to tell VS not to track file access (otherwise VS will not compile objects it should, will compile objects it shouldn’t, and will delete the cache when cleaning).

   1. The project property TrackFileAccess needs to be set to false.

      1. There are multiple ways to do so, including adding the property to the vcxproj file and setting an environment variable named TrackFileAccess (capitalization unimportant) with a value of 'false' (no quotes).

   2. Note that setting TrackFileAccess to 'false' will hamper Visual Studio’s ability to determine when objects need to be recompiled, and I have encountered situations in which objects that should be recompiled are not, leaving stale '.obj' files that cause link failures due to LNK2001, unresolved external symbol. Such errors disappear with a 'clean' or 'rebuild', which forces VS to rebuild all object files. Since clcache is caching the object files, rebuilding in this way is a short process.

      1. I recommend 'rebuilding' instead of 'building' when TrackFileAccess is false.

5. Check build flags required for clcache to actually cache.

   1. Do not use /Zi or /ZI flags.

      1. Replace /Zi and /ZI flags with /Z7. (Alternatively, just append the /Z7 flag since Visual Studio will ignore any previous debug flag).

         1. /Zi and /ZI create separate '.pdb' files that contain the debug symbols for all the objects in a project. These files are incrementally constructed, which makes caching them nontrivial.

         2. /Z7 puts the debug symbols inside the object files (which will make the '.obj' files larger).

   2. Use the /showIncludes flag.

      1. Files included by the source (and files included by those, etc.), must be considered part of the source when clcache is determining whether or not the result of a potential compilation is already cached.

      2. To determine which files are included in the source, clcache relies on the real compiler. The /showIncludes flag instructs the real compiler to provide that information.

      3. If the /showIncludes flag is not part of the call to the compiler, clcache will not cache the result and will log the reason.

         1. The /showIncludes flag will cause VS to output many lines of included filenames. It may be preferable to set the output verbosity to 'Quiet' (Tools→Options→Projects and Solutions→Build and Run).

6. Optionally specify which directory should hold the cached files, via CLCACHE_DIR environment variable.

7. Optionally (but recommended) make clcache print a few messages for each compile, via CLCACHE_LOG environment variable (value irrelevant).

8. Optionally set the maximum cache size, by calling 'cl -M newSizeInBytes' or 'clcache -M newSizeInBytes'.

   1. If 'cl -s' shows increasing numbers for 'evicted misses' then cached objects are being evicted/deleted because there is not enough room, and then later clcache is trying to recall those objects, so a larger maximum cache size may improve the probability of a successful cache hit.

9. If Visual Studio is open, close and restart it, so it picks up the new environment variables.

10. Build normally in Visual Studio. If CLCACHE_LOG is set, you should see messages about 'Adding file X to cache…​' and in future builds 'Reusing cached object…​'.

clcache.py was written by Frerich Raabe with a lot of help by Slava Chigrin and other contributors.

This program was heavily inspired by ccache, a compiler cache for the GNU Compiler Collection.