Usually, it is a good idea to make software which you built relocatable. This means that created binaries can be moved around and they still work after the location change. This includes the ability to still find the correct versions of dynamic libraries. One simple way to achieve this is by setting the rpath relative to the current location of the binary. In CMake this can be achieved like this:

# for all binaries created in a CMake project:
SET(CMAKE_INSTALL_RPATH "\$ORIGIN/../lib:\$ORIGIN/")
# for certain targets
SET_TARGET_PROPERTIES(target 1 target2 ...
    PROPERTIES INSTALL_RPATH "\$ORIGIN/../lib:\$ORIGIN/")

Please note that the backslash is required.

The Jenkins continuous integration server uses the notion of file fingerprints for this purpose. The upstream job is built by Jenkins and produces one or several so called artifacts, the results of the build process. The artifacts are archived by Jenkins and fingerprints (hash sums) for each artifact are created and stored along with the build number of the job. When the downstream job starts to build it downloads the (most recent) artifacts from the upstream job and uses them for its purposes, i.e. building and running the own source code. By comparing the fingerprints of the downloaded artifacts with the stored fingerprints Jenkins knows which version of each upstream job was involved in a build and can track which upstream build number broke the downstream job. Jenkins will only issue notifications if this fingerprinting mechanism is properly configured, triggering a build after another is not sufficient to receive these notifications. Moreover, the Blame Upstream Commiters plugin needs to be used and enabled for each downstream job or the global property hudson.upstreamCulprits (will this ever be renamed?) needs to be set.

The rational behind this rather complex mechanism is that it enables a high amount of parallelism for building jobs. While the downstream job builds, the upstream job can already operate again without affecting the downstream job. This would be the case if e.g. a central installation location would be shared between both jobs. If the upstream job installs new files while the downstream job is still building, this will certainly result in hard to debug errors. Moreover, this also allows to run the downstream job on a different build salve (assuming similar systems), which also would not be the case with a central installation location in a file system.

For Java projects (where Jenkins comes from) the explained mechanism usually works well. The upstream job produces one or several jar files containing all resources for the project like images, fingerprints them, no preprocessor is involved which configures the Java code according to the installation setup, and no source code was generated based on this setup. For C++ projects this is usually different, because the language already includes a preprocessor and it is common practice to set certain code lines according to the installation location, e.g. to find additional files like images, because they cannot be packaged in the jar file. Also, C++ projects usually consists of much more files considering all headers compared to Java. This provides more chances to mix something up.

So assuming an upstream C++ job A (using CMake, other build solutions are not covered in this post but the techniques can be applied there, too) which is built in Jenkins, it usually will be configured with an installation location, e.g. inside the job’s workspace like /jenkins/workspace/A/install.

1. Often, CMake will use this location, e.g. to generate a config.h which tells that images are found at /jenkins/workspace/A/install/share/A/images etc.
2. To use the CMake dependency mechanism, it will generate a AConfig.cmake file and install it also to the share folder (cf. the CMake documentation for find_package). The file might look like:
   

   SET(A_LIBRARIES "/jenkins/workspace/A/install/lib/libA.so")
   SET(A_INCLUDE_DIRS "/jenkins/workspace/A/install/include/A")
   

After building the project the job will e.g. use a compression tool to create a single archive and compress all contents of /jenkins/workspace/A/install/, archive this artifact and generate the fingerprints for it.

Both issues mentions above will prevent the dependency tracking of Jenkins to function properly, because the downstream job will download the artifacts to its own workspace, e.g. to /jenkins/workspace/B/upstream/A and unpack them. Cf. the issues:

1. The upstream project A will not find external files at all or will use wrong versions, because in the meantime a new build of job A might have started and hence the workspace of this job is currently changing.
2. The downstream job B will not build at all or might use a wrong version of A because the contents of AConfig.cmake point to A’s workspace and not the downloaded artifacts.

To enable reliable dependency tracking in Jenkins, the solutions are:

1. Do not use this technique at all. The software is generally more flexible if not hard locations are assumed and more situations are covered without recompiling.
2. The idea here is to make all paths given in the config file (AConfig.cmake) relative to its current location on the disc. This will look like this:
   

   GET_FILENAME_COMPONENT(CONFIG_DIR "${CMAKE_CURRENT_LIST_FILE}" PATH)
   SET(A_LIBRARIES "${CONFIG_DIR}/../../lib/libA.so")
   SET(A_INCLUDE_DIRS "${CONFIG_DIR}/../../include/A")
   

   Now the CMake script of B will use the correct downloaded headers, libraries etc. for A from the own workspace

The two aspects make it possible to use fingerprinting in Jenkins for dependency tracking with notifications for upstream committers. Especially the first aspect includes taking care while designing the project but there is no other solution I can think of.

Please note that for executing any tests in downsteam job B you have to set the LD_LIBRARY_PATH to find the right upstream libraries as well.

Random Comments

Some more care needs to be taken to not mix up the dependency tracking again:

• The downstream job needs to make sure that the latest downloaded artifact is really used to build its own source code. So it is a good idea to simply remove the upstream directory as the first step of the build.
• The downloaded artifacts (as explained above the generated archive files) need to be kept after extracting them, because the downstream job also has to generate fingerprints for them (and not for the extracted files) to create a match with the fingerprints stored for the upstream job.
• In order to enable the downstream CMake project to find the upstream project use the _DIR variable for the CMake call as defined in the CMake documentation, e.g. -DA_DIR="${WORKSPACE}/upstream/A/share/A"
• If your upstream project contains a version or revision number in the extracted folder (e.g. ${WORKSPACE}/upstream/A-0.35/) and you want your downstream job to be resilient against version changes in the upstream project you can use some find-magic on UNIX for automatically finding the folder:
  A=`find "${WORKSPACE}/upstream" -maxdepth 1 -type d -name "A-*";`
• If you are using pkg-config instead of or in addition to the CMake config file mechanism, you can use the --define-variable command line argument to achieve similar flexibility, assuming that all your absolute paths depend on a single prefix-variable in the pc file.

One thing which I was not happy with so far was the missing integration of open-source coverage tools for Linux. Here, Gcov can be used to generate more or less precise coverage reports for projects compiled with GCC. Unfortunately, Gcov itself does not provide tools to export the results in any common or even nicely readable format. Until now, the only working solution I found was to use the Gcov front-end LCOV to generate a HTML report. This report is nice to read but it cannot be tracked by Jenkins with the drawback that no trend report for the code coverage can be generated. Nevertheless, I’ve wrapped the creation of such a HTML report in a CMake function and worked with it so far.

Today, I searched again for cheap solutions to overcome this drawback (this means without writing a custom Jenkins plugin for Gcov coverage files). While searching the net, I found the gcovr script, which parses Gcov result files and is able to convert them into XML files that satisfy the format generated by Cobertura, which is a coverage tool for Java with an existing plugin for Jenkins.

As far as I tested it, this script works well with the Jenkins plugin, so I integrated the execution of this script in my existing coverage function for CMake, which is available in the RSC library. This library also contains additional CMake wrappers for tools that can be used to generate trend reports in Jenkins, like cppcheck. Now our Jenkins can also generate coverage trend reports for C++ projects.

def foo(arg=MyClass()):
    pass

As I am currently programming a lot in C++ this did not look suspicious to me. But from python’s point of view it absolutely makes sense that the default value for arg is already constructed at module load time and not only on a call to that functions. Moreover it is important to remember that this default argument is constructed only once for all calls to a function as stated here.

boost::shared_ptr<Foo> p(new Foo);
boost::thread t(boost::bind(&Foo::method, p.get()))

This prevents the livecycle management of shared_ptr or any other smart pointer to be effective in the thread. Hence, p may get destructed even though the thread is still active.
Boost bind can handle smart pointers, so instead use the smart pointer itself as the instance argument for bind:

boost::thread t(boost::bind(&Foo::method, p))

Frühlingsröllchen, Chili-Pfeffer-Dip, Reis, Eisberg mit Möhren, Kokosquark

-Wall -Wextra

will generate a lot of useful warning messages about unused parameters etc.
Unfortunately, this is not the common practice and often the own compiler settings concerning the warning level results in dozens of warnings from system headers on which the own code relies, making it impossible to spot warnings from your own code in the endless mass of console output.
Fortunately, GCC has a way to ignore warnings from foreign headers. Instead of using -I to specify an include path, -isystem tells the compiler to treat the includes from the given path as system headers where no warnings should be reported.

If CMake is used to create the Makefile, a special argument to the INCLUDE_DIRECTORIES function generates these compiler flags:

INCLUDE_DIRECTORIES(SYSTEM /usr/include /and/other /system/paths)