Jenkins builds

Our Jenkins instance has the ability to run a variety of automated jobs. This document describes the appropriate setup and usage of such jobs.

Import jobs

Jenkins can automatically synchronize every private repo with our public one. A job gets triggered whenever a push to our stable branch (hydro-devel) occurs. If you would like to sync a private repo with the public one, you should ask the administrators to add a new import job. Import jobs can be found here.

Documentation jobs

Jenkins automatically produces Doxygen (or other) documentation for our code. Documentation includes ROS and non-ROS packages. The documentation gets updated every time a new commit is pushed to the stable branch (hydro-devel) of a repo. If you want to automatically produce documentation for a repo, you should ask from an administrator to add a new job. If you are documenting a non-ros package, you should also add a manifest.yaml file, like this one. Products of the documentation jobs can be found here. All documentation jobs are listed here.

Build and test jobs

Jenkins can run build jobs for our private repos. Build jobs run in a vanilla environment. That means for every build a new VM gets spawned that runs stock Ubuntu images, in order to achieve a fresh environment each time. Builds get triggered for every push on every branch of a private repo. In order to automatically run a build for your repo, you should ask from an admin to create a new job and also include a .jenkins.yml file in the root directory of your repo. A list of all test jobs can be found here.

If a branch does not contain a .jenkins.yml file, builds for that branch will be skipped.

The .jenkins.yml file

The .jenkins.yml file is the configuration file of the build and consists of two parts: the install part and the script part. Usually in the install part we run commands that configure our system and prepare for the actual build. If one command fails the whole build aborts instantly, as there is no point continuing. In the script part we run commands that compile and test our code. All commands must exit with code 0 on success, anything else is considered failure. If one command fails we continue with the next one, we do not stop the build immediately.

A typical .jenkins.yml file, for a package without any source dependencies, would look like:

# Generic continuous integration configuration file, used by jenkins
install:
  - export ROS_DISTRO=hydro
  - export CI_SOURCE_PATH=$(pwd)
  - sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu precise main" > /etc/apt/sources.list.d/ros-latest.list'
  - wget http://packages.ros.org/ros.key -O - | sudo apt-key add -
  - sudo apt-get update -qq
  - sudo apt-get install -qq -y python-catkin-pkg python-rosdep python-wstool ros-$ROS_DISTRO-catkin ros-$ROS_DISTRO-ros
  - sudo rosdep init
  - rosdep update
  # Create workspace
  - mkdir -p $BUILD_HOME/catkin_ws/src
  # Add this folder to the workspace
  - ln -s $CI_SOURCE_PATH $BUILD_HOME/catkin_ws/src
  - cd $BUILD_HOME/catkin_ws
  # Install debian dependencies
  - rosdep install --from-paths src --ignore-src --rosdistro $ROS_DISTRO -y
  - source /opt/ros/$ROS_DISTRO/setup.bash
script: # All commands must exit with code 0 on success. Anything else is considered failure.
  - catkin_make
  - source devel/setup.bash
  - catkin_make roslint
  - catkin_make run_tests
  - catkin_test_results build

If our repo needs other repos in order to build properly, we have to use pandoradep to resolve the dependencies. Example, data_fusion:

# Continuous integration configuration file, used by Jenkins
# Author: Chris Zalidis
install:
  - export ROS_DISTRO=hydro
  - export CI_SOURCE_PATH=$(pwd)
  - export REPOSITORY_NAME=${PWD##*/}
  - sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu precise main" > /etc/apt/sources.list.d/ros-latest.list'
  - wget http://packages.ros.org/ros.key -O - | sudo apt-key add -
  - sudo apt-get update -qq
  - sudo apt-get install -qq -y python-catkin-pkg python-rosdep python-wstool ros-$ROS_DISTRO-catkin ros-$ROS_DISTRO-ros python-pip
  - sudo pip install pandoradep
  - sudo rosdep init
  - rosdep update
  # Create workspace
  - mkdir -p $BUILD_HOME/catkin_ws/src
  - cd $BUILD_HOME/catkin_ws/src
  - wstool init .
  # Find and download non-debian dependencies
  - pandoradep scan $CI_SOURCE_PATH > deps.rosinstall
  - wstool merge deps.rosinstall
  - wstool update
  # Delete the .rosinstall version of this repo and use the one of the branch we are testing
  - rm -rf $REPOSITORY_NAME
  - ln -s $CI_SOURCE_PATH . # Link the repo we are testing to the new workspace
  - cd ..
  # Install debian dependencies
  - rosdep install --from-paths src --ignore-src --rosdistro $ROS_DISTRO -y
  - source /opt/ros/$ROS_DISTRO/setup.bash
script: # All commands must exit with code 0 on success. Anything else is considered failure.
  - catkin_make --only-pkg-with-deps $REPOSITORY_NAME
  - source devel/setup.bash
  - catkin_make roslint_pandora_alert_handler
  - catkin_make roslint_pandora_sensor_processing
  - catkin_make roslint_pandora_sensor_coverage
  - catkin_make tests # Build all tests, even from other packages. Its not optimal, but...
  - catkin_make run_tests_pandora_alert_handler
  - catkin_make run_tests_pandora_sensor_processing
  - catkin_test_results build

In the above example you should notice several things:

• We are using the catkin_make --only-pkg-with-deps command to build the packages. This is done in order to build only desired packages and save time and accuracy in case of failure.
• We are running manually roslint targets for each package. If we just run catkin_make roslint we would check other packages also, which is not the purpose of that test.
• We are running manually test targets for each package. Same as above, we want to run test only for packages we are testing.
• The last command catkin_test_results build is mandatory because sometimes tests fail without returning an exit code different than 0.

If you have a branch that deals only with a specific package or you have some tests that are failing, you should consider removing them from the script part. When you are ready to merge with hydro-devel thought, you should bring them back. As a rule-of-thumb, in hydro-devel every test and roslint should be included.