Sometimes we need to run tests on different machines because of kinds of reasons:
- run tests on different environment, e.g. run tests against different OS, DB
- distributed test in order to get fast feedback, e.g. split tests into n parts, and only run one part on one machine
- …
The following is a real screenshot of Jenkins in one of my projects, we created several jobs for the same functional tests, and the only difference between them is what test tag(s) need to run on this job.
The Problem
The settings for each job are almost same:
- Limit the build agent to run the tests using job label
- Clean up test environment before test
- Copy zipped artifact from upstream
- Extract zip file
- Run tests specified by tag(s)
- Publish html reports
- Publish xml reports
The problem is any time the process need to be changed, we need to change all of them which is simple a copy paste work but need to repeat 7 times.
The Solution
Jenkins provides a Multi-configuration project support when create a new job, The only difference is to add an user-defined axis, and we can also add a Slave axis in order to limit the tests only runs on the desired agents.
Click the question icon if you don’t understand any setting because Jenkins provided a very helpful document.
The Multi-configuration job just likes a job template: it will start several sub jobs based on the configuration, aggregate all the test results and show in the status page:
Even better is to not configure things like this in Jenkins at all, but rather utilize DevOps in combination with Jenkins Job Builder (JJB) and Zuul to make the creation of jobs done in a simple text file that you can manage and track in your source code control systems (e.g. git).
http://ci.openstack.org/jenkins-job-builder/
https://github.com/openstack-infra/zuul