Continuous Integration Systems are one of the most important tools for agile software development. They automate the process of building and testing. A lot of people seem to have realized their importance and there are quite a few products in this arena. I already used Hudson and CruiseControl, but for OpenMRS we need to find one which is best suited to our needs. So I started out about 3 weeks back to create this roundup of continuous integration servers. This should be a useful roundup for any project with similar requirements.
Why does OpenMRS need a Continuous Integration System?
Any software development effort needs to take care that regression doesn’t happen with new code changes. Often a change in the API/module core results in breaking of modules dependent on an earlier method. A Continuous Integration System will rebuild OpenMRS after a change is committed and provide information on how that change is affecting related code.
OpenMRS would also benefit from an easy to understand UI that Continuous Integration Systems provide for number of failing tests. The number of passing or failing Unit Tests will indicate the quality of a build and help implementers/testers realize the stability of a build. We can also set some goals on how many test methods we need to write before an API method can be finalized or deprecated.
The work done in different branches and modules can be monitored and looked at easily by the community.
Building of OpenMRS Installer using NBI can be automated and new users can directly test with the latest build of OpenMRS using the cross-platform installer.
Thus to summarize, a continuous integration system will bring better release quality, more transparency, quicker bug finding and fixing, simplicity and TDD frame-of-mind.
Disadvantages for OpenMRS in using Continuous Integration System
- Additional Load on Servers
- Not every developer is motivated to write unit tests ;-)
Features that OpenMRS needs (Not exhaustive)
- Easy to monitor tests and easy to understand dashboard
- Support for SVN and Ant
- Dependency integration
- Email/RSS/IRC notifications when a build fails or bad code is committed
- Warning flags when a committed code doesn’t follow coding rules (naming, newline format,etc.)
- Allow code committers to modify build and test parameters from the GUI
- Optimal Performance
- Price & Open-source development
Comparison of Continuous Integration Systems:
CruiseControl | Continuum | TeamCity | Bamboo | Hudson | |
Monitoring UI | Dashboard introduced since v2.7 is not intuitive | Dashboard only shows tests | Advanced UI & dashboard. | Advanced UI, Detailed reporting out-of-the-box, Intuitive | Simple Dashboard, Plugins enhance reporting, Intuitive, somewhat detailed |
SCM Support | All Support SVN | ||||
Dependency Integration | Scripts need to be written for each new dependency. Tracking different versions of dependency jars is very complex | Easy for Maven2.0+ projects, but not so easy for other types of projects | Dependency can be managed easily. Advanced UI for dependency management | Dependency management is easy and intuitive. Different versions of same library not automated. Creating test/build plans allows dependency of different versions | Dependency management is easy to configure. file fingerprinting simplifies identification of different versions. Automatically can detect and build project dependencies |
Email/RSS/IRC | Emails. Plugins - RSS, blog, IM with Jabber | Email, IRC, IM with Jabber, MSN | Email, Jabber, RSS, external HTML widget | Emails, RSS, IM Notification using Jabber or OpenFire | Plugin – Emails, RSS, IRC, Jabber, Google Calender, Twitter |
Code Quality and Patterns | Not very easy to define | Could not find a way | Can be defined with plugin for IntelliJ IDEA | Managed through test plans. Manual test plans have to be created | Plugin provides UI. Test plans can be created manually out-of-the-box |
Security and User Management | Easy to configure with different roles | Roles can be easily defined | Roles can be easily defined | Simplistic UI for user build plan management. Easy integration with JIRA | Easy to configure roles for users |
Performance | Fast | Fast | Somewhat slower in comparison, but includes a lot of features | Fast in build and integration. Slightly slower in reports. Includes lots of features that may not be used. Distributed builds with slaves speeds up performance | Comparatively lightweight out-of-the-box, but requires plugins. Distributed builds with slaves speeds up performance |
Pricing | Free & OpenSource. Paid version called Cruise available. | Free & OpenSource | Professional version is free, but enterprise is paid | Free for opensource projects | Free & OpenSource |