When building an enterprise-scale CI system, one has to be extremely careful with the plugins that one installs in the CI system. This post describes the issues that plugins create, recommends keeping the number of plugins to a minimum, and discusses some examples of where plugins may provide value without compromising the maintainability of your CI system.

Issues Created by Plugin Use

While plugins seem useful at first, they can create a number of maintainability issues for your CI system if you aren’t careful. This section describes the three main ways in which a plethora of plugins leads to a less maintainable CI system.

Increased Coupling Between Projects

In the enterprise, a CI system is used by many development teams. Each team may have one or more projects in the CI system. In the ideal case, the projects of one team are completely isolated from the projects of another. Containerized builds should be the default, and build agents should have installed the minimal set of dependencies.

Plugins muddy this picture: they are, by nature, single-instance global dependencies. They are shared across the entire CI system, and therefore become a common dependency of all projects in the system.

What happens when a development team wants functionality provided by a new version of a plugin? The plugin must be upgraded. Since plugins are a common dependency of all projects, the upgrade affects all other projects in the system. How should the upgrade be tested? What happens if the new version of the plugin breaks some functionality needed by a different team? Upgrade it or not, some team will be unhappy.

Perhaps CI system vendors could alleviate this problem somewhat by allowing multiple versions of a plugin to be installed in a build system at a given time so that each project could choose the version to use. Unfortunately, this does not appear to be possible on current systems, and is likely technically impossible to implement satisfactorily.

Increased Difficulty Keeping Build System Up-to-Date

Like any software, it’s important to keep a build system up-to-date. The latest version fixes known security problems and provides exciting new functionality to keep users happy. Plus, if your enterprise has a support contract with the vendor of the CI system, staying current may be a requirement to receive on-going support.

Plugins make this harder. They are essentially third-party add-ons that customize¹ the build system in myriad ways. There is a combinatorial explosion that virtually guarantees no two build systems have the same set of plugins. How is the vendor of the CI system supposed to test this? They don’t.

That means, when it comes time to upgrade your CI system, if you have even a moderate number of plugins installed, it is likely that you are the first person in the world to perform exactly this upgrade. That’s just a little too exciting for the enterprise.

Furthermore, what if it is discovered that the new version of the CI system isn’t supported by a plugin that your development teams are already using? Do you wait to upgrade the CI system until a new version of the plugin is released? For every plugin, you must ensure the third-party provider of the plugin produces frequent, well-tested updates–or at least that the plugin source code is open so that you can update it should the third-party be unreliable.

Plugins Can Encourage Poorly-Designed Builds

Your build code should be able to run in places other than just the CI system. When you’re developing a build, and when you’re troubleshooting a build gone wrong, you need to be able to do so in an isolated environment as similar to the main build system as possible. Like any troubleshooting, the edit-run-debug cycle time is important, as is the ability to step through and inspect state, and use debuggers.

It’s easy to troubleshoot if you can run the build on your own machine. Trying to troubleshoot something on the live CI system is much harder. If you’re willingly suffering through the extended edit-run-debug cycle that happens when you troubleshoot on the CI system, you’re either being a moron or you’re masochistic. You’re in good company though: this author has spent many hours stuck in such a rut (it is left as an exercise to the reader to determine whether this author is a moron or merely masochistic).

Plugins get in the way of making a build that developers can run on their own machines. When a plugin provides some piece of build functionality, that part of the build can only be done on the CI system. When the build needs troubleshooting, this functionality will have to be replicated locally in some other way, making it more difficult to run your build locally. It’s important that you maintain architectural separation between the CI system and the builds. A CI system is a distributed job runner with VCS and artifact support. It should not implement your build logic.

CI system plugins can make certain tasks easy to set up and run in the CI system itself, but hinder you the moment you need to run the build anywhere else.

Acceptable Kinds of Plugins

There are, of course, exceptions to every rule. Not all plugins create the issues above; some kinds of plugins are acceptable and even encouraged. Generally, if a plugin does something that cannot be accomplished by way of script from within a build step, then that plugin may be a good candidate. Typically, such plugins are those that modify the behaviour of the CI system itself, rather than providing functionality to individual builds.

Examples

An example of an acceptable plugin is a plugin that allows the build system to store build artifacts in a cloud object storage service like Amazon S3. Such a plugin makes the CI system easier to maintain because cloud storage is elastic and scales to whatever size you need. If artifacts are stored on a disk volume in the CI system, then the maintenance burden is increased since you are forced to actively manage the size of that storage. There is little downside to installing a plugin that relieves you of this burden.

Another example of an acceptable plugin is the Jenkins Pipeline plugin, which among other things allows the build steps to be configured as code. This is a fantastic modification of the CI system itself, and adds significant value. You should still be careful with such a plugin; it’s best to not rely too much on all the pipeline features and instead put the build logic in a shell script that can be run anywhere.

An example of a plugin to be concerned about is a plugin that uploads code to a separate server and initiates security threat analysis. This is best done as part of the build script–use a Maven or Gradle plugin, instead.

Conclusions

Try to keep the number of plugins in your CI system to a minimum. If you need a few plugins to extend the OOTB build system to make it do something it otherwise couldn’t do, fine (make sure the third-party providing the plugin releases new versions frequently, or at least ensure the plugin is open source). On the other hand, if the main functionality of the CI system plugin is something that your development teams could achieve from inside their build scripts, don’t install plugin. Instead, write a script or library that provides the same functionality. It will be easier to upgrade your CI system, different projects can use different versions of your script, and your development teams will be better able to troubleshoot and maintain their builds.