The Benefits of Continuous Integration in the Cloud

Matthew SetterSeptember 1st, 2017Last Updated: August 31st, 2017

0 40 7 minutes read

There are a large number of technologies and concepts that can dramatically improve software quality, while simultaneously reducing its development time, cost, and effort. The question, though, is which combination is the right one for both your organization and your project?

Like so many topics, the answer to this one is, “It depends.” So today, I’m going to help whittle that list down somewhat by looking at several of the main advantages that continuous integration (CI) in the cloud can bring. However, before I do that, let’s quickly recap what CI and the cloud are, starting with the cloud.

What Is the Cloud?

The cloud is often seen as a very amorphous, even nebulous, concept, one often applied to anything to do with computing infrastructure that you can’t either directly or physically touch and control. As a result, it’s become a somewhat derided term that marketing departments seem to use with increasing abandon. This derision is rather unfortunate as, when well understood, the cloud is a potent tool, one which can be very enabling and transformative.

So let’s suitably define what it is. Of all the cloud descriptions I’ve heard, the best one is this one from an interview with Scott Hanselman:

The cloud is hosting generalized and made elastic. It’s hosting with an API wrapped around it. It’s choice. It’s an API. It’s flexible billing…The cloud is about gaining control of your business. It’s about gaining knowledge. It’s not a loss of control.

You can listen to the longer version of his description, which starts from about 14 minutes into the interview.

For myself, the cloud is computer infrastructure that scales dynamically with you, whether up or down. It’s infrastructure that has the flexibility to adapt to your changing needs — and budgets — continuously.

Given the benefits that this brings, over recent years, a whole range of vendors have appeared, offering a variety of services designed to support a broad range of infrastructure needs. These include hosted database and logging providers such as ElephantSQL and Loggly, email providers such as Zoho, and more generalized computing infrastructure such as Amazon Web Services, Google Cloud Platform, and Microsoft Azure.

What Is Continuous Integration?

Next, what is continuous integration (CI)? According to Codeship’s CI Essentials guide, continuous integration is:

…a development practice where developers integrate code into a shared repository frequently, preferably several times a day. Each integration can then be verified by an automated build and automated tests. While automated testing is not strictly part of CI, it is typically implied. One of the key benefits of regularly integrating code is that you can detect errors quickly and locate them more easily. As each change introduced is typically small, pinpointing the specific change that introduced a defect can be done quickly.

Continuous integration needn’t be complicated nor sophisticated. At its simplest, it could consist of just a few things:

A script that launches a testing tool, such as PHPUnit, JUnit, or NUnit, which runs the application’s test suite once new code has been integrated into a Git repository’s master branch.
A script that builds the software after the test suite has succeeded.
An email notification sent to the developers alerting them to the success or failure of the two previous steps.

In reality, however, CI pipelines often have much more sophisticated requirements, such as database, queueing, caching, and web servers, which are often required for acceptance and integration tests.

This complexity can be driven by a range of factors, such as the industry that the software serves, regulatory requirements and constraints, budgetary constraints, and the project’s technical requirements.

Benefits of CI in the Cloud

Now that we’ve recapped what these terms mean, why should you put your CI pipeline in the cloud instead of hosting it yourself? What benefits will you accrue by doing so that could outweigh a homegrown solution? To answer these two questions, let’s consider four key advantages of CI in the cloud.

Reduced cost

The first benefit of CI in the cloud that we should consider is lower cost. This might sound counterintuitive when you’re using an external service, as you’ll likely be paying for the service at some point. But calculating cost, as I’ve argued previously, is usually neither a quick nor easy process. Let’s step through an example so that you see what I mean.

Codeship Basic offers three paid plans: starter, essential, and power. There’s also a free plan (not just a free trial), a good option if you’re just getting started or you’re building a project that’s either for a hobby or an open-source project. You can get started for free, then as your budget grows, you can scale up to get more functionality and power.

Assuming that you went with one of those plans, after signing up, you’d just need to do a bit of light-touch account configuration, and you’re ready to build your first CI pipeline. Contrast this with building a solution that either you or your team created, or one based on an open-source tool such as Jenkins. If you take that path, you’ll have to:

Invest in both hardware and staffing resources to build and document the initial solution.
To maintain and adapt it as your software grows and changes over time.
To replace faulty or aging hardware, manage performance tuning.
To adjust it to stay up with industry trends.

Depending on just how advanced and sophisticated your solution needs to be, your costs, amortized over a decade or more, could be quite substantial. Given that, which approach costs less, not only in a monetary sense but also in respect to time and overhead?

Minimal configuration

One thing that can’t be removed by hosting a CI solution in the cloud is the need for configuration. However, it can be significantly reduced. By using a hosted solution, the configuration requirements are reduced just to those required for your CI pipeline(s). On top of that, a range of default settings is typically available to suit the nature of the software being developed.

All you have to do is choose the one that best fits your software and make supplementary changes. As an example, Codeship provides basic commands for such languages and frameworks as Go, Java, PHP, Python, Node.js, Vue.js, Symfony, Laravel, Ruby on Rails, and Sinatra. In addition, a default pipeline will be created to match the default build commands.

Despite this level of support and assistance, no cloud provider will ever be able to create a solution that exactly suits the needs of every software project. So you may find that the default commands and build pipelines aren’t always suitable. In these situations, you should expect to be able to provide custom build scripts and configurations, as well as to customize and extend the default options as you need to, on a project-by-project basis, as and when the need arises.

So either way, whether the defaults, extending the defaults, or a custom script set is the path you take, you would still have far less to configure than in a solution you created yourself.

Dynamic scalability

Perhaps the greatest advantage of using the cloud is its elastic nature, one that can scale both up and down as the needs arise. If you’re a small development organization, perhaps consisting of only two developers on the way to deploying your first release, you can get started with only the features and computing power that you absolutely need.

This lets you keep your costs to a minimum and avoid over-investing in hardware that you don’t yet need. However, as your application’s complexity or user base grows, you can dynamically scale up the available resources to meet demand.

This flexibility is also available if you have intermittent periods of load, periods that may only last for a few days — or hours — at most. It lets you dynamically scale to handle unpredictable traffic spikes or seasonal holidays.

But concerning CI, hosting providers can also provide elasticity around testing by offering a configurable number of parallel test pipelines. Parallel test pipelines allow you to split builds into multiple pipelines, which in turn reduce the test suite’s execution time.

Flexibility

Next, let’s consider flexibility. Consider that you designed a homegrown solution when your organization or software was quite young. It started out needing a rather simple software stack, perhaps composed of Apache or NGINX, PHP, and PostgreSQL.

However, you’ve rapidly gained a strong following and consequently, have added an increasing number of technologies into the mix, such as Redis, Memcached, and ELK. What’s more, because of the demand, you’re now considering transitioning to a Go-based microservice architecture.

What will it take and what will it cost you to refactor and maintain your internal CI solution? Can you find the staff with sufficient experience in the technologies required to both adapt — or to recreate — and maintain it?

Contrast this with a Cloud solution which works with projects almost regardless of their technology mix, one which comes with a host of infrastructure ready to go, including a range of database, queuing, cache, and search servers.

What’s more, in certain cases, multiple versions of these servers are simultaneously available. As a result, you have the flexibility of testing your application in a wide array of combinations and permutations. All you may need to do is to provide the appropriate environment settings to your application, and it can alternate between the different versions as necessary.

In Conclusion

And that’s been a look at four benefits of using cloud-based continuous integration over an in-house, homegrown solution. Can you think of any more? Do you feel that any of the four mentioned either aren’t applicable or could be extended? If so, I’d love to hear your thoughts in the comments.

If you’ve been on the fence for some time about the cloud, whether what it is or whether it’s worth including in your software stack, I hope that these benefits have helped resolve any lingering doubt that you may have. I also hope that you now see that regardless of your organization’s size or your application’s maturity and sophistication, a cloud-based CI solution may be the more viable choice.