Creating a ‘NoOps’ CI Pipeline with the Apprenda PaaS on OpenStack

By Atos Apprenda Support


Earlier this year, I wrote about PaaS and continuous integration (CI). In it, I demonstrated how PaaS can make a CI pipeline more efficient by removing the transient app instances that were deployed for the purposes of build verification or testing as soon as they are no longer required. This feature allows for a more efficient use of the compute resource pool during app deployments.

Another benefit of relying on PaaS in CI is the consistency across all phases of the pipeline. The app verification happens in exactly the same runtime that supports the production workloads, because the platform provides a uniform abstraction layer between the guest apps and the underlying infrastructure. Speaking of infrastructure, one of the common challenges with CI is that it requires significant operational overhead. A meaningful pipeline is typically supported by multiple logical areas of infrastructure: a few build servers and several distinct smaller production replicas (e.g., integrated DEV, QA, UAT, staging), whose number and purpose depends on the process adopted by the organization. These environments vary in size and have different uptime requirements. While the SLAs are not as rigid as those for production, the associated maintenance and monitoring still requires the attention of the IT department.

For example, to support a truly continuous build and test process, IT needs to worry about high availability of the build servers and other environments. To avoid interruptions in the pipeline and undesirable maintenance windows, additional instances of build servers need to be in a warm standby mode, which requires even more infrastructure. The irony of the situation is that all this equipment and compute resources are there to host transient application instances that do not have a long shelf life. Wouldn’t it be nice to support these types of app instances with a more elastic infrastructure, capable of expanding when apps need to be tested, and shrink when the testing is idle?

Traditional CI setup with dedicated environments

Traditional CI setup with dedicated environments

This is how the problem is addressed by an integration of the Apprenda PaaS with Jenkins and OpenStack.

In my earlier article, we’ve already seen how a single instance of Apprenda can securely host Dev, QA, UAT and production instances of the same application through its deployment policy engine. Platform operators (IT/DevOps) can segment the underlying infrastructure into the necessary environments and write policies ensuring that containers created for testing purposes will only land on the infrastructure segment designated for QA.

When deployed on OpenStack, Apprenda can now project its logical segmentation of the resource pool onto the underlying infrastructure and further secure the separation between container environments at the network level. This is accomplished through the ability to map Apprenda’s logical deployment areas (e.g., Dev, QA, Prod) to specially designated security groups in OpenStack. The multi-tenancy is also preserved as Apprenda maintains the mapping of its own tenants to OpenStack tenants.

In addition, the infrastructure does not have to pre-exist for the testing (or any other function of the pipeline) to be performed. VMs with appropriate security characteristics are created on the fly when necessary and later removed once they have fulfilled their purpose. This way, there is no longer the need to maintain dedicated VMs to host transient application instances. When Jenkins pushes an application that needs to be deployed to the QA area of the platform, for example, Apprenda checks to see if it has enough capacity for this particular CI step before it accepts the app. If the remaining capacity is insufficient, Apprenda will use OpenStack API to create new VMs and attach them to the appropriate security group that is mapped to the QA deployment area of the platform. Once the VMs are built and joined to the Apprenda pool, the platform will tag and associate them with the QA deployment area and only then accept the application from Jenkins for deployment. During the deployment, the policy engine will ensure that the app containers are routed to the QA deployment area of the platform.

Once Jenkins completes the testing, it can issue a promotion request to Apprenda with configurable directives to clean up existing artifacts, deployed earlier for testing. Apprenda will follow the same process to deploy the validated app upstream in accordance with its deployment policies. After the promotion is complete, Apprenda will remove the app containers used earlier for QA and call the OpenStack API to delete the underlying VMs if they are not in use by other applications or CI processes.

In summary, the Apprenda PaaS integrated with Jenkins and OpenStack provides an efficient foundation for a seamless and secure CI pipeline. The solution significantly reduces the overhead and cost of maintaining CI through the elasticity of the infrastructure and efficient container management. These benefits are achieved without compromising the mandatory secure separation between the software development environments with respect to multi-tenancy.

Check out the video below to see a concrete implementation of the CI pipeline with Jenkins and Apprenda on OpenStack:



Atos Apprenda Support

View Comments
  1. John GolkeAugust 17, 2016

    i miss the music 🙂 Nice job Sasha – really important that utilization be emphasised throughout CI process.

Comments are closed.