Purpose of the DevOps Stack

Since about 2006, the IT industry has been totally disrupted by the conjunction of several factors, including the virtualization of resources and the possibility of obtaining on-demand resources. This new way of managing IT resources is widely known under the term "Cloud Computing". It has become a kind of standard, or at least an objective pursued also by people wishing to maintain on-premises infrastructures. The long-awaited dream of finally being able to consider IT as a simple commodity. Well that’s what many believed.

The concrete consequences of this revolution for IT companies like Camptocamp have mainly been the incredible growth in the number of resources to be provisioned and managed. Automating our processes was not an option, it was just mandatory to avoid losing the control, maintain quality service and remain competitive.

We had no respite in this automation journey, open source projects have emerged from all sides to address all of the needs related to this paradigm shift. We have learned a lot during all these years and the DevOps Stack is our answer to implement our best practices in a containerized world where Kubernetes clusters became kings of application platforms.

Nowadays, the question is not so much whether there is software to solve a problem, but which one to use. Like a Linux distribution, we have container orchestrators (Kubernetes as a kernel) and many other projects revolving around and offering different features. These features are sometimes oriented for developers, operators or both (e.g. observability tools).

As for a Linux distribution, what we need is consistency, components that work together, easy to install and maintain. But unlike a Linux distribution which is by definition server-centric, the promise of containers and Kubernetes is portability.  The possibility of moving from one cloud (or on-prem infrastructure) to another, of distributing the workloads on different platforms. At the end, this DevOps Stack is simply a "Cloud Agnostic Kubernetes Distribution".

The illustration above tries to easily show that if it is relatively simple to manage one Kubernetes project, the difficulties become significant when you add more and more projects and you want to keep a certain homogeneity between the projects. The main problems observed are:

This is not a new IT problem. Two decades ago, it was not really easy to manage hundreds, or even thousands of machines in a homogeneous way. This problem has been successfully addressed by solutions like CFEngine, Puppet or more recently by Ansible. Today, we must adapt these principles for Kubernetes. There are many open source components out there, we just need a smart way to interconnect them.

By creating this DevOps Stack (a kind of abstraction layer), it becomes possible to instantiate all the components through a logical and coherent process. The main advantages are:

In the end, the DevOps Stack provides:

With the following main features:

A summary of good practices used daily and experiences accumulated over the time by the DevOps experts at Camptocamp.