Resilient

"Container as a Service" services enable you to get closer to "Serverless" without fundamentally impacting the way you organize or develop your applications.

Today, most Cloud Providers offer CaaS (e.g., CloudRun on GCP, AWS Lambda/ECS on AWS, ACS on Azure). These are ideal for teams that do not require extensive customization. Some of their features (e.g., scale to 0) enable significant maintenance and cost savings. 

Using a CaaS service today can be a great first step in modernizing your applications or creating a new one. 

CaaS is a real alternative to :

• Kubernetes is complex to set up and maintain.
• Serverless, because of the architectural transformation it implies

Another advantage of CaaS is that it is, by definition, less vendor-locked than other hosting services. Your application is packaged via a market standard (an OCI image) and can be deployed on any service supporting these OCI images, such as a future Kubernetes cluster.

We've observed several different behaviors when using these services: either they're entirely adopted and perfectly suited to our customers' needs, or they lack customization, and our customers are naturally pushed towards using Kubernetes. This is no longer an issue today, but 1 year ago, Cloud Run didn't support the CPU being "always allocated," and we couldn't use it for applications with background processes.

These services have become indispensable, and we recommend them. If using a Cloud Provider isn't an option, building your own CaaS using open-source technologies like Knative is always possible.

Karpenter is an autoscaling node for Kubernetes. It differentiates itself from its peers by offering to create its autoscaling configuration within the cluster itself.

In the world of node autoscaling for Kubernetes, two significant solutions exist today:

• Technology managed by its Cloud Provider, including GKE and its Autopilot mode
• Kubernetes Cluster Autoscaler (KCA) is deployed in its own cluster and is still widely used for AWS EKS.

AWS is the last major Cloud Provider not to provide managed autoscaling functionality. Their response to this lack was the development of Karpenter.

Unlike KCA, Karpenter will not rely on the Cloud Provider to create node groups but will manage them using CRDs. Combined with a GitOps tool such as ArgoCD, Karpenter brings a new level of flexibility to node group configuration.

As soon as at least one pod is in the Pending state and the Kubernetes scheduler cannot assign it to a node, Karpenter will provide the right capacity to accommodate this workload. This may involve one or more nodes with sufficient characteristics.

In addition, Karpenter comes with some exciting features:

• TTL for nodes: allows you to renew the nodes in your infrastructure, mainly to update them continuously.
• consolidation: Karpenter regularly calculates the consequences of deleting a node, which is committed if the cluster size is reduced as a result
• drift-detection: dynamic marking of nodes whose configuration is no longer aligned with its associated provisioner

Open source, KEDA enables resources deployed in Kubernetes to be scaled based on external events.

One of our main objectives as DevOps is ensuring that infrastructures can handle the load quickly. However, scaling our resources in anticipation is difficult, as we generally rely on CPU and RAM consumption. This is where KEDA (Kubernetes Event-Driven Autoscaling) makes the task much easier.

KEDA is a component that extends Kubernetes' event-based autoscaling capabilities. 

It monitors:

• queues 
• data flows
• messaging systems

By monitoring these, it is possible to trigger application scaling based on the event load of numerous services such as Kafka, RabbitMQ, Azure Service Bus, AWS SQS, and PubSub. It is, therefore, possible to start scaling a resource when many messages arrive upstream of a stream, for example, or even scaling to 0 when no messages are present.

KEDA is an excellent choice for event-driven autoscaling. There are alternatives, such as proprietary solutions offered by Cloud Providers like AWS Lambda, Azure Functions, or Google Cloud Functions. However, KEDA stands out for its open-source approach and compatibility with Kubernetes.

Kubernetes is the current community standard for scalable containerized application deployments.

Deploying an application in production involves solving a number of technological challenges, such as:

• Ensuring the reproducibility of deployments
• Address multiple replicas of your application to ensure resilience
• To be able to adapt treatment capacity to the load
• Incorporate external components into your deployment at a lower cost (i.e., without maintaining a whole series of additional deployment scripts).

Before Kubernetes and the era of containers, we would have used an army of bash or python scripts, Ansible playbooks, or even a setup to ensure application deployment. Today, all this is elegantly replaced by a CI pipeline to produce container images and Helm Charts for deployment.

We would also have used a separate VIP system for load balancing. This is now supported:

• Natively by Kubernetes for cluster-internal load balancing
• Via integrations with Cloud Providers for external load balancing
• Via extensions such as MetalLB on an on-premise infrastructure

The management of processing capacity and the installation of external tools is also facilitated by the use of its YAML interface and the various tools that have emerged from it, such as Kustomize and Helm.

Even so, Kubernetes maintenance does entail a certain burden, even when using a Cloud Provider's managed service. A CaaS will be lighter to maintain but less scalable. Indeed, it is often not possible to install additional controllers on CaaS.

A business continuity plan, or BCP, is the most pragmatic cloud architecture pattern for ensuring high resilience, exploiting the strengths of cloud providers.

A business continuity plan (BCP) guarantees infrastructure availability in the event of a disaster. It is essential for any infrastructure wishing to maintain a high level of availability and deliver an uninterrupted user experience. Before designing an architecture and considering implementing a BCP, you need to estimate a Recovery Time Objective (RTO) and a Recovery Point Objective (RPO) for your application.

• The Recovery Time Objective (RTO) is the maximum acceptable time between service interruption and service restoration.
• The recovery point objective (RPO) is the maximum acceptable time since the last data recovery point.

Public Cloud Providers offer several ways of guaranteeing a BCP. For example: 

• AWS regions are located in countries. There can be several per country, and they are subdivided into availability zones. 
• Availability zones group together several data centers. These data centers host the Cloud Provider's resources.

Depending on the objectives, the application will be hosted on :

• An availability zone within a region
• Several availability zones within a region, which is the most common approach  
• Multiple availability zones within multiple regions. This guarantees the highest availability but is also the hardest to maintain, as you need to ensure data consistency across several regions while keeping response times low.

At Padok, we recommend the implementation of BCPs rather than DRPs (Disaster Recovery Plans). These are costly to test, and rarely result in activatable levers, and we are convinced that it is vital that they are carried out at the Cloud Provider level.

The more resilient and partition-tolerant an infrastructure becomes, the more complicated data consistency becomes. This is the CAP theorem.

A technique for monitoring your applications that involves simulating a real user with robots to detect malfunctions.

Synthetics Monitoring is a technique for monitoring your applications that involves simulating a real user with robots to detect malfunctions. It contrasts with the classic but now outdated technique of checking infrastructure availability. This method no longer makes sense in highly distributed cloud architectures, where self-healing is present by design.

Generally speaking, priority is given to testing the critical paths of an application. This is the user path that represents the greatest business value. For example, for an online sales site, this would be the purchasing tunnel:  

• product search
• add to basket
• payment

This test will ensure that your customers can actually buy on your site. It also validates that your backend services, such as search, session storage, etc., are all operational during the test.

From a simple call to a backend API to a multi-step process, many tools are available to set up Synthetics Monitoring. The choice is vast, from paid tools such as Datadog, NewRelic, and DynaTrace to open-source tools like Blackbox Exporter (Prometheus Stack). Please note, however, that it's best to run these scenarios from outside the infrastructure to position yourself as a "real" client to your application and thus detect network malfunctions (outages, latency).

Beware of external dependencies, however: they can generate alerts you can't act on. Nevertheless, it is important to be aware if one of your suppliers is unavailable. In this case, we recommend setting up specific procedures to deal with such events and implementing a circuit breaker system in your applications. Your application will automatically go into maintenance, and you'll be able to have a specific treatment during your on-call periods. You can imagine being alerted only if the service is inaccessible for more than 1 hour.

Application monitoring should be a standard part of your development cycle. As with security and performance, you need to set up a monitoring phase to ensure your services run smoothly. Don't neglect maintenance and evolution either. You should never go into production without probes to warn you of malfunctions. Nothing is more frustrating than being warned by your customers without realizing it beforehand.

We recommend an "automatic" approach, creating probes for each deployed application. This is what we do on our projects, with the flexibility of the Kubernetes API and tools such as Blackbox exporter or Datadog via Crossplane. As a result, none of our projects goes into production without adequate monitoring to validate that the service is being delivered.

Synthetic Monitoring is essential to ensure that your application is up and running. You shouldn't consider going into production without this kind of monitoring.