Kubernetes - Your one stop shop for all things K8S

Kubernetes - Your one stop shop for all things K8S

Table of Contents

What is Kubernetes ?

If you're a developer or IT professional working with containers, chances are you've heard of Kubernetes. But what exactly is it, and why has it become such a popular choice for container orchestration? Kubernetes, often referred to as "K8s," is an open-source platform that automates the deployment, scaling, and management of containerized applications. It was originally designed by engineers at Google, who had years of experience running production workloads at scale using a system called Borg. When containers first started gaining traction, it became clear that there was a need for a tool that could help manage these lightweight, portable packages of software. Enter Kubernetes - a powerful platform that allows you to deploy your containerized apps across a cluster of machines, ensuring they run consistently and reliably. One of the key benefits of Kubernetes is its ability to abstract away the underlying infrastructure. Whether you're running your apps on-premises, in the cloud, or in a hybrid environment, Kubernetes provides a consistent way to manage them. This portability is a game-changer for organizations that want to avoid vendor lock-in and maintain flexibility in their infrastructure choices. But Kubernetes is more than just a container orchestrator. It also provides a rich set of features for managing the lifecycle of your applications, including:

  • Automatic scaling: Kubernetes can automatically scale your apps up or down based on usage, ensuring you always have the right amount of resources available.
  • Self-healing: If a container fails or a node goes down, Kubernetes will automatically reschedule the affected containers and bring your app back to a healthy state.
  • Load balancing: Kubernetes can distribute traffic across multiple containers, ensuring your app can handle high loads without crashing.
  • Declarative configuration: You define the desired state of your app in a configuration file, and Kubernetes takes care of making it a reality.

With all these features and more, it's no wonder Kubernetes has become the de facto standard for container orchestration. It's used by companies of all sizes, from startups to large enterprises, to run their mission-critical applications in production. If you're new to Kubernetes, the learning curve can be steep. But once you get the hang of it, you'll find that it makes managing your containerized apps a breeze. And with a thriving ecosystem of tools and services built around it, there's never been a better time to dive in and start learning.

Components

1: Kubernetes Architecture
Figure 1: Kubernetes Architecture. Image Courtesy of: simform.com

1. Cluster

A Kubernetes cluster is a set of worker machines, called nodes, that run containerized applications. It is managed by the control plane, which includes components like the API Server, Scheduler, and Controller Manager [1][2][3].

2. Nodes

Physical or virtual machines that make up the Kubernetes cluster. Nodes are the worker machines that run containerized applications. Each node hosts pods, which are the smallest unit provided by Kubernetes to manage containerized workloads.

2.1. Worker Node

A worker node is a node that runs pods and is managed by the control plane. It includes the following components:

2.1.1. kubelet

The kubelet is an agent that runs on each node in the cluster. It ensures that containers are running in a pod and manages the lifecycle of containers. It periodically interacts with the API server to retrieve the desired state of containers and pods and reconciles any differences between the desired and current states[1][2][3].

2.1.2. kube-proxy

Kube-proxy is a network proxy that runs on each node in the cluster. It maintains network rules, which allow network communication to pods from network sessions inside or outside of the cluster. It also provides load-balancing features by dispersing incoming requests over different service endpoints[1][2][3].

2.1.3. Container Runtime

The container runtime is a fundamental component that empowers Kubernetes to run containers effectively. It is responsible for managing the execution and lifecycle of containers within the Kubernetes environment[1][3].

3. Control Plane

The control plane is responsible for managing the entire Kubernetes cluster. It includes the following components:

3.1. Controller Manager

The Controller Manager is a control loop that monitors and regulates the state of a Kubernetes cluster. It receives information about the current state of the cluster and objects within it and sends instructions to move the cluster towards the cluster operator's desired state [1][4].

3.2. Scheduler

The Scheduler is responsible for scheduling pods on specific nodes according to automated workflows and user-defined conditions. It ensures that pods are assigned to nodes with the necessary resources and constraints [1][4].

3.3. etcd

etcd is a distributed key-value store that stores and replicates the Kubernetes cluster state. It is the primary datastore of Kubernetes [1][2][4].

3.4. API Server

The API Server provides an API that serves as the front end of a Kubernetes control plane. It handles external and internal requests, determining whether a request is valid and then processing it. It can be accessed via the kubectl command-line interface or other tools[1][4].

4. Pods

Pods are the smallest unit provided by Kubernetes to manage containerized workloads. A pod typically includes several containers, which together form a functional unit or microservice. Pods are hosted on worker nodes and are managed by the control plane [1][3][4].

Analogy

Control Plane

The control plane is like the main headquarters of the military operation. It's where the generals and commanders work together to manage the entire army and make sure everything is running smoothly. They give orders, coordinate the different units, and ensure the army is ready for battle.

Nodes

Nodes are like the individual soldiers in the army. They are the actual machines or computers that do the work and carry out the orders from the control plane. Each soldier (node) has their own equipment and supplies to get the job done.

Pods

Pods are like the small groups or squads of soldiers working together. In Kubernetes, pods are the smallest units that can be deployed and managed. They are made up of one or more containers (soldiers) that work together to complete a specific task or mission.

API Server

The API server is like the communication center of the army. It's where all the messages and orders are sent and received. It makes sure only authorized people can send orders and that everyone gets the right information.

Scheduler

The scheduler is like the logistics officer who decides where each soldier should go and what they should do. They look at factors like what equipment is needed, where the soldiers work best together, and where they shouldn't be placed.

etcd

etcd is like the central storage room where all the important documents and plans are kept. It's a secure place to store all the information about the army and the missions they are working on.

Kubelet

The kubelet is like the squad leader who is in charge of a small group of soldiers. They make sure their soldiers (containers) are ready for battle, have the right equipment, and are following orders from the control plane.

Kube-proxy

Kube-proxy is like the traffic controller who makes sure all the soldiers can communicate with each other and with the enemy (other networks) as needed. They set up the rules for how the soldiers can talk to each other.

Container Runtime

The container runtime is like the equipment manager who makes sure all the soldiers have the right gear and supplies to do their jobs. They provide the containers (soldiers) with the resources they need to run properly.

Citations:

The Different Types of Objects in Kubernetes

Kubernetes is a powerful container orchestration system that manages and organizes clusters of containers. It uses various types of objects to achieve this task. These objects can be categorized into two main groups: Primitive Objects and Controller Objects.

Primitive Objects

Primitive objects are the basic building blocks of Kubernetes. They include:

Namespaces

Namespaces are used to separate groups of objects by categories such as user, team, or project. Each namespace contains its own objects and serves as a basis for access control and resource quotas.

Nodes

Nodes represent physical or virtual worker machines where the kubelet, kube proxy, and Docker run. Nodes can be annotated and labeled to specify workload affinity and constraints.

Pods

Pods are groups of containers that are co-located, co-scheduled, and run on the same node. Each pod is assigned a unique, virtual IP address that allows pods to communicate with each other over the cluster network. Pods are immutable, meaning they cannot be changed or moved once created.

Volumes

Volumes are abstractions that allow different block and file storage types to be presented and mounted to pods. Volumes support various file storage systems such as AWS EBS, Azure block store, Git, NFS, GlusterFS, and Ceph.

ConfigMaps and Secrets

ConfigMaps and Secrets are objects that contain key-value pairs used to configure applications. ConfigMaps are used for non-sensitive configuration data, while Secrets are used for sensitive data like passwords, certificates, and private keys.

Controller Objects

Controller objects are used to manage and orchestrate the cluster. They include:

ReplicaSets

ReplicaSets ensure that a specified number of replicas of a pod are running at any given time. They manage the availability of the required number of identical pods.

Services

Services define a set of pods that work together and provide a network identity and load balancing for accessing the pods. Services can be used for both stateless and stateful applications.

StatefulSets

StatefulSets are used to manage stateful applications, ensuring uniqueness and ordering of the instances of a pod. They are necessary for applications that require persistent storage and consistent ordering of instances.

DaemonSets

DaemonSets ensure that a specified number of replicas of a pod are running on each node in the cluster. They are used for applications that need to run on every node in the cluster.

Deployments

Deployments manage the rollout of new versions of an application by creating and scaling replicas of pods. They provide a way to manage the deployment of applications in a controlled manner.

These objects work together to provide a robust and scalable container orchestration system that can manage complex applications and ensure high availability.

Kubernetes Concepts

Deployments:

In Kubernetes (k8s), a deployment is a resource object that manages the rollout of new versions of an application. It ensures that the desired state of the application is maintained by creating and managing replicas of pods. Deployments provide a way to automate the deployment process, ensuring that the application is always running the desired version [1][3][5].

Key Features of Deployments

  1. Declarative Configuration: Deployments use a declarative configuration style, where you define the desired state of the application and Kubernetes ensures that the state is maintained [1][3].
  2. Replica Management: Deployments manage replicas of pods, which ensures that the desired number of instances of the application is running at all times [1][3].
  3. Rollout and Rollback: Deployments provide a way to rollout new versions of the application and roll back to previous versions if needed [1][3].
  4. Self-healing: Deployments can automatically restart pods if they fail, ensuring that the application remains available [1][3].

Deployment Strategies

Kubernetes provides several deployment strategies, including:

  1. Recreate Deployment: This strategy kills the currently running pods and creates new ones with the latest version [1][3].
  2. Rolling Update Deployment: This strategy updates pods one by one, ensuring that the application remains available during the update process [1][3].

Benefits of Deployments

  1. Automation: Deployments automate the deployment process, reducing the need for manual intervention [1][3].
  2. Scalability: Deployments provide a way to scale the application horizontally and vertically, ensuring that it can handle increased traffic and load [1][3].
  3. High Availability: Deployments ensure that the application remains available even if individual pods fail, by automatically restarting them [1][3].

Creating a Deployment

To create a deployment, you can use the Kubernetes command-line tool, kubectl. You can define the deployment using a YAML or JSON file and then apply it to the cluster using kubectl apply [3][5].

Citations:

Namespaces

What are Namespaces?

Namespaces are a way to divide a Kubernetes cluster into multiple virtual clusters. Each namespace is a logical isolation of resources within the cluster. Namespaces are used to separate resources and manage them independently, making it easier to manage and organize resources within a cluster[1][2][3].

How do Namespaces Work?

Namespaces work by providing a scope for names. Names of resources must be unique within a namespace, but not across namespaces. This means that you can have the same name for a resource in different namespaces. Namespaces also provide a mechanism for attaching authorization and policy to a subsection of the cluster. This allows you to control access to resources within a namespace[1][2][3].

When to Use Multiple Namespaces?

Namespaces are intended for use in environments with many users spread across multiple teams, or projects. For clusters with a few to tens of users, you should not need to create or think about namespaces at all. Start using namespaces when you need the features they provide, such as isolation of resources and control over access to resources[4].

Benefits of Namespaces

Namespaces provide several benefits, including:

  1. Isolation of Resources: Namespaces provide a way to isolate resources within a cluster, making it easier to manage and organize resources.
  2. Control over Access: Namespaces provide a mechanism for attaching authorization and policy to a subsection of the cluster, allowing you to control access to resources within a namespace.
  3. Organization of Resources: Namespaces help to organize resources within a cluster, making it easier to manage and track resources.
  4. Security: Namespaces provide a way to isolate resources and control access to resources, making it easier to maintain security within a cluster[1][2][3].

Creating and Managing Namespaces

Namespaces can be created and managed using the kubectl command-line tool. You can create a namespace using the following command:

kubectl create namespace <namespace-name>

You can also create a namespace using a YAML file:

namespace.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: <namespace-name>

You can then apply the YAML file to create the namespace:

kubectl apply -f namespace.yaml

Viewing Namespaces

You can view the current namespaces in a cluster using the following command:

kubectl get namespace

Setting the Namespace Preference

You can set the namespace preference for a current request using the --namespace flag:

kubectl run nginx --image=nginx --namespace=<insert-namespace-name-here>

You can also set the namespace preference permanently using the kubectl config command:

kubectl config set-context --current --namespace=<insert-namespace-name-here>

Citations:

Pods

Kubernetes pods are the smallest deployable computing units in Kubernetes, an open-source system for container scheduling, orchestration, and management. A pod encapsulates one or more containers running instances of an application. Pods are ephemeral by nature, meaning that if a pod (or the node it executes on) fails, Kubernetes can automatically create a new replica of that pod to continue operations.

Ephemeral Nature of Pods

Pods are ephemeral because they are designed to be short-lived and can be easily recreated if they fail. This ephemeral nature allows Kubernetes to manage pods efficiently and ensure high availability. When a pod fails, Kubernetes can automatically create a new replica of that pod to continue operations, ensuring minimal disruption to the application.

Worker Containers and Sidercar/Helper Containers in Pods

A pod can contain multiple containers, including worker containers and helper containers. Worker containers are the primary containers that run the application, while helper containers, also known as sidecars, provide additional services or functionality to the application. For example, a pod might contain a worker container running a web server and a sidecar container running a load balancer.

Pod Creation Lifecycle

2: Pod Creation Lifecycle
Figure 2: Pod Creation Lifecycle. Image Courtesy of: belowthemalt

Here is the step-by-step explanation of the pod creation lifecycle in Kubernetes, involving the API Server, kubelet, etcd, and container runtime:

  1. User creates a pod definition (e.g. using kubectl run or a YAML file) and submits it to the Kubernetes API server.
  2. The API server validates the pod definition and writes it to etcd (the cluster's storage backend).
  3. The API server returns a success response to the user, indicating that the pod has been accepted into the cluster.
  4. The API server notifies the scheduler that a new pod is available.
  5. The scheduler selects a node for the pod to run on, based on available resources and scheduling policies.
  6. The scheduler updates the pod definition with the selected node and writes it back to etcd.
  7. The kubelet on the selected node watches etcd for new/updated pod definitions.
  8. The kubelet notices the new pod assignment and pulls the necessary container images.
  9. The kubelet creates the pod's cgroups and network namespace using the container runtime interface (CRI).
  10. The kubelet invokes the container runtime (e.g. containerd, CRI-O) to create the containers.
  11. The container runtime creates the containers using the OCI (Open Container Initiative) runtime (e.g. runc).
  12. The kubelet starts the containers and streams their logs and status back to the API server.
  13. The API server updates the pod status in etcd to "Running".
  14. The kubelet periodically runs liveness and readiness probes to ensure the containers are healthy.
  15. If the pod or containers fail, the kubelet reports the failure to the API server.
  16. The API server updates the pod status in etcd to "Failed".
  17. If the pod is part of a higher-level controller (e.g. Deployment), the controller creates a new pod to replace the failed one.
  18. When the pod completes successfully, the containers exit and the kubelet reports the "Succeeded" status to the API server.

Pod Deletion Lifecycle

3: Pod Deletion Lifecycle
Figure 3: Pod Deletion Lifecycle. Image Courtesy of: blog.heptio

The pod deletion lifecycle involves the following steps:

  1. User Requests Deletion: The user submits a request to delete a pod using the kubectl delete command.
  2. API Server Validates: The API server validates the request and ensures that the user has the necessary permissions to delete the pod.
  3. API Server Updates Etcd: The API server updates etcd to reflect the pod as deleted.
  4. Kubelet on the Node: The kubelet on the node where the pod is running receives the update from etcd and attempts a graceful pod shutdown.
  5. PreStop Hook: If a preStop hook is defined in the pod configuration, the kubelet runs the hook to allow the pod to perform any necessary cleanup before termination.
  6. Container Termination: The kubelet terminates the containers in the pod. If the containers are running, they are stopped. If they are not running, they are removed.
  7. PostStop Hook: If a postStop hook is defined in the pod configuration, the kubelet runs the hook to allow the pod to perform any necessary cleanup after termination.
  8. Pod Deletion: The pod is fully deleted from the cluster, and its resources are released.
  9. Garbage Collection: Kubernetes performs garbage collection to reclaim the freed resources, including releasing IP addresses, removing associated volumes, and updating relevant status.

Pod Deletion States

A pod can be in one of the following states during deletion:

  • Pending: The pod is being deleted, but the deletion process has not yet completed.
  • Running: The pod is being deleted, and the containers are still running.
  • Succeeded: The pod has been fully deleted, and its resources have been released.
  • Failed: The pod deletion failed, and the pod remains in the cluster.

Pod Deletion Options

The pod deletion process can be influenced by various options, including:

  • Termination Grace Period: The time allowed for the pod to shut down before being forcefully terminated.
  • PreStop Hook: A hook that runs before the pod is terminated to allow cleanup.
  • PostStop Hook: A hook that runs after the pod is terminated to allow cleanup.
  • Force Deletion: The option to force the deletion of the pod without attempting a graceful shutdown.

Init Container

Init containers are specialized containers in a Kubernetes Pod that are designed to run before the main application containers start. They are used to perform setup tasks, such as initialization, configuration, or data preparation, before the main application containers are launched. Init containers are executed sequentially, and each init container must complete successfully before the next one starts. They are typically short-lived and handle initialization tasks, while main containers handle the core application logic.

Init containers are useful for performing tasks such as database schema setup, data preparation, or any other initialization steps required before the main application can start. They help ensure that the main container doesn't start until its dependencies are ready.

Key Features of Init Containers

  • Sequential Execution: Init containers are executed sequentially, and each init container must complete successfully before the next one starts.
  • Short-lived: Init containers are typically short-lived and handle initialization tasks, while main containers handle the core application logic.
  • Initialization Tasks: Init containers are used to perform setup tasks, such as initialization, configuration, or data preparation, before the main application containers are launched.
  • Error Handling: If an init container fails, the pod enters a failure state, and the init containers are re-executed before the main container starts.

Use Cases for Init Containers

  • Database Schema Setup: Init containers can be used to set up the database schema before the main application starts.
  • Data Preparation: Init containers can be used to prepare data for the main application.
  • Configuration Setup: Init containers can be used to set up configuration files or environment variables for the main application.
  • Dependency Management: Init containers can be used to manage dependencies required by the main application.

Selectors

What are Selectors in Kubernetes?

Selectors are used by users to select a set of objects based on their labels. They are a core grouping primitive in Kubernetes and are used to filter objects based on their labels. There are two types of selectors in Kubernetes: equality-based selectors and set-based selectors.

What are Labels in Kubernetes?

Labels are key-value pairs that can be attached to Kubernetes objects such as pods, nodes, and deployments. They serve to provide attributes that help create meaningful Kubernetes objects. Labels can be added to an object at creation time and can be added or modified at runtime. They are used as identifying attributes for objects such as pods and replication controllers.

Equality-Based Selectors

Equality-based selectors allow filtering by key and value. Matching objects should satisfy all the specified labels. For example, you can use an equality-based selector to select pods with the label environment set to test:

bashkubectl get pods --selector environment=test

Set-Based Selectors

Set-based selectors allow filtering of keys according to a set of values. For example, you can use a set-based selector to select pods with a label service-type set to either frontend or backend :

bashkubectl get pods --selector service-type in (frontend, backend)

Example Yaml files illustrating selectors:

selectors-example.yaml
---
apiVersion: v1
kind: Pod
metadata:
  name: my-pod
  labels:
    app: my-app
    type: frontend
spec:
  containers:
  - name: my-container
    image: nginx:latest
    ports:
    - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: my-service
spec:
  selector:
    app: my-app
    type: frontend
  ports:
  - port: 80
    targetPort: 80

When you apply this YAML to your Kubernetes cluster, it will:

  1. Create a pod named my-pod with the specified labels.
  2. Create a service named my-service that selects the my-pod pod based on the labels.
  3. The service will forward traffic from port 80 to the my-pod pod on port 80.

Sidenote on nodeSelectors:

A nodeSelector is a feature in Kubernetes that allows you to specify which nodes a pod should run on based on the labels attached to those nodes. It is a simple way to constrain the Kubernetes scheduler to schedule pods on specific nodes that match the specified labels.

Here is a detailed breakdown of how nodeSelectors work:

  1. Labeling Nodes: You need to label the nodes in your Kubernetes cluster with the desired labels. This can be done using the kubectl label nodes command.
  2. Specifying NodeSelector: In the PodSpec, you can specify a nodeSelector that matches the labels on the nodes. This is done by adding a nodeSelector field to the PodSpec and specifying the key-value pairs that match the labels on the nodes.
  3. Scheduling: When the Kubernetes scheduler tries to schedule a pod, it checks the nodeSelector specified in the PodSpec and schedules the pod on a node that matches the specified labels.

Here is an example of how to use a nodeSelector:

nodeselector-example.yaml
apiVersion: v1
kind: Pod
metadata:
  name: my-pod
spec:
  containers:
  - name: my-container
    image: nginx:latest
  nodeSelector:
    disktype: ssd

In this example, the pod will only be scheduled on nodes that have a label with the key disktype and the value ssd.

NodeSelectors are useful when you need to schedule pods on specific nodes based on the labels attached to those nodes.

Multi-Container Pods

In Kubernetes, a multi-container pod is a pod that runs multiple containers. This allows you to run multiple services or applications within a single pod, which can be useful for a variety of scenarios.

Sidecar Pattern

The sidecar pattern is a design pattern where a container runs alongside another container in the same pod. This is useful for scenarios where you need to run a service or application that provides additional functionality to another service or application.

Real-life Scenario: Imagine you have a web application that generates logs. You can use the sidecar pattern to run a log-shipping container alongside your web application container. The log-shipping container can continuously read logs from the web application container and forward them to a centralized logging system.

Here is an example of a sidecar pattern in YAML:

sidecar-example.yaml
apiVersion: v1
kind: Pod
metadata:
  name: my-pod
spec:
  containers:
  - name: my-app
    image: my-app:latest
  - name: my-sidecar
    image: my-sidecar:latest

Adapter Pattern

The adapter pattern is a design pattern where a container acts as an adapter between two other containers. This is useful for scenarios where you need to run a service or application that provides additional functionality to another service or application.

Real-life Scenario: Imagine you have a legacy application that expects data in a specific format, but your new application generates data in a different format. You can use the adapter pattern to run a container that converts the data format between the two applications.

Here is an example of an adapter pattern in YAML:

adapter-example.yaml
apiVersion: v1
kind: Pod
metadata:
  name: my-pod
spec:
  containers:
  - name: my-app
    image: my-app:latest
  - name: my-adapter
    image: my-adapter:latest
    env:
    - name: MY_APP_PORT
      value: "8080"

Ambassador Pattern

The ambassador pattern is a design pattern where a container acts as an ambassador for another container. This is useful for scenarios where you need to run a service or application that provides additional functionality to another service or application.

Real-life Scenario: Imagine you have a database application that needs to be accessed by multiple clients. You can use the ambassador pattern to run a container that acts as an ambassador for the database application. The ambassador container can handle connection pooling, load balancing, and other functionality that the clients need.

Here is an example of an ambassador pattern in YAML:

ambassador-example.yaml
apiVersion: v1
kind: Pod
metadata:
  name: my-pod
spec:
  containers:
  - name: my-app
    image: my-app:latest
  - name: my-ambassador
    image: my-ambassador:latest
    ports:
    - containerPort: 8080

Refer Top Kubernetes Design Patterns for more !

Workloads

A Kubernetes workload is a set of pods and containers that run an application or service on a Kubernetes cluster, defining how the pods and containers should be created, updated, exposed, and configured.

  1. Deployments: These define how to create and update a set of pods. They provide rolling updates and rollbacks, ensuring zero-downtime deployments and easy failure recovery.
  2. Services: These define how to access a group of pods. They provide load balancing and service discovery features, enabling communication and interaction between workloads and external clients or systems.
  3. ConfigMaps and Secrets: These store and manage configuration data and sensitive information, such as environment variables, properties, or files. They allow for decoupling configuration from the application code and can be mounted as files or environment variables.
  4. ReplicaSets: These ensure that a specified number of pods are running at any given time. They provide basic scaling and availability features, acting as a self-healing mechanism by automatically replacing failed or deleted pods.
  5. StatefulSets: These provide guarantees around the ordering and unique identity of pods in a set. They are used to manage stateful applications, such as databases, where each pod in a set must have a unique identity and persistent storage.
  6. Jobs: These are used to run a finite number of pods to completion, such as for batch processes or one-time tasks. They ensure that a specified number of pods run to completion and clean up after themselves.
  7. CronJobs: These allow you to run a Job on a schedule, such as running a Job every hour. They provide a way to run Jobs at specific times or on a recurring schedule

ReplicaSet, StatefulSet, DaemonSet, Deployment, Tasks (Jobs and Cron Jobs)

ReplicaSets Example:

replicaset-example.yaml
apiVersion: apps/v1
kind: ReplicaSet
metadata:
  name: nginx-replicaset
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80

This ReplicaSet ensures that 3 replicas of the nginx:1.14.2 container are running at all times. It uses the matchLabels selector to identify the pods it manages, and the template section defines the pod specification.

The key differences between ReplicaSets and StatefulSets are:

  • StatefulSets provide stable network identities and persistent storage for each pod, while ReplicaSets do not.
  • StatefulSets create pods in a specific order and assign unique identifiers, while ReplicaSets create pods in parallel with random names.
  • StatefulSets are suitable for stateful applications like databases, while ReplicaSets are more commonly used for stateless applications.

StatefulSet

In Kubernetes, StatefulSets rely on several key concepts to provide stable network identities and persistent storage for stateful applications:

Headless Service

A headless service is a special type of Kubernetes service that does not load balance or provide a cluster IP. Instead, it provides a stable network identity for each pod in a StatefulSet.

When you create a StatefulSet, you must also create a corresponding headless service. The headless service is responsible for the network identity of the pods in the StatefulSet. It provides a stable domain name for each pod, allowing other applications to reliably connect to the pods.

PersistentVolumeClaim (PVC)

A PersistentVolumeClaim (PVC) is a request for storage by a user. It specifies the size and access mode of the desired storage volume.

In a StatefulSet, each pod has a corresponding PVC that is used to claim persistent storage. The PVC is defined in the volumeClaimTemplates section of the StatefulSet specification. When a pod is created, a new PVC is dynamically provisioned based on the template.

The PVC remains bound to the pod even if the pod is rescheduled to a different node, ensuring that the pod always has access to its persistent storage.

StorageClass

A StorageClass is a way for administrators to define different classes of storage. Each StorageClass has a provisioner that determines how volumes are created and what type of storage they use[3].

When defining a PVC in a StatefulSet, you can specify a StorageClass. The StorageClass determines the type of storage that will be provisioned for the PVC. For example, you might have a "fast" StorageClass that uses SSD storage and a "slow" StorageClass that uses HDD storage.

If no StorageClass is specified, the default StorageClass will be used. The default StorageClass can be set by an administrator using the storageclass.kubernetes.io/is-default-class annotation.

In summary, headless services provide stable network identities for pods in a StatefulSet, PVCs claim persistent storage for each pod, and StorageClasses define the type of storage that will be provisioned for the PVCs. These concepts work together to enable StatefulSets to manage stateful applications in Kubernetes.

StatefulSet Example

statefulset-example.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: web
spec:
  serviceName: nginx
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: k8s.gcr.io/nginx-slim:0.8
        ports:
        - containerPort: 80
        volumeMounts:
        - name: www
          mountPath: /usr/share/nginx/html
  volumeClaimTemplates:
  - metadata:
      name: www
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 1Gi

This StatefulSet creates a stable network identity for each pod by using the serviceName and a predictable naming convention. It also provides persistent storage for each pod using volumeClaimTemplates.

Job

Jobs are used to run batch processes or one-time tasks. They ensure that a specified number of Pods successfully terminate. Jobs can be run in serial or parallel mode, depending on your requirements. Serial mode ensures that Pods are executed one after the other, while parallel mode allows multiple Pods to run concurrently.

Job Example

job-example.yaml
apiVersion: batch/v1
kind: Job
metadata:
  name: my-job
spec:
  template:
    spec:
      containers:
      - name: my-container
        image: my-image
      restartPolicy: OnFailure
  backoffLimit: 4

This Job runs a single container named my-container from the my-image image. The restartPolicy is set to OnFailure, which means that the container will be restarted if it fails. The backoffLimit is set to 4, which means that the Job will retry the container up to 4 times if it fails.

These YAML examples demonstrate how to define Jobs and CronJobs in Kubernetes.

CronJobs

CronJobs are an extension of Jobs that allow you to schedule recurring tasks. They are based on the cron format and can be used to run tasks at specific intervals, such as daily, weekly, or monthly. CronJobs are useful for automating tasks like backups, cleanup scripts, or any other recurring processes.

CronJob Example

cronjob-example.yaml
apiVersion: batch/v1
kind: CronJob
metadata:
  name: my-cronjob
spec:
  schedule:
    - cron: 0 0 * * * * # Run every day at midnight
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: my-container
            image: my-image
          restartPolicy: OnFailure
  successfulJobsHistoryLimit: 3
  failedJobsHistoryLimit: 1

This CronJob runs a Job every day at midnight. The Job runs a single container named my-container from the my-image image. The restartPolicy is set to OnFailure, which means that the container will be restarted if it fails. The successfulJobsHistoryLimit is set to 3, which means that the CronJob will keep track of the last 3 successful runs. The failedJobsHistoryLimit is set to 1, which means that the CronJob will keep track of the last 1 failed run.

Kubernetes Updates: Keeping Your Applications Fresh

Kubernetes provides various update strategies to help you manage application updates seamlessly. The most common update strategy is rolling updates, which allow you to update your application gradually without downtime.Rolling updates work by creating a new ReplicaSet with the updated image, scaling up the new ReplicaSet, and scaling down the old ReplicaSet. This process continues until all Pods have been updated. Rolling updates ensure that your application remains available during the update process, making them suitable for most use cases.

Here are YAML examples for rolling updates, recreating updates, and blue-green updates in Kubernetes:

Rolling Update Example

rollingupdate-strategy-example.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: rollingupdate-strategy
spec:
  replicas: 3
  selector:
    matchLabels:
      app: rollingupdate-strategy
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 1
  template:
    metadata:
      labels:
        app: rollingupdate-strategy
    spec:
      containers:
      - name: rollingupdate-strategy
        image: hello-world:nanoserver-1809
        ports:
        - containerPort: 80

Recreate Update Example

recreate-strategy-example.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: recreate-strategy
spec:
  replicas: 3
  selector:
    matchLabels:
      app: recreate-strategy
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: recreate-strategy
    spec:
      containers:
      - name: recreate-strategy
        image: hello-world:nanoserver-1809
        ports:
        - containerPort: 80

Blue-Green Update Example

blue-green-strategy-example.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: blue-green-strategy
spec:
  replicas: 3
  selector:
    matchLabels:
      app: blue-green-strategy
  strategy:
    type: BlueGreen
  template:
    metadata:
      labels:
        app: blue-green-strategy
    spec:
      containers:
      - name: blue-green-strategy
        image: hello-world:nanoserver-1809
        ports:
        - containerPort: 80

Kubernetes Services: What, Why and When?

In Kubernetes, a Service is a logical abstraction that defines a set of Pods and a network interface to access them. This abstraction allows for load balancing, network policies, and other features that make managing and scaling applications easier. In this section, we will explore the different types of Services in Kubernetes, including ClusterIP, NodePort, and LoadBalancer Services, as well as Persistent Volumes and StorageClasses for storage and persistence.

Service vs Pod

A Service is a higher-level abstraction than a Pod. While a Pod represents a single instance of a containerized application, a Service represents a set of Pods that can be accessed as a single unit. This allows for load balancing, network policies, and other features that make it easier to manage and scale applications.

Services Target Pods Using Selectors

Services target Pods using selectors, which are key-value pairs that match specific labels on the Pods. This allows Services to automatically discover and manage the Pods that make up the application.

Services Load Balance Requests to Pods

Services load balance requests to Pods, which ensures that the application remains available even if one or more Pods fail or become unavailable. This is particularly important for applications that require high availability and scalability.

ClusterIp Service

The ClusterIP Service is the default internal cluster service in Kubernetes. It provides a stable virtual IP address that can be used to access the Pods that the Service targets. This virtual IP address is only accessible from within the cluster, making it ideal for internal communication between different components of an application.The ClusterIP Service uses selectors to identify the Pods it should target, and it also specifies a target port that the Pods should be listening on. This allows the Service to load balance requests across the selected Pods, ensuring that the application remains available even if one or more Pods fail or become unavailable.One of the key features of the ClusterIP Service is its ability to provide a stable network interface for the Pods it targets. This is particularly important for applications that need to communicate with each other, as it allows them to rely on a consistent network address regardless of the underlying infrastructure.

TargetPort

The TargetPort is the port on the Pods that the Service should forward traffic to. This allows the Service to abstract away the specific port that the application is listening on, making it easier to manage and scale the application.For example, if your application is running on port 8080 inside the Pods, you can configure the ClusterIP Service to use a TargetPort of 8080, even if the Service itself is listening on a different port (e.g., port 80 for HTTP traffic).

Example of ClusterIP Service Using YAML Files

Here are examples of YAML files for a Deployment and a ClusterIP Service:

Deployment YAML File

deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-deployment
spec:
  replicas: 2
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
      - name: my-deployment-container
        image: nginx

ClusterIP Service YAML File

cluster_ip_service.yaml
apiVersion: v1
kind: Service
metadata:
  name: my-service
spec:
  selector:
    app: my-app
  ports:
  - name: http
    port: 80
    targetPort: 8080
  type: ClusterIP

Explanation of the YAML Files

  1. Deployment YAML File:

    • This YAML file defines a Deployment named my-deployment with two replicas.
    • The Deployment targets Pods labeled with app: my-app.
    • The Pods run the nginx image.

  2. ClusterIP Service YAML File:

    • This YAML file defines a Service named my-service of type ClusterIP.
    • The Service targets Pods labeled with app: my-app.
    • The Service exposes port 80 and forwards traffic to port 8080 on the Pods.

Creating the Deployment and Service

To create the Deployment and Service, you can use the following commands:

kubectl create -f deployment.yaml
kubectl create -f cluster_ip_service.yaml

Checking the Service

After creating the Service, you can check its status using the following command:

kubectl get svc

This will display the details of the Service, including its IP address and port.

NodePort Service

The NodePort Service is another type of Service in Kubernetes that provides both internal and external visibility. Unlike the ClusterIP Service, which is only accessible from within the cluster, the NodePort Service exposes a port on each node in the cluster, allowing external traffic to reach the Pods.When a client makes a request to the NodePort, the request is forwarded to the appropriate Pod based on the Service's load balancing rules. This can be useful for exposing an application to external users or for integrating with other systems that need to access the application.One of the key advantages of the NodePort Service is its simplicity. It doesn't require any additional infrastructure, such as a load balancer, to provide external access to the application. This can make it a good choice for small-to-medium sized applications or for development and testing environments.

NodePort

In addition to the internal ClusterIP, the ClusterIP Service also supports the NodePort configuration. This allows the Service to be accessed from outside the cluster by exposing a port on each node in the cluster. The NodePort is a range of ports (30000-32767) that can be used to access the Service from outside the cluster.When a client makes a request to the NodePort, the request is forwarded to the appropriate Pod based on the Service's load balancing rules. This can be useful for exposing an application to external users or for integrating with other systems that need to access the application.

Example of Nodeport Service Using YAML Files

Here is an example of a NodePort Service in Kubernetes:

Deployment YAML File

deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
      - name: my-app
        image: lvthillo/python-flask-docker
        ports:
        - containerPort: 8080

NodePort Service YAML File

nodeport_service.yaml
apiVersion: v1
kind: Service
metadata:
  name: my-service
spec:
  selector:
    app: my-app
  type: NodePort
  ports:
  - name: http
    port: 80
    targetPort: 8080
    nodePort: 30080

Explanation of the YAML Files

  1. Deployment YAML File:

    • This YAML file defines a Deployment named my-app with two replicas.
    • The Deployment targets Pods labeled with app: my-app.
    • The Pods run the lvthillo/python-flask-docker image and expose port 8080.

  2. NodePort Service YAML File:

    • This YAML file defines a Service named my-service of type NodePort.
    • The Service targets Pods labeled with app: my-app.
    • The Service exposes port 80 and forwards traffic to port 8080 on the Pods.
    • The nodePort field specifies that the Service should be exposed on port 30080.

Creating the Deployment and Service

To create the Deployment and Service, you can use the following commands:

kubectl create -f deployment.yaml
kubectl create -f nodeport_service.yaml

Checking the Service

After creating the Service, you can check its status using the following command:

kubectl get svc

This will display the details of the Service, including its IP address and port.

output
NAME         TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP          1d
my-service   NodePort   10.105.152.19   <none>        80:30080/TCP     1m

Accessing the Service

To access the Service, you can use the following command:

curl http://<Node IP>:<NodePort>

This will display the output of the application.

Deleting the Service

To delete the Service, you can use the following command:

kubectl delete -f nodeport_service.yaml

LoadBalancer Service

The LoadBalancer Service is a type of Service that provides external visibility and load balancing for an application. It works by creating a load balancer in the cloud provider's infrastructure (e.g., AWS, GCP, Azure) and exposing a public IP address that can be used to access the application.When a client makes a request to the LoadBalancer Service, the request is forwarded to the appropriate Pod based on the Service's load balancing rules. The load balancer is responsible for distributing the traffic across the Pods, ensuring that the application remains available and responsive even under high load.One of the key advantages of the LoadBalancer Service is its ability to provide a highly available and scalable solution for exposing an application to the internet. By leveraging the cloud provider's load balancing infrastructure, the LoadBalancer Service can automatically scale up or down based on the incoming traffic, ensuring that the application can handle spikes in demand.

Example of LoadBalancer Service in Kubernetes

Here's an example of a LoadBalancer Service in Kubernetes:

Deployment YAML File

deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
      - name: my-app
        image: nginx
        ports:
        - containerPort: 80

LoadBalancer Service YAML File

loadbalancer_service.yaml
apiVersion: v1
kind: Service
metadata:
  name: my-loadbalancer
spec:
  type: LoadBalancer
  selector:
    app: my-app
  ports:
  - port: 80
    targetPort: 80

Explanation of the YAML Files

  1. Deployment YAML File:

    • This YAML file defines a Deployment named my-app with three replicas.
    • The Deployment targets Pods labeled with app: my-app.
    • The Pods run the nginx image and expose port 80.

  2. LoadBalancer Service YAML File:

    • This YAML file defines a Service named my-loadbalancer of type LoadBalancer.
    • The Service targets Pods labeled with app: my-app.
    • The Service exposes port 80 and forwards traffic to port 80 on the Pods.

Creating the Deployment and Service

To create the Deployment and Service, you can use the following commands:

kubectl create -f deployment.yaml
kubectl create -f loadbalancer_service.yaml

Checking the Service

After creating the Service, you can check its status using the following command:

kubectl get svc

This will display the details of the Service, including its external IP address.

Accessing the Application

Once the LoadBalancer Service is created, you can access the application using the external IP address of the LoadBalancer. The cloud provider (e.g., AWS, GCP, Azure) will automatically provision a load balancer and assign an external IP address to the Service.

For example, if the external IP address of the LoadBalancer Service is 123.45.67.89, you can access the application by visiting http://123.45.67.89 in your web browser.

The LoadBalancer Service will distribute incoming traffic across the three Pods created by the Deployment, providing load balancing and high availability for the application.

Storage and Persistence

Persistent Volumes and Persistent Volume Claims

Persistent Volumes (PVs) and Persistent Volume Claims (PVCs) are Kubernetes resources that provide a way to manage storage and persistence in your applications. PVs are resources that represent a piece of storage, while PVCs are requests for storage resources.PVs are created by administrators and can be of different types, such as local storage, network-attached storage (NAS), or cloud-based storage. PVCs, on the other hand, are created by users and specify the storage requirements for their applications, such as the size, access mode, and storage class.When a PVC is created, Kubernetes will automatically find a suitable PV that matches the PVC's requirements and bind them together. This allows the application to use the storage provided by the PV, without needing to know the details of the underlying storage infrastructure.

Reclaim Policies

Reclaim policies determine what happens to a PV when a PVC is deleted. There are three reclaim policies:

  1. Retain: The PV will remain in the cluster, and the data will be preserved. This is useful for scenarios where you want to reuse the same storage for a different application.
  2. Delete: The PV and the underlying storage will be deleted when the PVC is deleted. This is the default reclaim policy for many cloud-based storage providers.
  3. Recycle: The PV will be cleaned up (e.g., by running rm -rf /thevolume/*) and made available for reuse.

The choice of reclaim policy will depend on your specific use case and the requirements of your application.

Example of Persistent Volumes in Kubernetes

Here is an example of Persistent Volumes and Persistent Volume Claims for a Deployment:

Persistent Volume YAML File

pv.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
  name: my-pv
spec:
  capacity:
    storage: 5Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  local:
    path: /mnt/data
  storageClassName: local-storage

Persistent Volume Claim YAML File

pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 2Gi
  storageClassName: local-storage

Deployment YAML File

deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-deployment
spec:
  replicas: 2
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
      - name: my-app
        image: nginx
        volumeMounts:
        - name: my-pvc
          mountPath: /var/www/html
      volumes:
      - name: my-pvc
        persistentVolumeClaim:
          claimName: my-pvc

Explanation of the YAML Files

  1. Persistent Volume YAML File:

    • This YAML file defines a Persistent Volume named my-pv with a capacity of 5Gi and an access mode of ReadWriteOnce.
    • The Persistent Volume is provisioned locally on the node at the path /mnt/data.
    • The persistentVolumeReclaimPolicy is set to Retain, which means that the volume will be retained even after the pods using it are deleted.

  2. Persistent Volume Claim YAML File:

    • This YAML file defines a Persistent Volume Claim named my-pvc that requests a storage capacity of 2Gi with an access mode of ReadWriteOnce.
    • The storageClassName is set to local-storage, which matches the storage class of the Persistent Volume.

  3. Deployment YAML File:

    • This YAML file defines a Deployment named my-deployment with two replicas.
    • The Deployment targets Pods labeled with app: my-app.
    • The Pods run the nginx image and mount a Persistent Volume Claim named my-pvc at the path /var/www/html.

Creating the Persistent Volume and Persistent Volume Claim

To create the Persistent Volume and Persistent Volume Claim, you can use the following commands:

kubectl create -f pv.yaml
kubectl create -f pvc.yaml

Creating the Deployment

To create the Deployment, you can use the following command:

kubectl create -f deployment.yaml

Checking the Persistent Volume and Persistent Volume Claim

After creating the Persistent Volume and Persistent Volume Claim, you can check their status using the following commands:

kubectl get pv
kubectl get pvc

This will display the details of the Persistent Volume and Persistent Volume Claim, including their status and the pods that are using them.

StorageClasses

StorageClasses are used to define the characteristics of a PV. They can be used to define the type of storage, the size of the storage, and other properties, such as the underlying storage provider, the access mode, and the reclaim policy.StorageClasses provide a way to abstract the details of the underlying storage infrastructure, making it easier to manage and provision storage for your applications. By defining different StorageClasses, you can provide different levels of storage performance and durability to your applications, based on their specific needs.

Example of StorageClasses in K8S

Here is an example of a StorageClass, PVC, and Deployment:

StorageClass YAML File

storageclass.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: local-device
annotations:
  openebs.io/cas-type: local
  cas.openebs.io/config: |
    - name: StorageType
      value: device
    - name: FSType
      value: xfs
provisioner: openebs.io/local
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer

Persistent Volume Claim YAML File

pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: local-device-pvc
spec:
  storageClassName: local-device
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi

Deployment YAML File

deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mysql
labels:
  app: mysql
spec:
  selector:
    matchLabels:
      app: mysql
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
      - image: mysql:5.6
        name: mysql
        env:
        - name: MYSQL_ROOT_PASSWORD
          value: "password"
        ports:
        - containerPort: 3306
        name: mysql
        volumeMounts:
        - name: mysql-persistent-storage
          mountPath: /var/lib/mysql
      volumes:
      - name: mysql-persistent-storage
        persistentVolumeClaim:
          claimName: local-device-pvc

Explanation of the YAML Files

  1. StorageClass YAML File:

    • This YAML file defines a StorageClass named local-device that uses the openebs.io/local provisioner to create Persistent Volumes.
    • The StorageClass specifies the WaitForFirstConsumer binding mode and the Delete reclaim policy.

  2. Persistent Volume Claim YAML File:

    • This YAML file defines a Persistent Volume Claim named local-device-pvc that requests a storage capacity of 5Gi with an access mode of ReadWriteOnce.
    • The PVC uses the local-device StorageClass to provision the Persistent Volume.

  3. Deployment YAML File:

    • This YAML file defines a Deployment named mysql that runs the mysql:5.6 image.
    • The Deployment mounts a Persistent Volume Claim named local-device-pvc at the path /var/lib/mysql in the container.

Creating the StorageClass, PVC, and Deployment

To create the StorageClass, PVC, and Deployment, you can use the following commands:

kubectl apply -f storageclass.yaml
kubectl apply -f pvc.yaml
kubectl apply -f deployment.yaml

Checking the Persistent Volume and Persistent Volume Claim

After creating the Persistent Volume and Persistent Volume Claim, you can check their status using the following commands:

kubectl get pv
kubectl get pvc

This will display the details of the Persistent Volume and Persistent Volume Claim, including their status and the pods that are using them.

When to Use PV and When to Use SC

The choice between using PVs and StorageClasses will depend on your specific use case and the level of control you need over the storage infrastructure.If you have a specific storage requirement that needs to be managed at the cluster level, such as a shared file system or a high-performance storage volume, then using a PV may be the best approach. This allows you to configure the storage details directly and ensure that the application has access to the required storage resources.On the other hand, if you need to provide different levels of storage performance and durability to your applications, or if you want to abstract away the details of the underlying storage infrastructure, then using StorageClasses may be the better choice. This allows you to define and manage the storage characteristics at a higher level, making it easier to provision storage for your applications.In general, PVs are best suited for scenarios where you need to manage storage and persistence at the cluster level, while StorageClasses are better suited for scenarios where you need to provide different levels of storage performance and durability to your applications.

Application Settings

ConfigMap

ConfigMaps are API objects that store non-confidential data in key-value pairs. They are designed to decouple environment-specific configuration from container images, making it easier to manage and deploy applications across different environments. ConfigMaps can be consumed by pods as environment variables, command-line arguments, or as configuration files in a volume.

ConfigMaps do not provide secrecy or encryption, making them unsuitable for storing sensitive data. If you need to store confidential information, such as passwords or API keys, use Kubernetes Secrets instead. ConfigMaps are also vulnerable to unauthorized access if not properly secured. To mitigate this risk, implement role-based access control (RBAC) to manage who can access your ConfigMaps and use auditing to track changes.

  • ConfigMap
  • CM with Volumes for Dynamic Env Variables (Are variables secure this way) ?

ConfigMaps with Volumes

ConfigMaps can be used to provide dynamic environment variables to pods by mounting them as volumes. This allows pods to consume the ConfigMap data as environment variables, command-line arguments, or as configuration files. To use a ConfigMap with a volume, you need to create a ConfigMap and then mount it as a volume in your pod.

ConfigMap Example

Here is an example of a ConfigMap YAML file:

ConfigMap
apiVersion: v1
kind: ConfigMap
metadata:
  name: my-config
data:
  MY_VAR1: "value1"
  MY_VAR2: "value2"

ConfigMap with Volumes Example

Here is an example of a ConfigMap with volumes YAML file:

ConfigMap
apiVersion: v1
kind: Pod
metadata:
  name: my-pod
spec:
  containers:
  - name: my-container
    image: my-image
    volumeMounts:
    - name: config-volume
      mountPath: /config
  volumes:
  - name: config-volume
    configMap:
      name: my-config

Secrets

Secrets are designed to store sensitive information such as passwords, OAuth tokens, and SSH keys. They are base64 encoded but stored unencrypted by default. To enhance security, you can toggle "Enable Encryption at Rest for Secrets" to store them encrypted. Secrets are specifically designed for sensitive information and can be configured with a limited lifespan using Kubernetes Secret rotation policies.

Best Practices for Using ConfigMaps and Secrets

  1. Use Secrets for Sensitive Data: Store sensitive data such as passwords, API keys, and OAuth tokens in Secrets, not ConfigMaps. This ensures that your sensitive data is properly encrypted and protected.
  2. Implement Role-Based Access Control (RBAC): Use RBAC to manage who can access your ConfigMaps and Secrets. This ensures that only authorized users can view or modify your configuration data.
  3. Use Immutable ConfigMaps: Mark ConfigMaps as immutable to prevent accidental or malicious changes to their data. This feature is particularly useful for ConfigMaps that contain crucial configuration details.
  4. Store ConfigMaps Securely: Store ConfigMaps securely by using a key management system (KMS) to manage their encryption keys. This ensures that your configuration data remains protected even during a security breach.

Secrets Example

Here is an example of a Secrets YAML file:

secrets-example.yaml
apiVersion: v1
kind: Secret
metadata:
  name: my-secret
type: Opaque
data:
  MY_SECRET_VAR: <base64 encoded secret value>

Using ConfigMap with Volumes and Secrets

Here is an example of how to create Secrets:

# Create the ConfigMap
kubectl create configmap my-config --from-literal=MY_VAR1=value1 --from-literal=MY_VAR2=value2

# Create the Secret
kubectl create secret generic my-secret --from-literal=MY_SECRET_VAR=<base64 encoded secret value>

# Create the Pod
kubectl create -f pod.yaml

# Verify the Pod
kubectl get pods

Kubernetes Observability: Understanding Probes

Kubernetes provides several tools to ensure the health and availability of your applications. One of the key components in this process is the probe. Probes are used to check the health of a container by performing specific actions. In this article, we will explore the different types of probes, their configuration options, and how they are used in Kubernetes.

Types of Probes

Kubernetes supports three types of probes: startup probes, readiness probes, and liveliness probes.

Startup Probes

Startup probes are used to check if a container has started successfully. They are typically used in conjunction with readiness probes to ensure that a container is fully initialized before it starts accepting traffic.

Readiness Probes

Readiness probes are used to check if a container is ready to accept traffic. They are typically used to ensure that a container has completed its initialization and is ready to handle requests.

Liveliness Probes

Liveliness probes are used to check if a container is still running. They are typically used to detect if a container has crashed or is no longer responding.

Probe Configuration Options

Probes have several configuration options that can be used to customize their behavior.

InitialDelaySeconds

The initialDelaySeconds option specifies the delay before the probe is executed. This is useful for ensuring that the container has enough time to start before the probe is executed.

PeriodSeconds

The periodSeconds option specifies the interval at which the probe is executed. This is useful for ensuring that the probe is executed at regular intervals to check the health of the container.

FailureThreshold

The failureThreshold option specifies the number of consecutive failures before the container is considered unhealthy. This is useful for ensuring that the container is not considered unhealthy if there are occasional failures.

Probe Actions

Probes can be configured to perform different actions based on the outcome of the probe.

ExecAction

The ExecAction is used to execute a command inside the container. This is useful for checking the health of the container by executing a specific command.

TCPSocketAction

The TCPSocketAction is used to check if a container is listening on a specific port. This is useful for checking if a container is running and listening on a specific port.

HTTPGetAction

The HTTPGetAction is used to check if a container is responding to HTTP requests. This is useful for checking if a container is running and responding to HTTP requests.

Probe Example

Here is an example YAML file that demonstrates the use of all three types of probes in Kubernetes:

probe-example.yaml
apiVersion: v1
kind: Pod
metadata:
  name: my-app
spec:
  containers:
  - name: my-container
    image: my-image
    ports:
    - containerPort: 8080
    startupProbe:
      httpGet:
        path: /healthz
        port: 8080
      initialDelaySeconds: 30
      periodSeconds: 5
      failureThreshold: 10
    readinessProbe:
      tcpSocket:
        port: 8080
      initialDelaySeconds: 10
      periodSeconds: 3
      failureThreshold: 3
    livenessProbe:
      exec:
        command:
        - /bin/sh
        - -c
        - "curl -f http://localhost:8080/healthz"
      initialDelaySeconds: 15
      periodSeconds: 10
      failureThreshold: 5

Let's break down the different probe configurations in this example:

  1. Startup Probe:

    • Type: httpGet
    • Path: /healthz
    • Port: 8080
    • initialDelaySeconds: 30 (wait 30 seconds before starting the probe)
    • periodSeconds: 5 (check every 5 seconds)
    • failureThreshold: 10 (allow 10 consecutive failures before considering the container unhealthy)

  2. Readiness Probe:

    • Type: tcpSocket
    • Port: 8080
    • initialDelaySeconds: 10 (wait 10 seconds before starting the probe)
    • periodSeconds: 3 (check every 3 seconds)
    • failureThreshold: 3 (allow 3 consecutive failures before considering the container not ready)

  3. Liveness Probe:

    • Type: exec
    • Command: curl -f http://localhost:8080/healthz
    • initialDelaySeconds: 15 (wait 15 seconds before starting the probe)
    • periodSeconds: 10 (check every 10 seconds)
    • failureThreshold: 5 (allow 5 consecutive failures before considering the container unhealthy)

In this example, the startup probe checks the /healthz endpoint using an HTTP GET request, the readiness probe checks if the container is listening on port 8080 using a TCP socket, and the liveness probe executes a curl command to check the /healthz endpoint.

The different configuration options, such as initialDelaySeconds, periodSeconds, and failureThreshold, are set to ensure that the probes are executed at the appropriate times and with the desired level of tolerance for failures.

AutoScaling

Kubernetes Horizontal Pod Autoscaling (HPA): A Comprehensive Guide

Kubernetes Horizontal Pod Autoscaling (HPA) is a powerful tool for managing workload resources efficiently in cloud-native environments. It automatically scales the number of pods in a deployment, replication controller, replica set, or stateful set based on observed CPU utilization (or, with custom metrics support, on other application-provided metrics). In this article, we will explore the key points of HPA and provide a detailed guide on how to use it effectively.

Understanding HPA

HPA is designed to ensure that your applications maintain performance levels without wasting resources. It works by continuously monitoring resource utilization and adjusting the number of replicas to match the observed usage. This ensures that your applications are always running at optimal levels, without overloading or underutilizing resources.

Key Components of HPA

  1. Metrics Server: HPA relies on a metrics server to provide the necessary metrics for scaling decisions. This can be either the default metrics server or a custom metrics server.
  2. Custom Metrics: HPA supports custom metrics, which can be used to scale based on application-specific metrics such as requests per second.
  3. ScaleTargetRef: This configuration block sets up HPA’s target object, which can be a Deployment, replicaSet, replication controller, or custom resource.
  4. Metrics: This configuration block sets up the type of metric and when the target object should scale.

Configuring HPA

To configure HPA, you need to define a HorizontalPodAutoscaler manifest. Here is an example of a basic HPA configuration:

autoscaling-example.yaml
apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
  name: myapp-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: myapp
  minReplicas: 1
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 50

Advanced HPA Configuration

HPA supports advanced configurations such as custom metrics, external metrics, and multiple metrics. Here is an example of an advanced HPA configuration:

advanced-hpa-example.yaml
apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
  name: myapp-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: myapp
  minReplicas: 1
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 50
  - type: Pods
    pods:
      metric:
        name: packets-per-second
      target:
        type: AverageValue
        averageValue: 1k
  - type: Object
    object:
      metric:
        name: requests-per-second
      describedObject:
        apiVersion: networking.k8s.io/v1
        kind: Ingress
        name: main-route
      target:
        type: Value
        value: 10k

Best Practices for HPA

  1. Understand and Set Appropriate Metrics: Metrics serve as the cornerstone for HPA to make informed decisions about scaling. Ensure that your metrics server is properly configured and that you are using the right metrics for your application.
  2. Configure Cluster Auto-scaling: HPA depends on cluster auto-scaling being enabled to ensure that nodes are added to the cluster to fulfill any extra capacity required when HPA creates new pod replicas.
  3. Avoid Mixing HPA and VPA: HPA and Vertical Pod Autoscaling (VPA) are mutually exclusive and should not be used together for the same set of pods.
  4. Monitor and Adjust: Continuously monitor your HPA configuration and adjust it as needed to ensure optimal performance and resource utilization.

Side-Notes and Terminologies

  • What’s a Daemon in containerization and Kubernetes ?

    A Daemon, in the context of containerization and Kubernetes, is a type of process that runs in the background and performs tasks without manual intervention. In Kubernetes, a DaemonSet ensures that all (or some) Nodes run a copy of a Pod. As nodes are added to the cluster, Pods are added to them. As nodes are removed from the cluster, those Pods are garbage collected. Deleting a DaemonSet will clean up the Pods it created.

  • What’s Service Discovery?

    Service discovery in the context of cloud computing and services is the automatic detection of devices and services offered by these devices on a network. It allows applications and microservices to find and communicate with each other without hardcoding specific IP addresses or ports. This is particularly useful in a microservices architecture where services are often ephemeral and their locations change rapidly.

    • How does Service Discovery work?

      Service discovery works by using a central registry, where services register themselves and clients lookup services in the registry. When a service starts up, it registers with the service registry, providing details like the IP address, port, and the type of service it provides. The service registry then keeps track of all the services and their details. When a client needs to use a service, it queries the service registry to find the appropriate service. The client then uses the details returned by the registry to connect to the service. The registry also handles the removal of services when they stop running, ensuring that clients get up-to-date information.

  • Container Orchestration?

    Container orchestration automates different operational tasks required to run and manage containers in an environment. It involves provisioning, deploying, scaling, managing, load balancing, and networking containers. This process is typically performed using one or more tools, such as Docker Swarm or Kubernetes, which define the configurations and necessary criteria to execute these tasks and automatically handle the container lifecycle.

  • Access Parameters?

    Access parameters in Kubernetes refer to the ability to retrieve and use parameters stored in a secure location, such as Amazon Parameter Store or AWS Secrets Manager, within a Kubernetes cluster. This allows applications running in the cluster to access sensitive data without having to hardcode it or store it insecurely. Access parameters can be used for various purposes, such as:

    1. Environment Variables: Parameters can be used to set environment variables for containers, allowing them to access sensitive data without hardcoding it.
    2. Configuration Files: Parameters can be used to generate configuration files for applications, ensuring that sensitive data is not stored in plaintext.
    3. Secrets Management: Parameters can be used to manage secrets, such as API keys or database credentials, securely within the cluster.

    To access parameters in Kubernetes, you typically need to create a policy that grants access to the parameters and then use a tool like the ASCP to retrieve and mount the parameters as files in your pods

  • What are service endpoints?

    Service endpoints in Kubernetes are objects that represent the IP addresses and ports of the pods that a service routes traffic to. These endpoints are used by the service to direct traffic to the correct pods.

  • What’s one of the first commands you should run when trouble shooting a pod?

    kubectl describe pod <pod_name>

  • What are Probes?

    In Kubernetes, probes are diagnostic tools used to inspect the health of running containers within pods. They are essential for maintaining the health and reliability of applications running in containers. There are three types of probes:

    1. Liveness Probes:
      • Check if a container is still running. If a probe fails, the kubelet kills the container and restarts it according to the configured restartPolicy [1][2][3].
      • Used to ensure that applications remain healthy and accessible by automatically restarting containers that are not functioning correctly [1][2].
    2. Readiness Probes:
      • Check if a container is ready to accept traffic. If a probe fails, the endpoints controller removes the Pod's IP address from the endpoints of all Services that match the Pod [2][3][5].
      • Used to determine when a container is ready to start accepting traffic, ensuring that traffic is not sent to an unhealthy container [2][5].
    3. Startup Probes:
      • Check if an application within a container has started. If a probe fails, the kubelet kills the container and restarts it according to the configured restartPolicy [2][3][4].
      • Used to ensure that containers are fully started and operational before they begin to receive traffic [2][4].

    Each probe can be configured with various options such as initialDelaySeconds, periodSeconds, timeoutSeconds, successThreshold, and failureThreshold to fine-tune the health checks [2][3][4].

  • What is Port Forwarding?

    Port forwarding in Kubernetes is a method that establishes a temporary connection between a local machine and a specific pod or a container. This connection enables local access to a service that runs within the pod or container. This can be useful for debugging or accessing a service directly on a pod.

    Example:

    The command: kubectl port-forward service/serviceA 8080:80

    is used to forward traffic from a local port (8080) to a specific port on a Kubernetes service named serviceA. This allows you to access the service from your local machine. Here's a breakdown of the command:

    • kubectl port-forward: This is the command to forward ports from a Kubernetes resource to your local machine.
    • service/serviceA: This specifies the Kubernetes service to forward ports from. In this case, it's serviceA.
    • 8080:80: This specifies the port mapping. The first number (8080) is the local port on your machine where you want to access the service. The second number (80) is the port on the service where the traffic will be forwarded.

    When you run this command, it will create a tunnel from your local machine to the Kubernetes service. Any traffic sent to localhost:8080 will be forwarded to serviceA:80.

  • Sticky Identity

    In Kubernetes, a "sticky identity" refers to the unique and predictable naming convention used for pods in a StatefulSet. This naming convention ensures that each pod has a distinct and persistent name, which is crucial for stateful applications that require consistent and reliable access to data. In a StatefulSet, each pod is assigned a unique ordinal index, which is used to construct its name. For example, if a StatefulSet named web has three replicas, the pods will be named web-0, web-1, and web-2. This naming convention ensures that each pod has a distinct and predictable name, which is essential for managing stateful applications.

  • Stateful vs Stateless?

    Stateless applications do not require persistent storage of data. Each instance of the application can be restarted or redeployed without affecting the overall state of the application. Stateless applications are typically designed to be highly scalable and can be easily replicated across multiple nodes. Examples of stateless applications include:

    • Containerized microservices apps
    • CDN
    • Print services
    • Short-term workers

    Stateful applications require persistent storage of data. Each instance of the application maintains its own state and data, which is preserved even when the application is restarted or redeployed. Stateful applications are typically designed to maintain a consistent state across multiple interactions with users. Examples of stateful applications include:

    • Databases (e.g., PostgreSQL, MySQL, MongoDB)
    • Messaging systems
    • Big data platforms
    • Applications that require a consistent and reliable data storage layer

    Key Differences

    • Persistence: Stateful applications store state information persistently, while stateless applications do not.
    • Scalability: Stateful applications can be more challenging to scale than stateless applications because the state information needs to be managed.
    • Reliability: Stateful applications need to be designed to handle failures and errors gracefully.
    • Security: Stateful applications may need to handle sensitive user data and require appropriate security measures.

Videos