Mastering Kubernetes by Sayfan, Gigi -- Read -- Imperial Library of Trantor

Index

Mastering Kubernetes

Table of Contents Mastering Kubernetes Credits About the Author About the Reviewer www.PacktPub.com

eBooks, discount offers, and more

Why subscribe?

Customer Feedback Preface

What this book covers What you need for this book Who this book is for Conventions Reader feedback Customer support

Downloading the color images of this book Errata Piracy Questions

1. Understanding Kubernetes Architecture

Understanding container orchestration

Physical machines, virtual machines, and containers Containers in the cloud Cattle versus pets

Kubernetes concepts

Cluster Node Master Pod Label Annotation Label selector Replication controller and replica set Service Volume StatefulSet Secret Name Namespace

Diving into Kubernetes architecture in depth

Distributed systems design patterns

Sidecar pattern Ambassador pattern Adapter pattern Multi-node patterns

The Kubernetes APIs

Kubernetes API Autoscaling API

Batch API

Kubernetes components

Master components

API server Etcd Controller manager Scheduler DNS

Node components

Proxy Kubelet

Kubernetes runtimes

The runtime interface Docker Rkt

App container Rktnetes Is rkt ready for production usage?

Hyper containers

Hypernetes

Continuous integration and deployment

What is a CI/CD pipeline? Designing a CI/CD pipeline for Kubernetes

Summary

2. Creating Kubernetes Clusters

Quick single-node cluster with Minikube

Getting ready Creating the cluster Troubleshooting Checking out the cluster Doing work Examining the cluster with the dashboard

Creating a multi-node cluster using kubeadm

Getting ready Preparing a cluster of vagrant VMs Installing the required software

The hosts file The vars.yml file The playbook.yml file

Creating the cluster

Initializing the master

Setting up the pod network Adding the worker nodes

Creating clusters in the cloud (GCP, AWS, Azure)

The cloud-provider interface GCP AWS Azure

Creating a bare-metal cluster from scratch

Use cases for bare-metal When should you consider creating a bare-metal cluster?

The process Using virtual private cloud infrastructure Summary

3. Monitoring, Logging, and Troubleshooting

Monitoring Kubernetes with Heapster

cAdvisor

InfluxDB backend

The storage schema

CPU Filesystem Memory Network Uptime

Grafana visualization

Performance analysis with the dashboard

Top-level view

Admin view Workloads Services and discovery

Adding central logging

Planning central logging Fluentd Elasticsearch Kibana

Detecting node problems

Node problem detector DaemonSet Problem Daemons

Troubleshooting scenarios Designing robust systems

Hardware failure Quotas, shares, and limits Bad configuration Cost versus performance

Managing cost on the cloud Managing cost on bare metal Managing cost on hybrid clusters

Summary

4. High Availability and Reliability

High-availability concepts

Redundancy Hot swapping Leader election Smart load balancing Idempotency Self-healing

High-availability best practices

Creating highly available clusters Making your nodes reliable Protecting your cluster state

Clustering etcd

Static discovery Etcd discovery DNS discovery

The etcd.yaml file Verifying the etcd cluster etcd 2 versus etcd 3

Protecting your data Running redundant API servers Running leader election with Kubernetes

Leader election for your application

Making your staging environment highly available Testing high-availability

Live cluster upgrades

Rolling upgrades

Complex deployments

Blue-green upgrades Managing data-contract changes Migrating data Deprecating APIs

Large-cluster performance, cost, and design trade-offs

Availability requirements Best effort Maintenance windows Quick recovery Zero-downtime Performance and data consistency

Summary

5. Configuring Kubernetes Security, Limits, and Accounts

Understanding Kubernetes security challenges

Node challenges Network challenges Image challenges Configuration and deployment challenges Pod and container challenges Organisational, cultural, and process challenges

Hardening Kubernetes

Understanding service accounts in Kubernetes

How does Kubernetes manage service accounts?

Accessing the API server

Authenticating users Authorizing requests Using admission control plugins

Securing pods

Using a private image repository ImagePullSecrets Specifying a security context Protecting your cluster with AppArmor

Requirements Securing a pod with AppArmor Writing AppArmor profiles

Pod security policies

Managing network policies

Choosing a supported networking solution Defining a network policy

Using secrets

Storing secrets in Kubernetes Creating secrets Decoding secrets Using secrets in a container

Running a multi-user cluster

The case for a multi-user cluster Using namespaces for safe multi-tenancy Avoiding namespace pitfalls

Summary

6. Using Critical Kubernetes Resources

Designing the Hue platform

Defining the scope of Hue

Hue components

User profile User graph Identity Authorizer External service Generic sensor Generic actuator User learner

Hue microservices

Plugins Data stores Stateless microservices Queue-based interactions

Planning workflows

Automatic workflows Human workflows Budget-aware workflows

Using Kubernetes to build the Hue platform

Using Kubectl effectively Understanding Kubectl resource configuration files

ApiVersion Kind Metadata Spec

Container spec

Deploying long-running microservices in pods

Creating pods Decorating pods with labels Deploying long- running processes with deployments Updating a deployment

Separating internal and external services

Deploying an internal service Creating the Hue-reminders service Exposing a service externally

Ingress

Using namespace to limit access Launching jobs

Running jobs in parallel Cleaning up completed jobs Scheduling cron jobs

Kubectl get pods Mixing non-cluster components

Outside-the-cluster-network components Inside-the-cluster-network components Managing the Hue platform with Kubernetes

Using liveness probes to ensure your containers are alive

Using readiness probes to manage dependencies

Employing init containers for orderly pod bring-up

Sharing with DaemonSet pods

Evolving the Hue platform with Kubernetes

Utilizing Hue in the enterprise Advancing science with Hue Educating the kids of the future with hue

Summary

7. Handling Kubernetes Storage

Persistent volumes walkthrough

Volumes

Using emptyDir for intra-pod communication Using HostPath for intra-node communication Provisioning persistent volumes

Provisioning persistent volumes statically Provisioning persistent volumes dynamically

Creating persistent volumes

Capacity Access modes Reclaim policy Volume type

Making persistent volume claims Mounting claims as volumes Storage classes

Default storage class

Demonstrating persistent volume storage end to end

Public storage volume types - GCE, AWS, and Azure

AWS Elastic Block Store (EBS) AWS Elastic File System (EFS) GCE persistent disk Azure data disk Azure file storage

GlusterFS and Ceph volumes in Kubernetes

Using GlusterFS

Creating endpoints Adding a GlusterFS Kubernetes service Creating pods

Using Ceph

Connecting to Ceph using RBD Connecting to Ceph using CephFS

Flocker as a clustered container data volume manager Integrating enterprise storage into Kubernetes

Torus – the new kid on the block

Summary

8. Running Stateful Applications with Kubernetes

Stateful versus stateless applications in Kubernetes

Understanding the nature of distributed data-intensive apps Why manage states in Kubernetes? Why manage states outside of Kubernetes?

Shared environment variables versus DNS records for discovery

Accessing external data stores via DNS Accessing external data stores via environment variables

Creating a ConfigMap Consuming a ConfigMap as an environment variable

Using a redundant in-memory state Using DaemonSet for redundant persistent storage Applying persistent volume claims Utilizing StatefulSet

When to use StatefulSet The components of StatefulSet

Running a Cassandra cluster in Kubernetes

Quick introduction to Cassandra The Cassandra Docker image

Exploring the run.sh script

Hooking up Kubernetes and Cassandra

Digging into the Cassandra configuration The custom seed provider

Creating a Cassandra headless service Using statefulSet to create the Cassandra cluster

Dissecting the stateful set configuration file

Using a replication controller to distribute Cassandra

Dissecting the replication controller configuration file Assigning pods to nodes

Using DaemonSet to distribute Cassandra

Summary

9. Rolling Updates, Scalability, and Quotas

Horizontal pod autoscaling

Declaring horizontal pod autoscaler Custom metrics Autoscaling with Kubectl

Performing rolling updates with autoscaling Handling scarce resources with limits and quotas

Enabling resource quotas Resource quota types

Compute resource quota Storage resource quota Object count quota

Quota scopes Requests and limits Working with quotas

Using namespace-specific context Creating quotas Using limit ranges for default compute quotas

Choosing and managing the cluster capacity

Choosing your node types Choosing your storage solutions Trading off cost and response time Using effectively multiple node configurations Benefiting from elastic cloud resources

Autoscaling instances Mind your cloud quotas Manage regions carefully

Considering Hyper.sh

Pushing the envelope with Kubernetes

Improving the performance and scalability of Kubernetes

Caching reads in the API server The pod lifecycle event generator Serializing API objects with protocol buffers

Measuring the performance and scalability of Kubernetes

The Kubernetes SLOs Measuring API responsiveness Measuring end to end pod startup time

Testing Kubernetes at scale

Introducing the Kubemark tool Setting up a Kubemark cluster Comparing a Kubemark cluster to a real-world cluster

Summary

10. Advanced Kubernetes Networking

Understanding the Kubernetes networking model

Intra-pod communication (container to container) Inter-pod communication (pod to pod) Pod to service communication External access Kubernetes networking versus Docker networking Lookup and discovery

Self-registration Services and endpoints Loosely coupled connectivity with queues Loosely coupled connectivity with data stores Kubernetes ingress

Kubernetes network plugins

Basic Linux networking IP addresses and ports Network namespaces Virtual Ethernet devices Bridges Routing Maximum transmission unit Pod networking Kubenet

Requirements Setting the MTU

Container networking interface

Container runtime CNI plugin

Kubernetes networking solutions

Bridging on bare metal clusters Contiv Open vSwitch Nuage networks VCS Canal Flannel Calico project Romana Weave net

Using network policies effectively

Understanding the Kubernetes network policy design Network policies and CNI plugins Configuring network policies Implementing network policies

Load balancing options

External load balancer

Configuring an external load balancer

Via configuration file Via Kubectl

Finding the load balancer IP addresses Identifying client IP addresses

Annotating the load balancer for client IP address preservation

Understanding potential in even external load balancing

Service load balancer Ingress

HAProxy Utilizing the NodePort Custom load balancer provider using HAProxy Running HAProxy Inside the Kubernetes cluster Keepalived VIP

Writing your own CNI plugin

First look at the loopback plugin

Building on the CNI plugin skeleton Reviewing the bridge plugin

Summary

11. Running Kubernetes on Multiple Clouds and Cluster Federation

Understanding cluster federation

Important use cases for cluster federation

Capacity overflow Sensitive workloads Avoiding vendor lock-in Geo-distributing high availability

The federation control plane

Federation API server Federation controller manager

Federated resources

Federated ConfigMap

Creating a federated ConfigMap Viewing a federated ConfigMap Updating a federated ConfigMap Deleting a federated ConfigMap

Federated DaemonSet Federated deployment Federated events Federated ingress

Creating a federated ingress Request routing with a federated ingress Handling failures with federated ingress

Federated namespace Federated ReplicaSet Federated secrets

The hard parts

Federated unit of work Location affinity

Strictly-coupled Loosely-coupled Preferentially-coupled Strictly-decoupled Uniformly-spread

Cross-cluster scheduling Federated data access Federated auto-scaling

Managing a Kubernetes cluster federation Setting up cluster federation from the ground up

Initial setup Using the official hyperkube image Running the federation control plane Registering Kubernetes clusters with federation Updating KubeDNS Shutting down the federation Setting up cluster federation with Kubefed

Getting Kubefed Choosing a host cluster Deploying a federation control plane Adding a cluster to a federation

Naming rules and customization Secret name Removing a cluster from a federation Shutting down the federation

Cascading delete of resources Load balancing across multiple clusters Failing over across multiple clusters Federated service discovery Federated migration

Running federated workloads

Creating a federated service Adding backend pods Verifying public DNS records Discovering a federated service

DNS expansion

Handling failures of backend pods and whole clusters Troubleshooting

Unable to connect to federation API server

Summary

12. Customizing Kubernetes - API and Plugins

Working with the Kubernetes API

Understanding OpenAPI Setting up a proxy Exploring the Kubernetes API directly

Using Postman to explore the Kubernetes API Filtering the output with httpie and jq

Creating a pod via the Kubernetes API Accessing the Kubernetes API via the Python client

Dissecting the CoreV1API group Listing objects Creating objects Watching objects Invoking Kubectl programmatically Using Python subprocess to run Kubectl

Extending the Kubernetes API

Understanding the structure of a third-party-resource Developing third-party-resources Integrating third party resources

Writing Kubernetes plugins

Writing a custom scheduler plugin

Understanding the design of the Kubernetes scheduler

The scheduler Registering an algorithm provider Configuring the scheduler

Packaging the scheduler Deploying the custom scheduler Running another custom scheduler in the cluster Assigning pods to the custom scheduler Verifying that the pods were scheduled using custom scheduler

Writing an authorization plugin

Writing an admission control plugin

Implementing an admission control plugin Registering an admission control plugin Linking your custom admission control plugin

Writing a custom metrics plugin

Configuring the pod for custom metrics Specifying a target metric value

Writing a volume plugin

Implementing a volume plugin Registering a volume plugin Linking a volume plugin

Summary

13. Handling the Kubernetes Package Manager

Understanding Helm

The motivation for Helm The Helm architecture Helm components

The Tiller server The Helm client

Helm versus. Helm-classic

Using Helm

Installing Helm

Installing the Helm client Installing the Tiller server

Installing Tiller in-cluster Installing Tiller locally

Finding charts Installing packages

Checking installation status Customizing a chart Additional installation options Upgrading and rolling back a release Deleting a release

Working with repositories Managing charts with Helm

Taking advantage of starter packs

Creating your own charts

The Chart.yaml file

Versioning charts The appVersion field Deprecating charts

Chart metadata files Managing chart dependencies

Managing dependencies with requirements.yaml Utilizing special fields in requirements.yaml

Using templates and values

Writing template files

Using pipelines and functions

Embedding predefined values Feeding values from a file Scope, dependencies, and values

Summary

14. The Future of Kubernetes

The road ahead

Kubernetes releases and milestones Kubernetes special interest and working groups

Competition

The value of bundling Docker Swarm Mesos/Mesosphere Cloud platforms AWS Azure Alibaba Cloud

The Kubernetes momentum

Community GitHub Conferences and meetups Mindshare Ecosystem Public Cloud providers

OpenShift OpenStack Other players

Education and training Dynamic plugins Summary

Index

← Prev
Back
Next →

← Prev
Back
Next →