Welcome to Hacking Kubernetes, a book for Kubernetes practitioners who want to run their workloads securely and safely. At time of writing, Kubernetes has been around for some six years, give or take. There are over one hundred certified Kubernetes offerings available, such as distributions and managed services. With an increasing number of organizations deciding to move their workloads to Kubernetes, we thought we’d share our experiences in this space, to help make your workloads more secure and safe to deploy and operate. Thank you for joining us on this journey, and we hope you have as much fun reading this book and applying what you learn as we had writing it.
In this preface, we will paint a picture of our intended audience, talk about why we wrote the book, and explain how we think you should go about using it by providing a quick content guide. We will also go over some administrative details like Kubernetes versions and conventions used.
To get most out of the book, we assume that you either have a DevOps role, are a Kubernetes platform person, a cloud native architect, a site reliability engineer (SRE), or something related to being a chief information security officer (CISO). We further assume that you’re interested in being hands-on—while we discuss threats and defenses in principle, we try our best to demonstrate them at the same time and point you to tools that can help you.
At this point we also want to make sure you understand that the book you’re reading is targeting advanced topics. We assume that you’re already familiar with Kubernetes, and specifically Kubernetes security topics, at least on a surface level. In other words, we don’t go into much detail about how things work, but summarize or recap important concepts or mechanisms on a per-chapter basis.
We wrote this book with Blue and Red Teams in mind. It goes without saying that what we share here is to be used exclusively for defending your own Kubernetes cluster and workloads.
In particular, we assume that you understand what containers are for and how they run in Kubernetes. If you are not yet familiar with these topics, we recommend that you do some preliminary reading. The following are books we suggest consulting:
Kubernetes: Up and Running by Brendan Burns, Kelsey Hightower, and Joe Beda (O’Reilly)
Managing Kubernetes by Brendan Burns and Craig Tracey (O’Reilly)
Kubernetes Security by Liz Rice and Michael Hausenblas (O’Reilly)
Container Security by Liz Rice (O’Reilly)
Cloud Native Security by Chris Binnie and Rory McCune (Wiley)
Now that we have made clear what this book aims to achieve and who will, in our view, benefit from it, let’s move on to a different topic: the authors.
Based on our combined 10+ years of hands-on experience designing, running, attacking, and defending Kubernetes-based workloads and clusters, we, the authors, want to equip you, the cloud native security practitioner, with what you need to be successful in your job.
Security is often illuminated by the light of past mistakes, and both of us have been learning (and making mistakes in!) Kubernetes security for a while now. We wanted to be sure that what we thought we understood about the subject was true, so we wrote a book to verify our suspicions through a shared lens.
We have both served in different companies and roles, given training sessions, published material from tooling to blog posts, and we have shared lessons learned on the topic in various public speaking engagements. Much of what motivates us here and the examples we use are rooted in our experiences in our day-to-day jobs and/or things we observed at our clients’ companies.
This book is a threat-based guide to security in Kubernetes, using a vanilla Kubernetes installation with its (built-in) defaults as a starting point. We’ll kick off discussions with an abstract threat model of a distributed system running arbitrary workloads and progress to a detailed assessment of each component of a secure Kubernetes system.
In each chapter, we examine a component’s architecture and potential default settings and review high-profile attacks and historical Common Vulnerabilities and Exposures (CVEs). We also demonstrate attacks and share best-practice configuration in order to demonstrate hardening clusters from possible angles of attack.
In order to aid you in navigating the book, here’s a quick rundown on the chapter level:
In Chapter 1, “Introduction” we set the scene, introducing our main antagonist and also what threat modeling is.
Chapter 2, “Pod-Level Resources” then focuses on pods, from configurations to attacks to defenses.
Next up, in Chapter 3, “Container Runtime Isolation”, we switch gears and dive deep into sandboxing and isolation techniques.
Chapter 4, “Applications and Supply Chain” then covers supply chain attacks and what you can do to detect and mitigate them.
In Chapter 5, “Networking” we then review networking defaults and how to secure your cluster and workload traffic.
Then, in Chapter 6, “Storage” we shift our focus to aspects of persistence, looking at filesystems, volumes, and sensitive information at rest.
Chapter 7, “Hard Multitenancy” covers the topic of running workloads for multitenants in a cluster and what can go wrong with this.
Next up is Chapter 8, “Policy”, where we review different kinds of policies in use, discuss access control—specifically role-based access control (RBAC)—and generic policy solutions such as Open Policy Agent (OPA).
In Chapter 9, “Intrusion Detection” we cover the question of what you can do if, despite controls put in place, someone manages to break in.
The last chapter, Chapter 10, “Organizations”, is somewhat special, in that it doesn’t focus on tooling but on the human side of things, in the context of cloud as well as on-prem installations.
In Appendix A, “A Pod-Level Attack“, we walk you through a hands-on exploration of attacks on the pod-level as discussed in Chapter 2. Finally, in Appendix B, “Resources“ we put together further reading material on a per-chapter basis as well as a collection of annotated CVEs relevant in the context of this book.
You don’t have to read the chapters in order; we tried our best to keep the chapters as self-contained as possible and refer to related content where appropriate.
Note that at the time of writing this book, Kubernetes 1.21 was the latest stable version. Most examples shown here work with earlier versions, and we’re fully aware that by the time you’re reading this book, the current version will potentially be significantly higher. The concepts stay the same.
With this short guide on what to expect and a quick orientation done, let’s have a look at conventions used in the book.
The following typographical conventions are used in this book:
Indicates new terms, URLs, email addresses, filenames, and file extensions.
Constant widthUsed for program listings. Also used within paragraphs to refer to program elements such as variable or function names, databases, data types, environment variables, statements, and keywords.
This element signifies a tip or suggestion.
This element signifies a general note.
This element indicates a warning or caution.
Supplemental material is available at http://hacking-kubernetes.info.
If you have a technical question or a problem using the code examples, please email bookquestions@oreilly.com.
This book is here to help you get your job done. In general, if example code is offered with this book, you may use it in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission.
We appreciate but generally do not require attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: “Hacking Kubernetes by Andrew Martin and Michael Hausenblas (O’Reilly). Copyright 2022 Andrew Martin and Michael Hausenblas, 978-1-492-08173-9.”
If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact us at permissions@oreilly.com.
For more than 40 years, O’Reilly Media has provided technology and business training, knowledge, and insight to help companies succeed.
Our unique network of experts and innovators shares its knowledge and expertise through books, articles, and our online learning platform. O’Reilly’s online learning platform gives on-demand access to live training courses, in-depth learning paths, interactive coding environments, and a vast collection of text and video from O’Reilly and 200+ other publishers. For more information, visit http://oreilly.com.
Please address comments and questions concerning this book to the publisher:
We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at https://oreil.ly/HackingKubernetes.
Email bookquestions@oreilly.com to comment or ask technical questions about this book.
For news and information about our books and courses, visit http://oreilly.com.
Find us on Facebook: http://facebook.com/oreilly
Follow us on Twitter: http://twitter.com/oreillymedia
Watch us on YouTube: http://youtube.com/oreillymedia
Thanks go out to our reviewers Roland Huss, Liz Rice, Katie Gamanji, Ihor Dvoretskyi, Mark Manning, and Michael Gasch. Your comments absolutely made a difference and we appreciate your guidance and suggestions.
Andy would like to thank his family and friends for their unceasing love and encouragement, the inspiring and razor-sharp team at ControlPlane for their assiduous insight and guidance, and the continually enlightening cloud native security community for their relentless kindness and brilliance. Special thanks to Rowan Baker, Kevin Ward, Lewis Denham-Parry, Nick Simpson, Jack Kelly, and James Cleverley-Prance.
Michael would like to express his deepest gratitude to his awesome and supportive family: our kids Saphira, Ranya, and Iannis; my wicked smart and fun wife, Anneliese, and also our bestest of all dogs, Snoopy.
We would be remiss not to mention the Hacking Kubernetes Twitter list of our inspirations and mentors, featuring alphabetized luminaries such as @antitree, @bradgeesaman, @brau_ner, @christianposta, @dinodaizovi, @erchiang, @garethr, @IanColdwater, @IanMLewis, @jessfraz, @jonpulsifer, @jpetazzo, @justincormack, @kelseyhightower, @krisnova, @kubernetesonarm, @liggitt, @lizrice, @lordcyphar, @lorenc_dan, @lumjjb, @mauilion, @MayaKaczorowski, @mikedanese, @monadic, @raesene, @swagitda_, @tabbysable, @tallclair, @torresariass, @WhyHiAnnabelle, and @captainHλ$ħ𝔍A¢k.
Last but certainly not least, both authors thank the O’Reilly team, especially Angela Rufino, for shepherding us through the process of writing this book.