Log In
Or create an account ->
Imperial Library
Home
About
News
Upload
Forum
Help
Login/SignUp
Index
Puppet: Mastering Infrastructure Automation
Puppet: Mastering Infrastructure Automation
Credits
Preface
What this learning path covers
What you need for this learning path
Who this learning path is for
Reader feedback
Customer support
Downloading the example code
Errata
Piracy
Questions
1. Module 1
1. Writing Your First Manifests
Getting started
Introducing resources and properties
Interpreting the output of the puppet apply command
Dry-testing your manifest
Adding control structures in manifests
Using variables
Variable types
Controlling the order of evaluation
Declaring dependencies
Error propagation
Avoiding circular dependencies
Implementing resource interaction
Examining the most notable resource types
The user and group types
The exec resource type
The cron resource type
The mount resource type
Summary
2. The Master and Its Agents
The Puppet Master
Puppet Master and Puppet Server
Setting up the master machine
Creating the master manifest
Inspecting the configuration settings
Setting up the Puppet Agent
The agent's life cycle
Renewing an agent's certificate
Running the agent from cron
Performance considerations
Tuning puppetserver
Using Phusion Passenger with Nginx
Comparing Passenger with puppetserver
Completing the stack with PuppetDB
Troubleshooting SSL issues
Summary
3. A Peek under the Hood – Facts, Types, and Providers
Summarizing systems with Facter
Accessing and using fact values
Extending Facter with custom facts
Simplifying things using external facts
Goals of Facter
Understanding the type system
The resource type's life cycle on the agent side
Substantiating the model with providers
Resource types with generic providers
Summarizing types and providers
Putting it all together
Summary
4. Modularizing Manifests with Classes and Defined Types
Introducing classes and defined types
Defining and declaring classes
Creating and using defined types
Understanding and leveraging the differences
Structured design patterns
Writing comprehensive classes
Writing component classes
Using defined types as resource wrappers
Using defined types as resource multiplexers
Using defined types as macros
Exploiting array values using defined types
Using iterator functions
Including classes from defined types
Establishing relationships among containers
Passing events between classes and defined types
Ordering containers
Limitations
The performance implications of container relationships
Mitigating the limitations
The anchor pattern
The contain function
Making classes more flexible through parameters
The caveats of parameterized classes
Preferring the include keyword
Summary
5. Extending Your Puppet Infrastructure with Modules
An overview of Puppet's modules
Parts of a module
Module structure
Documentation in modules
Maintaining environments
Configuring environment locations
Obtaining and installing modules
Modules' best practices
Putting everything in modules
Avoiding generalization
Testing your modules
Safe testing with environments
Building a specific module
Naming your module
Making your module available to Puppet
Implementing basic module functionality
Creating utilities for derived manifests
Adding configuration items
Allowing customization
Removing unwanted configuration items
Dealing with complexity
Enhancing the agent through plugins
Replacing a defined type with a native type
Naming your type
Creating the resource type interface
Designing sensible parameter hooks
Using resource names
Adding a provider
Declaring management commands
Implementing the basic functionality
Allowing the provider to prefetch existing resources
Making the type robust during provisioning
Enhancing Puppet's system knowledge through facts
Refining the interface of your module through custom functions
Making your module portable across platforms
Finding helpful Forge modules
Identifying modules' characteristics
Summary
6. Leveraging the Full Toolset of the Language
Templating dynamic configuration files
Learning the template syntax
Using templates in practice
Avoiding performance bottlenecks from templates
Creating virtual resources
Realizing resources more flexibly using collectors
Exporting resources to other agents
Exporting and importing resources
Configuring the master to store exported resources
Exporting SSH host keys
Managing hosts files locally
Automating custom configuration items
Simplifying the configuration of Nagios
Maintaining your central firewall
Removing obsolete exports
Overriding resource parameters
Saving redundancy using resource defaults
Avoiding antipatterns
Summary
7. New Features from Puppet 4
Upgrading to Puppet 4
Using Puppet 3.8 and environment directories
The new Puppet 4 Master
Updating the Puppet agent
Testing Puppet DSL code
Using the type system
Learning lambdas and functions
Creating Puppet 4 functions
Leveraging the new template engine
Handling multiline with HEREDOC
Breaking old practices
Converting node inheritance
Dealing with bool algebra on Strings
Using strict variable naming
Learning the new reference syntax
Cleaning hyphens in names
No Ruby DSL anymore
Relative class name resolution
Dealing with different data types
Summary
8. Separating Data from Code Using Hiera
Understanding the need for separate data storage
Consequences of defining data in the manifest
Structuring configuration data in a hierarchy
Configuring Hiera
Storing Hiera data
Choosing your backends
Retrieving and using Hiera values in manifests
Working with simple values
Binding class parameter values automatically
Handling hashes and arrays
Converting resources to data
Choosing between manifest and Hiera designs
A practical example
Debugging Hiera lookups
Implementing the Roles and Profiles pattern
Summary
2. Module 2
1. Puppet Essentials
The Puppet ecosystem
Why configuration management matters
Puppet components
Installing and configuring Puppet
Puppet in action
Resources
The resource abstraction layer
Nodes
Classes and defines
Class inheritance
Resource defaults
Resource references
Variables, facts, and scopes
System's facts
External facts
User variables in Puppet DSL
Type-checking
User variables in an ENC
User variables in Hiera
Puppet built-in variables
Variables scope
Meta parameters
Managing order and dependencies
Run stages
Reserved names and allowed characters
Conditionals
Comparison operators
Iteration and lambdas
The in operator
Expressions combinations
Exported resources
Virtual resources
Modules
Paths of a module and auto loading
Templates
Restoring files from a filebucket
Summary
2. Managing Puppet Data with Hiera
Installing and configuring Hiera
The hiera.yaml configuration file
Global settings
Backend specific settings
Examples
Working with the command line on a YAML backend
Using Hiera in Puppet
Dealing with hashes in Puppet code
Puppet 3 automatic parameter lookup
Evolving usage patterns for class parameters
Additional Hiera backends
Hiera-file
Encryption plugins
Hiera-gpg
Hiera-eyaml
Hiera-http, hiera-mysql
Using Hiera as an ENC
Summary
3. Introducing PuppetDB
Installation and configuration
Installing PuppetDB
PuppetDB configurations
Init script configuration
Configuration settings
Logging configuration
Configurations on the Puppet Master
Dashboards
PuppetDB performance dashboard
Puppetboard—Querying PuppetDB from the Web
PuppetDB API
Querying PuppetDB (read)
PuppetDB commands (write)
Querying PuppetDB for fun and profit
/facts endpoint
/resources endpoint
/nodes endpoint
/catalogs endpoint
/fact-names endpoint
/metrics endpoint
/reports endpoint
/events endpoint
/event-counts endpoint
/aggregated-event-counts endpoint
/server-time endpoint
/version endpoint
The puppetdbquery module
Query format
Querying from the command line
Querying from Puppet manifests
The PuppetDB Hiera backend
How Puppet code may change in the future
Summary
4. Designing Puppet Architectures
Components of a Puppet architecture
Definition of the classes to include in each node
Definition of the parameters to use for each node
Definition of the configuration files provided to the nodes
Definition of custom resources and classes
The Foreman
Roles and profiles
The data and the code
Sample architectures
The default approach
Basic ENC, logic on site module, Hiera backend
The Foreman and Hiera
Hiera-based setup
Foreman smart variables
Facts-driven truths
Nodeless site.pp
Summary
5. Using and Writing Reusable Modules
Modules layout evolution
Class parameters—from zero to data bindings
Params pattern
Data in modules
Files and class names
The anchor pattern
The parameters dilemma
Naming standards
Reusability patterns
Managing files
Managing configuration hash patterns
Managing multiple configuration files
Managing users and dependencies
Managing installation options
Managing extra resources
Summary
6. Higher Abstraction Modules
The OpenStack example
Component (application) modules
Raising abstraction – the official OpenStack module
Raising abstraction – Puppet Labs OpenStack module
Taking an alternate approach
An approach to reusable stack modules
Tiny Puppet
Summary
7. Puppet Migration Patterns
Examining potential scenarios and approaches
New infrastructures
Existing manually managed infrastructures
Node migration
Node update
Existing automated infrastructures
Patterns for extending Puppet coverage
Raising the bar, step by step
Knowing the territory
Defining priorities
Evaluating solutions
Coding
Applying changes
Things change
Puppet friendly infrastructure
Puppet-friendly software
Summary
8. Code Workflow Management
Write Puppet code
Geppetto
Vim
Git workflows
Code management using Git
Git hooks
Environments and branches
Branch based automatic environments
Simplified developer workdir environments
Code review
Gerrit
Online resources for peer review
Testing Puppet code
Using rspec-puppet
Vagrant
Beaker and beaker-rspec
Deploying Puppet code
Using librarian-puppet for deployments
Deploying code with r10k
Propagating Puppet changes
Puppet continuous integration
Travis
Jenkins
Summary
9. Scaling Puppet Infrastructures
Scaling Puppet
As simple as puppet apply
Default Puppet Master
Puppet Master with Passenger
Installing and configuring Passenger
Puppet Master based on Trapperkeeper
Multi-Master scaling
Managing certificates
Managing SSL termination
Managing code and data
Load balancing alternatives
Masterless Puppet
Configurations and infrastructure optimizations
Traffic compression
Caching
Distributing Puppet run times
File system changes check
Scaling stored configs
Measuring performance
Puppet Metrics
Optimizing code
Testing different Puppet versions
Summary
10. Extending Puppet
Anatomy of a Puppet run, under the hood
Puppet extension alternatives
ENC and Hiera Extendibility
Modules pluginsync
Puppet indirector and its termini
Custom functions
Custom facts
Ruby facts distributed via pluginsync
External facts in facts.d directory
Custom types and providers
Custom report handlers
Custom faces
Summary
11. Beyond the System
Puppet on network equipment
Proxy mode with Puppet device application
Native Puppet on network equipment
Cisco onePK
Juniper and the netdev_stdlib
Puppet for cloud and virtualization
VMware
VM provisioning on vCenter and vSphere
vCenter configuration
vSphere virtual machine management with resource types
Amazon Web Services
Cloud provisioning on AWS
AWS provisioning and configuration with resource types
Managing CloudFormation
Cloud provisioning on Google Compute Engine
Puppet on storage devices
Puppet and Docker
Summary
12. Future Puppet
Changing the serialization format
Direct Puppet
File sync
Other changes
Certificate authority in Clojure
Package management
Changes in faces
Beyond Puppet 4.x
Summary
3. Module 3
1. Dealing with Load/Scale
Divide and conquer
Certificate signing
Reporting
Storeconfigs
Catalog compilation
puppetserver
Building a Puppet master
Certificates
systemd
Creating a load balancer
Apache proxy
TLS headers
SELinux
Configuring masters
Keeping the code consistent
rsync
NFS
Clustered filesystem
Git
One more split
One last split or maybe a few more
Conquer by dividing
Creating an rpm
Using Puppet resource to configure cron
Creating the yum repository
Summary
2. Organizing Your Nodes and Data
Getting started
Organizing the nodes with an ENC
A simple example
Hostname strategy
Modified ENC using hostname strategy
LDAP backend
OpenLDAP configuration
Hiera
Configuring Hiera
Using hiera_include
Summary
3. Git and Environments
Environments
Environments and Hiera
Multiple hierarchies
Single hierarchy for all environments
Directory environments
Git
Why Git?
A simple Git workflow
Git hooks
Using post-receive to set up environments
Puppet-sync
Using Git hooks to play nice with other developers
Not playing nice with others via Git hooks
Git for everyone
Summary
4. Public Modules
Getting modules
Using GitHub for public modules
Updating the local repository
Modules from the Forge
Using Librarian
Using r10k
Using Puppet-supported modules
concat
inifile
firewall
Logical volume manager
Standard library
Summary
5. Custom Facts and Modules
Module manifest files
Module files and templates
Naming a module
Creating modules with a Puppet module
Comments in modules
Multiple definitions
Custom facts
Creating custom facts
Creating a custom fact for use in Hiera
CFacter
Summary
6. Custom Types
Parameterized classes
Data types
Defined types
Types and providers
Creating a new type
Summary
7. Reporting and Orchestration
Turning on reporting
Store
Logback
Internet relay chat
Foreman
Installing Foreman
Attaching Foreman to Puppet
Using Foreman
Puppet GUIs
mcollective
Installing ActiveMQ
Configuring nodes to use ActiveMQ
Connecting a client to ActiveMQ
Using mcollective
Ansible
Summary
8. Exported Resources
Configuring PuppetDB – using the Forge module
Manually installing PuppetDB
Installing Puppet and PuppetDB
Installing and configuring PostgreSQL
Configuring puppetdb to use PostgreSQL
Configuring Puppet to use PuppetDB
Exported resource concepts
Declaring exported resources
Collecting exported resources
Simple example – a host entry
Resource tags
Exported SSH keys
sshkey collection for laptops
Putting it all together
Summary
9. Roles and Profiles
Design pattern
Creating an example CDN role
Creating a sub-CDN role
Dealing with exceptions
Summary
10. Troubleshooting
Connectivity issues
Catalog failures
Full trace on a catalog compilation
The classes.txt file
Debugging
Personal and bugfix branches
Echo statements
Scope
Profiling and summarizing
Summary
A. Bibliography
Index
← Prev
Back
Next →
← Prev
Back
Next →