Title Page Copyright and Credits Hands-On Markov Models with Python Packt Upsell Why subscribe? packt.com Contributors About the authors About the reviewer Packt is searching for authors like you Preface Who this book is for What this book covers To get the most out of this book Download the example code files Download the color images Conventions used Get in touch Reviews Introduction to the Markov Process Random variables Random processes Markov processes Installing Python and packages Installation on Windows Installation on Linux Markov chains or discrete-time Markov processes Parameterization of Markov chains Properties of Markov chains Reducibility Periodicity Transience and recurrence Mean recurrence time Expected number of visits Absorbing states Ergodicity Steady-state analysis and limiting distributions Continuous-time Markov chains Exponential distributions Poisson process Continuous-time Markov chain example Continuous-time Markov chain Summary Hidden Markov Models Markov models State space models The HMM Parameterization of HMM Generating an observation sequence Installing Python packages Evaluation of an HMM Extensions of HMM Factorial HMMs Tree-structured HMM Summary State Inference - Predicting the States State inference in HMM Dynamic programming Forward algorithm Computing the conditional distribution of the hidden state given the observations Backward algorithm Forward-backward algorithm (smoothing) The Viterbi algorithm Summary Parameter Learning Using Maximum Likelihood Maximum likelihood learning MLE in a coin toss MLE for normal distributions MLE for HMMs Supervised learning Code Unsupervised learning Viterbi learning algorithm The Baum-Welch algorithm (expectation maximization) Code Summary Parameter Inference Using the Bayesian Approach Bayesian learning Selecting the priors Intractability Bayesian learning in HMM Approximating required integrals Sampling methods Laplace approximations Stolke and Omohundro's method Variational methods Code Summary Time Series Predicting Stock price prediction using HMM Collecting stock price data Features for stock price prediction Predicting price using HMM Summary Natural Language Processing Part-of-speech tagging Code Getting data Exploring the data Finding the most frequent tag Evaluating model accuracy An HMM-based tagger Speech recognition Python packages for speech recognition Basics of SpeechRecognition Speech recognition from audio files Speech recognition using the microphone Summary 2D HMM for Image Processing Recap of 1D HMM 2D HMMs Algorithm Assumptions for the 2D HMM model Parameter estimation using EM Summary Markov Decision Process Reinforcement learning Reward hypothesis State of the environment and the agent Components of an agent The Markov reward process Bellman equation MDP Code example Summary Other Books You May Enjoy Leave a review - let other readers know what you think