Principles of Data Science by Ozdemir, Sinan -- Read -- Imperial Library of Trantor

Index

Principles of Data Science

Table of Contents Principles of Data Science Credits About the Author About the Reviewers www.PacktPub.com

eBooks, discount offers, and more

Why subscribe?

Preface

What this book covers What you need for this book Who this book is for Conventions Reader feedback Customer support

Downloading the example code Downloading the color images of this book Errata Piracy Questions

1. How to Sound Like a Data Scientist

What is data science?

Basic terminology Why data science? Example – Sigma Technologies

The data science Venn diagram

The math

Example – spawner-recruit models

Computer programming Why Python?

Python practices Example of basic Python

Example – parsing a single tweet Domain knowledge

Some more terminology Data science case studies

Case study – automating government paper pushing

Fire all humans, right?

Case study – marketing dollars Case study – what's in a job description?

Summary

2. Types of Data

Flavors of data Why look at these distinctions? Structured versus unstructured data

Example of data preprocessing

Word/phrase counts Presence of certain special characters Relative length of text Picking out topics

Quantitative versus qualitative data

Example – coffee shop data Example – world alcohol consumption data Digging deeper

The road thus far… The four levels of data

The nominal level

Mathematical operations allowed Measures of center What data is like at the nominal level

The ordinal level

Examples Mathematical operations allowed Measures of center Quick recap and check

The interval level

Example Mathematical operations allowed Measures of center Measures of variation

Standard deviation

The ratio level

Examples Measures of center Problems with the ratio level

Data is in the eye of the beholder Summary

3. The Five Steps of Data Science

Introduction to data science Overview of the five steps

Ask an interesting question Obtain the data Explore the data Model the data Communicate and visualize the results

Explore the data

Basic questions for data exploration Dataset 1 – Yelp

Dataframes Series Exploration tips for qualitative data

Nominal level columns Filtering in Pandas Ordinal level columns

Dataset 2 – titanic

Summary

4. Basic Mathematics

Mathematics as a discipline Basic symbols and terminology

Vectors and matrices

Quick exercises Answers

Arithmetic symbols

Summation Proportional Dot product

Graphs Logarithms/exponents Set theory

Linear algebra

Matrix multiplication

How to multiply matrices

Summary

5. Impossible or Improbable – A Gentle Introduction to Probability

Basic definitions Probability Bayesian versus Frequentist

Frequentist approach

The law of large numbers

Compound events Conditional probability The rules of probability

The addition rule Mutual exclusivity The multiplication rule Independence Complementary events

A bit deeper Summary

6. Advanced Probability

Collectively exhaustive events Bayesian ideas revisited

Bayes theorem More applications of Bayes theorem

Example – Titanic Example – medical studies

Random variables

Discrete random variables

Types of discrete random variables

Binomial random variables Poisson random variable, Continuous random variables

Summary

7. Basic Statistics

What are statistics? How do we obtain and sample data?

Obtaining data

Observational Experimental

Sampling data

Probability sampling Random sampling Unequal probability sampling

How do we measure statistics?

Measures of center Measures of variation

Definition Example – employee salaries

Measures of relative standing

The insightful part – correlations in data

The Empirical rule Summary

8. Advanced Statistics

Point estimates Sampling distributions Confidence intervals Hypothesis tests

Conducting a hypothesis test One sample t-tests

Example of a one sample t-tests Assumptions of the one sample t-tests

Type I and type II errors Hypothesis test for categorical variables

Chi-square goodness of fit test

Assumptions of the chi-square goodness of fit test Example of a chi-square test for goodness of fit

Chi-square test for association/independence Assumptions of the chi-square independence test

Summary

9. Communicating Data

Why does communication matter? Identifying effective and ineffective visualizations

Scatter plots Line graphs Bar charts Histograms Box plots

When graphs and statistics lie

Correlation versus causation Simpson's paradox If correlation doesn't imply causation, then what does?

Verbal communication

It's about telling a story On the more formal side of things

The why/how/what strategy of presenting Summary

10. How to Tell If Your Toaster Is Learning – Machine Learning Essentials

What is machine learning? Machine learning isn't perfect How does machine learning work? Types of machine learning

Supervised learning

It's not only about predictions Types of supervised learning

Regression Classification

Data is in the eyes of the beholder

Unsupervised learning

Reinforcement learning Overview of the types of machine learning

How does statistical modeling fit into all of this? Linear regression

Adding more predictors Regression metrics

Logistic regression Probability, odds, and log odds

The math of logistic regression

Dummy variables Summary

11. Predictions Don't Grow on Trees – or Do They?

Naïve Bayes classification Decision trees

How does a computer build a regression tree? How does a computer fit a classification tree?

Unsupervised learning

When to use unsupervised learning

K-means clustering

Illustrative example – data points Illustrative example – beer!

Choosing an optimal number for K and cluster validation

The Silhouette Coefficient

Feature extraction and principal component analysis Summary

12. Beyond the Essentials

The bias variance tradeoff

Error due to bias Error due to variance Two extreme cases of bias/variance tradeoff

Underfitting Overfitting

How bias/variance play into error functions

K folds cross-validation Grid searching

Visualizing training error versus cross-validation error

Ensembling techniques

Random forests Comparing Random forests with decision trees

Neural networks

Basic structure

Summary

13. Case Studies

Case study 1 – predicting stock prices based on social media

Text sentiment analysis Exploratory data analysis

Regression route Classification route

Going beyond with this example

Case study 2 – why do some people cheat on their spouses? Case study 3 – using tensorflow

Tensorflow and neural networks

Summary

Index

← Prev
Back
Next →

← Prev
Back
Next →