administrative data 156–7
advertising 125
algorithms 157–8, 159, 161–94, 280
biased datasets, training on 174, 176
effectiveness, testing 179, 186–9, 191, 193
fallibility 169–71, 172–3, 175–6, 179, 180
false positive issue 169–70, 187, 188
pattern-recognising 162, 164, 166, 168, 169, 193
racial bias 187–8
‘winter detector’ problem 164
Amazon (retailer) 156, 175, 185, 192
Anderson, Chris 165
Angwin, Julia 186
antidepressant medications 131–2
Argentina, inflation 205–6, 223
art forgeries 19–23, 30–3, 45–7
Asch, Solomon 142, 143, 145, 154, 275
Atkinson, Sir Tony 88
Attenborough, David 292
autism 56
Avogadro’s number 260
Babbage, Charles 233
Babcock, Linda 28
Babson, Roger 278
‘backfire effect’ 135
bail decisions 178–9
current offence bias 178
Bangladesh 64
Bank for International Settlements 106
Bannon, Steve 14
Bargh, John 126–7
barometer 182
base rates 267–8
basketball 121–2
Battistella, Annabelle (Fanne Foxe) 196, 197, 225
Bell, Vanessa 271
Bem, Daryl 115–17, 118–19, 121, 126, 127, 128
Berti, Gasparo 181
Bevacqua, Graciela 205, 206, 207, 225
Beyth, Ruth 262, 263, 264, 268
Bi Hua 65
biased assimilation of information 36–8, 137
optimism bias 101
publication bias 118, 119, 120, 121, 123, 125, 127, 128, 131, 133
selective attention 114
survivorship bias 115, 117, 118, 128–9, 130–1, 132
unconscious 41
big data 156–9, 160, 161–94, 280
accountability and transparency 190, 191, 192, 193
administrative data 156–7
algorithms see algorithms
missing data 157, 158–9, 173–4
N = All assumptions 158–9, 160
privacy issues 192
sinister aspects of 166–7
traps 171–2
big numbers 98–100
Bird, Sheila 72
‘bird’s eye view’ 49, 64, 65, 67–8, 279
Blastland, Michael 11, 72–3, 98, 226
Bloomberg TV 93
body temperature 174–5
Borges, Jorge Luis 123
Bredius, Abraham 19–20, 21, 22, 23, 30–3, 36, 45, 47, 83, 255, 277
Brettschneider, Brian 237–8
British Medical Journal 71
Brown, Derren 120
Brown, Zack 112–13
Buchanan, Larry 243
Buffett, Warren 274
business writing 128–9
Cairo, Alberto 241
Cambridge Analytica 166–7, 171
Cameron, David 154
Campbell, Donald T. 62
Campbell Collaboration 141
Canadian statistical agency 208, 224
cancers 3–7, 12–13, 16–17, 24–5, 41, 54, 293
capital punishment 36–7
cardiac surgery 60
Carter, Jimmy 198
cash-and-mentor schemes 64
cost-benefit analysis 210–11
response rates 155–6
sample bias 156
sample error 156
Chalmers, Sir Iain 139
Chambers, David 274
child benefit payments 149
child mortality 69–72, 95, 97, 104
child neglect or abuse 179–80
Covid-19 pandemic 8
economic data 214
famine 214
Gini coefficient 97
jam experiment 109–11, 115, 118, 129
navigating 111
cigarettes, and lung cancer 3–7, 12–13, 16–17, 41, 54, 293
clickbait headlines 112
climate change 35–6, 38–41, 94–5, 105, 283, 284
clinical outcomes 60
clinical trials 64, 125, 131–2, 133–4, 136, 139, 147–8, 191
preregistration 133–4
randomised controlled trials (RCTs) 5fn, 55, 64, 131, 139, 191
sex-dependant effects 147–8
under-representation of women 147
Clinton, Bill 199
Cochrane, Archie 138–9
Cochrane Collaboration 138–9
cognitive reflection test 43
Colbert, Stephen 289–90
Cold War 94
college rankings 61
common-sense principles 12
COMPAS algorithm 186–9
conformity pressure 142–5
context and perspective 91–108, 279
news content analysis 91–6, 100–7
time 93–6
Corbett-Davies, Sam 187
Corbyn, Jeremy 15
cortisol 126
Cotgreave, Andy 246
Covid-19 pandemic 7–9, 10, 27, 30, 72, 125fn, 159, 240fn
gendered effects 148
misinformation 27
mortality figures 72
news coverage 107
selective attention bias 114
wishful thinking and 30
Cowperthwaite, Sir John 212, 214–15, 216
Crawford, Kate 158
see also wishful thinking
crime
bail decisions 178–9
convictions and sentencing 177–8, 179
re-offending risk 186–7
trends 151
Crimean War 226–7, 233–4, 239, 240, 249, 250
crowdfunding see Kickstarter
Cruz, Ted 238
Cuban missile crisis 94
Cukier, Kenn 166
curiosity 279–94
incremental change 285
information gap theory of 286, 287
scientific curiosity 283–4
situational 285–6
sparking 285, 286, 288–9, 291, 292, 293
currency markets 272
dark data 154–5
data gathering and analysis 109–41, 279
‘backfire effect’ 135
disbelief 127
‘the garden of forking paths’ 123–4, 127, 133
‘interestingness’ filter 128
missing data see missing data
standard statistical methods 123, 124, 125, 135–6
statistical significance concept 135, 136
see also research practices and publications
data science code of ethics 190
data visualisation 226–51, 280
data density 243
decorative function 229–31, 235–7
ease of sharing 237–8
exploratory function 241–2
tone 246–7
understanding the basics 251
visual arguments 241, 242–3, 251
De Meyer, Kris 40–1
death penalty, deterrent effect 36–7
debt and derivatives markets 105–6
decision-trees 176
denialism 24, 25, 29, 37, 38, 40–1, 261–2, 283
Deutsch, Morton 275
diabetes 58
Dimson, Elroy 274
divorce rates 267–8
Dobelli, Rolf 106
Doll, Richard 3–7, 10, 12–13, 16–17, 239, 293
Dollar Street 65–7
doubt
Draeger’s 110
Dressel, Julia 188
Du Bois, W.E.B. 218
Duhigg, Charles 167, 168, 170, 172
Earth’s circumference 99
Eco, Umberto 1
economic forecasts 199–200
see also forecasting industry
The Economist 93
literacy 215–16
campaign finance 289–90
electric shock experiments 145–6
Elliott, Andrew 98–9
emotional response 19–48
cognitive reflection test 43
motivated reasoning and 29, 30, 33, 34–5, 36, 39, 40, 137, 282
noticing and reflecting on 26, 27, 42, 47–8, 107, 279
ostrich effect 25–6
trap of 23, 24, 26–7, 35, 36, 42, 82–3
wishful thinking 27–30, 33, 38, 42, 47
Empire State Building 99
The Empire Strikes Back (movie) 19
Facebook 12, 27, 58, 158, 166, 185, 192, 237
Farr, William 227–8, 234–5, 239–40, 247
fast statistics 59, 62, 63, 65
Feller, Avi 187
Fernbach, Philip 287
Festinger, Leon 252
Feynman, Richard 259
filtering information 260–2, 263, 283
financial news media 93, 105–6
Financial Times 93, 106, 149–50
Finland
Gini coefficient 97
Fischhoff, Baruch 262, 263, 264, 268
Fisher, Irving 252–6, 257–8, 263, 266, 269, 272, 274–7, 278
Fletcher, Harvey 259
folic acid supplements 168, 170
‘fooled by randomness’ 129
football games 29
forecasting industry 256, 263–78
base rates 267–8
Good Judgment Project 266
Intelligence Advanced Research Projects Activity 265–6
misremembering forecasts 262–3, 264, 268
open-mindedness 269
outside and inside views 268
public commitments 275
recording predictions 268, 275
superforecasters 266, 268–9, 275
training 266–7
Frank, Anne 44
Frederick, Shane 43
French Academy of Sciences 184
Frisch, Ragnar 252
Fuentes, Ricardo 81
Full Fact 150
Fung, Kaiser 169–70
‘fuzzy’ data 192
Gallup, George 152–3
‘the garden of forking paths’ 123–4, 127, 133
gender data gap 145, 146–9, 154, 160, 175
Georgiou, Andreas 202, 204–5, 206, 207, 225
Gerard, Harold 275
Germany
crime statistics 202
Nazi statistical system 214
refugees in 201–2
Gini, Corrado 96
global financial crisis (2007–08) 88, 89, 106, 203, 236
global populations 99
Global Wealth Report 85
Goel, Sharad 187
Goldin, Rebecca 74
Goldman Sachs 203
Goldstein, Jacob 120
Goodhart, Charles 62
Google Flu Trends 161–2, 163–4, 173–4
Gove, Michael 292fn
graduate earnings 2–3
Grant, Duncan 271
graphs see data visualisation
ELSTAT statistical agency 202–3, 204, 206
Greenpeace 44
Greifeneder, Rainer 111
Guardian 71, 78, 79, 81, 82, 106
gun deaths 76–8
Hand, David 154
HAR King (Hypothesising After Results Known) 123
Harper, Stephen 208
Hawking, Stephen 292
Hayek, Friedrich 61
hedge funds 271
herpes 25
Hewlett Packard 159
Hill, Austin Bradford 3–7, 10, 12–13, 16–17, 239, 293
Hillsborough football stadium disaster 77
Hitchhiker’s Guide to the Galaxy (Douglas Adams) 69
Hobbes, Thomas 183
Holmes, Nigel 231
Holocaust 44–5
Hong Kong 63
‘hot sauce paradigm’ 74
How to Lie with Statistics (Darrell Huff) 2–3, 6, 125
Huff, Darrell 2–3, 6, 9, 10, 16, 18, 76, 125, 281, 293
Hurricane Sandy 158
illusion of explanatory depth 287
immigration policies 75
IMPACT algorithm 172–3
In Search of Excellence (Peters and Waterman) 128–9
income inequality 65–7, 81–9, 243–5
Gini coefficient 96–7
highest-earning 1 per cent 87, 88, 96
Index Number Institute 255–6
India 96
unemployment statistics 207–8
income inequality 86–9
wealth inequality 81–6
World Inequality Database 88
infant and child mortality 69–72, 95, 97, 104
inflammatory memes 44
influenza
Google Flu Trends 161–2, 163–4, 165, 166
pandemic 95
infographics see data visualisation
innovation, rate of 12
insider trading 222
Instagram 158
intelligently open decisions 190
International Monetary Fund (IMF) 204, 277
Inuit 144
see also forecasting industry
Ioannidis, John 7, 125, 126, 127, 136
Ipsos Mori 57–8
Iraq 245–6
Iyengar, Sheena 109, 110, 111, 112, 118
jam experiment 109–11, 115, 118, 129
Jamieson, Kathleen Hall 290
Japan 231fn
judges and their judgements 177–8, 179
Kahan, Dan 281, 282–3, 285–6, 290
Kahneman, Daniel 43, 59, 64fn, 101, 126, 127, 137, 268
Kestenbaum, David 120
Keynes, John Maynard 253, 269–74, 276, 277
Kickstarter 112–14, 115, 117, 128
King’s College, Cambridge 269, 270, 272, 273, 274
Kirchner, Cristina Fernández de 205
Kirchner, Néstor 205
Korean War 94
Kosara, Robert 241
Kunda, Ziva 24
Kurov, Alexander 222
Kvitkin, Olimpiy 214
landmark numbers 98–100
Leave Means Leave 75
LePan, Douglas 269
Lepper, Mark 37, 109, 110, 111, 112, 118
Lewin, Kurt 275
Lewis, Martyn 104
Lewis, Michael 14
Lievesley, Denise 207
life expectancy 97, 98, 227, 250
Life Extension Institute 254
Limeburner, Shannon 113
LinkedIn 158
literacy 215–16
Literary Digest 151–2, 153, 173
Loewenstein, George 28, 286, 287
London Olympic Games (2012) 151
London Underground 49–50, 52, 53
Lord, Charles 37
Lovelace, Ada 233
Lundberg, Shelly 149
McDonald, Lynn 228fn
McGregor, David 113
McMaster, General H. R. 61
McNamara, Robert 61
mad cow disease 13
Madison, James 216–17
Malkiel, Burton 131
management metrics 60
Mars, planet 2–4
Martineau, Harriet 250
mass shootings 76–7
mathematical risk models 76
May, Theresa 15
Mayer-Schönberger, Viktor 156, 166
Mayraz, Guy 27–8
medical appointment waiting times 60–1
medical diagnoses 176
Meegeren, Han van 22, 23, 31, 32, 33, 44, 45–7, 48, 255
Meehl, Paul 176
Mellers, Barbara 266
Merkel, Angela 201
Mersenne, Marin 183–4
meta-research 125
microcredit schemes 64
Microsoft 185
Mills, Wilbur 196
mindfulness meditation 141
misinformation 26–7, 43, 237–8
missing data 115, 125, 130, 132, 134, 142–60, 279, 280
big data and 157, 158–9, 173–4
dark data 154–5
gender data gap 145, 146–9, 154, 160, 175
see also censuses; opinion polls
MMR vaccine 56–7
Modi, Narendra 207
Moore, Alan 109
Moore, Don 266
More or Less (BBC radio programme) 11, 12, 49, 76, 98, 106, 281, 291
Mosteller, Frederick 16fn
motivated reasoning 29, 30, 33, 34–5, 36, 39, 40, 137, 282
see also wishful thinking
movie audiences 156
Moy, Will 150–1
Mullainathan, Sendhil 178
Muller, Jerry Z. 60
N = All 156, 157, 158, 160, 163
see also big data
naive realism 57–8
Nasar, Sylvia 276–7
negativity instinct 100–1, 104
Nelson, Leif 124
net wealth 83–4
recommendation algorithm 192
New Yorker 243
New Zealand census 210–11
context and perspective 93–5, 100
frequency of engagement with 100–1
good news stories 104–5
negativity instinct 100–1, 104
surprising or dramatic news 101–2
Nightingale, Florence 226–9, 232–5, 239–41, 247, 249, 250–1
Nikon 159
Nosek, Brian 118–19, 120, 126, 127, 137
nuclear weapons 94
nurses’ pay 90
Nyhan, Brendan 135
official statistics 195–225, 280
censuses see censuses
cost-benefit analysis 210–11
fiddling the figures 201, 202, 203–4, 205–6, 219, 224
financial sensitivity 220, 221–2
government assertion of ownership 216–17, 219, 220
government misuse of 213–14, 219
managerial tools 217
partisan political issue 208
political sensitivity 220
pre-release access 220–3
private sector use of 217–19
public availability of 218–19
statisticians, undermining independence of 204–5, 206, 207, 208
trust in, undermining 201, 206, 219, 222–3
oil drop experiment 258
filtering information 260–2, 263, 283
superforecasting 269
dark data 154–5
representative sampling 153
response rates 153–4
sampling error 152
opioid crisis 12
optimism bias 101
ostrich effect 25–6
Oxfam 81–2, 83, 85, 86, 103, 112
Pac-Man 74
Pascal, Blaise 181, 182, 183, 184
pensions 85
Perez, Caroline Criado 146–7
performance targets 60
perinatal health 139
personal experience 49–68, 107, 279
‘bird’s eye view’ 49, 64, 65, 67–8, 279
naive realism 57–8
statistics in combination with 65, 67
statistics in conflict with 51, 52–3, 54, 56, 57, 58
‘worm’s eye view’ 49, 64, 65, 67–8, 279
Peter, Tom 128–9
Phillips Curve 231fn
Picasso, Pablo 231
Pierson, Emma 187
Piketty, Thomas 88
Planck’s constant 260
Planet Money (podcast) 290–1
polarisation 36, 282, 283, 287–8, 292
pothole-detecting app 159, 174
poverty 95–6
reduction in global poverty 103, 104, 105
World Bank definition 95
see also income inequality
Powell-Smith, Anna 142
Pratchett, Terry 91
precognition 116–18, 119, 126, 128, 134–5
see also forecasting industry
premature enumeration 73, 75–6, 78, 84–5
priming experiment 126–7
prisoner population 58
Proctor, Robert 14
psychological experiments 142–5
public transport occupancy rates 49–54
publication bias 118, 119, 120, 121, 123, 125, 127, 128, 131, 133
Puerto Rico
hurricane 209
statistical agency 209–10, 211
Quetelet, Adolphe 233
randomised controlled trials (RCTs) 5fn, 55, 64, 131, 139, 191
Rapid Safety Feedback 180
Rayner, Sir Derek 217, 219, 220
redistributive taxation 88
Reifler, Jason 135
Reischauer, Robert 197
Reiter, Jonathan 113
Rembrandt van Rijn 19
Remington Rand 257–8
reproducibility problem 111, 117, 118, 119, 127, 135, 137, 184, 185
research practices and publications
fluke results 116, 119, 121, 123, 128
fudged research practices 120, 121, 122, 123, 124–5, 132, 134, 258–9, 260
HARKing (Hypothesising After Results Known) 123
publication bias 118, 119, 120, 121, 123, 125, 127, 128, 131, 133
reproducibility problem 111, 117, 118, 119, 127, 135, 137, 184, 185
second opinions on 138
self-regulation 134
survivorship bias 115, 117, 118, 128–9, 130–1, 132
transparency 132–3
see also clinical trials
Riecken, Henry 252
Rivlin, Alice 197, 198, 199, 225
rolling news coverage 93
Rolodex 255
Rönnlund, Anna Rosling 65, 66, 67
Roosevelt, Franklin Delano 151, 152, 153
rose diagram 228, 247, 248, 249–50
Roser, Max 94
Ross, Lee 37
Royal Statistical Society 223, 227, 233
same-sex marriage 26–7
Samuelson, Paul 252
Santos, Alexis 209
Scarr, Simon 245–6
scepticism, healthy 10
Schachter, Stanley 252
Scheibehenne, Benjamin 110–11, 115, 119, 126
science journalism see research practices and publications
Scientific American 107
scientific curiosity 283–4
Scott Brown, Denise 230
screen time, well-being and 122–3
Second World War 44
Seehofer, Horst 202
self-reported data 3
sexual activity 97–8
Shane, Janelle 164
Sharot, Tali 101
Sheikh, Munir 208
Shenzhen 63
sick building syndrome 13
sildenafil 147
Simmons, Joseph 124
Simonsohn, Uri 124
Singh, Sameer 164
situational understanding 61
see also personal experience
Sloman, Steven 287
slow statistics 59, 60, 62, 63, 65
Smith, Adam 253
Smith, Dr Lucy 69–70
Smith, Wayne 208
smoking-related diseases 3–7, 12–13, 16–17, 41, 54, 101, 105, 293
Somerfeld, Eric 39
Soviet Union 214, 262, 263, 264
Space Invaders 74
species extinction 12
Spicer, Sean 201
Spiegelhalter, Sir David 171, 172
standard statistical methods 123, 124, 125, 135–6
standards for statistical
record-keeping 235
Starbucks 110
state pensions 85fn
statistical metrics 61–2
statistical significance testing 135, 136
Stereotypical Daydream 114, 118
stock market crash (1929) 257, 272, 273, 274, 275
stock markets 25–6, 27–8, 272–6
ostrich effect and 25–6
wishful thinking and 27–8
stock-picking 129–30
see also forecasting industry
storks and babies correlation 1–2, 16
stroke, incidence of 102–3
Subacchi, Paola 202–3
suicide 80
Sunstein, Cass 178
survivorship bias 115, 117, 118, 128–9, 130–1, 132
Sutherland, Dr John 228fn
Sweden 222
system 1 and system 2 thinking 64fn
Taft, William 254
Taiwan 7–8
Tanzania 207
Target department store 167–8, 170–1, 172, 185, 191
teaching quality, measurement of 172–3
teenage drinking 12
television personalities 57
Temne people 144
terrorism, deaths from 58, 59, 101
testosterone 126
Tetlock, Philip 263–4, 265, 266, 269, 275
Thaler, Richard 137 thalidomide 147
Thatcher, Margaret 217 Tierney, John 126
tobacco industry 12–13, 14, 16, 41
see also cigarettes, and lung cancer
Tooze, Adam 214
Torricelli, Evangelista 181, 183
Transport for London 50, 51, 52, 53–4
Trials (journal) 134
Trudeau, Justin 208
Trump, Donald 8, 15, 44, 84fn, 98, 135, 155, 167, 201–2, 209, 219, 220, 222
trustworthiness 189–90
Tufte, Edward 231, 237, 243 Tversky, Amos 101
2001: A Space Odyssey 161
Uganda, labour force growth 148–9
understanding what is being measured 69–90, 108, 279
premature enumeration 73, 75–6, 78, 84–5
unemployment statistics 11–12, 201, 207–8, 219, 220, 223–4
UNESCO 215–16
United Kingdom
Active Lives Survey 151
Brexit 291–2
British Election Study 153–4
child benefit payments 149
Crime Survey of England and Wales 151
demographics 99
divorce rates 267fn
elections 154–5
infant mortality rate 69–70, 71–2
London riots (2011) 177–8
London transport occupancy rates 49–54
lung cancer rates 4
medical appointment waiting times 60–1
Office for Budget Responsibility (OBR) 199–200
Office for National Statistics 200, 211, 219–22, 223
population 99
self-harming 78–80
sexual activity 97–8
stroke, incidence of 102–3
unemployment statistics 219, 221, 223–4
Universal Credit payments 150
United Nations Sustainable Development Goals 150
United States
census 155
Centers for Disease Control and Prevention (CDC) 161, 163
child neglect or abuse 180
climate change politics 35–6, 282, 283, 284
Congressional Budget Office (CBO) 196–7, 198–9, 200, 211, 216–17
GDP 99
Gini coefficient 97
gun deaths 76–8
gun violence 76–8
homicide rate 92–3
Hurricane Sandy 158
income inequality 243–5
infant mortality rate 70–1
influenza, spread of 161–2, 163–4, 173–4
Mexican border wall 98
national debt 100fn
9/11 101
official statistics compilation 200
Political Action Committees (PACs) 289–90
population 99
Prohibition 255
smoking-related deaths 101
teaching quality, measurement of 172–3
unemployment rate 201
Vietnam War 61
Yale graduate earnings 2–3
university rankings 61
‘unskilled immigration’ 75
US News and World Report 61
vacuum experiments 183
Venturi, Robert 230
Vermeer, Johannes 19–22, 23, 30–3
Christ at Emmaus 19–20, 21, 22, 23, 31–3
Christ in the House of Martha and Mary 31
Girl With a Pearl Earring 21
Woman Reading a Letter 21
The Woman Taken in Adultery 22
Viagra 147
Vietnam War 61
violence, video games and 73–5
Waterman, Robert 128–9
WebMD 140
well-being, measuring 122–3
Wheelan, Charles 2
‘When I’m Sixty-Four’ (Beatles) 124, 137
whistleblowers 190
Wilkinson, Norman 231–2
wishful thinking 27–30, 33, 38, 42, 47
Woolf, Virginia 271
World Health Organization (WHO) 8
World Inequality Database 88
‘worm’s eye view’ 49, 64, 65, 67–8, 279
Yale graduate earnings 2–3
Yong, Ed 188
YouTube 191
Yue Hen 65
Zelazny, Gene 242