Over the course of this book we have developed a strategy for data science projects which includes the beginning stages of uncovering what the customer really wants, through to understanding the roles that your team plays in delivering to meet that need. It is now time to take a step back and see the whole picture.
What’s the difference between a data science project that succeeds and one that fails? A successful data science project begins with a clear understanding of the customer’s needs and ends with results that can be understood in a usable platform, and along the way, the project leader must convince people that the project is worth doing.
Examining the process from start to finish enables you to not only understand what is needed at any particular moment in the journey, but to understand the bigger picture requires you to step back and think about how the different regions relate to each other and how to develop a common framework to apply to improving the efficacy of your data science team and your data science projects.
One word has reappeared constantly throughout this journey into better data science projects: trust. The big problem when you’re trying to get someone to trust you is that it doesn’t come overnight—instead, gaining someone’s trust is an incremental process, similar to the stages needed to create an oil painting, from first sketch to detailed coloring.
For data scientists, trying to gain the trust of others in your organization or trying to win the trust of customers can seem very slow. Making a model is very quick. Activities like data preparation are much slower, but still not as slow as convincing someone to trust you.
Fortunately, an individual project often provides many opportunities to build trust, as we have seen. There is a crucial opportunity when you first engage with someone who has a problem—you can win their trust by listening and understanding their problem, nearly to the same extent as you can by actually solving their problem.
Gaining trust means not wasting opportunities. A cookbook published around the beginning of the millennium advocated nose to tail eating.1 The philosophy of nose to tail eating is often summed up by the author’s slogan, “if you’re going to kill an animal, it’s only polite to eat the whole thing.” In data science, if you’re going to take someone’s time and their data, it’s only polite that you discover all the lessons that are available in doing so.
Every time someone lets you work on their problem, they are taking a risk. Repay that risk by understanding that working on someone’s problem is a golden opportunity. Any project you work on gives you multiple opportunities to convince someone to trust you, even if the project doesn’t make it through to final implementation.
Don’t waste those opportunities, especially the opportunities that seem like you missed. After all, it’s no great challenge to make the best of a project that turned out well. The trouble is that there will be projects that don’t turn out well, and you still need to make the best of those.
To make the most of those missed opportunities, you need to take as wide a view as possible of what the possible lessons are. If what you learn from a data science project is how to work better with your local database administrator, there’s some value there, and if the two of you now have a better working relationship, you owe it to the customer who helped you get there to find something of value in the data together.
It’s only polite to make the most of what people give you.
Building trust into your models is a virtuous circle. When you work as closely as possible with your customer, you build something they want. When you build something they want, the more freely they will talk to you about their real needs, and the more likely it is that you will build something they want.
To be sure, there are exceptions, and it can happen that success against reasonable expectations is rewarded by new unreasonable expectations. Even so, it is far easier to win people’s trust, and therefore ensure they give you their time by succeeding at what you do, and convincing them that you have even more to give them.
Losing trust goes the other way. If they don’t trust you, the more guarded they will become, the less open they will be about what they actually want, and the more likely that what you build doesn’t meet their requirements. Another outcome is that the choice of deadlines can become less reasonable—and less flexible if changed is needed—further deepening the trust deficit. This is a vicious cycle indeed.
Far better to be in a cycle that repays you by increasing your chances of success, rather than defeats it. Moreover, if nothing else, this book should have shown that you have considerable control over the way people react to your work.
At the center of data science success is the ability of your project to solve a problem for humans, which means understanding what humans somewhere want. It means avoiding reliance on algorithms and ensuring that you rely on the team’s own human intuition. It’s the human intuition that will bridge the gap between what the models can do by themselves and what people actually want.
Succeeding at the human side demands more human interaction instead of more time spent developing models.
The human side of data science is often also the hidden side. By understanding the way that your users will engage with what you have created, you will ensure that they will appreciate what you have created to its fullest potential. You will also make it far more likely that you will be asked back to help on more projects.
As long as your team remembers that being asked back isn’t guaranteed, but is dependent on how well they have convinced your users of the value of your work, your team will go far in being accepted as being on the side of their users and being lauded for their achievements.
As long as you remember how the team’s performance shapes how people value what you have done on your behalf, your reputation will grow and you will continue to enjoy increasingly exciting opportunities over the course of your career.
Data science means many things to many people. Regardless of where you come from, I hope—and believe—you can apply something from this book to making sure the data science products built in your organization succeed in making your users’ lives easier and promote data science as a great tool to solve problems in many contexts.