part0132

Chapter 7: Most Common Data Science Problems

Regardless of whether you pursue a full-time job in the field, or if you’re using data analytics in your pre-existing career, you’ll face certain problems with your work.

You can’t always have a flawless and efficient workflow, no-one can, if you could, you’d soon enough become obsolete because there’d be countless people like you.

While some parts of working in data science are utterly amazing, there are still some issues.

You can easily get frustrated, especially since most of your superiors won’t know in detail what you do.

It’s very difficult to communicate with non-data-analysts precisely what you do.

Because of this, the post is prone to misunderstandings and mismanagement.

While all that’s true, some of the problems you’ll have can be managed and resolved.

In this section, we’ll look at the most common complaints that people working with data analytics have had in the past, as well as how to resolve them without much consequence.

This issue is especially prevalent in positions that require you to do a degree in data modeling.

Most of data modeling concerns gathering and cleaning the data so that it’s actually usable.

This is obviously quite a bit of an issue on the manager’s fault, as many of them will just suddenly come up with an idea and expect it to be done last-minute.

Obviously, sometimes the modelers are at fault, but unfortunately, more often than not, it’s managers simply not understanding the job.

In the management world, it’s quite common to insert things last-minute, but in data analytics that’s basically impossible.

Your manager might just pop in and say, “Hey, we’re going to include a social media history in our latest analysis. Cool? Cool, I’ll see you in 15 minutes when it’s done.”

Now, if you sigh at this kind of request, then at least there are some solutions for it.

Not resolving this issue is bound to either cause serious delays, or some serious dissatisfaction from your managers.

The worst thing here is that both sides of the argument are entirely understandable.

The data scientists simply can’t deal with this in such a short time span, and managers will have a hard time understanding that.

Serious complaints about managers being unreasonable and expecting the world are quite common in most technical fields, especially those concerning programming and AI.

Fortunately, some solutions exist, and most of them are concerned with improving your communication skills while at the same time being clear about the possibilities of what is possible, and what isn’t.

First of all, you should keep communication open, but keep a firm “no changes” date.

After that date, make sure your manager is aware no changes will be processed.

Unfortunately, some managers will not be swayed by this.

You can’t expect your manager to be perfectly well versed in data analytics.

The main mistake managers make here is expecting data scientists to utilize datasets which either contain bad, little, or no data and actually have something to show for it at the end of the day.

It’s imperative to explain to your manager what you can and cannot do.

Give them a few useful articles to read about what ML and AI can actually accomplish, rather than what they’ve probably read.

These days ML and AI are being hyped up to be essentially omnipotent and capable of turning any dataset into extremely valuable information.

Unfortunately, as you know, this is quite far from the truth.

The analysis you make has a limit on how good it can be, and that limit is the quality of the data you’re given.

Naturally, you can use interpolation and extrapolation to “plug” the holes in a dataset, but it’s not like there’s a magic wand you can just point at the computer to create data.

If you’re given a week of sales info, it doesn’t matter how good you are; you won’t be able to predict the sales of next year accurately.

The best thing you can do about this is to pay attention to what kind of company you apply to.

Do they already have many data scientists on board?

Do they collect a lot of good data already?

Are they maybe adaptable enough to start collecting it as soon as you join?

If the answer to these questions is no, you might want to reconsider working there.

It’s important to address this early on so it doesn’t affect you in the future.

Besides that, try to explain to your manager that last-minute alterations are very difficult, and try to use phrases like “Yes, I could totally do that, it’s just going to add about options days to the schedule.”

Your manager’s going to be singing a different tune soon.

Generally, people think of data as a set of information, a truth if you will.

Data is merely facts until someone comes by and puts some context into it.

This is an issue that can affect basically everyone; your boss, your manager, even you might fall into this faulty mindset.

Being careful not to think about data as the information is one of the most crucial parts of being a data analytics expert.

Fundamentally, it’s extremely important to remember that even if your title is “data analyst” when it comes to actual work, the analyst comes before the data.

Fostering a data-first culture in the workplace is a surefire way to have every one of your endeavors heralded by utter failure.

It’s easy to forget that data needs context to be useful, and it is so, so easy to fall down the slippery slope of worshiping data.

Giving the context is your job; your job is to think about the data, to frame it.

The data itself is like a wench is to a mechanic.

You don’t go praising the wench for fixing the car, so you shouldn’t rely on the data too much either.

You need to know the broader conditions, for example, market trends which aren’t in the data need to be considered.

While your managers might be most inclined to trust in the numbers, your job is to reveal where those numbers might be faulty, what might be affecting them, and what the truth is closest to.

Fortunately, this is an easy problem to solve; just let them have it.

If your manager gets burned for a few million because they trusted data more than you, then next time, you can be sure that they’re going to pay more attention to your words next time around.

Now, you also need to consider the bias of data collection when dealing with work.

All data collection processes are susceptible to certain biases.

Let’s say that you’re analyzing a market based on how many people buy from the company site.

In this case, the bias is on younger, more tech-savvy people, as older people are more likely to buy from brick and mortar stores.

Unfortunately, when it comes to being a data scientist, your recommendations are likely to end up in one of three ways: A bonus, a promotion, or expulsion from work.

The danger of working as anyone that concerns themselves with data analytics is that you will often have to profess the bad news to your bosses.

Unfortunately, not all of them have read Sun Tzu’s book of war and refuse to shoot the messenger.

If your data analysis shows that there are serious problems in the company, or even that the company is headed towards its own destruction, it’s quite likely your bosses will be less than kind.

Presenting this information can feel very awkward and uncomfortable, and can sometimes end up in disastrous consequences.

In most cases, you won’t be to blame for this, but you are an easy link to scapegoat.

Any manager can easily put the blame on you, and your boss might not be well versed enough to see through it.

Now, ultimately, this is an issue you cannot precisely solve.

If your boss is blaming you for what you find out after digging through the company data, you’ll probably want to check if your resume is up to date as soon as possible.

You don’t need, and shouldn't put up with being attacked for doing your job.

If you’re really committed to solving the issue, try assigning blame to yourself.

You can’t hold your job if you sidestep telling your boss about things that were others’ responsibilities.

Even if companies are trying to be modern and adapt to changes in the industry swiftly, fundamentally, most companies are still run in an old-fashioned manner.

The complaint that Strand has articulated is extremely common in data scientists, and the chances that you aren’t going to run into an example of this in your career are close to nil.

A recent study has shown that ⅔ of all managers distrust data, and would rather hand over decision making onto their intuition, rather than trusting scientists.

Unfortunately, these are generally mid-level managers, who have just enough power to feel like they’re important, but not enough power to affect the decisions made on a broader, company-wide scale.

Most data scientists get stuck with working for one of these at least at one point in their careers.

You’ll find that you have to convince the management of practically every new decision you have to make.

Are you trying to make a financial model of the company spending so you can budget accordingly?

Well, too bad, because Steve from management has decided that his intuition tops that.

Even in the case that you’ve actually gotten approval for your project, you’ll still face challenges with getting management to well...act accordingly.

Even if your model showed that your company spends too much on marketing, good luck convincing your managers of that.

This is why skills in communications are so useful for any role related to data science. All of the analytical skills in the world are going to be useless if there’s nobody to take action upon them.

Your results won’t have even the slightest impact on the firm unless you’re able to engage upper management enough with your speech, data presentation, etc.

This is why it’s important to keep in mind soft skills, as well as your ability to do presentations and visualizations of projects.

It’s much easier to convince management if you’re showing them shapes and figures, rather than Excel spreadsheets.

Try running your presentation by a friend that has absolutely no technical skills; this will prove to you whether your presentation is fine.

Pay attention to what questions are posed to you, and try to address them more clearly in the presentation.

It’s also sometimes useful to try to explain your ideas to an inanimate object.

This lets you pay attention to how you talk, as well as how you communicate the data without the need to have an actual person with you there.

With that being said, don’t feel too bad if it doesn’t work out.

Sometimes your managers will simply elect not to listen to the data, or decide that something is simply more important.

A relatively recent case of this was when data analytics showed that Grace & Frankie’s promotional images worked the best without the show’s star.

The team of executives at Netflix then had to think about the pros and cons of excluding the lead, Jane Fonda, from the images.

In the end, they elected not to, partly not to anger the lead, and partly because the show would be more “iconic” if the lead was present, rather than if promotional images were used exclusively as an advertisement.

The only fortunate thing here is that this is a bit of a cascading issue.

If you fail a few times, management is unlikely ever to trust you again.

On the other hand, if you bring success a few times, you’ll build their confidence in your data, and they’ll be much more likely to trust you with important projects.

It is a matter of picking your battles, so to speak, try to only engage where you are absolutely sure you can succeed.

You might have noticed that the overarching theme here is communication, and it is.

While your data analytics and portfolio are the things that will let you get the job and perform it well, mere performance isn’t enough.

To make your day to day life better, and your career more successful, you have to practice communication and learn how to speak to your managers in the most effective ways possible.

If you’re looking to hone your communication skills, look no further than those same managers you take issue with.

They tend to be quite good at communicating with their bosses, talking to them, and paying attention to the terms and tactics they use can be an excellent way to learn communication skills.

Try to make your every email sound more professional, your every message to be more concise and effective.

The same way you analyze in your job, analyze your approach to your job, think about what the most effective words to use are, and when to use them.