What you can learn from these 8 conversations between a Data Scientist and his boss

I’m sure many of you analytics folks have been confronted with situations where your boss, your client, your wife or husband -just kidding-, in general the one you report to, comes up with questions you think are pretty far away from what the scope of your job is. You might be “hiding” within a big team and your job might just consist of preparing something for a more senior data scientist to deliver it to business… but one day, you might be in this more senior role, confronted with the same set of questions… I think this post is also relevant for you.

On the other hand, if you are not on the supply side… aka. you are “the client”, it’s important to understand the Data Scientists’ point of view and what they think of the demand side.

In this post, I dissected 8 usual conversations between both sides, analyzing the background and providing some tips I find very useful on how to handle them.

1. The what the heck went wrong conversation

Targets are high, campaign is running for a few weeks already, bucks are massively spent in all sorts of advertising, the offer is damned good… Yet the numbers are not quite the ones we expected. Why!?!?

Obviously this is a quite important and legitimate question. You are going to hear it very often, unless you are working for a booming business where the pulse of the market doesn’t leave room for bad results. But these booming businesses usually neglect the need for analytics, so you probably aren’t. 🙂

You most probably are going to be doing a root-cause analysis à la most classical descriptive analytics.  In your conversation with your boss, following points can be very useful:
Open up the scope.. you guys are not alone in the market and your competitors don’t sleep. Be certain similar campaigns to yours have been running in parallel taking their piece of the cake. In your conversation you should mention that and rise the need for competition monitoring.

Understand how the targets were set.. that’s right, you must be under performing against a set of predefined targets for the campaign. Who set them? Which procedure was used to define them? Over which time period? You better tell your boss, that  that you’d like to understand or be involved in the targets setting process next time it takes place.
Consider the market clock ticking pace: the demand for your product might be subject to strong seasonality trends. The campaigning folks might have taking them into account, but you might want to proactively offer a seasonality analysis if they don’t have any with the right level of rigor yet.

Understand the precision of  the measuring system: sometimes and quite often when it comes down to brand campaigns, measuring the results of a campaign is everything but trivial. Are you sure your data is right? If not, you better offer to perform an audit. If you can’t do it short term, make sure at least your measuring system is consistent (-even consistently wrong would help-), otherwise you can’t compare results with previous campaigns.

Tell the whole story of the campaign: you probably report against hard KPIs, like items sold, number of contract extended, number of upgrades or cross-sales… You can introduce a new set of KPIs to measure engagement, reach, increase of average touch-points with your brand per customer, etc. Maybe your campaign underperforms in terms of sales but delivers awesome results for additional soft KPIs.

2. The what did we do to get such an awesome results conversation

That’s indeed the other side of the coin. Your results are way better than expected! Everybody is over excited and nobody is going to have time to stop and try to understand what worked out so well.

The chance of this conversation to take place is quite low, but you should try to make it happen. Again, it calls again for a root cause analysis, where you consider ALL the points I provided in the previous conversation, but where you need to justify better why you should do things in a different way:

If your competitors are going through a tough time, you have to take advantage of the situation and increase your marketing budget to take the most, now that you can.

If the market is so favorable -it might be reporting big successes for your competitors as well-, you should press the gas pedal even harder.

Your measuring system might be somehow inflating the numbers, so be certain it’s accurate.

Soft KPIs are equally important to understand if the campaign has been even more successful than what you thought.

In any case, you need to identify what worked out so well, extract the best practices and make sure you apply the learnings the next time. If you create an executive summary with all these points, everybody is going to see the need for the analysis of not only failing but also successful campaigns and the benefits of having a best practices catalogue

3. The your forecast vs. my targets conversation

Let’s say you are already part of the forecasting process. Let’s say your analysis takes into account everything you need to be comfortable making following statements:  According to our analysis, we are going to make 130K sales in the next 3 months with the current campaigns with an estimated error of +/-10% .

But when you think this time the targets defined for your campaign are defined according to a much more thorough, systemic and scientific approach, you might be hearing back from your boss something like I sat down with the steering board and I got my new targets. We need to hit the 200K barrier. How can we do that?

Relax! but not too much! Your boss is not telling you he doesn’t trust your analysis, your boss is telling you that he is in big trouble if your analysis is right! Of course he can go back to the board and challenge the targets -be certain he is going to try that, but be certain it is not going to change much the targets definition-… in any case you need to adjust to the new targets.

It’s about identifying which keys need to be hit, understanding the levers and running an scenario analysis for each lever or combination of levers. But it doesn’t stop here… you also need to understand the cost of pulling a particular lever and the mid term and long term implications of the short term push.

I know your time is finite and you have A LOT of things to do, like reading inspiring blogs, but consider anticipating the scenario analysis, providing it with the initial forecast analysis… so that your boss is armed with the right weapons to negotiate better the targets. You both are going to have a better life with achievable targets, believe me.

4. The these are my targets, tell me what to do conversation

Well this is a different formulation of the previous conversation, but making even more evident the need for a Data Scientist to know the business domain, the industry, the market, the department and everything around!

I really like the article The one language a Data Scientist must master… the language meant there is not R -although for me it’s a real must-master as well-, but “business”.

Sorry to disappoint your, but you are not a data cruncher, an R programmer, a Hadoop specialist or an Excel freak…  your ultimate goal is to contribute the best you can to the success of your company! Think of a Data Scientist working for the extinct Encarta; do you remember this company, probably not… it was the Microsoft owned proprietary software encyclopedia quite popular prior to the online open-source community driven Wikipedia. Your goal as a Data Scientist there wouldn’t have been anything but helping design a better product, increasing the users base, advising on interactive content, etc… You could have predicted  the success of Wikipedia or at least detected the adoption pace to react promptly enough… to develop a strategy to avoid being completely flashed away from the market. But they disappeared and with them all the good or bad data science they might have been producing. Do you see the point? Don’t limit your job to your title or to what’s written in your job description. Your boss’s targets are the same as yours!

And what you need to deliver in these case is called prescriptive analytics... Basically making the data tell your boss what’s the decision that should be taken next and of course, what happens if the decision is not taken. This is the most difficult task all along your department, because it requires you to understand very well each and every part of your business… but come on! that’s why Data Scientist is the sexiest job of the 21st century, isn’t it?

5. The we’ve been doing it this way for years conversation

Your job as a Data Scientist in residence, is supporting the transformation from Hippo’s (Highest Paid Person’s Opinion) decision making to data-driven decision making. Obviously a transformation usually is everything but smooth.

You might be challenging well-established ways of working, standard reports, maybe core KPIs your department has been reporting to the company board for quite a long time. You might be breaking the principle of continuity or consistency over time. Let’s take something as simple as the online conversion rate…

But let’s say you acknowledge the fact that not everybody coming to your site is interested in buying… maybe the visit purpose is just checking the account credit or downloading the monthly bill. Let’s say you put a mechanism in place to strip out these visitors and your conversion rate massively improves.

Improving is not a bad thing, but you might have other KPIs going in the ‘getting worst’ direction, so you better have a good explanation behind and show the advantages of the way you are suggesting of computing the KPIs.

6. The can you give me something now conversation

You are extremely worried about the quality of your deliverables… accuracy is part of your reputation as analyst. You know that the worst enemy of quality is rush… or more elegantly expressed, you need to deal with the quality – time-to-market trade-off…

Your boss is going to push for time-to-market, that’s clear… you are going to be in a kind way “laughed at” for being the accuracy freak.  You think accuracy is an hygiene factor.  What can you do?

Be sure you have a method to assess the accuracy of your analysis. I’m sure you have one, but make sure you make it part of your deliverables.

Find a way to express the accuracy value in business terms. How can your boss understand something like MSE or MASE? You better come up with a confidence interval (+/-5%) or something everybody can read.

But also see your boss’s point… What’s the purpose of your analysis? Which decisions are going to be made upon that? And what are the implications of not hitting the accuracy threshold you’d love to have?

7. The cool, but what can I do with that conversation

This time you came up with something really cool. Your d3.js animations are worth it an entry in the Mike Bostock gallery (bl.ocks.org/mbostock).  Moreover, you created 2 new KPIs that are going to change the way the decisions are taken in your department. Your big moment is there… Your really put a big show presenting your stuff… And then, the “Awesome, but what can we do with that?”

Obviously you missed 2 essential points… the actionable part and the benefits of actioning your insights.  Don’t expect everybody to immediately see what great things can be done with your results… You need to explain the benefit of your analysis for the business… make it crystal clear and wow your audience.

Don’t take your boss wrong… of course he or she is amazed and be certain your work is going to have a much more visibility than you think, but help her or him selling it. Ever tried to put a money value behind your analysis? How much money can you make or save actioning your insights?
Money talks and everybody understands this language! I learned it from my boss and good friend.

8. The cool, I need something sexy for my boss conversation

Everybody needs some marketing, that’s clear… Actually a surprisingly high number of people in your organization, if you work in a big one, have made self-marketing their full time job… To survive in this jungle, your boss and yourself need to play a bit according to these rules.

Power Point has been the self-marketing instrument par excellence for many years. You meet real slide-ware masters, who have been polishing their PPT skills for years. It’s very difficult to wow an audience with PPT… whatever special eye-catching effects you try, you are not going to be the first one…

Interactive playable charts are kind of in trend… javascript is sexier than powerpoint and Data Visualization an essential instrument to sell Data Science.  And it’s still a bit exclusive… not everybody who can put together a good looking PPT can create a cool charting visualization.

That’s why your boss is asking you to help here! Make sure your visualization is compelling and everybody is in a position of using it intuitively, that it works on a iPad or a tablet -management devices- and the benefit of your insights is conveyed.

If you’ve read this post until this point -I hope you enjoyed it-, you probably realized that the scope of a Data Science job is much broader than what you thought… That’s the set of skills you need go beyond the mere machine learning, statistical, visualization and programming… you need to be an expert in the business you are working for.

P.S.: I think memes are an quite evolved and powerful form of micro-blogging… that’s why I like to use them to illustrate concepts or situations. I created them with memegenerator.net