Data science consulting: Part 1 of however many I feel like
|Aug 6||Public post|| 12||1|
In one of my last newsletters, I linked to a Reddit thread, “How does data science work in the consulting space?” and said that if there was enough interest, I’d cover some aspects of data science consulting in the newsletter from time to time. This is the first of those pieces.
Last week, Pardis published a great piece on how data science teams can be organized in various companies:
Designing and building a data science team is a complex problem; so is determining the nature of interactions between data scientists and the rest of the organization.
and covers several ways that data science work can take place in an organization. She ultimately comes to the conclusion that the best model is a hybrid model, where data scientists are part of product teams and report into a centralized data science function.
This debate around what a data scientist is and where data scientists belong has been going on ever since the job title was defined. It’s hard enough to figure out where and how data science belongs in-house.
But how about consulting? How do data scientists work in a consulting capacity? To understand that, it makes sense to first understand what consulting is.
Some years ago, I put together an insanely awesome data analysis. If I told you the SQL gymnastics I had to go through to get the data (window functions, cursors, multiple databases on different servers), you would give me a standing ovation.
I then made the most beautiful Powerpoint you’ve ever seen in your life. Bubble charts, sparklines, a master slide deck theme, the works.
I still think about that presentation some days.
I handed it to my boss gingerly, like a newborn baby. “I made this per your request,” I said with bated breath. He skimmed it. “This looks great! Want to present it at our exec meeting?”
Me? Present? To executives? The people that wear ties and Bluetooth earpieces and say things like, “Sharon, hold my 2 o’clock, I have a meeting at 1:30 and I’m coming in hot,” unironically? He might as well have asked me to present to the UN Security Council.
“I’ll help you tweak it a bit,” my manager said, encouragingly. “Ok,” I said. “First, take out all the slides where you talk about how you did the analysis. Put those in the back. Move the charts forward. Delete these numbers. Make the headlines bigger.”
I gulped. But that was all my work. What were we doing to it? We were rewriting everything! I wouldn’t even get a chance to talk about my confidence intervals! My churn visualizations! My beautiful window queries!
When I got to the executive meeting, I was a nervous wreck. All of the Bluetooth earpieces turned in my direction. “Ok, so I’m going to run through this analysis,” I said. “This is the first slide,” and I proceeded to describe everything on the slide, word for word. I saw their attention waning. I went through the second set of figures, laid out the percentages, and I started seeing people looking at their phones.
By the end of the presentation, I could tell that I’d lost them.
“Don’t get discouraged,” my boss said. But I understood something intuitively then that I wasn’t able to put into words until much later:
Half of your job, regardless of what that job is, is being able to sell your work.
This principle also forms the basis of consulting. Consulting, at its core, is about two things:
1) Being an expert at something and
2) Convincing other people that you know something well enough that they need to pay you for their expertise, in either money or time.
Last week, I tweeted:
Joel 🌧@jhooksI set up a calendly link for an hour consultation. Didn’t expect anybody to actually use it tbh. Somebody did! Looking forward digging i to their business and offering my thoughts. https://t.co/96rysZqJDL
which was an extension of this principle.
It’s something I didn’t understand at all until I joined a consulting company, but the ability to convince someone of something is probably the strongest superpower you can ever have in your career.
In an in-house data science position like the ones discussed in the piece, the ability to being convince people that what you do is valuable is mostly separated from the actual money of the business, unless you work very hard at it.
For example, let’s say you create a model that predicts when your customers are unhappy with your SaaS application, and have customer service call them and offer them coupons. They aren’t so unhappy that they’ve cancelled yet, but there’s the potential to cancel. How do you calculate how much money you’ve saved the company? How does that figure into a company’s bottom line for the year?
Or, let’s say you build a machine learning model that predicts code autocompletion and saves developers two seconds per command when they’re writing their code. How do you calculate that time savings in a way that makes sense for managers?
Tracing money within a company can be really hard, which is why, usually, data science in companies is rewarded instead with attention. A good data science (or engineering, really) team will get more:
Seats in important meetings
Time to present specific analyses
Shout-outs in emails from higher-ups
More complex analyses entrusted to you
Promotions and bonuses
Budget for software and hardware
If you do a good job as an in-house data scientist, and convince people that you know what you’re doing, that usually manifests itself in additional attention for you.
How do you know you’re a bad data scientist? You never get to present to executives. Your analyses go unused. People don’t send you emails with questions about your numbers. A good in-house data scientist does a good job by putting together good data, and, just as importantly, commanding attention to that data.
The same goes for consulting, but with an additional extra step. As an external party, you’re not shielded by the largess of the company’s budget. A company can’t survive on attention and emails alone - it needs revenue.
As a consultant, you have to constantly be selling actual work - otherwise your consultancy dies. An army marches on its stomach. Any given consultancy, whether it’s one person as an LLC, or a company with thousands of employees, marches on sales. If you’re not selling contracts, your company won’t exist, because you don’t have work.
One of the reasons I decided to go into consulting was that I wanted to see how the money moved, because in a company, it’s not as visible.
In consulting, you can very clearly see how budget gets allocated to projects, moves through the company, and how decisions are made based on that money.
And the way the money moves from companies to consultancies is consultants convincing companies that they have the expertise to fix problems, that they know something, more than them, that they’re willing to pay for.
But what happens once you actually sell some work? I’ll cover that in a future newsletter.
Art: Portrait of a Businessman K. Artsybushev, Mikhail Vrubel 1896
What I’m reading lately:
Books I’ve finished recently:
I Heart Logs by Jay Kreps - Does this count as a book? It was literally 30 pages.
Dad is Fat by Jim Gaffigan - Awesome maternity leave read. Hilarious
On how to tackle loneliness
From 2012, how writing a novel is like having a child
Alex Stamos, formerly of Facebook, is on the move:
About the Author and Newsletter
I’m a data scientist in Philadelphia. This newsletter is about tech and everything around tech. Most of my free time is spent wrangling a preschooler and a newborn, reading, and writing bad tweets. I also have longer opinions on things. Find out more here or follow me on Twitter.
If you like this newsletter, forward it to friends!