The First Question I Have For Every Data RequestAnd how I use it to build partnerships with cross-functional teamsRandy AuBlockedUnblockFollowFollowingMar 13This is about building a data-driven culture at a company from the ground up, in this case, by building partnerships across teams.
In the lives in the same context as my earlier succeeding as a data scientist in startups post.
There are very few absolutes in the world, death, taxes, and dirty data sets easily come to mind — but when it comes to internal experts in support positions like researchers, data analysts, and data scientists, I’d like to nominate another one: their reaction when someone comes to them with a request.
“Tell me what problem you’re trying to solve, not the solution you think you want.
” — All of us, all the timeLofty Cat, totally not judging anyone at allThe majority of the time, especially if there’s not a lot of cross-functional experience, people will come and make a specific request for an analysis or data that they believe will help them do whatever it is they’re doing.
Often it’s couched as a “small data request” or “just a chart or two”.
Very often their request has little context.
This behavior isn’t even restricted to the data fields, I catch even myself doing similar things for repairmen and engineers.
We’re all susceptible to this.
This happening once in a while is not a big deal.
But when it happens enough times, it’s maddening.
Oftentimes you find battle-scarred researchers who have seen this a bit too often and have strong feelings about it.
Still, remember that people are doing the best they can, so forgive them.
I’m pretty sure they’re not doing it out of malice or ego.
They’re typically trying to be respectful of your time, or have thought about their problem a great deal before coming to you for help and their request reflects that line of thinking.
Occasionally it’s because they’ve heard of a technique or analysis and are overly excited about it without understanding all the pros and cons.
“So, what decision are you trying to make?”When faced with this situation, I normally ask some variation of this, for two reasons.
First, just about every question in an industry setting can be formulated as one or more decisions that need to be made so it’s not out of place to ask.
Decisions can be specific, like choosing between a set of alternatives, or more abstract like “it will help inform the design of the next version”.
It also gets them thinking more about what they want to do.
Since I always encourage people to have data conversations early on (too early is better than late), so it’s expected that ideas aren’t fully baked.
Second, and much more importantly, it gets them thinking more in terms of a partnership model of working together instead of a simple “ask and receive report” model.
This is a critical habit you need everyone to get used to — come with questions, not proposed solutions.
Start the conversationThe whole point of asking their actual problem statement and decision point is because it starts a conversation.
Conversations are good.
Its your chance to learn what they’re up to, and for them to better learn about how to work with data.
Long term partnerships don’t happen in a day, this is how they start.
Their starting question should be able to translate into some kind of hypothesis that we can test using research toolbox, the decision frames the conversation and your future recommendations.
Often it just takes a small nudge.
“How many times to people click this button?” is just a context-free request, but can easily turn to “we think no one uses that delete button because they delete using the keyboard, we want to remove the button”.
Now you have enough context to check if people really do use the keyboard more, or make a case for whether the button is used by a specific small set of users for accessibility reasons.
From there you can work with them figuring out what data is available to answer their questions.
When some data is missing, you can make a case for implementing instrumentation to get data they really want (e.
g.
should we log keyboard shortcuts?).
At the same time, you might know of a proxy measure that already exists that may yield a “good enough” result (e.
g.
deletes with no button clicks is probably close enough).
Never assume that an exact answer is necessary or even desired, an approximation can be just as useful (especially quick ones).
Next, you can work with them on understanding the methodology you’ll be using.
It’s often a good idea to teach people about the potentials and limitations of methods, even if you have to gloss over deep technical details.
The reason being that if people understand how things work at a high level, they’re better able to understand when you later tell them “the data let’s you say this, but not that”.
Then, keep having conversationsFinally once the initial analysis has been done, you can reinforce the partnership by bringing the stakeholders in to go over the findings, answer questions, and help brainstorm next steps.
Much like how almost every academic paper ends with “and more research needs to be done”, there are always more questions that could be answered.
It’s up to the team as a group, with your help, to decide if they have enough confidence to proceed.
Keep iterating!Repeat until it’s a habitOver time, as this people go through this process repeatedly, everyone should start getting the hang of it and automatically come to you with more questions and involve you in partnering to explore the problem before committing to big decisions.
When that happens, you’ll know you’ve gotten to a good point in data-driven decision making culture.
.