Documenting Ad Hoc Data Analysis Matters

Ad Hoc Data Analysis 🔬 #

If you’re a data scientist or analyst, ad hoc analysis is key par of your job. And you have probably developed some impressive technical skills to help you answer a variety of different questions.

You are likely not effectively answering the questions you’ve been asked to investigate though. And that’s because you’re not answering the root of the questions you’ve been asked to answer.

In my experience, the people coming to you with questions don’t fully understand what they need, or how they should be looking at solving the business problem they’re asking you to investigate. When you are close to the data, you arguably know more about the business than those removed from it. As a result, it’s important you lead the dialogue around setting the request’s scope.

Ask Why❓ #

The first few thing I ask when starting ad hoc work are…

• Why is this needed?
• What’s the objective?
• What questions are you looking to answer?
• How will this help you and the business?

These are important as they help establish the scope of what needs to be done. And the answers to these questions will also help you better understand what root question you should be looking to answer. It will be your guiding star as you query data and work through your analysis. Here are a few specific reasons this is crucial:

Asking why helps you make better decisions. #

This should be a no-brainer. If you understand the objective, you are better able to find the data needed to address or answer it. Non-technical people are not that great at knowing what data they need, nor do they know what analysis is possible. Therefore it is the role of the analyst, to steer the ship right. But that can’t be done without understand exactly what’s at stake.

Asking why makes it easier to produce more impactful work. #

Asking why usually leads to more in-depth questions. For example, “how do we measure the impact of this promotion” rather than, “how many people clicked our advertisement”.

I’m sure most of us would agree we should always ask why. But in practice, I’ve found analysts often dive into answering the question asked rather than stimulating more discussion.

Document it all 📝 #

Once you’ve figured out what it is you are solving for, make sure you write it all down. Again, this will help you build out your project’s scope. You should document the following items in particular:

1. The root problem / business objective.
2. Your proposed approach.
3. That data used.
4. The findings from the data used.
5. Complications or shortcomings with the findings/data.
6. Decisions made from the analysis.

Try to build an analysis template that is structured to answer all of the above items. I’ve found that a Word document saved on a shared drive or a Google Doc shared with your team works best. The key idea here is you want your entire team to have access to the document. So pick a sharing tool that works best for you.

Sharing is Caring 👍 #

Documenting helps both you and your team. Everyone can leverage the work done for future inquiries and updates. In a sense, this gives your work scale.

When you receive similar ad hoc inquiries in the future, you’ll be able to share what was done and the basis for the decision made. This is huge and prepares you to answer, “why did we do it this way before” types of questions.

Finally, in the event that your team changes and you gain/lose an employee - having a centralized repository of your work makes it easier and faster to on-board new employees and/or transition team members into new roles.

By always asking why first and implementing documentation practices for you and your team, you can make everyone’s time spent more impactful, rewarding, and lasting.