When it Comes to Data, Embrace the Chaos

What if your data isn’t perfect? What if you have to make a host of assumptions that people can challenge and use to discredit your conclusions? If you are debating whether this can be avoided, let me end the suspense for you: It cannot. In fact, it’s going to happen over and over again. The real question is what are you going to do about it? I’ve seen teams/projects get slowed to a stop because they were afraid to make claims that could be disrupted.

Here’s a tip for when you find yourself in this situation: get in front of the data challenges, don’t try to hide them. When you present your data do so with the acknowledgement that you know certain assumptions had to be made. If a lawyer knows there’s a bit of evidence that hurts their case, they will always ask their own clients about it before the opposing lawyer has a chance to do so. Why? Because then it doesn’t look like they are hiding something, and they can frame the issue so it doesn’t sound so negative. Take this same approach when presenting data.

I worked in a health care organization a few years ago that was trying to keep track of utilization of space in their various clinics. We developed a system for that, but then the doubts crept in. Should we count total patient time in room or only the portion in which a doctor present? What if a patient left the room for a test and then returned… should the intervening time count as utilization since the room was effectively tied up? What if two doctors are seeing the patient- who gets credit for utilization and how do we know when the changeover occurred? And so on and so on. When we presented our results to each division we started with the following disclaimer: “Here are the assumptions we made to get this information, and of course you could dispute those- we did it ourselves. But while our data isn’t exact enough to tell us whether is precisely 91 degrees versus 92 degrees outside, its plenty good enough to tell us its hot.” That was enough to get us past the challenge phase and on to discuss improvement opportunities, which is of course what we were shooting for.

Don’t let the fear of challenges to your data unnecessarily impede your progress.

—Ralph Smith