Attempting to measure intangibles, ambiguous and the seemingly impossible
“I love you.”
“I love you more” I replied with a big smile to my wife (KD).
“Really? How much more?”
“What? I don’t know. You can’t measure these things. You just feel it.”
“Ok, so you don’t know how much more you love me but you know it is definitely more than I love you, right?”
“Yes”, I said cautiously (I felt that I was about to be trapped)
“Hmm. So if the feeling of love can be observed to be of more or less, then the amount by which it is more or less can be measured too, right?”
“I guess” (busted!), I replied half heartedly.
“So, how much more then?”
“Umm… Oh! “Friends” is starting. Let’s talk after this (phew!).”
She has a point. To summarize,
If we care about “X”, it means we can observe it when it changes. Either it becomes more or less. And if change in “X” can be observed in some amount (i.e X is more than what it used to be), then the amount by which it changed, must be measurable.[1]
When we measure, we get data which reduces the uncertainty and eventually helps make better decisions. That begs the question, what to measure and how to measure it.
Fortunately, there are heaps of well established metrics available for measuring the success of digital products. So I won’t talk about that but if you want to read more about it, please refer to this resource.
Let’s talk about things which seem a bit hard to measure at first glance. Have you ever been asked to measure something ambiguous? In my role as Software Developer, Project Manager, Agile Coach and Delivery Principal, I have been witness to many discussions where what we want to measure doesn't seem as straightforward as performance of digital products.
For example how to measure
performance of IT security?
a team's happiness?
a team's agile maturity?
effectiveness of account leadership? etc
The fundamental intent behind any measurement is that by knowing more about what you care about would inform some decision making. The more you know, the less uncertainties there are,which leads to improved decision making. So simply put, when faced with a challenge to measure something, which may seem ambiguous, we first need to clarify what exactly we need to measure. The following two questions can help with that.
What do you mean by _____ ? and
Why do you care?
Replace _____ with what you have been asked to measure? For example, what do you mean by IT security? The person who asked for it to be measured could clarify that it means reduction in unauthorized access, reduction in malware and phishing attacks etc. Now we know what to measure.
Asking why we care about IT security, is a way to identify if that measurement has any economic value. We want to measure unauthorized access because based on the measurement, there will be a decision which will be affected. If we can’t identify a decision that could be affected by a proposed measurement, then the measurement simply has no value. This eliminates the need for measuring it.
The two questions above are critical before we start thinking about how to measure. In most cases, once we know what to measure, we will find that there is already a precedent for that. There are already methods devised on how to measure those variables. Brilliant.
Although if, for any reason, you end up deciding to measure a variable which has not been measured before, then try asking yourself “does the variable I am trying to measure leave behind any trail?” You need to observe and measure those trails.
Let's assume there is no direct way to measure effective leadership (thankfully there are) then data from exit interviews, Glassdoor review can be seen as traces.
Almost all events leave some kind of trail. But if you can’t find any, then you have to attach a tracer. For example, many e-commerce websites provide free gift wrapping to track items which were purchased as a gift.
If you can’t even attach a tracer for some reason then you need to create conditions where an event can occur and you can observe the variable you want to measure. Basically creating an experiment. For example, if you wish to measure when you get more attendance, have two community events under the same circumstances, one event where meals are provided and at others not. See if that reveals anything.
Measurement is not just measuring the length of the microwave to see if it would fit on your kitchen benchtop. In fact, apparently difficult-looking measurements will involve indirect deductions and inferences. For this type of measurement, any reduction in uncertainties will improve decision making. When you know nothing, any measurement will tell you something which you didn’t know before.
Anything can be measured, even at first it seems ambiguous, intangible and difficult to measure. Remember, if it can be observed, it can be measured.
My wife makes my favorite meal once a month while I cook her favorite meals on alternate weekends. With those measurements I announced to my wife that I love her twice as much as she loves me.
[1] Clarification Chain - as explained in How to Measure Anything
Disclaimer: The statements and opinions expressed in this article are those of the author(s) and do not necessarily reflect the positions of Thoughtworks.
