Measurement, again: Assessment is greater than measurement

Why might things be “beyond measure?”

If I am right, that not everything is measurable, why might that be the case?  There might be a lot of answers to that question.  Here is one response. 

            We live our lives as a continuity, as an on-going experience of every moment.  Life is continuous.  Our appreciation of life is in analog, like an old-fashioned vinyl LP or a truly analog watch (the kind you have to wind up).  Your love of family or your passion for your work isn’t divided into “chunks.”  You live them as a sweep of inseparable experiences.

In order to measure things, you have to make them digital, discrete.  Measurements of life have to fit with the “chunked” digits on our scale.  Inevitably, when you measure something that is part of our lived experiences, you will lose some acuity.  This reductionism is inevitable.

In addition, some aspects of our experiences are inherently qualitative.  They require a “perceiver of quality” to exist.  In other words, there is no experience of quality without someone who actually experiences that quality.  But to measure things, we want to eliminate the impact of that perceiver. 

In the social sciences and in epidemiology we call this impact of the perceiver “bias.” We want to eliminate the bias so that we are left with the “objective truth.”  But when you eliminate bias, when you eliminate the perceiver of quality, you are no longer assessing a qualitative experience.  You have reduced it to only quantity.  But bias is the essence of quality.  Eliminate the perceiver of the quality, and you have eliminated the quality.

Assessment is Larger Than Measurement

Inevitably, by this far into the argument, some reader is thinking “But if we can’t measure things, then how can we know we are doing a good job?”  The assumption is that the only adequate assessment is measurement.

But that, too, is simply not true.  There are plenty of good ways to assess success without measuring that success.  You know who your good teachers are, without a measurement.  You know who your friends are, without being able to put your friendship against a yardstick.  For those of you who have had staff working for you, you know who your good workers are, and who your bad workers are.  In the hospital, if you want to know who provides the quality of care I was talking about, just ask the nurses.  They know. And without measuring anything.

Who knows who the good teachers are and who the bad teachers are?  The students.  The principals.  The parents.  The alumni.  And the other teachers.  We have an abundance of evidence with which to assess our teachers, all of it based on an abundance of experience.  Yet we want to disregard all of that and instead stake our evaluation on a three hour test that the students take on one day during the year.  Really?  Does that make sense to you?

So, we can assess.  We can evaluate.  We can judge.  Sometimes there will be inputs that are measurable, and sometimes we won’t have that.  Most often, it will be a mix of both.  And we can then make the decisions that we have to make, as managers and as policy-makers and as political leaders.  In short, assessment is larger than measurement. 

Is this “scientific?”  Well, if you think that the sine qua non of science is measurement, then maybe this isn’t good science.  But if the essence of science is the careful examination of our experiences in the world, and our measurements are merely more or less good representations of those experiences, then this is quintessentially scientific.

Why not try to measure everything?

            But, you might ask, so what?  What is the harm in trying to measure things, even if they are not measurable?

            It’s about the guy who sees someone, obviously drunk, on his hands and knees patting the ground under a lamp post.  “What is wrong?” asks the first man.

            “I dropped my keys a couple of blocks down the road, and I am looking for them,” says the drunk.

            “But why here?”

            “Ah, the light is better here!”

If, when we think we can measure something, that belief has consequences.  We start contorting our standards to fit what measurements we have, even if they are admittedly inadequate or wrong.  We look for measurable things, and miss the unmeasurable, because, as the drunk says, “the light is better here.”

When we fetishize measurement we find, to rethink my colleague’s maxim, that “only that which is measured gets managed.”  We start teaching to the test.  We accomplish what we try to measure, rather than what we really want to achieve.  We train our staff to meet the metrics, rather than educate them to fulfill the organization’s mission and their own passions.  (The verb “to train” comes from the notion of “dragging behind us.”  The verb “to educate” comes from “to bring up, to raise.”  Which is a better metaphor for managing staff?)  You don’t get good cooks by focusing on only the measurable aspects of being a chef.  You don’t get good parents by focusing on the measurable competencies of parenting.

By attempting to measure what can’t be measured, we waste precious resources chasing illusions.  And we run the risk of missing that which brought us to the effort in the first place.  We misplace our potential, and we by-pass our passion.

That can’t be the right thing to do.