When the industry dismisses qualitative metrics


My post yesterday about computers not feeling faster, despite performing better in certain benchmarks, made me think about qualitatitve metrics in IT in a broader sense. Things like usability, accessibility, even attractiveness and whether a system is nice to use. Barring very specific circumstances, I consider these as core to a system’s success as whether they technically function.

I’m glad to see these metrics getting more attention, especially in free and open source software communities. But go to any technical forum, bug tracker, or mailing list, and they’re still often seen as fringe ideas to be ridiculued, discounted, or dismissed. And we all end up paying for it in engineering time and helping confused people down the line.

I can empathise to an extent. Qualitative metrics are, by their very definition, hard to measure. You can apply a rational, scientific method to verify and improve them, but it takes far more work and the outcome can still be frustratingly ambugious. It’s easy then to dismiss the entire exercise as an open-ended waste of time, and an expensive opportunity cost when technical features can be worked on instead. Throw into the mix that so many of these projects are volunteer efforts, and an outsider coming in saying the usability and appearance of their software could be improved is a recipe for resentment.

But this cuts both ways. Qualitiative metrics are still routinely seen as a bolt-on you can add later, like where to put the steering wheel after you’ve made a dashboard. Unless you’re thinking about how the system will be used when designing it, you end up with software like PGP that’s technically excellent and entirely unusable by anyone in the real world. Dismissing people for not reading the manual or Wikipedia article on public key crptography isn’t just unhelpful, it’s how you breed resentment on the other side. It also holds back meaningful progress, in this case with decentralised, secured communications.

Accessibility is fortunately now being taken more seriously, and is becoming harder to defend when your software ignores it. But other metrics like ease of use and appearance are still routinely dismissed as shallow and unimportant. Spend five minutes talking to an industrial engineer about SAP and see how far you get with saying such concerns are foofoo, and limited to those you deem unintelligent. Because that’s really what so much of this comes down to: I can use the system, so therefore anyone else who struggles is a luddite who only appreciates form over function. Lulz Apple, amirite?

We’ll never convince some people that it’s a worthwhile endeavour making the world a nicer place. So at the very least a starting point should be that improving qualitiative metrics will lead to better use of the system. Because if you can’t use it, or don’t want to, what’s the point?

Author bio and support


Ruben Schade is a technical writer and infrastructure architect in Sydney, Australia who refers to himself in the third person. Hi!

The site is powered by Hugo, FreeBSD, and OpenZFS on OrionVM, everyone’s favourite bespoke cloud infrastructure provider.

If you found this post helpful or entertaining, you can shout me a coffee or send a comment. Thanks ☺️.