In his recent The Counterintuitive World post, Kevin Drum tells the following story:
Back during World War II, the RAF lost a lot of planes to German anti-aircraft fire. So they decided to armor them up. But where to put the armor? The obvious answer was to look at planes that returned from missions, count up all the bullet holes in various places, and then put extra armor in the areas that attracted the most fire.
Obvious but wrong. As Hungarian-born mathematician Abraham Wald explained at the time, if a plane makes it back safely even though it has, say, a bunch of bullet holes in its wings, it means that bullet holes in the wings aren’t very dangerous. What you really want to do is armor up the areas that, on average, don’t have any bullet holes. Why? Because planes with bullet holes in those places never made it back.
Applying the above to UI/UX confirms a lingering feeling I have about software metrics and usage statistics.
Metrics are undeniably useful when designing sign up forms, improving conversion rate on merchant sites, increasing upload success rate… They’re essential to web design. But, to some extent, they do not make sense for apps.
Some users are dealing with your UI blunders but they keep using the app.
Back home with bullet holes.
Some just can’t cope with it.
Tweaking your design based on statistical usage is armoring up the plane that made it back.
Doing so is becoming blind to your interface real pain points and ending up strengthening your user deviations.
You’re adding armors where it’s not needed.
Oh! 80% of our users click on this reply button rather than the other.
You’ll make the other one disappear, make the 80% button more prominent. By getting along with the figures, you’ll only amplify the observed behavior.
Let’s cross reference all the data on all buttons, all actions and we’ll have a comprehensive map of our user’s interactions.
Same thing. It’s like looking at a mouse in a labyrinth. The left/right decision at each corner doesn’t matter. It’s the labyrinth design that matters and affects the behavior. Not an isolated choice within the design.
Of course, you can tweak it but it means entering in an infinite loop of modifications impacting each other.
Application UI/UX design does not obey the same rules than web design. Designing an app means selecting a few things and making them work as flawlessly as possible.
Application design is organic.
An app is a whole, a living organism. You can’t subtract or add without hampering its core or killing it. The only way out if it doesn’t work out is to start from scratch. Not tweaking based on metrics and stats.
One of the best example I know of this is Path.
Path, as we know it today, is not a 2.0. It’s not an iteration over a first version.
It is a new app.