Verification asymmetry

My first job was designing the UI for a few 3D printers. I shared every version and every iteration with the team. But there was no way to put those screens on the actual machines, so nobody could use them. People looked at static Figma screens, imagined the rest, and gave me some light feedback.

Later, as engineers built the UI piece by piece, I started to see that some of my designs didn't work well for users, and others could have been better. I hadn't seen any of it coming while designing. Nobody could have told me, and I had no way to verify it myself. By then, redesigning was expensive. I could make the designs. I just couldn't tell if they were good, so it was hard to get better.

At the time I thought I just lacked experience. Then I started writing code myself, and noticed that software engineers rarely have this problem. A feature might take a week to build, but tests tell you in minutes whether it works, and you can run them whenever you want. With test-driven development, you even start with a clear target. So engineers keep improving their work, and themselves along the way.

In software engineering, verifying is often easier than building. Design often works the other way: making something is easy compared to knowing whether it's good. I call this verification asymmetry.

Take two extremes. A login page is easy to make and easy to verify. People can either log in or they can't, and we know which analytics to watch to confirm it works. A landing page is hard on both counts. It takes real creativity, and whether it works only shows in the data after a while, tangled with so many other factors that it's hard to attribute. Most design work sits somewhere in between, but leans the same way: verifying takes more effort and more time than making.

Why design reviews go in circles

The most direct cost of this asymmetry is the design review. Reviews are slow, and they rarely produce good decisions.

Everyone looks at static screens or a simple prototype and imagines what it would feel like to use. Each person imagines something different. Everyone can explain their reasoning, and nobody can prove they're right. The meeting runs long, people run out of energy, someone makes one more revision, and next week it starts again.

We usually blame this on design being subjective. I think subjectivity is fine. User experience is a subjective feeling, after all. The problem is that nobody has used the thing yet, so all those feelings are imagined.

When nobody has used it, the only thing left to verify a design with is the taste you built up beforehand. Taste is a prediction: someone experienced looks at a design and can roughly guess how it will feel to use. It comes from years of watching, using all kinds of products and working on projects, a bit like a model trained on a lot of examples.

If you're lucky, someone on the team has good taste. Obviously that isn't always the case. And without anything better to go on, decisions tend to follow seniority.

Another way

It doesn't have to be like this. Teams can build better workflows, and at Wealthsimple I saw something completely different.

While we designed and built, changes showed up live in the real Wealthsimple app on our own phones. We'd make a change and feel it right away. If you wanted a coworker to try it on their phone, you didn't even need a PR review. They scanned a QR code and they were in.

Some problems show up the moment you hold the phone: how an animated gradient looks under indoor light and in sunlight, or whether a transition feels snappy. Others take days to surface, like an order flow with enough steps to feel tedious, until using it stops being enjoyable.

So everyone dogfooded every day. We used many features for weeks or even months before rollout. After launch, we were all on Reddit reading what users said, and we enjoyed it, because it meant seeing people talk about what we'd made. Even the complaints were fun.

This kind of design infrastructure gave us a steady stream of real feedback to verify designs with, and that feedback was convincing. Seniority settled far fewer arguments. Discussions came back to the experience itself, there was less back-and-forth, and working together got a lot easier and more fun.

What I noticed at Wealthsimple is that we never made design more objective. It stayed subjective, often very opinionated. The difference was that those opinions came from many people using the real thing over a long time. The more people used a design, and the longer they used it, the closer their opinions got to real verification. And every round of it trained our taste.

Now

Software engineering verifies with tests, and the same tests verify everyone's code. That's a big reason the field is so lively and so welcoming to newcomers. Design has no tests. We've mostly relied on luck, hoping someone on the team has enough taste to make the call.

Wealthsimple showed me one way to shrink the asymmetry: let taste and real use form a loop. Use taste to verify quickly along the way, then let real use give the final verdict. Real use trains taste, so next time you take fewer wrong turns. Good design teams have a culture of verification. They value it as part of the work, get designs running in the real product early, and get more people using them sooner and for longer. That's how taste builds up across a whole team and becomes its most valuable long-term asset.

The flip side is that without real use as the final check, taste doesn't build up. Many new grads and junior designers, and even some mid-level designers with years behind them, have worked on projects and sat through plenty of meetings, yet rarely seen their designs verified. Taste takes a long time to build and can't be handed to someone else, so it stays rare.

Shrinking verification asymmetry is what I most want to do as a design engineer: make verification faster and easier, keep decisions grounded in the user experience, and spend less time on unproductive debates.

We haven't found a way for agents to verify design yet. An agent can't be the designer with taste, because taste is still too fuzzy. It can't be the user either, because it can't give feedback that matches a real user's.

Looking back at my first job, the company had no shortage of printers. I just didn't realize I should have started by building something that let me send designs to the team quickly for everyday use, so I could learn and improve.