Design Exercises are Bad Interviewing Practice

Recruiting and hiring is among the most difficult and time-consuming aspects of a design manager's job, and wherever they gather and share experiences, the subject of design exercises inevitably comes up. We wrote about it briefly in our book:

Design Tests?

A topic of some controversy within product design circles is whether candidate interviews should involve some kind of design test or challenge akin to what happens in engineering interviews. Our firm, resolute response to this is “no.” Design tests set up an unhealthy power dynamic in the interview environment, when instead you should be fostering collegiality. The context in which the challenge is given (typically narrowly time-boxed and with only a little information and little support) is wholly artificial—and so whether a candidate succeeds or fails is not a meaningful indicator of actual practice. There is nothing you will find out in such a test that you couldn’t better learn through probing the candidate about their portfolio.

I had hoped that this would be sufficient and never need to be discussed again. Judging by lengthy multi-party threads on Twitter, I was wrong. Forthwith, a lengthier set of reasons for why design exercises are bad interview practice.

Design Firms Don't Do Them

At Adaptive Path, we hired world-class designers without ever having them conduct a challenge. Same thing back at the first design firm I worked for, Studio Archetype, which was a standard-bearer for early digital design. These are companies whose sole purpose was the delivery of superlative design, and where the value was the talent of the people on staff. How were we able to assess their abilities? As alluded to in the passage above, portfolio reviews, including discussions of how they tackled design challenges.

They are a waste of time

If there's nothing you can get from a design exercise that you can't get from a portfolio review and a well-structured thoughtful interview, then it follows they are a waste of time. I call this out because recruiting and hiring is already monumentally time consuming, and anything that needlessly takes up time should be excised from the process.

Design Is Not Engineering

I can't say for certain, as I haven't done the research as to where design exercises emerged as an interviewing practice (it's not from traditional design practice), but my guess is that they came about in technology companies where software engineering was the dominant practice. Design had to overcome its perception as squishy, soft, "make it pretty," by demonstrating rigor, relying on data, and generally making the practice of design operate more like engineering.

And engineering hiring interviews involve technical exercises (coding challenges and the like), so shouldn't design hiring interviews?

The thing is, coding challenges are waaaaay more straightforward than design exercises. There are demonstrably better ways to solve engineering problems. And in most coding exercises, the outcome is predetermined -- it's a matter of how would you realize it?

The same is not at all true for design. You're not applying process to realize an already known outcome. You're taking in a massive amount of input in order to navigate your way through the problem space. Unlike engineers, you need to consider business context, user needs, goals, and capabilities, brand concerns, technical constraints, channels of use, and god knows what else. And good designers know that there are many potential solutions to a problem, and require testing and iteration to get to anything like a good solution.

Design Exercises Bias Towards Facile Problem-Solving

Designers don't all solve problems the same way. Some take in a lot of data, go off into a cave, noodle on it for a while, and come out with something great. Others iterate and prototype almost from the get go, uncovering solutions through refinement. Some require thinking out loud, and deep collaboration to get their best work. A great design organization has people with a variety of problem-solving modes and approaches, which enables the organization to better tackle a wide array of challenges.

The artificial constraints of design exercises (typically time-limited; a problem that the candidate isn't prior familiar with, but which the interviewers are; performing under the scrutiny of others) biases toward a narrow range of problem solving.

A design exercise, by its very nature, is inclined towards facile solutions, and so biases teams towards facile designers. There's not really any room for grokking depth.

Design Exercises Exacerbate An Already Problematic Power Dynamic

Design exercises ask candidates to perform on demand. In the context of a job interview, this only heightens the fraught power dynamic between an employer and prospective candidate. Even in markets where talent is in high demand, job interviews place candidates in a vulnerable situation. Being expected to perform on demand only adds to the candidate's stress and anxiety, and makes for a suboptimal candidate experience. This Twitter exchange between my friends Ryan and Jared touches on this...

As Jared notes later in the thread, design exercises introduce cultural bias, too:

Shown to be highly biased against women and people of color, especially those who come from cultures where questioning or interrupting authoritative strangers is seen as social insubordination.

— Jared Spool (@jmspool) May 10, 2018

What about take-home exercises?

This is often the response to my ranty diatribes against design exercises. What if they're take-home? Then people have all the time they need, and it's the pressure cooker of performing-on-demand is.

Beyond the obvious problem, that's still at the root of all of my issues with design exercises (for the people in the back: THEY ARE ARTIFICIAL CONSTRUCTS THAT DON'T REFLECT HOW DESIGN ACTUALLY HAPPENS), they introduce new issues... Namely, now you're asking this person to do unpaid work. Young people with savings (i.e., don't need money) and free time will be able to put a lot more effort into take-home exercises than, say, a single parent whose at-home time is focused on their children, and can only do "homework" after the kids asleep and when they're likely exhausted.

Recognizing this, some companies do offer to pay people for taking the time to do a take-home exercise (which can help defray costs like child care), and that's better than not doing so, but even better...? No exercises. Because you don't need them. Because they add nothing to the recruiting and hiring process that can't be figured out through thoughtful, experienced-based interviews, a savvy portfolio review, and speaking with people the candidate has worked with.