Quality Standards: Overview
In design, quality is a funny word. For most designers, when asked how they gauge the quality of a design, they'll respond with something like "I know it when I see it." Quality is expected to just be understood, appreciated, intuited. Quality is learned through practice and review, and judged based on experience.
This contrasts with quality in engineering. Software engineering has very clear standards: number of bugs; executing to requirements; performance; uptime; security, code maintainability. Product engineering considers dimensional and material conformance; reliability; durability; safety and hazard compliance, defect rates, repairability.
In engineering, the concept of 'quality' is basically the same across companies and contexts. These standards are largely uniform and predictable. A memory leak is a memory leak regardless of who's running the software. "Quality Assurance" emerged as a robust practice to manage this. Businesses may differentiate on their degree of quality (where you pay more for that which is 'higher quality'), but the framework for defining it holds.
The stability of the definition of engineering quality is because it's simply determined by how well the thing that's built works for the systems it runs on. Those systems may be complicated, but they are knowable.
Design quality emerges from a much more chaotic context, because it is determined by how something works for people. It is inherently relational. Unlike engineering, design mediates between a system and the humans who use it. In order to determine quality, humans must be part of the evaluation. And those humans are a heterogeneous mass with variable perception, cognition, ability, need, culture, and expectation.
Thoughtful practitioners have recognized this gap. Dieter Rams, the legendary designer for Braun, drafted "10 principles of good design" (originally in German: "Zehn Thesen für gutes Design"), listing qualities for the work to aspire to, including "innovative," "make a product useful," "long-lasting."
Jakob Nielsen's 10 usability heuristics have proven durable, update-able, and broadly relevant for over 30 years. 20 years ago, Peter Morville created the UX Honeycomb in an effort to expand beyond usability. It never really caught on, but the attempt for something more robust is admirable.
Ultimately, though, these approaches fall short. Nielsen's heuristics are solid, but limited—there's more to design quality than usability. Rams' 10 principles and Morville's honeycomb are noble, but neither are operational—there's no clear criteria as to whether the designs are satisfying these aspirations, leaving any assessment to be subjective. And while subjective assessment might be sufficient within the UX/Design organization, it falls apart when placed alongside the clear standards that other functions employ.
The relational reality of standards means that designers have it harder than engineers. Unlike technical standards, design standards can't be purely universal. While industry-wide standards exist, they address only the functional aspects. Capturing the essential experiential and expressive aspects of our work requires standards specific to your users and their contexts. And even that's not sufficient. Designed things are made by and for humans, so there will always some subjectivity in the evaluation. Developing norms for discussing this becomes important.
What this means is that to adequately define design quality requires a multi-pronged approach. You may not need all these components at all times, but you will likely need all of them at some time, and having them ready will make you stronger.
My consulting, coaching, and research work led me to the following framework for defining quality standards. What this lacks in conceptual purity of the other models, I hope it makes up for in rigor and operability.
Industry standard criteria
Over decades of doing this work, there have emerged a set of standards that are design's analogy to engineering quality. These are mature, solid, functional, and inarguable components of design quality:
- Generic UX metrics
- Attitudinal and self-reported (SUS, UMUX)
- Behavioral measures of basic usability (error rate, time on task, task completion)
- Heuristic evaluation (which can be scorecarded with an approach like PURE)
- Accessibility standards
Generic UX metrics are industry-wide guides for understanding the ability of our target users to successfully use what we've built.
Accessibility standards ensure that all our target users can use what we've built. (hat tip to Thomas Sutton for some of the language around usability and accessibility.) Many tools exist to evaluate the accessibility of interfaces.
Company specific criteria
While "good" engineering quality is basically the same across all companies, "good" design quality is highly specific to the company, reliant on context. Each of following requires bespoke definitional work before quality can be evaluated.
Brand characteristics. How well does the design communicate the desired brand attributes, voice, tone, and values?
Experience principles. How well does the design deliver on customer-centered desires for the product or service?
Product, service, business metrics. How well does the design satisfy requirements and deliver desired business value?
Case studies and their discourse
The prior elements provide means by which design's quality can be evaluated, understood, and communicated through measurement of criteria. But design should not solely be reduced to metrics: it's a creative practice for which there will always be some subjectivity in how it's engaged.
As such, in creating quality standards, it's essential to develop a language of discourse around design, which will be used in critique, review, and other situations where design is discussed.
Case studies are a good vehicle for this. Identify exemplars of "good design," and write up what makes them good. While much of the case may refer to the kinds of metrics discussed above, there should also be subjective commentary, addressing aesthetics, sense, fit, feeling, flow, and other descriptors of the experience.