3 Kinds of Product Consistency (how not to design the world’s worst elevators)

I work in an amazing building... with awful elevators.

Visitors tend not to notice the problems.  The elevators are new.  They look modern.  They have touch screen interfaces.  They’re reasonably fast in carrying people up and down.  Plus who really cares about how an elevator is designed?  But people who work here feel the design problems deeply.  I’ve lived with the aggravation, I’ve surveyed my coworkers, and I can’t help but write about this #productfail as the perfect cautionary tail of what happens when you break certain kinds of consistency expectations in product design.

I’d use these three categories of “Must Fix Problems” to cover the very worst kinds of design flaws for elevators:

Our elevators fail at #3 all the time.  Many of us are taken on rides up and down without reaching our intended floor.  Many of us have been stuck inside (briefly) waiting for someone new to call the elevator.  We can’t seem to re-train ourselves to the UI well enough to avoid the problems.  And whenever these things happen, we’re helpless to correct the problem from inside the elevator.  We have to get out on the wrong floor to fix the problem, then get back in.  I’ve been here 18 freaking months, and just last week I went on another “ride of shame” to the wrong floor.  My coworker has been here 7 months and confessed to me that she got stuck inside the elevator recently and waited 5 minutes for another employee to call it to a floor.

Why are we suffering?  Because someone made the design decision that there would be no controls to direct the elevator inside the elevator.  But as human beings who have visited and lived in cities, we’ve been trained our entire lives that you can safely get into an elevator without thinking, then tell it your intended destination.

I’m sure there were valid reasons for making this design choice.  These elevators could be 12% more efficient because people going to the same floors are better grouped into elevators, leading to fewer stops on average.  But as users, we’d really like to stop being stuck, sent to the wrong place, and made to feel stupid.  Our elevators are inconsistent with all the other elevators we’ve ever used, and re-training is no fun at all.

Does consistency matter for products?

The importance of consistency is well-known in product management, design, and marketing.  When your product is “inconsistent,” it hurts usability and leads to confusion.  It can aggravate your loyal users (like these elevators) and hurt your overall retention rates.  It can even result in ground-swell community backlash efforts which in turn scare away new users and limit growth.

This isn’t new.  It was a hot topic in the Microsoft hallways 20 years ago when we planned user experience updates for new versions of Office.  The biggest design offense of the web 1.0 era was a lack of consistency across web sites (remember Flash-based websites for restaurants?).  In 2012-13, a desire for consistency in our everyday products was at the core of user backlashes against Apple’s change to the flat, content-first design of iOS 7 and Microsoft’s changes to drop the start menu and better support touch-screen computers with Windows 8.  And a healthy respect for consistency in design is just as relevant today.  Snapchat just aggravated 83% of their user base with their January 2018 UX changes.  

If you are working on software and interacting with designers you are probably aware of their efforts to define and adhere to consistent design patterns.  If you work with marketers, you’ve  probably heard them defend the importance of consistent marketing messages, content, and tone.  If you work on a platform, you’re probably providing common controls and enforcing guidelines for developers that use your platform… all so that the end users can have a consistent experience.

What does consistency mean, exactly?

Consistency in product design is a bit subtle.  There are 3 important kinds of consistency worth striving for:

#1 is straightforward to get right as long as someone on the team is putting in the effort.  Most designers strive for this automatically, pointing out consistency flaws in your current product design and making their new designs consistent with each other.  Engineers may tend to reuse existing UX code in their area, perpetuating inconsistencies across a large project, but these inconsistencies will be visible with a little testing and some usability work.  If you prioritize consistency fixes from time to time, you should stay on track.

#2 is harder, because there are often very good reasons to make substantial product changes.  You may have a product with a stable user base that’s not growing.  Bold new ideas can jump-start growth, but they require substantial changes to both functionality and design.  You may want to rebrand your product in a minor way to reach new people and/or re-engage users.  Nearly every major mobile app has done this (most more than once).  Deciding which changes are worth the “costs” of the consistency breaks, and how these changes should be rolled out is one of the key ways you’ll earn your paycheck in innovative releases.

#3 really deserves your attention though, because chances are that no one else on the team is thinking deeply about it.  Assume from the start of any project that you need a broad, real-world understanding of your users and the world they’ve been living in.  Supplement your team’s instincts with user research and go through thought exercises on user empathy and expectations as early as possible.  “Where within our product should we innovate and where within our product should we copy what people are used to?” is one of the questions you should be ready to answer at every stage of your product development cycle.

How does a savvy product leader balance consistency and innovation?

Very carefully :-)  

You have some real obligations when you break product consistency from version to version (type #2):

User obligations: I’m sure you’re already planning to test major changes with a fraction of your users to understand and usability problems and positive/negative behavioral impacts.  You might also want to consider pre-announcing the change and your reasons for the change before pushing it to a majority of customers.

Organizational obligations: You’re probably already on top of stakeholder agreement on vision, priorities, and goals.  You have your team buy-in for your execution plan too.  Now put some work into understanding the risks of breaking consistency across versions and ensure the organization is bought into these risks as being justified, depsite the likely difficulty in winning over users, press, etc.

Minding your funnels: Change can be good, but it comes with risk.  Take stock of the likely range of outcomes for your upcoming break with consistency and whether your product can survive the change.  For example, if you have a mobile product where your churned user numbers are close to your retained new user numbers, then any meaningful drop in new user sign-ups or retention will lead to a decline in overall active users.  Or, if your monetization funnel is dominated by a few percent of long-term, happy customers - you may see a disproportionate impact to revenue by breaking with consistency in your new version.

Mistakes I’ve learned from:

Product consistency decisions are hard.  I’ve messed them up on more than one occasion.  

A version to version screw-up: In one case, we sought to 10x our mobile app’s success through significant changes and new capabilities, but the organization was not willing to risk the current user base.  Instead of making a major version to version change, we should have spun up a new mobile app with the new features and design and used hooks in our current mobile app to encourage users to try the new app.  By the framework here, I didn’t correctly assess my org’s commitment to a type #2 break with consistency.

Internal product consistency arguments: I’ve seen more than one team get into a temporarily dysfunctional state over disagreements on the importance of “design debt.”  This kind of problem results from lack of communication and team buy-in to the priority of type #1 consistency.  At times it’s not the right call to prioritize small user experience changes (and it’s often hard to measure the impact of these fixes in the short run).  A good technique here is to set a fixed budget (e.g. schedule a “design debt” week every six months) to address these types of issues.

User expectation problems: Having worked on several new versions of Microsoft Office where our biggest competitor was the last version of Microsoft Office, I’ve had the importance of #3 beaten into me from the start of my career.  For those product updates, external user expectations and version to version consistency expectations often amounted to the same thing.   When I see these types of flaws in other products like our elevators above (or software products with gratuitous new design paradigms for common tasks), they are often real killers.  

Nevertheless, these mistakes aren’t that rare.  Got an example you find particularly aggravating?  Let me know about it.

Related Reading: