ThoughtWorks
  • Kontakt
  • Español
  • Português
  • English
  • 中文
Übersicht
  • Delivery Mindset trifft Software-Exzellenz

    Verfolgen Sie einen innovativen Ansatz in der Softwareentwicklung, um noch schneller erfolgreich zu sein.

    Erkenntnisgestützte Entscheidungsfindung

    Nutzen Sie Ihre Datenbestände, um neue Geschäftsmöglichkeiten zu erschließen.

  • Betriebsmodelle ohne Reibungsverluste

    Verbessern Sie die Fähigkeit Ihres Unternehmens, auf Veränderungen zu reagieren.

    Plattform Strategie

    Entwicklung dynamischer Technologieplattformen, die sich an Ihre Geschäftsstrategie anpassen.

  • Experience Design und innovative Produkte

    Liefern Sie schnell außergewöhnliche Produkte und Kundenerlebnisse. Entwickeln Sie Design und Funktion kontinuierlich weiter.

    Partnerschaften

    Nutzung unseres Netzwerks aus vertrauenswürdigen Partnern, um noch bessere Ergebnisse für unsere Kunden zu erzielen.

Übersicht
  • Automobil
  • Clientech, Energie und Versorgung
  • Banken und Versicherungen
  • Gesundheit
  • Medien
  • Non-Profit
  • Öffentlicher Sektor
  • Handel und E-Commerce
  • Reise und Transport
Übersicht

Unsere Empfehlungen

  • Technologie

    Ausführliche Betrachtungen neuer Technologien.

  • Business

    Aktuelle Business-Insights, Strategien und Impulse für digitale Querdenker.

  • Kultur

    Insights zu Karrieremöglichkeiten und unsere Sicht auf soziale Gerechtigkeit und Inklusivität.

Digitale Veröffentlichungen und Tools

  • Technology Radar

    Unser Leitfaden für aktuelle Technologietrends.

  • Perspectives

    Unsere Publikation für digitale Vordenker*innen

  • Digital Fluency Model

    Ein Modell zur Priorisierung digitaler Fähigkeiten, um für das Unvorhersehbare bereit zu sein.

  • Decoder

    Der Technology-Guide für Business Entscheider

Alle Insights

  • Artikel

    Expertenwissen für Ihr Unternehmen.

  • Blogs

    Persönliche Perspektiven von ThoughtWorkern aus aller Welt.

  • Bücher

    Stöbern Sie durch unsere umfangreiche Bibliothek.

  • Podcasts

    Spannende Gespräche über das Neueste aus Business und Technologie.

Übersicht
  • Bewerbungsprozess

    Finde heraus, was dich in unserem Bewerbungsprozess erwartet.

  • Hochschulabsovent*innen und Quereinsteiger*innen

    Dein Einstieg in die IT-Welt.

  • Stellenangebote

    Finde offene Stellen in deiner Region.

  • In Kontakt bleiben

    Abonniere unsere monatlichen Updates.

Übersicht
  • Konferenzen und Events
  • Diversity und Inclusion
  • Neuigkeiten
  • Open Source
  • Management
  • Social Change
  • Español
  • Português
  • English
  • 中文
ThoughtWorksMenü
  • schließen   ✕
  • Unsere Services
  • Unsere Kunden
  • Insights
  • Karriere
  • Über uns
  • Kontakt
  • Zurück
  • schließen   ✕
  • Übersicht
  • Delivery Mindset trifft Software-Exzellenz

    Verfolgen Sie einen innovativen Ansatz in der Softwareentwicklung, um noch schneller erfolgreich zu sein.

  • Experience Design und innovative Produkte

    Liefern Sie schnell außergewöhnliche Produkte und Kundenerlebnisse. Entwickeln Sie Design und Funktion kontinuierlich weiter.

  • Betriebsmodelle ohne Reibungsverluste

    Verbessern Sie die Fähigkeit Ihres Unternehmens, auf Veränderungen zu reagieren.

  • Erkenntnisgestützte Entscheidungsfindung

    Nutzen Sie Ihre Datenbestände, um neue Geschäftsmöglichkeiten zu erschließen.

  • Partnerschaften

    Nutzung unseres Netzwerks aus vertrauenswürdigen Partnern, um noch bessere Ergebnisse für unsere Kunden zu erzielen.

  • Plattform Strategie

    Entwicklung dynamischer Technologieplattformen, die sich an Ihre Geschäftsstrategie anpassen.

  • Zurück
  • schließen   ✕
  • Übersicht
  • Automobil
  • Clientech, Energie und Versorgung
  • Banken und Versicherungen
  • Gesundheit
  • Medien
  • Non-Profit
  • Öffentlicher Sektor
  • Handel und E-Commerce
  • Reise und Transport
  • Zurück
  • schließen   ✕
  • Übersicht
  • Unsere Empfehlungen

  • Technologie

    Ausführliche Betrachtungen neuer Technologien.

  • Business

    Aktuelle Business-Insights, Strategien und Impulse für digitale Querdenker.

  • Kultur

    Insights zu Karrieremöglichkeiten und unsere Sicht auf soziale Gerechtigkeit und Inklusivität.

  • Digitale Veröffentlichungen und Tools

  • Technology Radar

    Unser Leitfaden für aktuelle Technologietrends.

  • Perspectives

    Unsere Publikation für digitale Vordenker*innen

  • Digital Fluency Model

    Ein Modell zur Priorisierung digitaler Fähigkeiten, um für das Unvorhersehbare bereit zu sein.

  • Decoder

    Der Technology-Guide für Business Entscheider

  • Alle Insights

  • Artikel

    Expertenwissen für Ihr Unternehmen.

  • Blogs

    Persönliche Perspektiven von ThoughtWorkern aus aller Welt.

  • Bücher

    Stöbern Sie durch unsere umfangreiche Bibliothek.

  • Podcasts

    Spannende Gespräche über das Neueste aus Business und Technologie.

  • Zurück
  • schließen   ✕
  • Übersicht
  • Bewerbungsprozess

    Finde heraus, was dich in unserem Bewerbungsprozess erwartet.

  • Hochschulabsovent*innen und Quereinsteiger*innen

    Dein Einstieg in die IT-Welt.

  • Stellenangebote

    Finde offene Stellen in deiner Region.

  • In Kontakt bleiben

    Abonniere unsere monatlichen Updates.

  • Zurück
  • schließen   ✕
  • Übersicht
  • Konferenzen und Events
  • Diversity und Inclusion
  • Neuigkeiten
  • Open Source
  • Management
  • Social Change
Blogs
Wählen Sie ein Thema
Alle Themen ansehenschließen
Technologie 
Agiles Projektmanagement Cloud Continuous Delivery  Data Science & Engineering Defending the Free Internet Evolutionäre Architekturen Experience Design IoT Sprachen, Tools & Frameworks Modernisierung bestehender Alt-Systeme Machine Learning & Artificial Intelligence Microservices Plattformen Sicherheit Software Testing Technologiestrategie 
Geschäft 
Financial Services Global Health Innovation Retail  Transformation 
Karriere 
Karriere Hacks Diversity und Inclusion Social Change 
Blogs

Themen

Thema auswählen
  • Technologie
    Technologie
  • Technologie Überblick
  • Agiles Projektmanagement
  • Cloud
  • Continuous Delivery
  • Data Science & Engineering
  • Defending the Free Internet
  • Evolutionäre Architekturen
  • Experience Design
  • IoT
  • Sprachen, Tools & Frameworks
  • Modernisierung bestehender Alt-Systeme
  • Machine Learning & Artificial Intelligence
  • Microservices
  • Plattformen
  • Sicherheit
  • Software Testing
  • Technologiestrategie
  • Geschäft
    Geschäft
  • Geschäft Überblick
  • Financial Services
  • Global Health
  • Innovation
  • Retail
  • Transformation
  • Karriere
    Karriere
  • Karriere Überblick
  • Karriere Hacks
  • Diversity und Inclusion
  • Social Change
Experience DesignInnovationTechnologieGeschäft

User Research Will Destroy Your Product

Ted Nielsen Ted Nielsen

Published: May 25, 2016

Properly used, user research will destroy your product. This is a feature, not a bug.

You have an idea, maybe even a great one. You’ve done your design sprint, your visioning session, your intense few days of white-boarding. You’ve built your draft roadmap, have a few personas and a well-worn copy of the Lean Startup. No plan, though, survives its first contact with the enemy.

Lean methodologies praise the power and impact of user research without actually talking through how to use it. Product managers, particularly in the enterprise, have piles of competitive, quantitative outputs, and analytics reports. Startups have budgets to hand out gift cards to participants in coffee shops, and both think they’re doing it right.

Talking to your customer is hardly a novel idea. Keep in mind, mold is over a billion years old. Penicillin was only released in the 20th century. Just because something exists doesn’t mean it’s useful to you, especially if you’re not using it the right way.

In that vein, get ready for a bitter pill.

The Lies We Tell Ourselves

In too many process implementations, 'value' is a convenient fiction. Agile and Lean teach us to drive to value and to frame everything in the context of the user’s need. This results in backlogs full of stories that start with the dubious 'as a system' and with UI tweaks. Noble ideals have a way of leading us astray if we don’t think critically about them.

Our understanding of value in this context is anemic and cheap.
Tweet this

To reclaim the word, a product is a promise of shared value: lasting commercial and competitive benefit for the provider and ongoing unique fulfillment for the consumer. Like a handshake, it only achieves its purpose in the moment that the two collide.

I’ve already talked about the importance of 'unique' in that definition: Markets crowded with me-too solutions are reduced to bowls of oatmeal — homogenized, boring, and different only in incidentals.

Value, though, is how we crack open 'fulfillment' in the above definition.

Risk and Reward

Pirate metrics (Acquisition - Activation - Retention - Revenue - Referral) were popularized by Dave McClure of 500 Startups as a funnel to describe an engagement model for B2C companies. While each stage is a fundamental change in the nature of the relationship of the user to the product, I want to focus on the middle, Retention. Stickiness and the realization of value in the product lives there.

Hypothesized value is where you test, and the model for measuring the return of user research is iterations of Risk and Reward. Your risk is composed of two costs: the cost of staging the test (whether as ethnographic observation, qualitative interviews, panels, sending out a survey, setting up the clickstream analytics, etc.) and the cost of being wrong (your sunk cost in what you’re testing). Reward is pretty simple; it’s the benefit of being right.

The ratio between risk and reward is not stable over the course of taking a product from concept to customer. Like a functional test smoke suite, the first few tests are highly valuable: Do people need this? Does this product create fulfillment? Have I made what makes my company different and amazing a critical part of this product and how it works?

Over time, your focus narrows as you pass those broader, qualitative tests: Does this feature work? Can a different design or interaction paradigm improve the performance of this KPI? As you move from 'should' to 'how', your questions get smaller, and so your reward gets smaller too. Like all iterative testing frameworks, your process suffers from diminishing returns.

The relationship of risk and reward over the lifecycle of a product is therefore described logarithmically — rapid advancement and value from the early stages of the investigation to stable and shrinking return in the later stages. Because of this, both the beginning and the end of the journey encourage the most dangerous behavior.

Big Bang

The first tests are unquestionably the riskiest; the fate of your idea hangs in the balance. You use them to establish the fitness for need for the idea. These tests establish the boundaries within which the fitness for purpose for the product’s features and capabilities live and develop. Your first tests are the gladiatorial arena for your idea.

In organizations without a core testing principle, the risk borne by the first tests balloons the longer they don’t vet their ideas. This is never worse than when they commit the cardinal sin of a Big Bang release. The sin of the Big Bang is either fueled by fear of being wrong (so why wouldn’t you stop earlier?) or a case of mistaken identity: the idea has been conflated with its implementation.

There’s an inherent amount of risk in the first tests—don’t make it worse by expanding that risk beyond the potential reward value. You end up shouldering higher premiums on that risk for the same information that other organizations are getting on the cheap.

The Narrows

In a psychology paper published last year, Wagenmakers et al. said, “A single experiment cannot overturn a large body of work. […] A strong anvil need not fear the hammer[.]” A healthy research capability develops the 'strong anvil' of your product – you can develop and evolve your product with confidence, and validate your instincts against customer feedback and behavioral analysis. With an established track record of tests and successful reactions to the test results, no single test or line of inquiry is going to break the world.

But as your line of inquiry narrows from idea to implementation, and from implementation to options within an implementation, your potential benefit faces dramatically reduced returns on the costs of the research. There are only so many 'red vs. blue buttons' and 'tabs or hamburger menu' tests that you can put into the field.

There is a minimum cost for any research initiative, whether the developer time to implement new tags for your analytics platform, the time to write a script to walk a customer through the application flow, or the cost of that Starbucks gift card to thank your customer for their time and consideration—you’re paying someone something for that information. You must ensure that the information you’re getting is worth what you’re paying for it.

The Narrows, as the risk-reward loop of research becomes neutral and smaller, are the more dangerous location on our curve – it’s unusual for digital organizations never to have heard of user testing and willfully do big-bang releases (accidentally is another matter entirely). Overusing testing and falling prey to the narrows is all too common, though.

The danger of the Narrows is simple: If you’re spending all your time and money in your research budget fine-tuning a finely tuned instrument, your competitors are already looking for the next opportunity.

The Narrows are just another kind of stagnation, and it’s a sure sign that a major pivot or breakthrough capability set is needed. When dealing with the chilling effects of a stagnated roadmap, a product manager should purposefully introduce risk as a way to unlock potential new reward. Escaping the Narrows requires a step change to leap out of it: modifying your product to break into adjacent markets or evolving from one product into a platform play to leverage network effects.

Your competitors are hungry and trying to out-innovate you.

First Contact

No matter how clever or well-conceived the roadmap, every user test I’ve been a part of has produced insightful, interesting, and actionable data. The produced product always was different from the planned product—and better too. That plan didn’t survive first contact with the user, and the user research—happily—destroyed my product.

The usefulness of any given test is measured against its subject, scope, and the outcomes of all the prior tests.

This means that the user testing framework for any product incurs diminishing returns over its lifecycle. While modeling this return as iterations of risk-reward, there are two dangerous periods in the testing harness: the earliest and the last ones.

Early tests run the risk of having uneven distribution of risk, especially considering more sophisticated solutions that have advanced too far into their implementation. Big bang releases, surprise launches, and other isolationist tactics bear significantly higher risk premiums than collaborative and iterative investigations advocated by standard Lean practices.

Later tests, with narrower scope, run the risk of stagnating the product’s advancement of value by confusing activity (staging tests and analyzing results) with progress (incremental or dramatic production of value). The narrowing risk-reward cycles mean a step-change (pivot, launch, or new market) should be introduced to drive the product to higher levels of lasting value.

Lean on research smartly—answer the big questions for your stakeholders and checkbook-holders (should we build this thing at all?). Connect with your customer in ways that ensure they benefit from all the hard work you and your teams have done (be the penicillin, not the mold). Understand when you have a stagnating product and push it out of the nest; find a new market, a new segment, or re-imagine your role in your customer’s life.

Properly used, user research will break your product. It also will show you the way to pick up the pieces.
Tweet this

Weitere Blogposts
Experience Design

Business, Design and Technology: Joining Forces for a Truly Competitive Advantage

Ben Melbourne
Mehr hier
Innovation

Practical Product Innovation

Ben Melbourne
Mehr hier
Experience Design

Conducting User Testing Remotely

Suganth Chellamuthu
Mehr hier
  • Unsere Services
  • Unsere Kunden
  • Insights
  • Karriere
  • Über uns
  • Kontakt

WeChat

×
QR code to ThoughtWorks China WeChat subscription account

Presseanfragen | Datenschutz | Impressum | Modern Slavery statement ThoughtWorks| Barrierefreies Webdesign | © 2021 ThoughtWorks, Inc.