ThoughtWorks
  • Contact
  • Español
  • Português
  • Deutsch
  • 中文
Go to overview
  • Engineering Culture, Delivery Mindset

    Embrace a modern approach to software development and deliver value faster

    Intelligence-Driven Decision Making

    Leverage your data assets to unlock new sources of value

  • Frictionless Operating Model

    Improve your organization's ability to respond to change

    Platform Strategy

    Create adaptable technology platforms that move with your business strategy

  • Experience Design and Product Capability

    Rapidly design, deliver and evolve exceptional products and experiences

    Partnerships

    Leveraging our network of trusted partners to amplify the outcomes we deliver for our clients

Go to overview
  • Automotive
  • Cleantech, Energy and Utilities
  • Financial Services and Insurance
  • Healthcare
  • Media and Publishing
  • Not-for-profit
  • Public Sector
  • Retail and E-commerce
  • Travel and Transport
Go to overview

Featured

  • Technology

    An in-depth exploration of enterprise technology and engineering excellence

  • Business

    Keep up to date with the latest business and industry insights for digital leaders

  • Culture

    The place for career-building content and tips, and our view on social justice and inclusivity

Digital Publications and Tools

  • Technology Radar

    An opinionated guide to technology frontiers

  • Perspectives

    A publication for digital leaders

  • Digital Fluency Model

    A model for prioritizing the digital capabilities needed to navigate uncertainty

  • Decoder

    The business execs' A-Z guide to technology

All Insights

  • Articles

    Expert insights to help your business grow

  • Blogs

    Personal perspectives from ThoughtWorkers around the globe

  • Books

    Explore our extensive library

  • Podcasts

    Captivating conversations on the latest in business and tech

Go to overview
  • Application process

    What to expect as you interview with us

  • Grads and career changers

    Start your tech career on the right foot

  • Search jobs

    Find open positions in your region

  • Stay connected

    Sign up for our monthly newsletter

Go to overview
  • Conferences and Events
  • Diversity and Inclusion
  • News
  • Open Source
  • Our Leaders
  • Social Change
  • Español
  • Português
  • Deutsch
  • 中文
ThoughtWorksMenu
  • Close   ✕
  • What we do
  • Who we work with
  • Insights
  • Careers
  • About
  • Contact
  • Back
  • Close   ✕
  • Go to overview
  • Engineering Culture, Delivery Mindset

    Embrace a modern approach to software development and deliver value faster

  • Experience Design and Product Capability

    Rapidly design, deliver and evolve exceptional products and experiences

  • Frictionless Operating Model

    Improve your organization's ability to respond to change

  • Intelligence-Driven Decision Making

    Leverage your data assets to unlock new sources of value

  • Partnerships

    Leveraging our network of trusted partners to amplify the outcomes we deliver for our clients

  • Platform Strategy

    Create adaptable technology platforms that move with your business strategy

  • Back
  • Close   ✕
  • Go to overview
  • Automotive
  • Cleantech, Energy and Utilities
  • Financial Services and Insurance
  • Healthcare
  • Media and Publishing
  • Not-for-profit
  • Public Sector
  • Retail and E-commerce
  • Travel and Transport
  • Back
  • Close   ✕
  • Go to overview
  • Featured

  • Technology

    An in-depth exploration of enterprise technology and engineering excellence

  • Business

    Keep up to date with the latest business and industry insights for digital leaders

  • Culture

    The place for career-building content and tips, and our view on social justice and inclusivity

  • Digital Publications and Tools

  • Technology Radar

    An opinionated guide to technology frontiers

  • Perspectives

    A publication for digital leaders

  • Digital Fluency Model

    A model for prioritizing the digital capabilities needed to navigate uncertainty

  • Decoder

    The business execs' A-Z guide to technology

  • All Insights

  • Articles

    Expert insights to help your business grow

  • Blogs

    Personal perspectives from ThoughtWorkers around the globe

  • Books

    Explore our extensive library

  • Podcasts

    Captivating conversations on the latest in business and tech

  • Back
  • Close   ✕
  • Go to overview
  • Application process

    What to expect as you interview with us

  • Grads and career changers

    Start your tech career on the right foot

  • Search jobs

    Find open positions in your region

  • Stay connected

    Sign up for our monthly newsletter

  • Back
  • Close   ✕
  • Go to overview
  • Conferences and Events
  • Diversity and Inclusion
  • News
  • Open Source
  • Our Leaders
  • Social Change
Blogs
Select a topic
View all topicsClose
Technology 
Agile Project Management Cloud Continuous Delivery  Data Science & Engineering Defending the Free Internet Evolutionary Architecture Experience Design IoT Languages, Tools & Frameworks Legacy Modernization Machine Learning & Artificial Intelligence Microservices Platforms Security Software Testing Technology Strategy 
Business 
Financial Services Global Health Innovation Retail  Transformation 
Careers 
Career Hacks Diversity & Inclusion Social Change 
Blogs

Topics

Choose a topic
  • Technology
    Technology
  • Technology Overview
  • Agile Project Management
  • Cloud
  • Continuous Delivery
  • Data Science & Engineering
  • Defending the Free Internet
  • Evolutionary Architecture
  • Experience Design
  • IoT
  • Languages, Tools & Frameworks
  • Legacy Modernization
  • Machine Learning & Artificial Intelligence
  • Microservices
  • Platforms
  • Security
  • Software Testing
  • Technology Strategy
  • Business
    Business
  • Business Overview
  • Financial Services
  • Global Health
  • Innovation
  • Retail
  • Transformation
  • Careers
    Careers
  • Careers Overview
  • Career Hacks
  • Diversity & Inclusion
  • Social Change
InnovationTechnologyBusiness

Lessons learned from life as a chatbot: Part 1

Kylie Castellaw Kylie Castellaw

Published: Dec 26, 2017

This is the first in a two-part series of articles exploring some of our learnings from working with chatbots.
​
It was a blindingly sunny day in the Bay Area, but inside a corporate meeting room, the shades were drawn. Kylie was FaceTiming with a 16-year old cancer survivor, asking questions about platforms like Facebook Messenger. For a while, the conversation sounded like any you might expect from a 30-something asking a teenager about what the kids are using nowadays.


“Yeah, I use Messenger every day” he replied nonchalantly. “All my friends are on it.”

“Ok, good to know. Do you have your Messenger app open? We have something in the works that we’d like you to take a look at...”

Kylie glanced at Janise, sitting just out of view of the camera. Her hands hovered over the keyboard; knuckles cracked and ready to fly at a moment’s notice. The Facebook Messenger UI open and waiting, the Google Doc of pre-written responses at the ready… it was bot time.

 

At HopeLab, when the idea for a chatbot came out as a promising concept through extensive needs-based user research, we had a few options for next steps.
  • We could ask our audience what they thought about bots, and get them to anticipate how they might use one. But we’d already heard our audience didn't know what bots were, and we knew humans — even super-switched on teenagers — aren’t famous for their abilities to accurately predict the future.
  • We could build out our idea as a lightweight bot and then test that. But without some indication that it was the right thing to build, why waste even a week of developer time to put together our best guess at what would be helpful to our users?
  • Or we could go with the fastest, cheapest, and most accurate way to test the idea: we could pretend to be a bot.
So we recruited a small group of users and asked them to chat with our trial automated service, Cancer Mindshift, over Facebook Messenger — with the knowledge that it was early days and the service was only part automated, still part human. With just a handful of users and a few days of part-time side hustle to our normal jobs, our understanding of the product and our users skyrocketed. By pretending to be a bot, we learned more than we could have hoped for. Here are a few of our big takeaways:
 
  1. There's still plenty of confusion from people about what a bot even is. “Is it a human or a machine? Why isn’t it an app? What will happen when I talk to it?” Though bots are starting to become more common on platforms such as FB messenger, almost no one we spoke to admitted to having chatted with a “bot” or an automated text-based system before. And these answers were from the technologically savvy generation of millennials, who are very likely have interacted with chatbots in the past. As designers of technology, sometimes we forget how long it takes for new tech to really sink into a culture’s understanding. Though people have been talking about bots for years in Silicon Valley, the concept is still very new for many of our users. And pretending to be a chatbot helped us find out what exactly was confusing them and what they needed to get on board.
  2. People really want signals that they're heard. This is true even if they know that the chatbot may not understand them at this point and is still delivering generic responses. We found that bot responses don’t have to be long to be helpful — subtle responses that we as people use all the time in our spoken and text conversations are often sufficient. Examples include “Sorry to hear that” or “Sounds like you’re dealing with a lot” or even an emoji.
  3. People sometimes open up to bots even more than to other people. We think this happens because people know that the feedback and advice coming back is truly objective. Though hard to wrap their mind around at first, once they got going, our users really started to open up. Here’s something we heard from a test user in this crucial test phase after she asked for help with a stressful emotional situation she was experiencing. She was happy to let us share her feedback: “This is actually a true thing that is happening, and it was almost better to talk to the bot about it... Friends just say, ‘damn, that sucks’; and it's hard. The bot reminded me to break it down into small manageable bits — that so much of it is not in my control. So what can I do to feel empowered? That was super important to hear.”


     
  4. Bots are good at serving up content quickly, but people pretending to be bots aren’t. We rapidly saw how hard it is to reply quickly as a human, even when you have a simple script to copy and paste from. This is something that bots were made for — and people weren’t. And our users noticed the lag from the second one, complaining that the system was too slow, even if it was only a two-second response.

So many lessons learned in so little time. We’re extremely grateful to our small but mighty group of testers, who were able to suspend disbelief in the concept of a bot long enough to give it a shot and to let us know what they thought as they interacted with it.

From just five 30-minute user sessions and a week of those five people texting intermittently with the system, we validated our product idea and found a plethora of ways to increase its likelihood of success going forward.

Stay tuned for Part Two, which will focus on lessons for designers and developers considering pretending to be a bot to validate their ideas.
Related blogs
Machine Learning & Artificial Intelligence

Why conversational UI? Why now?

Holden Bale
Neil Redding
Learn more
Machine Learning & Artificial Intelligence

It’s time to talk (to our devices)

Stephan Dowding
Learn more
Machine Learning & Artificial Intelligence

Interact or Die Trying

Neil Redding
Learn more
  • What we do
  • Who we work with
  • Insights
  • Careers
  • About
  • Contact

WeChat

×
QR code to ThoughtWorks China WeChat subscription account

Media and analyst relations | Privacy policy | Modern Slavery statement ThoughtWorks| Accessibility | © 2021 ThoughtWorks, Inc.