When visitors come to a website, it’s straightforward to determine insights that allow a company to improve and refine the experience for their visitors: where they went, where they ‘hovered’, if they ‘liked’ a post, and much more.
In the physical world, getting this level of insight is more challenging. How can a company get actionable feedback in the physical world in the same way they do for their online properties?
This was the challenge facing C4Media, the organizers of international software development conference QCon. Traditionally, QCon used a tray of red, yellow, and green pieces of paper outside a session. Attendees were asked to pick and put into a bucket when they left. This is simple, obvious, low-friction, and has a very high participation rate.
However, the data collected was in the aggregate: feedback without any understanding of a specific attendee's overall experience. Knowing when, where, and who gave feedback across sessions would enable the organizers to continually improve the attendee experience.
Leaving a talk early and voting yellow is different from sticking to the end and voting yellow. Because an individual was identified, patterns across multiple sessions could be seen. An attendee could be contacted later and asked if they would provide further feedback.
The organizers had already tried a mobile app in the past and found that the added friction of accessing the app massively reduced participation.
An idea was proposed using the NFC (near-field communication) tag built into the attendee’s badge. The tag could be used with a device that captured a ‘vote’ to create a system that is as easy and obvious to use as paper, but with the ability to record key details—the who, when and where—for the feedback.
This is not as simple as it seems. Conferences are temporary spaces, networks are unreliable and inconsistent. Installation of special equipment is often prohibited or very difficult to do. Staff are temporary and anything that requires significant training will fail. Any solution must be battery operated as power outlets and dangling wires won’t work out in the busy conference environment.
Making something that humans can intuitively use and is reliable is deceptively hard. With six weeks to go until QCon SF 2015, QCon asked ThoughtWorks to help them design and test such a system.
ThoughtWorks takes a human-centered design approach to these types of problems. The solution needs to be as frictionless as possible: not just for the attendee, but for the operators and organizers of the conference as well.
The first concern is the challenging temporary space of a conference hall. What technology would be robust and fail-safe? We broke the problem down into two distinct needs: collecting an attendee's vote at the exit point, and then the transmission of that vote to the conference organizers back-end systems.
Given the temporary nature of conferences, we realized we needed a solution that a volunteer could hold out to attendees as they left a session. It needed to always be on and as light as possible to reduce fatigue. It needed to be extremely robust—not only physically, but also in terms of capturing votes. Once one or more votes were captured,they needed to be transmitted to the back-end systems for aggregation and analysis.
Looking at various options, the best was to use a small, ultra-low-powered Bluetooth “System on Chip.” These are cheap, easy-to-prototype systems that have a radio to communicate to the outside world and a small, embedded processor to manage inputs and run the application. A solution could be built that would be light and last all day without recharging.
The Bluetooth radio would enable us to wirelessly communicate the results of a vote. Any radio communication, particularly in environments like a conference venue, are error-prone and subject to unreliability.
So we decided to employ an unusual application of Bluetooth and deploy it as a mesh network. Making all the voting devices part of this ad-hoc, self-healing ‘mesh’ network made managing the collection of data completely seamless and automatic. No wires, no complex setting up of a special wifi network. As a node comes into range of another node, it automatically connects.
As one of those nodes connects to a gateway—a laptop or iPad running the gateway software, the data is passed from one node to another until it reaches the gateway and is sent to the backend servers. (Read more here for details of the open-source mesh that ThoughtWorks developed for this project.)
As the technical platform was being decided and iterated on, an approach to developing the form factor was needed. The first decision was to make each voting option a standalone module with its own batteries. This enabled us to greatly simplify the creation of different form factors to hold them, a key need for the design process.
When it came to producing final production candidates, the cost of making three or four times the number of identical boards versus one more complex board is trivial for a small production run. The cost of setting up the assembly line and automated equipment for a run of a few hundred is typically the biggest item, rather than the per board overhead.
At the 2015 QCon in San Francisco, we chose a few small sessions where we could see how the concept would work in practice, identify any issues with the overall service design, and discover what needed to be refined and re-tested.
When a new idea meets reality, some of the things we discovered surprised us:
- It turned out that the NFC tags in each of the badges were somewhat randomly placed inside the badge holder, and there was no visual indication of where it was. This obviously led to frustration by both participants and volunteers as they searched for the “golden spot.”
- The initial prototype had a single LED indicator to show when a vote had been registered. The use of the actual badge at the conference often obscured the indication leading to more confusion (although double voting had already been taken care of in the software).
- By the third day, some attendees had stuffed their badge holder with enough business cards and random handouts to interfere with the NFC tag detection.
There were many minor industrial design items and other observations like these. Some we were able to retool or recode overnight and test the next day, some were logged for the next iteration. For example, we were able to create a version with a buzzer to see if the overall flow was better when both a visual and audible indication was used. It was—obvious in hindsight.
By using 3D printing, different form factors and prototypes could be developed and modified not only in the run-up to the first conference but during the conference after observing the system in use with attendees.
After the first conference, there was enough confidence for us to work with the ThoughtWorks office in Shenzhen and identify a good original design manufacturer (ODM) to develop a single board that would integrate all the chosen hardware and have some made for testing at the next conference.
Over the course of three conferences, the team was able to iterate, refine and live user test the designs until the hardware, software and industrial design was optimal and we could commit to a final production run.
The final design is a modular compact robust system that was more than a vote capturing device. For instance, a single node could be deployed with a different faceplate for attendees to register interest in a particular vendor or talk.
More than a one-off solution, the mesh is a platform for the conference organizers, allowing them to build multiple applications in the future and bridge the physical and digital worlds.
