Inside Quizlet's incredible feedback center
Today we'd like to share Quizlet's secret weapon: our custom-built feedback center.
When I started Quizlet, one of the first things I built was a small admin tool to receive and respond to feedback messages from users. I spent a lot of time talking directly to our early users, by email and sometimes by phone. Quizlet has grown a lot since then, and we now get around 20,000 feedback messages a month. We personally respond to almost every one of them, without using response templates.
Here are the highlights of our system:
- We can pinpoint user problems very quickly thanks to a deep integration with our database and logs.
- Our entire team uses this system as a vital part of doing their jobs, so knowledge of issues is distributed across the whole team.
- We've built the system to make it easy to identify patterns, bugs, and fast-emerging issues.
- We make lots of users very loyal by being human and personally responding to almost every message.
When someone writes to a company's user support with a vague message, like "the page broke," figuring out the problem can be a time-consuming back-and-forth process. But because our feedback system is deeply integrated into our site data and logs, we can often pinpoint the problem without any additional conversation. There's also no wall between our users and the people who build the products -- if someone writes in with a design issue, there's a good chance they'll get a response from our designer, Anand. Over the years we've tweaked the system hundreds of times, and made it an incredibly efficient process for our entire team to use. As a result, we get a huge amount of value from our user support messages with minimal resource drain. Here's what it looks like:
Some brief background: Quizlet is a top 100 U.S. website, with 20 million unique users per month. Quizlet is an essential service for students and teachers around the world, and we care a lot about solving the day-to-day problems they have with us. Our feedback center has become our most important tool in identifying problems with our service, and has scaled extremely well with our growth.
Everybody does feedback
Every employee on the team has unmediated access to the system, and frequently checks it for the latest news on how people are experiencing Quizlet. Our feedback system is the homepage of our admin center, so there's no separate system to log in to. Our developers all use it to monitor for bugs and keep in touch with users. Because they use the tool frequently, they're automatically motivated to continually tweak it and make it better.
Let's take a look at how this system works. First, we want to understand our current status at a high level. At the top of the page, a row of icons in the header shows how many messages each staff member has responded to in the past week:
Because everyone is encouraged to respond to some feedback every week, the distribution of knowledge about current issues on the product is spread across everyone on the team. A color bar at the top shows the latest 200 messages and who responded to each one. This lets us see at a glance who's been handling recent messages and how far behind we are:
An incoming message
When a new message comes in, we automatically categorize it by its source so we can see feedback messages from different areas of the site in one click. This means our Android team can quickly scan for problems in their app and our web teams can look at feedback on only recently launched features.
Now, let's look at each individual message. We've designed each message to show the maximum amount of information that will help us respond to the user quickly and accurately.
We show information about their device, their account, their location, their recent actions, and their history of feedback with us. This user is having problems with our Speller feature, but we also know they are in New York (geography can sometimes be the cause of an issue), using Internet Explorer 9, and they've gotten some errors logged. They've also written to us 3 times, so their previous messages might have some clues.
At the top of each message, we show how many times they've used all our study modes, and if they use our iPhone or Android apps. We include information like this, along with their profile icon, to give us a more personalized understanding of who they are. A heavy user of Quizlet might look like this:
Integrated log viewer
Finding patterns in feedback
One person understanding an issue is not enough; for everyone on the team to understand, we needed tools for cataloging groups of feedback and flagging actionable issues. For that, we built a system for creating "cases," where many feedbacks can be grouped together. It's useful for identifying trends and larger issues; it's also helpful for bugs we can't seem to figure out because we can group related messages together.
Any time you want to add a case to a feedback, you can click "+ Case" and it will give you an auto-completing list of current cases. For example, someone may be asking us to let them record audio on Quizlet:
Recent active cases are displayed at the top of every page, so everyone on the team knows the pulse of current issues.
If you click on a case, you can see all the feedback messages for that case, and see the frequency trend of that problem. In this case, people have been asking for this new feature a lot recently!
We use Github to track all the current bugs in Quizlet. If the case you're looking at is a bug that warrants being fixed, you can turn it into a Github issue in one click:
When you've been tasked with fixing the bug, you can view all the related feedback messages and their context in one click. This makes reproducing issues and identifying patterns that cause an issue extremely easy. And when you're done, you can click "Email everyone" and send a note to everyone who was affected by the bug, letting them know it's now a thing of the past!
Generating user loyalty
Because we manage the system ourselves, we've added lots of little niceties over time. For example, in the barebones email response template for each message, we automatically add a special thank you for paid subscribers who wrote in for the first time:
We see this tool as a primary way to generate warm fuzzies from our users. People aren't used to getting a personal, real message via online feedback forms. So we often surprise our users with the fact that they're dealing with real humans.
As Quizlet has grown, it's been a priority to maintain our deep understanding and involvement with the issues of our users. We spend a lot of time responding to feedback, but it's no one's full-time job. We have 3 people whose job description includes responding to feedback, but it's far from 50% of how they spend their time. Providing personal care for all of our users isn't easy, but the efficiency built into this tool makes it possible. Instead of seeing support as a necessary cost of running the business, we see this system as a main tool enhancing our ability to understand our business.
Many people tell us that we should package our feedback center up and sell it as a product. The main secret to its usefulness, however, is its level of custom integration with our product. So our response is usually, "You should just copy us instead!" Because we control exactly how it works, our developers can add and remove data for it in minutes. The tool has full access to all of our data and logs, letting us richly integrate any information we need. With each passing month, the tool's ability to provide context for our users' feedback messages increases.