Building Empathy-Driven Developer Documentation - Kat King - Write the Docs Portland 2018

Twilio’s developer education team built “empathy-driven documentation” by treating developer frustration as a measurable, research-backed signal—not an afterthought. The core shift: documentation quality couldn’t be judged by traffic or signups alone, because those metrics don’t reveal whether developers feel unstuck, confident, and able to complete real tasks. The team’s answer was to combine lightweight satisfaction scoring with deeper user research and continuous iteration, then wire the results directly into the documentation workflow.

The journey began with a reality check from real-time public feedback. Team members monitored Twitter for comments about the docs—compliments were nice, but complaints about duplicate pages, outdated content, and missing information were a “canary in a coal mine.” That prompted a bigger internal question: what does “good documentation” actually mean, and who exactly is it serving? The team realized it didn’t fully understand its audience across personas (from CTOs to junior developers) or how different content types were being used.

To measure quality beyond vanity metrics, the team launched a pop-up widget across documentation pages asking a single question: did this content serve your needs (yes/no). Over a couple months, the widget produced quantitative “percent happy” style data and helped pinpoint weak areas by content type—voice docs lagged, while error code documentation was “abysmal.” When users answered “no,” the team offered short follow-up calls, turning negative ratings into qualitative insight. Those conversations surfaced recurring pain points and also forced the team to confront an emotional truth: documentation work can feel personal when developers describe it as confusing or incomplete.

That empathy work deepened through one-on-one user research in September, where a researcher observed nine developers from different personas completing tasks or working through real scenarios. Watching people navigate the docs revealed a major information architecture mismatch. The docs were organized around content types (API references, tutorials, walkthroughs), but developers behaved task-first. With more than 20 products and thousands of URLs and code samples, the “tutorial vs. reference” framing didn’t help people find what they needed—developers kept asking practical questions like how to send a text message or make a phone call.

In October, the team upgraded its measurement system from yes/no to a custom ratings widget that allowed star ratings per page. Pages rated below four stars triggered prompts for more detail, and the feedback flowed into a dashboard and a Slack integration so contributors could act quickly. The team initially worried the tooling would create noise, but it instead helped surface trends like “can’t find what I’m looking for” and “not enough examples,” while also catching bugs and outdated guidance.

Iteration then became the operating model: investigate, test, measure, and only then build. A/B tests helped kill ineffective ideas—like adding a prominent “sign up now” button to QuickStart pages, which didn’t increase registrations and annoyed users. For cases that couldn’t be tested cheaply, the team rewrote content based on research. One QuickStart revision in Python rose from 3.67 to 4.75 stars after being rebuilt with clearer steps and up-to-date code samples. Alongside the quality metric, business indicators improved too: signups rose, bounce rate fell, and sessions increased.

The team’s concluding lessons were blunt: accurate code samples are the most important feature, content-type labels matter less than task success, and docs must support “information foraging” by giving developers clear signposts once they land from Google. The overall thesis was practical rather than sentimental: build mechanisms for feedback and research, iterate in small steps, and let empathy guide what gets fixed first—because that’s the only path to documentation that reliably helps developers succeed.