trolls.dev

The Night Shift · Department

AI User Interviews

Notes from building AI User Interviews — moving the cap on user research off the calendar so a team can talk to a hundred users as easily as one.

8 essays filed here

  1. 01

    Classic user research hears from whoever agrees to a call, and the people who agree are almost never a cross-section — they're the two loud ends: the fans who want to gush and the furious who want to vent. The quiet middle, the users for whom the product mostly just works, has no reason to give up an afternoon for a stranger's questions, so they don't. A sample built entirely from the willing reads as two camps with nothing between them — but that shape isn't your users, it's your recruiting. AI User Interviews drops the cost of answering to near zero, so the middle can finally answer without booking anything, and the two camps turn back into the tails of a distribution whose real mass was the people you were never hearing from.

    6 min read
  2. 02

    Ask 'so that was frustrating, right?' and nine people out of ten will agree, and you'll walk away with a tidy, confident number that measures nothing but the shape of your own question. A leading follow-up doesn't read the room — it moves it. The honest answer is messier, lower, and true, and the whole discipline of an AI interviewer is asking the question that lets it stand.

    6 min read
  3. 03

    A survey asks its questions and stops. An interview hears the answer and asks the one nobody wrote down — why? — and keeps asking until the reason underneath stops moving. The finding almost never lives in the scripted answer; it lives two or three follow-ups down, in the part that never scaled because a person asking for the hundredth time can't. AI User Interviews is built so the improvised part is the part that scales.

    6 min read
  4. 04

    Talk to six users and you'll be tempted to average them into one tidy 'typical' person — and build for them. Talk to a hundred and the average dissolves: two real groups appear with an empty valley between, and the mean turns out to describe someone who doesn't exist. The distribution was the finding all along. AI User Interviews is built so you can afford to see it.

    6 min read
  5. 05

    Two people answer the same research question differently, and everyone reads it as a finding. But a human interviewer never actually asks the same question twice — the wording drifts, the tone leads, the tenth session is wearier than the first. AI User Interviews holds the one instrument nobody thought to control still, so the variation that's left is the respondent's, not the asking's.

    5 min read
  6. 06

    Every research plan quietly dodges the same question: how many interviews is enough? The honest answer — keep going until you stop hearing new things — was useless for decades, because nobody could afford to run that far. AI User Interviews turns saturation from a hunch you defend into a shape you watch arrive.

    5 min read
  7. 07

    Drop the cost of a user interview far enough and you don't just get more interviews — you get a different thing entirely. Research stops being an event you schedule and becomes an instrument you leave running. That shift, not the raw count, is what AI User Interviews is really for.

    5 min read
  8. 08

    Teams don't run too few user interviews because they've run out of things to ask. They run too few because every interview has to be scheduled, sat through, and written up by a person — and that cost, not curiosity, is what caps the research. AI User Interviews is built to move the cap.

    5 min read