The Night Shift · Department
Research
How we choose research questions, recruit participants, and decide when a study has enough evidence.
9 essays filed here
- 01
The pilot you never ran
Every research plan has a pilot in it — a first pass where you find out your questions don't work the way you hoped. The catch is that in classic research the pilot is invisible: it's your first three real interviews, run at full price, and the wording you fix afterward can never be asked of the people who already answered the broken version. AI participants make the pilot a thing you can actually run — cheaply, repeatedly, before a single human hour is spent — so the instrument that reaches your real users is one you've already debugged.
6 min read - 02
The answers didn't match. The meanings did.
Ask a hundred people an open question and you get a hundred answers, no two worded the same. The naive way to make sense of them is to count words — group the ones that say 'slow', the ones that say 'expensive' — but people almost never reach for the same word for the same feeling, so keyword counting finds a dozen tiny piles and misses the four real themes hiding under the phrasing. The fix is to stop matching words and start placing meanings: put each answer where its sense sits, not where its vocabulary does, and let the groups appear on their own. This is the quiet engine under AI User Interviews' synthesis — the step that turns a pile of transcripts into a short list of findings you can act on.
6 min read - 03
The question before the question
Ask someone how satisfied they are and you'll get one number. Ask them to describe a recent frustration first, then ask the same satisfaction question, and the number drops — not because anything about the product changed, but because the question before it changed what 'satisfied' now means. Order effects are one of the oldest, best-documented traps in survey research, and the reason they stay invisible is simple: a study that runs one fixed order has nothing to compare its answer against. AI User Interviews can afford to run the same interview in several orders at once, which turns the bias from an unseen constant into a number you can measure — and subtract.
6 min read - 04
The middle never booked
Classic user research hears from whoever agrees to a call, and the people who agree are almost never a cross-section — they're the two loud ends: the fans who want to gush and the furious who want to vent. The quiet middle, the users for whom the product mostly just works, has no reason to give up an afternoon for a stranger's questions, so they don't. A sample built entirely from the willing reads as two camps with nothing between them — but that shape isn't your users, it's your recruiting. AI User Interviews drops the cost of answering to near zero, so the middle can finally answer without booking anything, and the two camps turn back into the tails of a distribution whose real mass was the people you were never hearing from.
6 min read - 05
The clean answer was the wrong one
Ask 'so that was frustrating, right?' and nine people out of ten will agree, and you'll walk away with a tidy, confident number that measures nothing but the shape of your own question. A leading follow-up doesn't read the room — it moves it. The honest answer is messier, lower, and true, and the whole discipline of an AI interviewer is asking the question that lets it stand.
6 min read - 06
The script was never the interview
A survey asks its questions and stops. An interview hears the answer and asks the one nobody wrote down — why? — and keeps asking until the reason underneath stops moving. The finding almost never lives in the scripted answer; it lives two or three follow-ups down, in the part that never scaled because a person asking for the hundredth time can't. AI User Interviews is built so the improvised part is the part that scales.
6 min read - 07
No one lives at the average
Talk to six users and you'll be tempted to average them into one tidy 'typical' person — and build for them. Talk to a hundred and the average dissolves: two real groups appear with an empty valley between, and the mean turns out to describe someone who doesn't exist. The distribution was the finding all along. AI User Interviews is built so you can afford to see it.
6 min read - 08
The interviewer was the variable
Two people answer the same research question differently, and everyone reads it as a finding. But a human interviewer never actually asks the same question twice — the wording drifts, the tone leads, the tenth session is wearier than the first. AI User Interviews holds the one instrument nobody thought to control still, so the variation that's left is the respondent's, not the asking's.
5 min read - 09
Enough is a shape you can see
Every research plan quietly dodges the same question: how many interviews is enough? The honest answer — keep going until you stop hearing new things — was useless for decades, because nobody could afford to run that far. AI User Interviews turns saturation from a hunch you defend into a shape you watch arrive.
5 min read