Skip to content
try reference course saved
English
← all Data & Analysts playbooks

playbook

Find out why the number moved

Answer "why did it drop?" for real — compare the suspect period to a baseline, break it down by the right dimension, and rule out a broken query or dirty data before you blame the business.

medium ~30 min

when to reach for this

A metric moved and someone needs to know why before the meeting. Signups are down, revenue is flat, churn spiked — and the worst possible answer is a confident guess that turns out to be a broken query or a missing day of data. This system finds the real cause: it frames the move against a baseline, breaks it down by the dimension that actually explains it, and — crucially — rules out a data artifact *before* concluding the business changed. The difference between "paid ads collapsed" and "someone dropped me a half-loaded export" is this discipline.

gather this first

  • The metric and the window: "signups, last week vs the four weeks before" — and the raw file or table behind it, e.g. signups.csv.
  • A baseline you trust: the prior weeks, the same week last year, or last month's report — so 'moved' has something to move *against*.
  • Any known events in the window — a campaign that ended, a pricing change, an outage, a tracking change — so a real-world cause isn't mistaken for a data bug, or vice versa.

the workflow

  1. Quantify the move against a baseline

    Before chasing causes, size the change precisely. "Down a lot" sends you hunting for a crisis; "down 9% week-over-week, within the normal swing" might be noise. Anchor it to a baseline so you know whether there's even a real move to explain.

    you ask 
    From signups.csv, give me last week's signup total versus each of the prior four weeks, plus the average of those four. State the change as an absolute number and a percent. Is this move outside the normal week-to-week variation you see in the baseline?

    what you get back "Last week 1,180 signups; prior four averaged 2,310 — down 49%. Weekly variation in the baseline was ±8%, so a 49% drop is well outside normal." Now you know it's a real signal, not noise, and how big it is.

    Always size the move against the baseline's *normal variation* first. Half of 'why did it drop' questions dissolve when you see the metric was inside its usual range.

  2. Rule out the data and the query first

    This is the step everyone skips and regrets. A 49% drop is exactly what a missing day, a tracking change, or a filter bug looks like. Clear those out before you tell anyone the business changed.

    you ask 
    Before we conclude the business changed, rule out a data artifact. Does last week have all 7 days present (no missing dates)? Did the row count or any column's blank-rate jump versus prior weeks? Could a tracking or schema change explain it? And is the query counting last week on the same basis as the baseline weeks?

    what you get back A clean bill or a smoking gun: "All 7 days present; blanks steady; but source is NULL on 40% of last week's rows vs 12% before — a tracking change likely undercounts, this may be partly an artifact." Either way you now know whether you're chasing a real cause.

  3. Break it down by the dimension that explains it

    A top-line move is a sum of segment moves. Decompose by the dimension most likely to carry the cause — channel, source, region, plan — so you isolate *what* moved instead of asserting *why*.

    you ask 
    Now break last week's drop down by source against the baseline. For each source, show last week, the baseline average, and the change. Which source accounts for most of the decline — is it concentrated in one, or broad across all of them?

    what you get back A breakdown that localizes it: "paid_ads fell from 1,040 to 95 (–91%); every other source held within ±6%." A drop concentrated in one segment points at a specific cause; a broad drop points at something systemic — and they're different investigations.

  4. Form the hypothesis and pressure-test it

    Now you can name a cause — but make Claude argue against it too. The goal is a conclusion that survives a skeptic, with the one check that would confirm or kill it.

    you ask 
    So paid_ads signups collapsed while everything else held. Give me the 3 most likely explanations for that specific drop, the single check that would confirm or rule out each, and state which explanation the data most supports — and what evidence would change your mind.

    what you get back "Most likely the paid campaign ended or its budget capped on Tuesday — confirmed by paid_ads going to near-zero mid-week, not gradually. Less likely: ad-tracking broke (but other sources are fine). Check the ad platform's spend for last week to confirm." A defensible cause with a verification, not a hunch.

make it your own

  • **The number went UP unexpectedly:** the exact same flow works for a surprise spike — and ruling out a double-counting bug or a duplicate import matters even more, since good news gets less scrutiny.
  • **You haven't profiled the file yet:** if you're staring at a fresh export, run *Interview a dataset before you touch it* first — the date-range and blank-rate it surfaces are half of the 'rule out the data' step here.
  • **It keeps moving:** if the metric swings often, fold this into a scheduled agent or a /why command (see the *Features* tab) that runs the baseline-and-breakdown automatically and flags the segment that moved — so you start the investigation already pointed at the cause.

watch out for

  • Don't skip the rule-out-the-data step to get to a story. A confident "the product is failing" that's actually a missing day of data is the most expensive mistake on this page — verify the data before you blame the business.
  • Correlation isn't cause. "paid_ads dropped the same week we changed pricing" is a hypothesis, not a finding — name the one check that would confirm it, and treat the conclusion as provisional until you've run it.
  • Claude can localize the move and rank the explanations; the *decision* about what's really going on is yours. Pressure-test its top hypothesis against what you know happened that week before you repeat it as fact.

you'll end up with A defensible answer to "why did it move" — sized against a baseline, cleared of data artifacts, localized to the segment that actually moved, and backed by the one check that confirms it — instead of a guess that falls apart under one question.