How is a quality audit different from profiling a dataset?

Profiling answers "what's in this file?" — a quick orientation before you analyze. A quality audit answers "is this data fit to feed a decision?" — a systematic test of completeness, validity, uniqueness, referential integrity, freshness, and drift against stated rules, ending in a go/no-go verdict. You profile every new file; you audit when the cost of a wrong number is high enough that "it looked fine" won't survive a challenge.

When is it worth running a full audit instead of a quick check?

When the data is about to feed something with real stakes — a board figure, a recurring report whose errors compound, a model, an external deliverable. The stakes set the bar: a rough internal exploration doesn't need this, but anything a stakeholder will act on or that an auditor might question does. The rule of thumb is the cost of being wrong — high cost earns the forty minutes.

Will Claude fix the data quality problems it finds?

No, and it shouldn't without your say-so. The audit's job is to surface problems with severities and examples; fixing them — deduping a key, dropping out-of-range rows, resolving a contradiction — changes the data and is a decision an owner makes on the record. Claude flags and proposes; a human decides which fixes to apply, so the scorecard never goes green because something was quietly altered.

Does passing the audit mean the numbers are correct?

No — it means the data obeys the rules you set, which is necessary but not sufficient. The audit can't tell you the rules capture what the business actually means, or that a well-formatted figure reflects reality. It catches the avoidable failures — missing rows, broken keys, stale loads — so they don't produce a confident wrong answer. You still reconcile results against known totals; the audit just removes one whole class of input errors first.

Audit a dataset's quality before you trust it

when to reach for this

A dataset is about to feed something that matters — a board number, a recurring report, a model, a stakeholder dashboard — and "it looks fine" is not a standard. Bad data never announces itself; it quietly produces a confident wrong answer that sails through review because nobody checked the inputs, only the output. A quality audit is the systematic version of that check: completeness, validity, uniqueness, consistency, referential integrity, freshness, and drift, each tested against a stated rule, producing a fit-to-use verdict with severities. It's a heavier instrument than a quick profile — you reach for it when the cost of a wrong number is high enough that "I eyeballed it" won't hold up.

gather this first

The dataset as it'll be used — the actual file or table that's about to feed the report or model, not a sample, since the issues you care about often live in the rows you didn't spot-check.
What it's about to feed, because the stakes set the bar — data good enough for a rough internal look is not good enough for a board figure or a model in production.
The rules the data should obey: which column is the unique key, what ranges and allowed values are valid, what a foreign key should reference, how fresh it must be. And the prior version if you want to check drift.

the workflow

Define what "good" means for this data — the rules

Open the folder with the dataset (and last load, if you have it) in Claude Desktop and ask in the chat — no terminal needed. Before testing anything, write down the expectations. An audit with no stated rules is just another profile; the rules are what turn "looks fine" into pass/fail.

you ask

I'm about to use this dataset to [feed the monthly board report]. Before checking anything, help me write the quality rules it must pass: which column is the unique key, which columns can never be null, the valid ranges and allowed values per column, any foreign key that must reference another table, and how fresh the data needs to be. Propose sensible rules from what you see, and flag any you're guessing so I can confirm them.

what you get back A written rule set — "order_id unique and never null; amount >= 0; status in {paid, pending, refunded}; customer_id must exist in customers; data must cover through yesterday" — with the guessed rules flagged for you to ratify. Now there's a bar to test against.

The rules are the audit. Without them you're back to eyeballing; with them, every check has a clear pass or fail a reviewer can trust.
Check completeness and validity

Start with the two cheapest, highest-yield dimensions: is anything missing, and is anything out of bounds. These catch the import that half-loaded and the column that holds a value it shouldn't.

you ask

Run the completeness and validity checks against the rules. For each column: the null count and percent (flag any that breaks a 'never null' rule); the type (flag values that don't match, like text in a numeric column); and any value outside the allowed range or set (negative amounts, statuses not in the allowed list, dates outside the expected window). Give me a table of every failure with a count and a few example rows.

what you get back A failures table: "amount: 31 negative values (refunds? confirm); status: 12 rows = 'cancelled', not in allowed set; signup_date: 4 rows in 1970 (epoch-zero bug). region null on 18% — allowed." Each failure with a count and examples so you can judge it, not just see it.
Check uniqueness, consistency, and referential integrity

Now the structural checks — the ones that silently break joins and double totals. Duplicate keys, foreign keys pointing at nothing, and fields that contradict each other within a row.

you ask

Check the structure: are there duplicate values in the key column (order_id), and any fully duplicate rows? Does every customer_id in this file exist in customers.csv — list the orphans? And check intra-row consistency: rows where the logic contradicts itself (status = 'refunded' but amount > 0, end_date before start_date, total that doesn't equal the sum of its parts). Count and show examples for each.

what you get back "47 duplicate order_ids (retries — dedupe or keep?); 130 customer_ids with no matching customer (orphans); 9 rows where status='refunded' but amount is positive (contradiction)." The structural problems that would have quietly corrupted any join or total, surfaced before they did.

Duplicate keys and orphan references are the errors that produce a clean-looking-but-wrong result downstream. They're exactly what a quick profile misses and a report inherits.
Check freshness and drift against the last load

A dataset can pass every static rule and still be wrong if it's stale or has silently changed shape. Compare against the prior version to catch the schema change, the volume cliff, or the distribution shift nobody announced.

you ask

Compare this load against last month's file. Is it fresh enough — does it cover through the date I expect, with no missing recent days? Did the row count change more than you'd expect? Did any column appear, disappear, or change type? And did any key distribution shift sharply — a category that was 5% of rows now 40%, a null rate that jumped? Flag anything that suggests an upstream change rather than a real business change.

what you get back "Fresh through yesterday. Row count +12%, plausible. But new column experiment_id appeared, and source is null on 40% of rows vs 12% last month — likely a tracking change upstream, not a business shift." The silent pipeline change caught before it's read as a trend.
Produce the fit-to-use scorecard and a go/no-go

Pull every finding into one scorecard, rank by severity, and force the verdict: can this data feed what it's about to feed, as-is, with fixes, or not yet? A list of issues isn't a decision — the verdict is.

you ask

Assemble a one-page data-quality scorecard: every check, pass or fail, with a severity (blocker / warning / note) and the count. Then give me a clear verdict for feeding [the board report]: go, go-with-fixes (list exactly what to fix first), or no-go (and why). Separate the blockers that must be fixed from the notes that are fine to ship with. Save it as data-quality-report.md.

what you get back A data-quality-report.md with a ranked scorecard and an explicit verdict: "GO WITH FIXES — blockers: dedupe 47 order_ids, resolve 9 refund contradictions. Warnings: orphan customers (expected, deleted accounts). Then safe for the board report." A decision you can defend, not a pile of findings.

The severity split is the point. Not every null is a blocker — forcing each finding into blocker / warning / note is what keeps the audit from either crying wolf or waving real problems through.

make it your own

Gate a pipeline (Power Track): once the rules are stable, hand them to a scheduled agent or a /audit command (see the Features tab) that runs the audit on every new load and pings you only when a blocker appears — so the recurring report never silently runs on broken data.
Right before a join: run this on each input before Turn two messy files into one trustworthy dataset — a join inherits every quality problem in its inputs, so auditing first means the orphans and dupes are known, not discovered in the combined file.
Reuse the rules as a contract: the rule set you wrote in step one is a reusable data contract — save it beside the dataset so the same expectations are checked every time, by anyone, and 'is this data good?' has a documented answer.
Bilingual hand-off: if the data owner reads Arabic, write the scorecard's verdict and blockers in Arabic so the people who must fix the upstream issue understand exactly what failed — author it in Arabic rather than handing them a translated checklist.

watch out for

An audit surfaces problems; it must never silently fix them. Deduping a key or dropping out-of-range rows changes the data — that's a decision an owner makes on the record, not something Claude does quietly to make the scorecard green. Flag, then let a human decide the fix.
Severity is everything, or the audit cries wolf. An 18%-null region might be totally expected while a single duplicated primary key is a blocker. Forcing each finding into blocker / warning / note is what makes the verdict trustworthy instead of a wall of red.
Fresh and clean is not the same as correct. The audit checks the data obeys its rules — it can't tell you the rules capture what the business actually means, or that a perfectly-formatted number reflects reality. It's a necessary gate, not a guarantee of truth.
Quality rules need an owner, like metric definitions do. A rule set nobody owns goes stale — the 'valid range' that was right last year quietly passes bad data this year. Keep the data contract beside the metric dictionary, owned and reviewed, not buried in one analyst's notebook.

you'll end up with A `data-quality-report.md` — every quality dimension checked against a stated rule, findings ranked by severity, and an explicit go / go-with-fixes / no-go verdict — so a dataset earns its way into a report or a decision on evidence, and the blocker that would have produced a confident wrong number is caught at the input, not the output.

Questions people ask

How is a quality audit different from profiling a dataset?: Profiling answers "what's in this file?" — a quick orientation before you analyze. A quality audit answers "is this data fit to feed a decision?" — a systematic test of completeness, validity, uniqueness, referential integrity, freshness, and drift against stated rules, ending in a go/no-go verdict. You profile every new file; you audit when the cost of a wrong number is high enough that "it looked fine" won't survive a challenge.
When is it worth running a full audit instead of a quick check?: When the data is about to feed something with real stakes — a board figure, a recurring report whose errors compound, a model, an external deliverable. The stakes set the bar: a rough internal exploration doesn't need this, but anything a stakeholder will act on or that an auditor might question does. The rule of thumb is the cost of being wrong — high cost earns the forty minutes.
Will Claude fix the data quality problems it finds?: No, and it shouldn't without your say-so. The audit's job is to surface problems with severities and examples; fixing them — deduping a key, dropping out-of-range rows, resolving a contradiction — changes the data and is a decision an owner makes on the record. Claude flags and proposes; a human decides which fixes to apply, so the scorecard never goes green because something was quietly altered.
Does passing the audit mean the numbers are correct?: No — it means the data obeys the rules you set, which is necessary but not sufficient. The audit can't tell you the rules capture what the business actually means, or that a well-formatted figure reflects reality. It catches the avoidable failures — missing rows, broken keys, stale loads — so they don't produce a confident wrong answer. You still reconcile results against known totals; the audit just removes one whole class of input errors first.