The cheapest data quality check is a row count

Every data quality tool I've evaluated wants a config file, a scheduler, and a taxonomy of expectations. They're fine. But the check that has caught the most real incidents for me, across four jobs, is a table with five columns and a nightly job that fills it.

CREATE TABLE ops.row_counts (
  checked_at  timestamptz NOT NULL DEFAULT now(),
  table_name  text        NOT NULL,
  day         date        NOT NULL,
  src_rows    bigint,
  dst_rows    bigint,
  PRIMARY KEY (table_name, day, checked_at)
);

Once a night, for each table that matters, count rows per day in the source system and in the warehouse for the last seven days (not just yesterday — late data changes yesterday's answer), and insert a row. Then two alerts:

• dst_rows < src_rows * 0.99 for any day older than a day: the destination is missing something.
• Today's src_rows is more than three standard deviations from the trailing eight-week mean for the same weekday: the source changed, and it's worth a look before the warehouse faithfully copies it.

The weekday-matched baseline matters. Sunday is not Tuesday, and a check that doesn't know that will page you every Sunday until someone turns it off, which is the usual fate of alerts.

What it catches

A broken pipeline (zero). A load that ran twice (double). A partial load after a timeout (0.6×). A source that quietly stopped emitting (flat). An upstream change that started filtering rows nobody told you about (a step down that stays down). That's most of the incident list from any year I can remember.

What it doesn't

Wrong values in the right number of rows. Nulls where there weren't nulls. A column that changed meaning. That's what the expensive tools are for, and they earn their keep there. This is for the other eighty percent, and it costs one table, one cron entry, and an afternoon.