Projects

Side projects, in the sense of things I built because I was annoyed. The statuses are honest: maintained means I still run it and fix it when it breaks; archived means it worked for me once and probably would again.

• pgshift maintained

  Diffs two Postgres schemas — a live database, a dump, or a directory of migrations — and writes the SQL to get from one to the other in dependency order. It exists because every tool I tried either emitted statements in an order that failed on foreign keys, or refused to generate DROP statements at all. It refuses too, by default, but you can ask nicely with a flag. About three thousand lines, a single binary, no daemon.

  gopostgressince 2021

• topictail maintained

  tail -f for Kafka topics. Decodes Avro and JSON against a schema registry, filters messages with a small expression language (.customer_id == 42 and .amount < 0), and prints one per line so you can pipe into jq or grep. Started as a sixty-line script in 2020 and is now the thing I run before I read any consumer's code, because the topic is usually not what the consumer thinks it is.

  pythonkafkaavrosince 2020

• dbt-freshness-exporter archived

  Reads the sources.json that dbt source freshness writes and exposes each source's max-loaded-at age as a Prometheus gauge, so "this source hasn't updated in six hours" can be an alert like any other, in the same tooling as everything else. Archived because warehouse metadata and dbt's own artifacts have since made this trivial without a sidecar process.

  pythondbtprometheus2021–2024

• dagcheck archived

  A linter for Airflow DAG files. Flags database calls at module top level (they run on every scheduler parse), tasks without retries, cron expressions that don't mean what the comment next to them says, and ExternalTaskSensors pointed at DAGs with a different schedule. Ran as a pre-commit hook at two jobs. Archived once dag.test() and newer parser warnings covered most of what it caught.

  pythonairflow2020–2023

• canalwatch maintained

  A Raspberry Pi on a shelf polls a public water-level feed for the canal at the end of my street every ten minutes and writes the reading to a very small Postgres. A weekly job renders a chart to a PNG that I look at more often than I'd admit. Maintained loosely. It has taught me more about idempotent ingestion, clock drift and SD card failure than any job has.

  pythonpostgresraspberry pisince 2019

None of these is published. pgshift and topictail are still tangled up with a work config I haven't had the patience to strip out, and canalwatch is a Pi on a shelf. I've made peace with that.