We asked AI to write a database migration. It passed every test. It would have silently dropped six months of customer data.

Problem

We used an AI assistant to draft a “routine” schema migration: dropping a column that looked unused. It compiled, passed CI, and reviewed clean. Nobody caught that the column was still read by a nightly export job outside the main codebase.

Why it happens

• AI tools are confident about the code they can see and blind to the code (and jobs, cron scripts, other repos) they can’t
• A migration that looks self-contained often isn’t: the risk lives in what references the schema, not the schema itself
• Reviewers trust AI-authored diffs more than human ones because they look tidy, so they skim instead of tracing usages
• “It passed the tests” gets treated as proof of safety, when the tests were written by the same blind spot

Better approach

• Treat AI-generated migrations like a junior engineer’s first PR to a critical system: assume it’s missing context, not that it has all of it
• Grep the entire org’s codebase (and cron/job configs) for the column or table name before merging, not just the service being changed
• Reserve AI for drafting the migration syntax, not for deciding what’s safe to remove: that call needs a human who knows what else touches the data
• Add a staging step that runs against a production-shaped dataset with all dependent jobs active, not just the service’s own test suite

Example

The column legacy_region looked dead inside the API service. It was the only field a finance export script used to reconcile tax reporting. The AI had no way to see that script; it lived in a separate ops repo it was never given access to.