Sunday, April 6, 2025

Every Change Might Be Breaking

We originally had the “automatic minor version upgrade” option active at Amazon RDS.  This option simply does not work very well.  Sometimes, for no clear reason (and without notification), it would stop applying upgrades, and require manual updates to get moving again.  We mostly lived with it, and then we hit the worst case scenario: it did perform the upgrade, and then one of our scripts stopped working.

Not only that, it managed to break while I was on vacation.

(Obligatory xkcd about spacebar heating.)

Since then, we don’t use that option.  When I’m good and ready, I peruse the changelogs, then schedule the update to happen when I will be in the office to handle unexpected issues.

For their part, AWS recommends testing the app against the new version of the database before performing any upgrades.  This is implicitly a recommendation against using automatic minor upgrades, because there is no automated process to test the upgrade first.

One knows an analysis tool is looking at AWS with a security-first paradigm when it recommends switching the automatic upgrade option back on for the database.  It is technically correct that new releases MAY contain security fixes, but upgrading to them MAY cause an automated denial of service.  It is not a simple, inconsequential task.