This is an excellent point. I once had to deal with an external contract firm on a project that I was hired to fix. We had issues of production code breaking so badly that it brought down the entire server (N+1 query issue that triggered 50k queries).
The tests passed. When I emergency patched the issue and deployed it to production, the contract firm got mad at me for breaking their tests...to fix a production emergency.
It’s put me on guard again militant test ideologues with no concept of real priorities ever since.
We generally wouldn't allow that either. We've ran into cases where emergency fixes cause even more damage (e.g., the system is up, but now it's processing payments wrong), so you have to prove beyond a shadow of a doubt that the test failures are irrelevant or less bad than the current incident.
Often times it's less effort/more expedient to make the change pass tests (or update the tests) than convince all of the stakeholders that what you're about to do is safe, but the break-glass is there if needed.
Maybe you'd call this a militant test ideology, but I think it's perfectly reasonable. Systems are complex, and people can get tunnel vision during a bad outage.
The way the tests were written in that case, it was hard coded to how the work was being done and not the result produced. Both the code and the tests were bad.
Fair point. In this particular case the primary developer for them wanted to “publicly shame” me in Slack. Seemed much more ideology driven at the time.
> The tests passed. When I emergency patched the issue and deployed it to production, the contract firm got mad at me for breaking their tests...to fix a production emergency.
I get the point, but it depends on which tests failed.
Tests for unreleased features and trivial UX stuff is not the same as breaking a test making sure not every customer gets a 50% discount.
The tests passed. When I emergency patched the issue and deployed it to production, the contract firm got mad at me for breaking their tests...to fix a production emergency.
It’s put me on guard again militant test ideologues with no concept of real priorities ever since.