OK, so the phrase “caching was a bad choice” isn’t exactly something you expect to hear about software development. Generally, caching is a good thing – it improves performance by reducing calls to your database or external services, saving time and resources by not re-querying or re-computing. What’s not to love? Well, as with everything else in life, the devil’s in the details, and if you don’t get those right then anything can become a failure.
I was working on hotfix that turned hotter than I wanted during the deployment, all because we missed an important test case that left us scrambling to resolve an issue during the deployment that we should have caught earlier. We got lucky and fixed it during the deployment, but we shouldn’t have been in that position because the scenario that failed was a known requirement, which means we should have tested it. So what happened? Well, point blank, we forgot a test case.
We all screw up, it’s part of life. What’s important that we learn from it all. I’ve found it’s generally better to document the learning, both to help internalize it, and to share it with who I can. I recently worked an issue stemming from automated API call retries that didn’t go nearly as well as it probably should have, and I wanted to take a little time to post-mortem on how everything went, and how it could have gone better. A lot of this is fairly specific to what I was doing with that particular problem, but there’s hopefully some general lessons that can be drawn from the whole ordeal.