- Joined
- Feb 6, 2008
- Messages
- 38,938
- Reaction score
- 3,937
- Points
- 203
- Location
- Wolfsburg
- Preferred Pronouns
- Sire
Finding a bug that causes a permanent off-by-one error in billable metrics is not great for your mental health. I'm on the fast track to becoming paranoid now... :shifty: At least the code wasn't in production yet.
Had the same off by one once...luckily it wasn't billable. It only decided if the server park was available or was considering itself out of sessions. So, maybe it was billable in a way.
Problem: The software redesign I did for fixing a major race condition the previous developer left me, assumed that there is never a prolonged period in which all second tier servers in the pipeline are physically not available to the first tier servers that interact with the "users" (or better, connected systems).
What I did not include was, that operations still shut down the second tier servers every night for maintenance, but did not shut down the first tier servers. So the first tier servers still accepted sessions, tried to contact every other second tier server and rolled back all second-tier sessions in that attempt per request. Except that, when all servers in that process did not respond at all, one last zombie session was not deleted, because the time for the long connection attempts inside the first tier server exceeded the response-timeout and bypassed the code to clean up all session resources properly. :facepalm:
Sadly I am no longer responsible for fixing this bug and shutting down the second tier servers sequentially for maintenance was a cheaper solution...
