Of course better checks will be put in place. Will we ever eliminate whoopsies? No.
You know that I work for a large, open source vendor, and I met the other day with a large customer of ours that has an IT budget in the billions (yes, billions), of which a reasonable percentage (mid-30s) is spent purely on high-availability and disaster recovery, to eliminate outages. They have a better than six 9s uptime requirement. The reason we were meeting? They’d had an outage; human error, of course. So if they can spend hundreds of millions of dollars on avoiding errors and still have a whoopsie, then what hope does SOTA have?
Now I don’t think you’d have been so bold as to assert this, but please don’t assume because we occasionally cock up, we’re not trying to avoid whoopsies whenever we do things.
I’m a big fan of giving SOTA participants a detailed description of what went wrong when RBNHole isn’t working, even if that makes me occasionally look a little dense or not farsighted enough. But, I’ve always believed in the principle, essentially enshrined in the culture of my company, that I am not the smartest person in the room. We learn from our mistakes, and we get better; and the difference in the IT systems between when I started in SOTA, from when I joined the MT, to now is immense: it’s just not all visible, because, if it’s visible, we’ve generally made a mistake.
At the recent SOTA MT meeting, we took further steps to get the IT systems of the SOTA program in better shape: more of that in a later reflector post once we’ve tidied up the message and agreed on the milestones. Continuous improvement; kaizen; it’s all happening. But we’ll still cock up occasionally