This should be a fun post.  I’m pretty proud of my server room and the infrastructure I’ve built.  Having worked in the I.T. industry specializing in enterprise compute, storage, virtualization etc, I felt my own setup is pretty resilient.  But then disaster struck.  On Friday last week, we had a power outage.  No problem, everything is on UPS backup batteries and good for a half hour.  I got notified in the middle of the night that I needed to shutdown some key systems to protect data.  Well, that’s where the problem came in.  I’m not gonna lie, I was out getting piss drunk and didn’t know my ass from my elbow.  The power outage was an extended one and a number of systems went down hard after their UPS ran out of juice.

Typically this isn’t an issue however when the systems came back up, my vSAN was corrupt and I lost a significant number of virtual machines.  Luckily, I always keep backups so I had to spend the entire weekend and the better part of this week to recover everything.  Happily I haven’t lost any data because of good backups.  It just takes a lot of time to recover (considering I house over 100 TB).  Anyway, I’m almost done rebuilding and should get back to the 2.0 plugin release soon.