Did Your Disaster Recovery and Business Continuity Plan Include a Pandemic?... Neither Did Mine.
Pandemic? Nope, the thought had not even crossed my mind. As an “IT guy”, I have designed, reviewed, and consulted on Disaster Recovery and Business Continuity (DRBC) plans for several companies, and never once did I see the word “pandemic” ever mentioned or thought to include it myself. It’s way overused these days, but “unprecedented” is the right word. An unprecedented global pandemic that, for many of us, means that we must rapidly convert our entire workforce to working from home for an undetermined amount of time. How do you even square that in your head? Well, many of us had to do it and in a very rapid fashion. This is how it took shape for BizStream.

Backstory

BizStream’s DRBC plan had been thought through. I had thought about what would happen if I lost a server, lost more than one server, the building had a fire, or our internet had been cut. What I had not thought about was being told your whole company can not come to the office and not only that, they can’t meet at an alternative location. In fact, by government fiat, they must remain at home for an undetermined amount of time. That doesn’t fit any of the models I had laid out at all. The closest scenario for me was “the building just burned down,” and there was nothing left. In that case, we were to set up a remote command center at a local shared office space, regroup, and start building/restoring our data into the cloud. The thought momentarily crossed my mind to start building out our cloud, but the time, effort, and expense that would have incurred were not what we needed. We still had access, and though we had to leave the office, the servers did not. 

Thankfully I was able to start planning for this a few days in advance. Here in Michigan, we had heard rumors about a “stay at home” order on a Thursday. I was thinking that this was maybe at best a 20% possibility on at that time. By Sunday I was thinking it was 60-70%. While I didn’t think it would last long, it was then that I really started thinking about what this would look like. The governor gave the order on Tuesday to stay at home for the next 21 days.

Groundwork

Luckily, BizStream had three big things going for us in this.
  1. We already had a Work From Home (WFH) policy in place. We allowed our team members to WFH on Tuesdays and Thursdays as long as they didn’t have any meetings where they needed to be in the office, a client visit, for instance. This policy was put in place to allow our team members flexibility and to add additional work-life balance. It had been put in place several years back and was fairly loose but we had given some thought as to how people would be able to connect, access business functions, and network data. 
  2. Most of our team members were already using laptops. We had made the decision many years back that we would move to laptops with docking stations as our primary option. This allowed us to have the multi-screen setup you would get with a desktop but with the flexibility to grab your laptop and walk into a meeting without needing a second device. There have been some bumps on that road with finding the right docks and living on the cutting edge of that technology, driving 3 monitors plus your laptop screen, but that is a conversation for a different time. 
  3. We had already purchased a Firewall/VPN that would allow for our entire team to be remote simultaneously. This was not because I had thought of this eventuality, it was because even though we are small (about 30 employees) we needed some more enterprise-type features/functions for some of the work we do.

Hurdles

You could look at all of that and say, “well, what more could you need? Your team all have laptops, you have a policy already in place, and your VPN is up to the challenge.” All of that is true, but it had never been tested to this “extreme.” Not only had it never been tested this way it had never been tested for this duration. Many of our team members (myself included) rarely, if ever worked from home. We have a really cool office, great sit-stand desks, lots of monitors, coffee, a fire pole, a slide, and really cool people. So for many of us, there was no reason not to come to the office. Personally, I always felt I could do my best work for the people around me when I could just walk up and help them so I didn’t work from home unless I had sick kids or I wasn’t feeling the best. This means that not everyone had a “home set up,” and I don’t think any of our team was really fully prepared to WFH for three weeks plus.

Gear Up

Our leadership team was ahead of the curve and had told everyone at our Monday company meeting that if they didn’t feel comfortable with being around others, they were welcome to work from home. We temporarily suspended the Tuesday/Thursday WFH policy limitation. We also told them if they needed something for home, (keyboard, mouse, extra monitor, office chair) to borrow it from the office and make their setup at home as nice as it could be. When the order came down from the governor on Tuesday this was reiterated to the team as we knew that it would be for at least 21 days (this has since been extended). We put a lot of time and effort into making our office comfortable and we feel this, in turn, makes our team more efficient. We decided this time is already stressful, we might as well make working remotely as reasonably comfortable as possible.

Freakout

Now, to say I was stressed out would be an understatement. I knew that there were going to be issues, I just didn’t know what they were yet. The first week I watched every metric I could consume like a hawk. What did the network look like, was the firewall CPU ok, was the stress of the VPN too much, could people access the internal file shares, was the threat monitoring going to freak out because of the large spike in traffic, and about a thousand other things? My stomach churned, I didn’t eat much, and I was fixated on the smallest oddity in the data.

Watching all of these things also came with their own set of challenges. My home office was really not much more than my laptop and a monitor. At the office, I normally have my laptop, three monitors, and several large TVs with dashboards running on them. I took advantage of our call to “gear up” and took extra monitors home and repurposed an old TV, which I attached an older laptop to try and give me some broad rotating metrics.

Making it work

After the initial freakout was over, everyone was working. It was bumpy, but it worked. Here are a few things we did to streamline our remote work.
  • We already were using Slack but we encouraged everyone to really use and update their statuses. This helps when you walk away to get coffee or lunch or are done for the day. Your computer could still be on and you look like you're still working but you don’t respond. It just helps keep communication flowing.
  • We strongly encouraged using video conferencing (Slack, Google Hangouts, Zoom, etc.). Typing works if it's just a quick question but if you are hammering out messages in Slack for 10 minutes trying to explain a problem, just get on a call. Talking it out is usually more efficient and screen sharing helps visually show the issue that you are troubleshooting. It also helps to ensure there is nothing lost in translation when you can see and hear the other person as opposed to trying to read between the lines.
  • I found the use of “Jump-boxes” invaluable. For those of you who are unaware, a “jump-box” is a node (in our case Windows 10 machines) that sits on your network that teams can RDP into. This helps as there are times when you have to be on our network to access certain client assets or even some of our internal assets.
  • Weekly virtual happy hours help our team stay connected. We really are like a family and do like hanging out with each other so we implemented a standing virtual happy hour on Fridays. We just kick back on a Zoom call and laugh and talk about whatever. It's great fun and is great for team building when you can’t be together.
  • Keeping up our weekly company meeting. To me this was key to keeping everyone informed on what was/is happening within the company. Just like in our normal company meeting we encourage people to jump in and ask questions. If you have a question, surely others have it as well. It helps keep the communication flowing in both directions.

Not a Hero

All in all, the transition to working fully remote, though unplanned, went fairly well for BizStream. I have friends whose transition was not nearly as smooth, and they are still on rocky footing. Our planning, willingness to be agile, and some good old fashioned luck allowed our team to rotate out to 100% remote in under 48 hours. But I am not, however, the hero here. The true heroes are our medical teams that are working tirelessly to help the sick and find a solution to this mess. They heard the call and stepped into the fray without hesitation, and for that, I am truly grateful. If you happen to see or know someone on the frontlines, remember to tell them thanks. We will get through this, and we will be stronger on the other side. Stay safe.

Share This Post:

Twitter Pinterest Facebook Google+
Click here to read more Advice posts
Start a Project with Us
Photo of the author, Blair Compston

About the author

Blair has been playing with computers as long as he can remember. He has worked in companies both big and small and in every position from helpdesk through manager, and now he brings his knowledge of all things IT to BizStream. As our IT and DevOps guy, he keeps our servers running, networks in line, and clouds fluffy. Outside of work, this family guy can be found spending some QT with his wife and four kids or tackling yet another home renovation project.

View other posts by Blair

Subscribe to Updates

Stay up to date on what BizStream is doing and keep in the loop on the latest with Kentico.