Scenario: Backing Up Core Systems
Tim Barbee, CIO, North Central Texas Council of Governments
Earlier this year, I initiated a project to develop a business-impact analysis and update me on the disaster-recovery requirements for each of the agency's departments. I did not like what I saw. Based on the sheer number of affected systems, we were looking at a price tag north of $1 million. As you can glean from headlines across the country, state government agencies are in a budget crisis and we're not in a position to request funds for disaster-recovery capabilities that we hope we never have to use. Even so, it is a critical service we must provide.
One of the big-ticket items in a disaster-recovery plan that meets our requirements is a backup data center, but the budget is not going to allow that. I am interested in creative, cost-effective ways to provision similar capabilities. For example, Microsoft's cloud business solution, Office 365, includes a hosted email platform we can access from the cloud. Theoretically, all we would need in a disaster would be a laptop and an Internet connection. The service includes data backup and recovery at no additional cost. So what other disaster-recovery approaches exist for things like email, telecommunications and core systems?
Advice: Allocate your backup workspace like TV time slots
Marty Gomberg, SVP and CIO, A&E Television Networks
The idea was simple. My end users need instant access to critical systems, but that does not mean they need access for the full eight-hour workday during a disaster. Rather, key personnel could complete their critical tasks in an assigned block of time. We took the number of seats in a conference room and scheduled each in one-hour blocks on a 24-hour schedule. We then asked the business units to assign slots to key personnel. For example, accounts payable could request three seats for three hours each to complete their priority tasks. These are all scheduled and prioritized.We applied the exact same time-allocation approach to our virtual services. For example, we bought a limited number of Citrix licenses to assign to virtual seats where remote staff could perform critical functions.
Through this combination of rotating physical and virtual access, I have been able to support hundreds of people doing their most critical activities. The model works well to reduce the cost of crisis operations. The advance planning it requires helps to identify which tasks are most important to continue in a crisis and which you can simply put on hold.
Advice Use All Your Resources, Not Just the Obvious Ones
Paul T. Cottey, CIO, Accretive Health
When creating contingency plans for a disaster of any type, it is easy to forget, in the midst of your worry about all your critical, multimillion-dollar systems, that your people are the most important (and least costly) resource to protect. We realized that our ability to contact key staff in the event of a disaster was dependent upon the very systems that could fail. For example, we might not be able to look up contact information in our email system. That's why we maintain a hard-copy list of personal email addresses and phone numbers for critical IT support staff. Even if technology fails, I still have a low-tech way to reach my guys.
As for the data center itself, we cast our lot with a tier-one provider -- and with their other customers. While we provision the backup servers for our needs, they provide the generators, diesel fuel, cold air and network-management resources. In a sense, our arrangement is a reservation for data center space that would likely be hard to come by in the event of a disaster. Cost is a factor, of course, and led us to ask key questions about disaster recovery when we were selecting our data center provider. Even though we are paying a relatively low rate, we get premium disaster-recovery capabilities because we are willing to share the space with larger companies running their critical systems. In our search, we specifically inquired about other clients and asked to share space with those who needed even higher recovery standards than ours. Sometimes, it pays to be the smaller fish in a big pond.
Barbee, Gomberg and Cottey are all members of the CIO Executive Council, a global peer advisory service and professional association of more than 500 CIOs, founded by CIO's publisher. To learn more, visit council.cio.com.