Disaster recovery trial by fire... literally
- — 22 February, 2011 02:44
On a Sunday morning last year, John Brooks received news no one wants to hear. There'd been an electrical fire in the basement of a New York City office tower - where his law firm has an office.
The firm's offices are on a higher floor and fortunately weren't directly damaged by the fire. But the entire building was locked down due to the fire, which was caused by the two main electrical feeds to the building. That meant no data links, no power to the IT infrastructure, and no opportunity to enter the building to retrieve the firm's servers, which contain documents and SQL databases for both of its New York offices.
Within an hour of hearing the news, the IT team and the firm’s management personnel were on the phone, planning a course of action, recalls Brooks, who is manager of network services at Marshall, Dennehey, Warner, Coleman & Goggin.
"In a matter of about two hours after our team meeting, we had the data restored,” Brooks says. “We restored a SQL database and the file repository. After our programmers created a custom application, the data was accessible to people in the office.”
It was a trial by fire - literally - of the Philadelphia firm's updated backup and recovery systems.
Marshall Dennehey had replaced its old tape-based backup systems with EVault Software, a disk-to-disk backup and disaster recovery product from i365 (a Seagate company).
I365's EVault software is deployed across the firm's 19 offices and protects more than 3TB of data. Agents on each remote server perform nightly backups of files and SQL databases, using deduplication technology to ensure that only new files and those that have been modified since the last backup are duplicated. The changes are compressed and encrypted, then sent to the main vault at the firm's Philadelphia headquarters.
As an added precaution, Marshall Dennehey also replicates the aggregate data to its suburban Philadelphia location using storage arrays and built-in replication technology. (See also: Disk or tape? How about both)
When the fire occurred, the firm restored the data using its primary vault in Philadelphia and made it available to attorneys and staff from a backup server until the IT team could get into the New York office and retrieve their servers. It was two weeks before the New York employees could resume working in the building, but work interruption was minimal since they could work remotely with full access to their digital workspaces and files.
When Marshall Dennehey first deployed the EVault software, the firm tested its new data backup technology, which made recovering from the fire a less daunting task.
"We had tested the recovery of our SQL databases and our Exchange databases, and we fully documented everything," Brooks says. "When the fire happened, we were ahead of the game. We knew exactly what to do, and that's why we were able to recover so quickly."
He might not have been so confident if the old tape-based backup system were still in place. It was a cumbersome, error-prone process, Brooks recalls. "We had local office personnel changing tapes on a daily basis, and it was easy to miss a backup," he says. There was also the risk of tapes getting lost en route from the remote offices to Marshall Dennehey's headquarters.
Using i365's disk-to-disk backup system, local personnel are freed from having to manually change tapes. Plus the firm is now able to perform backups seven days a week instead of Monday through Friday. "It's centrally managed now. There's no user intervention required whatsoever," Brooks says.
Recovery time is vastly improved, too. "I can recover a file in minutes whereas before it would have taken a minimum of four to eight hours by the time the tapes were sent out, we put them in the drive, indexed the information, and recovered the data," Brooks says. "Recovery is much faster, and it can be done remotely, which is the beauty of it."
When Marshall Dennehey decided to upgrade its backup and recovery infrastructure, it considered both disk-to-disk backup technologies and hosted backup services. A move to cloud-based backup services isn't in the cards anytime soon. "The concern for us is the retrieval of our data," Brooks says. "If you discontinue the cloud service, where does your data go?
Another potential roadblock with cloud-based backup is the increase in costs as data volumes grow. "Our data growth is increasing exponentially on a yearly basis," Brooks says, which means the volume of hosted data and its associated cost could continue to climb. "The costs can be ever-increasing, and we've taken that into consideration, too."
Read more about data center in Network World's Data Center section.