The massive power failure that paralyzed the northeast region of North America two weeks ago was caused by an aging, under-financed power infrastructure. The deregulation of the power industry means that the system now is used to transport electricity long distances, a task for which it was not designed, and that leaves it vulnerable.
So much for the good news.
Good news? Yes, simple because any cause, no matter how fundamental and troublesome, is better than what people most feared when the power went out: The blackout was not caused by another terrorist attack.
The bad news, of course, is that terrorism and sabotage remain massive threats. In the wake of 9-11, some enterprises increased efforts to secure their data centers. Although the blackout wasn't caused by terrorists, it's a good time for organizations to step back and assess the ways they can improve the physical security of their data centers.
Which brings us to the opposite disaster recovery book-end of last week's article: protecting the data center from deliberate, human-generated events.
Without a doubt, the most important step is to impress on everybody -- from the president down to the kids in the mailroom -- that their jobs and physical safety rely on the security of the workplace.
Common sense should be the first line of defense: Things out of the ordinary must be reported, doors that should be locked must not be propped open, and security cards and keys must not be loaned. At the executive level, the CFO must heed a consultant's explanation that a guardhouse manned 24/7 is necessary, but building the guardhouse and manning it only 16 hours a day is a sham.
The following are some of the many steps enterprises can take while planning, building, and operating a data center.
- In general, buildings holding mission-critical equipment should be as nondescript as possible. A strong fence as far away from the building as possible minimizes the potential that people will hurl incendiaries at the structure. Make sure that the fence itself doesn't attract attention, however.
- Building designers should make sure the room intended for the mission-critical equipment doesn't abut any outside walls. Use materials that are sturdier and than the dry wall and sheet rock used in most building. For additional security, sensors that signal tampering attempts can be embedded in walls. Materials should be fired-rated for at least four hours -- as opposed to common, one-hour rated materials -- recommends Philip Jan Rothstein, FBCI, principal of a security consulting firm that bears his name. Installing separate air systems to serve vital areas is another security measure to consider.
Construct the data center of slab-to-slab concrete. In most construction, concrete walls end a foot or two below the ceiling and above the floor. Finishing the job -- creating a six-sided box instead of a four-sided one, as Rothstein puts it -- will make it more difficult for intruders to get in.
Alarm all windows. Air ducts should not be wide enough for a person to shimmy through. If the facility has wide air ducts, consider purchasing devices known as "man traps" or alarms to make it impossible to use the ducts for passage.
Build a guardhouse with a perpetually open link to security personnel in the building, and install a perimeter camera system with night vision capabilities. Make sure the system is digital. Software that automatically finds a specific object (e.g., a red van) is available. Moving to IP -- perhaps on an intranet -- will make the video available in many places simultaneously.
- Make sure the data center doesn't become a shortcut from, say, the mailroom to the marketing department. If that happens, a door will end up unlocked or propped open, and trouble will be invited in. Rothstein advocates a layered approach whereby employees with the most legitimate reasons for visiting the server room often are seated the closest.
In addition, the entry doors themselves should be constructed in a way that makes it difficult for a second person to rush in with an individual legitimately gaining entry.
- Employee security cards should be programmed to deny employees entry to departments to which they shouldn't have access. The system should alert security as well as deny entry to anyone who tries to enter unauthorized areas.
Security cards and other systems should keep track of when an individual belongs in the building. It is unlikely that a mailroom attendant would need to be in the building at 3 a.m. If there is an occasion when that person's presence is required at an odd hour, special steps can be granted, but the default setting should not allow him or her in after hours.
It is equally important that a company have a termination process. Anything that can be duplicated -- such as cards and keys -- should be changed. A key is easily copied, so simply getting it back is meaningless. Be aware that some low-level employees (e.g., mailroom attendants) often have more physically spread out duties than executives. Such employees may have a lot of information on passwords and other access facilitators. React accordingly when one of these employees quits or, particularly, is terminated.
- Keep vital pieces of equipment in the most secure areas. Store key spare parts in these areas as well. While its usually not feasible to keep an entire spare-parts inventory in the server room, an effort should be made to keep the most difficult to get or sensitive pieces in the safest possible environment.
- Biometric security-retinal scans, thumb prints, and other hight-tech measures can add to security, if used correctly. Vital servers and other devices can be put in cages in the data center. (This often is done in telco collocation facilities where competitors' equipment sits side by side.) If this is overkill, consider secure cabinets that will at least slow down an intruder.
- Some emergencies, such as hurricanes or toxic spills, will leave the the data center itself untouched, and the equipment operational but personnel without access. For circumstances like this, the capability to remotely run the operation is an important addition. Keep in mind, however, that the telecommunications linking the emergency site to the main equipment must be continuously secure.
These steps and others like them are almost all rooted in common sense. For them to be effective, however, management must adhere to them.
For a while it seemed September 11 was the harbinger for security to be a top priority. Two years later the big question is whether the procedures put in place are being followed. "What I've seen in some cases is that there is a flurry of activity and then it's faded fast. In others there has been an overreaction, way too much emphasis on some issues, [while] other have been ignored," Rothstein says. "I think it's impossible to generalize. Some [enterprises] have done a great job, some lousy."
Carl Weinschenk writes a weekly server hardware series for ServerWatch.