Cyberterrorism: Network lessons from Ground Zero
Published: 28 Aug 2002 15:58 BST
As architects submit proposals for rebuilding the World Trade Center, teams of engineers are working deep below the streets of Manhattan to construct a project of their own -- one designed to keep the city connected to the rest of the world if disaster strikes again.
In a subterranean labyrinth of ageing pipes and bundled wires stretching for miles in every direction, these engineers are trying to make this city's densely packed communications networks less susceptible to the kind of widespread outages caused by the 11 September attacks.
What they have found, however, is something many had feared all along: New York's concentration of key network interchanges in one place makes its communications exponentially more vulnerable, yet they have no choice but to continue working with it.
"The benefits of connecting all these networks together are so huge that decoupling them would be unrealistic," said Anthony Townsend, an associate professor at New York University's Taub Urban Research Center. As a result, he added: "Failures in one type of infrastructure system are likely to cascade into others because information technology and telecommunications have been linked within individual networks and between different networks like power, telecommunications and transportation."
This unsettling realisation is forcing businesses to re-evaluate how to structure computer networks, data backup centres and links to the Internet. Companies, agencies and all manner of organisations are working on plans to connect their staffs to networks remotely in an emergency, all the while knowing that a single well-placed explosion could render their efforts futile.
If New York's situation seems impossible, that city is hardly alone. The concentration of networks was the product of a century of growth, an evolutionary process that every major metropolis has experienced. As a result, governments and businesses worldwide are observing the city's progress as a case study for ways to reinforce their own systems.
Diversification is key
Manhattan is an island crammed with many of the world's most powerful industries, all of which rely on digital networks to keep their global businesses running, but its systems are representative of major communications hubs everywhere. Last year's catastrophe exposed just how fragile all communications networks are, from the digital hubs that run American's cities right down to the most basic functions like using a telephone.
Wall Street firms of all sizes were shocked to discover that their well-laid plans for diversified networks and backup systems were little more than theories put to paper. Moreover, following the consolidation of the telecommunications industry, even phone companies did not have such critical information as the exact route of their networks.
The Bank of New York was one of many to learn of all this the hard way. The institution -- more like a bank for banks -- had multiple backup systems, dual-access in and out of each of its Manhattan buildings, and a resilient state-of-the-art network design known as "ring architecture." It still had a total communications breakdown.
"We had all the redundancies, only to find out that several central offices in the current configuration nationwide were connected to each other," said Donald Monks, senior vice president of the Bank of New York. "As a result of these dual connections, it's not buying you any redundancy from the position of failure of the senior central office. These problems were exposed by 9/11, and in some cases we think they still exist."
Now, major players on Wall Street are demanding that telecommunications carriers guarantee in writing that the network route is well diversified -- not just at the central office but throughout the whole infrastructure. Monks suggested that companies pay a fee for such documentation to bind it as a contractual agreement.
Faced with an exodus of companies that could erode its tax base, New York officials are reviewing various diversification proposals to reassure businesses that even if one network is destroyed, alternate systems can pick up the slack. One idea is to convert abandoned underground water pipes into fibre conduits.
"These huge pipes would be an excellent conduit for making the city's landline telecommunications infrastructure more robust," said Agostino Cangemi, the city of New York's deputy commissioner for technology and telecommunications. The cast-iron, high-pressure pipes run up and down every street below 34th Street and in Brooklyn.
The city also plans to harness its 20,000 light poles and municipal rooftops for wireless data and cellular communications. The previous mistake was to concentrate major antennas in one location at the World Trade Center: When the twin towers went down, so did all cellular communications in the area. Officials are already negotiating with Ricochet to resume its services from atop 3,000 Manhattan light poles.
Fibre all over the map
Cangemi said the city will also encourage the construction of so-called lateral fibre conduits. At present, shared fibre conduits run north and south under New York's avenues but not east and west along its streets.
"Each time a building on a side street needs a conduit, it costs about $200,000 to build a new one," Cangemi said. "This initiative is intended to help lower carrier costs associated with getting fibre into the building and to alleviate a reliance on copper connections to a single central office."
Some companies are turning to cutting-edge technologies to strengthen their businesses. The venerable New York Stock Exchange, which lost connections when falling debris severed its digital lines, is building a more resilient infrastructure for its Securities Industries Automation subsidiary.
That subsidiary, which provides data processing and communications services for the securities industry, is fortifying its infrastructure with dedicated fibre optic lines it can track. The goal is to create a metropolitan-area network that guarantees geographic diversity of its routes.
Instead of linking to just two of the system's data centres, member firms will connect to the metropolitan network initially at a minimum of four points -- two in Manhattan, one in neighbouring Westchester County and one in New Jersey. The route between these connection points will consist of interlocking fibre-optic rings; if one part is broken, traffic is routed around the damage. The NYSE hopes to have this infrastructure in place by the end of this year.
The exchange is also pushing an aggressive move to use the Internet because of its greater flexibility and self-healing properties, said Roger Burkhardt, the chief technology officer at the NYSE. "A large part of what we are investing in is a move to modern technologies. We started the move to a common IP-based infrastructure about two years ago so we would have much more flexibility," he said.
Others are investing in early-warning technologies such as embedded chips that could detect anything from leaks along waterways to structural weaknesses at a power facility. Such sensors could also sense explosives or unauthorised entries to a building.
Yet Rae Zimmerman, director of the Institute for Civil Infrastructure Systems at New York University, warns against information overload. Not only can this slow an operation's efficiency, but an elaborate security system can work against itself.
"My biggest problem is, how do you sort through all that information that comes from thousands of data points?" Zimmerman said. "False positives are a huge problem, sending the whole system out of whack. It could be a simple thing like a roach dropping in front of the sensor."
Technology as archeology
In many ways, engineers seeking to update the nation's communications infrastructure are doing battle with history. Most large US cities have grown by building new technologies upon older ones. Because of this haphazard construction, ultramodern digital lines often run precariously beside waterways, sewer pipes and aging telephone lines.
"If you look back to the evolution of the telegraph and telephone, they largely followed existing infrastructure networks like railroads -- and like the Interstate highway system followed the railroads," NYU's Townsend said. "These things don't come into being arbitrarily; they are laid down to reinforce existing economic links."
That industrial expediency can have serious consequences, as seen in last year's train derailment and fire in downtown Baltimore. The blaze brought down parts of WorldCom's UUNet network, interrupting Internet access along the Eastern corridor while causing scattered power outages.
Accidental damage to wires and cables by backhoes or other equipment during street work remains one of the most common reasons for floods, power outages and communication disruptions to this day. In addition, any efforts to remedy the situation are often exacerbated by rival companies unwilling or unable to share documentation that shows the exact location of all the wires.
"The level of competition and the tradition of secrecy as well as the fragmentation of the telco sector is a major reason no one has done a comprehensive inventory of telecommunications infrastructure before," Townsend said. "It continues to cause problems and scare people away from looking at it."
Not surprisingly, businesses are searching for ways to diversify their physical offices as well as their networks. But because communication lines are so heavily concentrated in Manhattan, they may need to set up auxiliary offices out of the area altogether to ensure their ability to use other networks and remain operational.
Credit Lyonnais, for example, thought it was safely out of reach on 11 September, having a branch on a different power grid in midtown Manhattan miles from its other office at Ground Zero. Still, the French bank's operations were disrupted.
"We always thought that we have two locations -- one in midtown and one downtown -- on separate power grids. We thought that was OK since we were mostly protecting against localised issues like fires or power outages," said George Levitt, the bank's chief information officer. "But after 9/11, you had to adjust your thinking: maybe one should be a little further away, outside of Manhattan."
The bank is setting up a backup centre in SchlumbergerSema Global Recovery Services' facilities about 30 miles from the city.
Others point to a more obvious reason to keep offices farther apart from one another: in any type of disaster, related to terrorism or not, staffs will inevitably face physical obstacles that have nothing to do with the company itself.
"You can have the most hardened systems with motor generators, UPS (uninterruptible power supply), all sorts of network redundancies. But what if you can't physically get into the building?" said Gino Menchini, New York City's commissioner of technology. "Do you have a facility outside that area which is prepared with adequate technology where you can send people to work? I think that kind of facility is a value to anybody, whether it is a government office or a corporation."
Click here for part one, "Cyberterrorism: The real risks".
Click here for part two, "Cyberterrorism: Security versus liberty".
Have your say instantly, and see what others have said. Go to the Security forum.
Let the editors know what you think in the Mailroom.
Did you find this article useful?
35 out of 73 people found this useful
- Share this article:
