ON 18 July 2001, a freight train derailed in the Howard Street tunnel running beneath downtown Baltimore, spilling 20,000 litres of hydrochloric acid. destroyed fibre-optic cables owned by eight major US internet carriers. Moments later, Verizon Communications, which operates key portions of the internet’s physical infrastructure in the US, lost links to two operations buildings and several other carriers’ networks. For many hours, internet traffic slowed to a crawl across the entire country. “That tunnel is basically the I-95 [the main US East Coast highway] for fibre,” one repair contractor told reporters. “It was a once-in-a-lifetime place for vulnerability.”
Eight years on, and events have proved otherwise. A series of catastrophic failures seems to suggest that the internet is rather more vulnerable to accidents, earthquakes or misplaced ships’ anchors than people thought. At tens, perhaps hundreds, of places around the world, the net seems to be hanging by a thread.
These days a major failure has the potential to cause far greater disruption than in 2001. Yet much of the internet’s physical infrastructure is decades old. It badly needs upgrading, but clearly we can’t just tear up sections of the network and rebuild them from scratch. Nor is it likely that governments and telecoms companies will bear the enormous costs of laying extra connections simply to insure against temporary problems. So how can we make the net more resilient?
Advertisement
Nick McKeown, a computer scientist at Stanford University in California, thinks he has the answer. He believes the key to a better net lies with a prosaic black box called a router.
Routers are the internet’s traffic controllers. There are millions in service, linking up the thousands of networks that make up the internet. They can direct huge flows of traffic for internet service providers, or just provide connectivity between a handful of computers. They check the addresses on data packets, direct them to the right destination and dictate which physical path they take to get there. When a connection breaks, they play a crucial role in helping divert data around it.
At the moment, though, routers are part of the problem, not the solution. For one thing, they can be very slow to find a way around a blockage, and in the many minutes it often takes, traffic backs up into jams so huge that much of the data is simply discarded.
Though numerous potential solutions to these problems exist, the other big sticking point is that there is nowhere to test them. Any update of router software ought first to be thoroughly tested on a large network – one that has all the complexity of the internet but which is physically isolated from it. Yet nothing like that exists.
Even if you could test it, says McKeown, it is very difficult to actually install new router software. Each router is pre-programmed according to international standards set 10 or 15 years ago largely by the manufacturers themselves. They contain proprietary circuits, and the software controlling the way data packets are routed operates in set ways, allowing little means for change.
Now McKeown, along with Stanford colleague Guru Parulkar, is developing the means to solve all these problems at a stroke: a system that can alter a router’s control software on the fly as well as providing the perfect place to test it safely.
Smoother surfing
Named , their system is already running on Stanford University’s network, and the first commercial products should reach the market this year. OpenFlow won’t solve the problem of cable bottlenecks or prevent the odd accidental failure, but if the technology is adopted as McKeown hopes, it will enable the internet to adapt to changing loads, dynamically altering pathways to cope with spikes in traffic and giving every surfer a smoother ride, regardless of earthquakes, terrorists, ships’ anchors and so on. “We are trying to enable a network that continually evolves and improves,” says McKeown.
Anything that makes the internet more resilient should be good news, and not just for the millions of ordinary people who use it to book holidays or twitter to their friends. The financial impact of a net outage can be huge: online commerce is now worth over $7 trillion annually, representing about 12 per cent of global GDP. A 2005 study by researchers at the Swiss Federal Institute of Technology in Zurich calculated that cutting off all links to the internet would cost Switzerland over $3 billion per week – around 1 per cent of its GDP. And with e-commerce expected to account for 18 per cent of global GDP by 2010, the impact of failure is set to grow.
Aside from that, critical parts of our infrastructure, such as power and water utilities, now rely on the internet for information exchange and remote diagnostics. Banks and stock exchanges around the world swap financial data via the internet, as well as using their own networks. Transport systems, too, such as the German railway system, rely on it to link ticketing and information networks.
In fact, it might seem miraculous, given the internet’s growing traffic density, that outages have not caused more problems. This is mainly down to the fact that the internet is a scale-free network, which is another way of saying that while it depends on a few highly connected nodes, most have just a few connections. That means an outage in one area has a limited impact elsewhere and it doesn’t take much to adjust traffic flow to keep things moving.
Routers play a key role in making this happen. Normally a router checks the address on any data packet it receives and sends it on according to predefined rules held in a set of tables. Two sets of data going to the same address, say, are usually sent along the same path. If this path becomes impassable for whatever reason, the router checks in with its neighbours, finds out which still work, and calculates the best way to redirect data.
To do this, routers run a complex algorithm, but it can take many minutes to complete. Because of the problems with testing and updating new software, technical improvements have come at a glacially slow rate. Any change must be made very carefully, says Tom Anderson at the University of Washington in Seattle. “You have to make sure that you’re not doing something that will create problems of its own.”
That was highlighted in February, when a for a router in the Czech Republic spread across the web, causing traffic to slow to a crawl across the entire internet for over an hour. This is by no means the first time such a mistake has caused chaos (see map). Yet there is no large-scale testbed or “virtual internet” on which to experiment.
In 2005, the US National Science Foundation (NSF) asked a team of researchers to find a way around this problem. Their solution was bold: construct a huge new national network, with much the same complexity as the internet, on which to test and refine novel concepts until they are ready to be transferred to the real thing. As if that wasn’t ambitious enough, they also wanted to slice up the traffic on this network. The idea was that each slice could run on the same infrastructure of routers, switches and cables, but remain isolated from every other slice. That way thousands of researchers could experiment with different approaches all at the same time.
The NSF liked the idea enough to stump up over $10 million in start-up funds for what is now called the . The new nationwide grid will take many years, and over $100 million, to complete. Now, though, McKeown and his colleagues have come up with a plan that will not only allow GENI to be deployed much more quickly and cheaply, but the project will largely be able to use existing routers, switches and cables.
The key is OpenFlow. With the cooperation of the manufacturer, a small OpenFlow program can be added to almost any router, where it acts like a remote control for the proprietary algorithms and hardware inside. By creating an interface to the router’s flow table – the thing that specifies the rules for handling traffic – it allows someone to take control of the way the router directs traffic, to make new routeing decisions and implement them.
The upshot is that OpenFlow gives software engineers and developers the ability to create their own routes for data packets, by writing the algorithms on a regular computer and sending them via a secure connection to the router. By controlling the flow table, it becomes straightforward to partition a network into any number of slices, each isolated from the rest, on which researchers can test or refine their ideas. With a “multiverse” of virtual networks available for experiment, it should at last be possible to kick-start the evolution of the internet.
“With a multiverse of virtual networks, it should be possible to kick-start the evolution of the internet”
To speed up the process, McKeown and his team decided to make their system , meaning that the software is free for redistribution. This should help stimulate new ideas and help get them deployed more quickly, he says. “You get the benefit of sharing and building on top, creating a rapid rate of innovation,” McKeown says. “That has never happened in networking.”
OpenFlow is already providing internet testbeds on the Stanford University network, and the team plans to install it on half a dozen other university networks in the US in the near future. Their aim is to allow students to experiment and try out new ideas on virtual networks. With a number of manufacturers on board already, the team hopes to see OpenFlow-compatible commercial routers, internet switches or Wi-Fi access points reach the market this year.
The idea so impressed Chip Elliott, GENI’s project director, that GENI is now one of OpenFlow’s chief funders. It is a really good way to open up the network for experimentation and innovation, he says. The alternatives would be a lot more expensive and take longer to implement. “What I really like is that they based it all on fast, cheap, commercial hardware.”
Success, however, will depend on convincing the major manufacturers that the long-term advantages of OpenFlow are worth the short-term investment. To help persuade them, last year the team at Stanford put by installing a “virtual server” running a shoot-’em-up game on a computer on Stanford’s network. Thanks to router software written using OpenFlow, players equipped with laptops found that even though they moved between wireless access points all round the campus, the game play was seamless. “No one lost connections,” McKeown says. Then, while the game was still going on, the researchers moved the virtual server from Stanford to a machine in Japan. The game continued without any interruption. “You couldn’t even tell that it had moved. That’s the kind of thing you can’t do in the current network.”
Beyond this kind of smooth rerouting, OpenFlow should offer some important benefits for network operators. It could let them alter their router’s rules so that particular kinds of data are sent along particular routes – to give emails priority over music downloads, say, or spread traffic over a large number of alternative connections when one path is broken. “It allows you to add new capabilities, new features to the network, without having to program inside that proprietary box,” says McKeown. “In essence it is .”
Another key problem that OpenFlow should help address is network security. After the recent router error in the Czech Republic, a group of experts took a close look at the software running on routers and switches. They found that there are vulnerabilities in every version of every router manufacturer’s software, allowing hackers to hijack a router, say. In other words, the very fabric of the internet itself is at risk.
Initially, OpenFlow could actually make networks less secure since it offers a route for attack, admits McKeown. But that should change rapidly as engineers develop new, more secure versions of router code that can be tested on existing systems without slowing or interrupting traffic. Then, when an update is ready to be installed, the fix can be accomplished simply by programming in the new instructions, rather than taking the router off-line and reprogramming by hand.
One of the most important benefits of OpenFlow could arise from the ability to alter the way that data packets travel across the network. At the moment, emails between two particular computers always take the same path. Problems arise when any one connection on that path fails. So a number of research groups are exploring in Daejeon. This involves splitting a message into several packets and sending each one via a different route to its destination, where they can be recombined. Lee and others are convinced that by spreading traffic more evenly, this approach will increase the internet’s reliability and reduce congestion. There are several competing multipath schemes, and with OpenFlow they could all be tested on the same network to quantify the advantages they offer.
Such a test may not be too far off. This spring, two more universities, Columbia University in New York city and Georgia Institute of Technology in Atlanta, have started teaching students OpenFlow. Meanwhile, electronics manufacturer NEC has announced it will begin making OpenFlow-enabled routers. Within five years, McKeown predicts, we could see a thriving community of developers creating open-source software to redefine how the internet works. He expects internet data centres to be the movement’s advance guard because they have vast arrays of routers and are used to creating their own software. “If in those same five years the net itself becomes software-defined,” McKeown says, “well, that would be nice, too.”
