Usenix: Google deploys IPv6 for internal network

Though the project is only halfway finished, Google's IPv6 network is already bearing fruit

In a project that has taken longer than company engineers anticipated, Google is rolling out IPv6 across its entire internal employee network.

Google network engineer Irena Nikolova discussed the company-wide implementation at the Usenix Large Installation System Administration (LISA) conference, being held this week in Boston. There, she shared some lessons that other organizations might benefit from as they migrate their own networks to the next generation Internet Protocol.

From the experience, Google has learned that an IPv6 migration involves more than just updating the software and hardware. It also requires buy-in from management and staff, particularly administrators who already are juggling too many tasks. And, for early adopters, it requires a lot of work with vendors to get them to fix buggy and still-unfinished code. "We should not expect something to work just because it is declared supported," the paper accompanying the presentation concluded.

"I think everyone who has tried to migrate to IPv6 has run into the same problems we have," Nikolova said.

The project, which has been under way for about four years, turned out to be a larger endeavor than the engineering team had anticipated. It is only half way finished. But the company has made significant gains in this time. About 95 percent of Google's engineers now have IPv6 access on their desks. Eventually, the company plans to have an IPv6-only network.

The project was started in 2008 by a small group of Google engineers, some of whom worked on it in the 20 percent of their work time that Google allots for its engineers to pursue their own pet projects. The goal was "IPv6 everywhere," Nikolova said.

Part of the interest in the upgrade was practical. Even though it was a private network, Google's internal network used public IP addresses, and Google was running out of internal IPv4 addresses. Also, Google engineers were developing IPv6 versions of Google's own tools and applications and needed to test this software internally before releasing it to the public.

Lastly, Google engineers realized they faced a chicken-and-egg problem with deploying IPv6. Like many organizations, the company has been slow to adopt IPv6 due to the lack of third party applications running on IPv6, which, in turn, are scarce because few organizations run IPv6 networks.

Google's internal network spans more than 200 offices worldwide, serving about 30,000 employees. It consists of a wide variety of devices from companies such as Cisco Systems, Juniper Networks, and Aruba Networks, hundreds of commercial and home-built applications, and a range of operating systems, including Linux, Mac OS X, and Microsoft Windows.

The engineers modeled the IPv6 network as closely as possible on the existing IPv4 one, to keep the routing and traffic flow largely the same. At first they ran IPv6 over the IPv4 networks, a process called tunneling. Then, in cases where it was feasible, they set up dual stacks, where IPv4 and IPv6 run side by side. Eventually, they want to make the network IPv6 only.

To assign IPv6 numbers to devices, Google followed the guidelines in the Internet Engineering Task Force's RFC 5375. Each campus or office got a /48 address block, which meant that it was allotted 280 addresses. In turn, each building got a /56 block of those addresses (or about 272 addresses) and each VLAN (Virtual Local Area Network) received a /64 block, or about 264 addresses. To assign numbers to specific devices, the engineers used the Stateless Address Auto-Configuration capability (SLAAC), which allows the devices to assign numbers to themselves. This approach eliminated the need to manually assign numbers to each device. It was also necessary in that at least some operating systems do not yet support DHCPv6, the version of Dynamic Host Configuration Protocol server-based addressing mechanism for IPv6 networks.

One of the major issues was inadequate support for IPv6 in network devices and software, Nikolova said.

Many network devices now only support IPv6 in software, meaning that much of the traffic processing is carried out in software, rather than with customized hardware. As a result, IPv6 network operations consume more processor cycles than IPv4 operations do. At least one wireless equipment vendor did not support ACLs (access control lists). Also, the network's WAN (wide area network) acceleration devices can not encrypt IPv6 traffic, because the protocol they use--WCCP (Web Cache Control Protocol)--does not yet work with IPv6. In addition to networking gear, printers remain problematic, in that many do not fully support IPv6.

Application and OS compatibility also proved to be a challenge. The company has been phasing out those applications that do not support IPv6, though many essential tools, such as databases and billing applications, remain in operation because they can not be modified or upgraded easily. And while the current versions of most OSes support IPv6, they do not do so by default, which causes additional work on the part of administrators and users.

"When it comes to technical problems, we can confirm that there is a lot of new, unproven and therefore buggy code, and getting our vendors aligned so that everything supports IPv6 has been a challenge," the Google paper stated.

Google also faced challenges with service providers, those companies that provided network connectivity to Google's offices. The SLAs (Service Level Agreements) were not as rigorous as those for their IPv4 services. They took a longer time connecting separate IPv6 points than they did with IPv4 peering sessions. Google itself had to rewrite its own network monitoring tools to work with IPv6 as well.

Despite these obstacles, Nikolova is confident that the team will achieve its goal of implementing an all-IPv6 network within Google's offices.

"At some point, we stopped talking about IPv6 as the 'new protocol' and started calling IPv4 the 'old protocol'," Nikolova said.

Joab Jackson covers enterprise software and general technology breaking news for The IDG News Service. Follow Joab on Twitter at @Joab_Jackson. Joab's e-mail address is

Join the Good Gear Guide newsletter!

Error: Please check your email address.

Tags networking hardwareGoogleNetworkingroutersswitchesinternet

Our Back to Business guide highlights the best products for you to boost your productivity at home, on the road, at the office, or in the classroom.

Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Joab Jackson

IDG News Service
Show Comments

Most Popular Reviews

Latest News Articles


PCW Evaluation Team

Azadeh Williams

HP OfficeJet Pro 8730

A smarter way to print for busy small business owners, combining speedy printing with scanning and copying, making it easier to produce high quality documents and images at a touch of a button.

Andrew Grant

HP OfficeJet Pro 8730

I've had a multifunction printer in the office going on 10 years now. It was a neat bit of kit back in the day -- print, copy, scan, fax -- when printing over WiFi felt a bit like magic. It’s seen better days though and an upgrade’s well overdue. This HP OfficeJet Pro 8730 looks like it ticks all the same boxes: print, copy, scan, and fax. (Really? Does anyone fax anything any more? I guess it's good to know the facility’s there, just in case.) Printing over WiFi is more-or- less standard these days.

Ed Dawson

HP OfficeJet Pro 8730

As a freelance writer who is always on the go, I like my technology to be both efficient and effective so I can do my job well. The HP OfficeJet Pro 8730 Inkjet Printer ticks all the boxes in terms of form factor, performance and user interface.

Michael Hargreaves

Windows 10 for Business / Dell XPS 13

I’d happily recommend this touchscreen laptop and Windows 10 as a great way to get serious work done at a desk or on the road.

Aysha Strobbe

Windows 10 / HP Spectre x360

Ultimately, I think the Windows 10 environment is excellent for me as it caters for so many different uses. The inclusion of the Xbox app is also great for when you need some downtime too!

Mark Escubio

Windows 10 / Lenovo Yoga 910

For me, the Xbox Play Anywhere is a great new feature as it allows you to play your current Xbox games with higher resolutions and better graphics without forking out extra cash for another copy. Although available titles are still scarce, but I’m sure it will grow in time.

Featured Content

Latest Jobs

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?