Cisco rations VPNs for staff as strain of 100,000+ home workers hits its network

Following the moon to find capacity and sticking to safe sites as Chuck Robbins leads new weekly briefings

Cisco was surprised by how quickly it needed to adopt a global working-from-home policy, amid the coronavirus pandemic, and is now rationing VPN use to safeguard its security.

Speaking at the Cisco Live virtual event for the Asia Pacific and Japan region today, vice president of Cisco Customer & Seller Bailey Szeto said Cisco initially had little trouble implementing work-from-home for individual countries or cities, and from February 3 to March 4 only had to do so in China and Australia. A torrent of office closures followed, and by March 16, Cisco adopted a mandatory work-from-home policy.

“That was certainly a challenge for IT to get it in place,” Szeto said. “We had to think about getting ready for 140,000 employees and partners. That mean we had to prepare 130,000 devices and 55,000 BYOD devices, across 498 offices in 94 countries.”

The sudden move saw Cisco scramble for resources. Szeto said the company’s pre-virus systems didn’t have the scale necessary to handle all of its newly remote workers. “Every group in the company” found some compute, storage, or networking capacity it could share, he said.

Szeto didn’t detail the device or data center provisioning efforts, but did say the IT titan's assessment of its security needs identified some shortages of VPN capacity. Essentially, staff at home were using the corporate VPN to connect into internal work systems as well as public-facing services.

Load balancing and follow-the-moon resource allocation therefore became important.

“Sometimes it impossible to get enough capacity in a location, so we take advantage of our global footprint to connect to other VPN resources," he said. "You use the time of day to your advantage – if an Indian site is too busy, we redirect VPN requests to another location."

“Split tunnelling” has also come in handy, we're told. This technique sees traffic that absolutely needs a VPN to reach internal resources take that route, while other traffic travels over the public internet to public-facing servers.

Szeto presenting during Cisco Live virtual. This is what conferences look like now.

Szeto said Cisco IT identified sites like its major SaaS providers that it feels can safely be accessed over the public internet without VPN, which keeps resources free for more sensitive traffic.

The switch giant's assessment of the tools its teams need followed similar thinking. Cisco is dogfooding with Webex during the coronavirus crisis, and uses Office 365. And because those sites are rated as safe, staff are reminded they don’t need to be on the company’s VPNs all day to remain productive. Which again means traffic that really needs to get from a staffer’s home and into a Cisco office can do so swiftly and securely.

To help things along, the corporation's IT team has created a new internal resource that offers what Szeto called "a single version of the truth" after after some inaccurate advice about VPN use and other matters circulated within the company.

CEO Chuck Robbins has even referenced the new IT rule-book in his now-weekly all-hands meetings, in which Cisco is updating staff about its operations during the COVID-19 coronavirus crisis. Szeto said the company has involved medical experts so that staff feel well-informed about how both Cisco, and the wider world, is handling the current pandemic.

Szeto said the business is also trying to preserve its culture, by having virtual watercooler chats or even team workouts over video (which Szeto thinks they’d never do in person, partly due to odors). The Register has also spotted a Work In The Time Of Coronavirus blog post from Cisco featuring a GIF of Cisco staffers mimicking some moves from Michael Jackson’s Thriller video. ®