This article is more than 1 year old

Internet root keymasters must think they're cursed: First, a dodgy safe. Now, coronavirus upends IANA ceremony

Pandemic lockdown forces new measures on crucial crypto process than underpins world's DNS

IANA – the body that oversees the internet's IP addresses and domain names – must think it's under a curse in its quest to protect the 'net. Last time it was a malfunctioning safe that blocked its important work to keep the global network glued together.

This time, coronavirus.

Every quarter, a small group of people cram inside a secure facility in either California or Virginia in America, get locked in, and spend the next two to three hours cryptographically signing the digital key pairs used to secure the internet’s root zone, the text file that shapes the 'net as we know it.

The integrity of the digital signing process is so critical that the organization that runs it, IANA, part of DNS overlord ICANN, flies in trusted internet community representatives from across the world, out of a pool of 14, to methodically run through the steps. These representatives each possess a set of physical keys required to gain access to the necessary equipment, held in safe deposit boxes in IANA's key-signing facilities.

Once inside one of these facilities, which requires the use of fingerprint and retinal scanners, those present use their physical keys to access electronic key cards that activate a special locked-away device – a hardware security module (HSM) – that signs the digital key pairs for the root zone file for the next three months. Every step is meticulously recorded, and no one is allowed to enter or leave until the job is done.

This Thursday, the 41st of these ceremonies will take place, and, as you may have already gathered, the ongoing coronavirus pandemic has thrown a spanner in the works.

First, it is not terribly easy to fly people into either California or Virginia due to global travel restrictions and virus safety concerns. The 41st ceremony is supposed to take place in Virginia – it alternates between the two, the 40th being in Cali. IANA staff are based in Los Angeles, California, for what it's worth.

Second, even if you can fly in the reps, how do you obey mandatory social distancing rules while squashed inside a metal cage?

Cage fighting

These were the questions that landed on the desk of Kim Davies, IANA’s point man, just two months after he was forced to deal with another minor crisis surrounding the ceremony: during a test run on the day before the 40th ceremony, the main safe containing some of the necessary key-signing equipment was found to have jammed.

They couldn’t get in and had to take emergency measures. Ultimately that comprised hiring a specialist locksmith to drill out the safe’s lock and put everyone up in hotels for an extra three nights (yes, it took that long to drill the lock out – 20 hours – with IANA staff taking rotating shifts to make sure the location was kept secure.) It was the first time in 10 years the ceremony had been delayed.

Incidentally, we have a picture of the poor locksmith who had to spend two full working days lying on the floor of a secure facility drilling out a government-grade lock. Here he is:


Ever spent 20 hours on your side drilling a lock? Don't do it ... Click to enlarge

This time, however, it wasn’t the equipment but the people that were the problem. Fortunately, California’s stay-at-home lockdown happened a few weeks rather than several hours before the ceremony so the IANA folks had time to figure out a solution and get an official sign-off from ICANN’s board for the new plan.

It is far from perfect but it’s the best they can do given the circumstances.

Dressed to the nines

Instead of the ceremony securing the internet's root zone file for the next three months, this ceremony will do so for nine months, because no one is sure when the next ceremony can take place in the proper way.


Internet's safe-keepers forced to postpone crucial DNSSEC root key signing ceremony – no, not a hacker attack, but because they can't open a safe


It will be held again in El Segundo, California, rather than Culpeper, Virginia, due to the location of IANA and ICANN staff. And the trusted reps will be replaced with ICANN staff. They had considered drilling out the three safe deposit boxes to get access to the electronic cards inside but the hassle from February’s safe episode pushed the next obvious route: get the selected reps to send their physical keys in tamper proof bags and secure mail to the ICANN reps in California.

“The TCRs have wrapped their deposit keys with opaque material, and then transmitted them in tamper-evident bags,” explained IANA. “This bag will not be opened until within the ceremony so that each TCR can witness their key is in the same condition as when they released it. At the conclusion of the ceremony, the four keys will be similarly wrapped and then entrusted to four staff members who will independently arrange for them to be couriered back to their respective TCRs.”

The trusted reps, who are based in Mauritius, Spain, Russia, Tanzania, Uruguay, and the east coast of the US, will then watch their keys be used to open the boxes via a YouTube livestream and a secure side-chat will see them take an active role in the ceremony. The independent auditor will also be watching the ceremony from YouTube, rather than sitting in the room.

Meanwhile those in the room will be dressed like emergency room doctors, covered in PPE, and doing their best to distance themselves.

What about getting to the facility in the first place? IANA had to get a special waiver to access the space and it has informed the “relevant government agencies” of its plans to break the rules over non-essential services in Los Angeles.

Extras from ET

If all goes according to plan, the IANA and ICANN staff, dressed like they are trying to kidnap ET, will be directed over YouTube chat by people thousand of miles away, while keeping as much distance from each other as possible, and then spend hours methodically digitally signing key pairs so the internet can be secure until early 2021. It will add a surreal layer to an already unusual ceremony.

IANA has already started thinking about what it can do differently in future: would extra locations outside the US help? Or would it increase the security risks? Does flying in people from across the world to a single location make sense any more in a post-pandemic world? Or should the internet’s custodians figure out a way to distribute the process, both in terms of people and equipment?

Should IANA create a standby key system in case something goes awry in future? And how would that be secured and/or used? And is there any point in performing the ceremonies at all in their current form?

Those are the questions that will be asked from this Friday. In the meantime, a ceremony that was designed to be so methodically carried out that it is supposed to be almost painfully, predictably dull [PDF] has again proven to be anything but. Let’s all pray the YouTube livestream doesn’t cut out. ®

More about


Send us news

Other stories you might like