One of the problems was the creeping commercialization of the internet. Back when it was just engineers in the early days of the internet, it was all about engineering, says Bill Manning, a veteran internet engineer and previous IETF working group chair.
"I remember the OSI/IP debates, the OSPF/ISIS discussions, the emergence of NAT and private address space, how IPv6 emerged and who crippled it so badly, the four phases of DNSSEC development," he remarked. "But I pretty much stopped attending when the IETF ossified and the legal/social issues became more relevant than actual engineering."
The huge success of the internet brought with it enormous issues largely unrelated to engineering. Most of them were pushed off into other arenas such as DNS overseer ICANN or UN telecoms body the ITU, but the IETF was far from immune.
The fact that attendees to IETF meetings have to pay several hundred dollars to register, as well as flights and accommodation and lost work time meant that companies were paying their employees to attend and with their business models increasingly tied up in the work that the IETF was doing, it immediately started impacting the organization.
"The biggest change I have seen over the years is the move from a pure technological focus to one that was very much directed by business and what businesses were willing, or wanting, to do with technology," notes Avri Doria, a researcher and another veteran of internet organizations. "It was quite blatant. At some point we needed to bring forward at least two companies willing to develop something before we could put a new tech project on the table. It has since softened a bit, but for a while it was quite absolute."
That softening has come largely as a result of the inclusion of more academics into the organization - something which Manning suggests will end up being the main purpose of the IETF going into the future. "[It will be] the object of academic papers on the history and evolution of the emergent global communications and data infrastructure," he predicts.
The open culture also led to a number of functional problems. For one, the larger the IETF got, the harder it became to move forward either logically or in a timely fashion. When companies started developing their own standards and then brought them to the IETF, they often found others had been working on the same problem. Typically with the internet there is no right or wrong way, just a different way. The result was frustrating repetition and sometimes stalemate.
The initial spec of the critical IPv6 standard - which is necessary to grown the basic internet infrastructure to handle the addressing needs of all users for the next century - was first published by the IETF in 1996 yet because of the organization's dynamics it ended up not being backwards compatible with the existing IPv4 standard. The result is that 20 years later, there is still only 10 per cent of the of global network using it - and this is despite the number of IPv4 addresses having already run out.
What's more, and ironically for an organization that is in theory completely open, the IETF started being dominated by a small number of high-engaged individuals.
One important figure in the internet world recalls being deeply involved at one point before withdrawing a little. "I remember being an area director and not having the tools server with all its mechanisms… oh boy," explains Patrik Fältström. "Otherwise it has been interesting to see such fights around for example chat protocols. The IETF ended up producing spec for four different ones, but a fifth, Jabber, did win. And how excellent technologies like Beep failed, because everyone uses HTTP anyway… And of course how so many loooooooong discussions on the IETF list involves the same individuals that already were established there when I started, and they continued when I stopped being super-active."
The NSA comes sniffing
And then there was the fact that the NSA started taking advantage of the IETF's peculiarities to push its own agenda.
Among documents leaked by Edward Snowden were notes on IETF meetings that NSA staffers had attended in 2006 in which they flagged not only that the slow rate of progress on new security protocols was assisting them but that they were actively working the system in an effort to undermine new protocols.
"New session policy extensions may improve our ability to passively target two sided communications by the incorporation of detailed call information being included with XML imbedded in SIP messages," said one NSA note. "On the security front, SRTP has been slow in the uptake to key management limitations."
The end result of all of these issues has been that the one-time powerhouse of everything Internet has become increasingly less relevant in recent years.
Manning again: "The IETF was a jewel: an organization for its time. But times have changed and much of the work has moved to specific organizations: M3AAWG, W3C, DNSOARC, and the IoT world."
The Internet of Things reference is particularly noteworthy since it is the abundance of different standards and protocols that is currently holding back what many believe will be the next big step for the internet.
The IETF has noticed and just this week, its chair Arkko published a blog post highlighting a seminar that the organization is running on this very topic. "I cannot think of a better example where interoperability is important than the Internet of Things. Without interoperability, lights won’t work with the switches, sensors can't be read by your smartphone, and devices cannot use the networks around them."
But he is forced to note that people are not exactly queuing up to have the IETF figure out the problem: "Of course, the IETF along with other relevant organisations is working on those standards."