The evolution of C#: Lead designer describes modernization journey, breaks it down about getting func-y

As C# 9.0 is released, a look back at functional programming adoption, what the language is for, and what was a mistake

Interview C# lead designer Mads Torgersen cracked open a window into the evolution of the language during his spot at the virtual QCon Plus developer conference last week, and was quite frank about what has worked and what has not.

The programming language is "about two decades old," said Torgersen, "and during that time it's [gone] through a journey of transformation… it started out as a very classic, turn of the century object-oriented language."

C# was created by Anders Hejlsberg (whose other achievements include Borland Delphi and most recently, Microsoft TypeScript); but Torgersen has been working on C# for around 15 years and has been lead designer for five years.

Records in C# 9.0 are considered equal if they are the same type and have identical properties

How has it changed? Torgersen spoke to QCon attendees about C#'s gradual adoption of features from functional programming as well as answering some questions we put to him. Even C# 1.0 had delegate types, "they're sort of function types, they're really crappy in many ways … but they get the job done," he said.

C# soon moved on though. Generics (C# 2.0); type inference (the var keyword) (C# 3.0); Lambda expressions (C# 3.0). In fact C# 3.0 in late 2007 was a big release, with anonymous types, extension methods, and query expressions, this last also known as Language-integrated query (LINQ).

"Many of the things that happened were inspired, borrowed, stolen from the functional world," Torgersen acknowledged.

How does C# compare to say, Scala, as a functional language? "I don't think we will get to where C# is a complete balance between functional and object oriented," said Torgersen. "We're always object-first, so we're never going to compete with a functional-first language … there's so many things from functional programming, so many idioms, such an amount of type inference that we could never get there from here."

Torgersen said that a big trend in recent years has been pattern matching, driven by cloud computing where "your data is in the cloud and is being shared across many different applications."

Object-orientation, he said, is not well suited to this scenario. "Object-oriented programming focuses on encapsulating functionality and data together," but in this new world, "packaging the functionality with the data doesn't make sense any more. You need to have the functions on the outside," by which he meant, "you need to write a function that takes some object in, the object doesn't know about the function, but the function does different things depending on the type of the object. That's what pattern matching is for."

Pattern matching arrived in C# 7.0, released in March 2017, and has been steadily enhanced since. C# 9.0, released this week with .NET 5.0, added 6 further pattern improvements, including Type patterns which match on the type of a variable.

C# 9.0 also introduces record types. "The main thing they give you is value semantics by default," said Torgersen. What he has in mind is the seemingly simple question of is A equal to B. Object orientation normally requires writing equality methods to override the defaults, which is "a maintenance nightmare," he said. Now with record types, "the synthesized methods for equality and hash codes consider two records equal if their properties are all equal," as the docs explain (they also have to be of the same type of course). "It's another step towards functional programming," said Torgersen.

What is C# good for?

Can C# be everything to everybody, or if not what is its niche, its particular value versus other language choices, we asked.

"C# started out because Windows needed a good modern app language that looked a little bit like C++ and Java," Torgersen said. "It was very specific in its initial targeting. It had function types that were there almost by mistake, for scenarios that had nothing to do with functional programming. We've been pushing into other scenarios because we were successful and people liked it, but it's also just spread on its own. If you look at the world of games, the majority are written in Unity which is based on a C# engine that was built outside Microsoft, partly because Mono happened to be open source and easy to tinker with, but also because they liked C# for that.

Mads Torgersen, C# Lead Designer

"One of the things we've been doing with every release of C# over the last five years was to have more low-level features. C# always had support for unsafe code, but they have more safe, efficient, low-level codes in various ways. It's an area we've invested a lot in, even though most people don't use it, because it means most of the lower levels of your architecture can be built in C# and run on .NET and be portable. But from that to saying that C# is a systems programming language, that's not something we would say, and to chase that would be to the detriment of the rest of the language. So there are boundaries. We try to just let it grow organically and chase things that are valuable now."

A decade ago, C# 4.0 introduced the dynamic type, where type is resolved at runtime. Was that a big deal for C#?

"It was a nice language feature and it was part of a dynamic pushback that didn't come to as much as we had felt. We were doing IronPython and IronRUby at the time, which were implementations of Python and Ruby on top of the .NET platform, and we wanted to have great interop between those as well as to JavaScript. So we built a whole infrastructure for resolving dynamic-ness on .NET.

"I have some regrets about exactly the way we built it, because it's expensive at runtime. It's beautiful, it runs the whole features of the compiler to do overload resolution at runtime instead of compile time… from a theoretical perspective that's super elegant and I'm proud of it. But the cost and overhead is prohibitive in some scenarios, so [the] dynamic doesn't mix well with performance.

"It's not been a blockbuster feature for C# and sometimes I regret it a little bit, but sometimes it's just the thing, that lets you stitch things together that don't otherwise put together… that makes me happy."

What about null safety, where work was done in C# 8.0 to add a nullable aware context to the compiler, in an effort to avoid null reference exceptions at runtime, but off by default. If enabled, reference types are non-nullable by default, but can be specifically declared nullable as required, though it is a compiler feature and not enforced by the runtime. Is C# where it should be in this respect?

"It was a very big and gnarly feature to do. The conceptual overhead isn't so great, but the engineering work and details are immense," said Torgersen. "It's been similar to the TypeScript design experience …. Where they have this very complicated type system that's designed to feel not complicated. For null specifically, we tried to do the same thing. We wanted to let you drop these little question mark annotations here and there, and it would beautifully flow through your scenario.

"We gave ourselves time through C# 9.0, maybe a little longer, to say OK, this is only about warnings, not errors. We will allow ourselves to tinker with it. I think we are in the long tail of it now. I think we are pretty much done."

Asked if there is anything in C# that he would like to retire, Torgersen said "that first shot at function literals, the anonymous methods, was a clunky mistake. We overwrote it immediately in C# 3.0. Of course we can't take it out."

That is the thing, you can add things to a programming language, but not remove them. "People… have code they've committed and if you take anything out or change what it means, you're breaking a lot: we can't do that," he said. ®