This is just going to be a bit of a brain dump.
I’ve been talking with my friend from Microsoft for the last couple of days about .NET and concurrency. Me being the language geek that I am, I was arguing that Erlang is, and will continue to be, the better choice for highly concurrent systems. .NET currently does not offer any reasonable equivalent to Erlang’s concurrency model. Sure, we have message-passing frameworks, we have locks, we have concurrent collections, we have the TPL (and TDF), but it all boils down to the same thing: Threads. Threads are evil.
A thread is no better than an OS process. In order to create a thread, a 1 MB stack (on Windows, anyway) needs to be allocated, and the OS needs to schedule the thread like it would schedule a process. This is heavier than it might sound. With Erlang, you can have millions of processes running at almost no cost; with OS processes you’d die at a few thousands.
Well, this happens to be so because Erlang processes are designed to be small. A few hundred bytes is all it takes to run code in an Erlang process. This is huge difference from OS threads/processes. There is one primary reason why Erlang is so economic with process resources: It uses the Actor model. If you’re an OO developer, you could think of actors as objects. In Erlang, to invoke a method on these objects, you’d send a message that the objects process completely isolated from each other, and thus, concurrently. In other words, by using messaging as the means of communication between objects, you get concurrency for free. This is quite different from how we view objects and threads from a language such as C#. In C#, we often have some sort of primary object that runs in a separate thread, or in the thread pool, which takes care of running a bunch of child objects, because it’s simply too expensive and inefficient to spawn too many threads or add too many callbacks to the thread pool. In Erlang, this is not the case; in fact, you’re encouraged to run thousands or even millions of small processes in order to be able to scale your application. There is a clear difference between Erlang and, for example, .NET here: Distributed concurrency is first-class in Erlang.
.NET is a fine platform and I love to use it. But the problem with .NET is that concurrency is something that wasn’t really regarded as being important. Sure, .NET has threads, as well as high-level abstractions on top of them, such as TPL, TDF, PLINQ, and so on, but there is virtually no support in the runtime itself for distributed concurrency. We don’t have location-transparent processes, we don’t have load-balancing, we don’t have lightweight processes, we don’t have continuations, we don’t have code hot swapping, we don’t have fault-tolerance… The list goes on. And it doesn’t look like we’re getting it anytime soon, either. To Microsoft, this apparently seems too “academic” and “unrealistic”. Apparently, Ericsson, Facebook, RabbitMQ, T-Mobile, and Telia are not realistic companies.
I know exactly what the problem is, however. The problem is that Erlang is not popular. If it’s not popular, surely nobody needs it to solve their problems, and it’s just an academic toy. The usual excuse for not implementing something. You can probably guess what I’m getting at: Microsoft has recently done almost nothing innovative when it comes to development tools and frameworks. They spend their time mimicking what the open source community or other companies have provided for years. ORMs have existed for a long, long time, yet Microsoft is still playing the catch-up game with Entity Framework. Cloud services have existed for years, yet Microsoft had to push Azure forward. IoC and extension APIs have existed for almost a decade, yet Microsoft is working on MEF. MVC web frameworks have existed for years, yet Microsoft is working on ASP.NET MVC. The only Microsoft division I can say is truly rolling out new tech is Microsoft Research, with their work on F#, Code Contracts, Pex/Moles, Singularity, etc.
It’s sad. Microsoft doesn’t seem to dare encourage any sort of paradigm shift these days, and shamelessly dismisses new technologies as “academic” if they’re not popular and in wide use. What they don’t seem to realize is that all they’re doing is reinventing wheels and that innovation is what needs to happen. Now.