Thus, Tony and I set out to design a simplistic functional programming language. It is heavily inspired by Haskell and OCaml, and has many similarities, though it does not have much in the way of OOP. In short, it’s a language with first-class functions, generics, records/discriminated unions, and finally, treats everything as a value (much like most FP languages). We call it Cesura.

So far, nothing of what I’ve mentioned is particularly new or innovative. In fact, very few FP languages don’t have the features I mentioned. However, one interesting feature that we thought up (and which, to our knowledge, hasn’t been used in any other language) is what we call overloaded functional types. I’ll spare you from any category theory and jump straight to the problem.

You’re given this code:

sqr x : Int -> Int = x * x
sqr x : Float -> Float = x * x
f = sqr

The problem is immediately obvious: Which of the two functions does f refer to?

You could perhaps solve the problem by doing:

f : Float -> Float = sqr

This, however, completely defeats type inference and has lots of corner cases. If you passed sqr directly to a function, what would you do then?

We came to the conclusion that in order to maintain type inference and still have overloaded functions, we’d need to carry the overloads directly in the type system. This means:

f = sqr // The type of f is now: Int -> Int | Float -> Float

Here, we introduce the concept of an overload set. The type of f is a function that has an overload set consisting of Int -> Int and Float -> Float.

There are several advantages in this sort of type system. One of them is the fact that any function value is implicitly convertible to any function type whose overload set is a subset of the overload set of the function value’s function type.

But that’s not all. You might ask: What happens if I try to call f (from above)? How will the compiler know which function in the overload set I intend to call? The answer to this question is actually very simple:

x = f 2
y = f 4.0

Given this code, look at what’s passed to f in the first call. Clearly, it’s an integer. Thus, we know that a call to the Int -> Int overload was intended. In the second call, we’re passing a floating-point value, and therefore we’re calling the Float -> Float overload. So, x is of type Int and y is of type Float.

You might ask why we didn’t just choose to incorporate type classes, considering I claim that we’re heavily inspired by Haskell. The answer to this boils down to simplicity: Cesura needs to be a simple language utilizing as many features of the MCI framework as possible. We’re creating a simple, practical language, not a language utilizing extremely complex (but certainly beautiful) type system techniques.

Lastly, we’d really like some input on this approach. We’re still not entirely sure that it is the right way to go about this, though it does seem to be the most promising solution that isn’t overly complicated to implement. Also, if you know of some other language that uses this technique, please let us know!

namespace DynamicExtensionMethods
{
    internal static class Program
    {
        private static int Square(this int i)
        {
            return i * i;
        }

        private static void Main()
        {
            ((dynamic)1).Square();
        }
    }
}

Answer: It crashes.

Unhandled Exception: Microsoft.CSharp.RuntimeBinder.RuntimeBinderException: 'int' does not contain a definition for 'Square'

Apparently, RuntimeBinder doesn’t take extension methods into account. On one hand, it’s surprising because the dynamic binder supposedly acts like the C# compiler’s binder. On the other, RuntimeBinder is a framework thing and extension methods are a language feature. This makes me wonder if there would be any problematic corner cases with supporting extension methods in dynamic method calls…

You might think that I like functional programming because of first-class functions. You might think that I like it because of purity and immutability. Hell, you might even think that I like it because of natural recursion and tail calls. But no. Below are the three reasons I like the paradigm.

Everything is an expression: Typing return everywhere is a pain in the ass. The fact that the last expression in a function will be its result really cuts down on typing and reading. Additionally, this means that what you would usually call an if statement is actually an if expression. This implies that the last expression in the true/false paths are the result of the if expression. The same goes for match expressions, loop expressions, and so on. This makes it possible to write very clear code compared to the imperative equivalent.

Pattern matching: Basically the switch statement on steroids. Some (me being among them) will argue that this is what the switch statement always should have been, and that the switch statement in its current form is a language design mistake. Being able to match any value against any value makes for much more readable code, as compared to an if/else forest.

Discriminated unions: Ever wanted C unions but with less insanity and more safety? DUs allow you to use any type for your cases and with well-defined behavior. That alone is already great, but DUs truly shine when combined with fluent pattern matching syntax.

Most of the features I described here are present in all functional languages in one form or another, but they are particularly important in F# and Erlang which are the functional languages I take most interest in.

It might surprise some people that I didn’t list first-class functions and immutability. I like those concepts too, but I don’t think they’re what makes functional programming truly beautiful. This is ironic, because they’re probably the most basic traits of a functional programming language. But, in the time I’ve worked with functional languages, I haven’t found those two features to be the most appealing to me (don’t take that the wrong way; they are still great).

First, I’ve just become an intern at Xamarin. I’ll be working on the documentation build system and integration into the website and the MonoDevelop IDE. I was hired after GSoC was over, following a recommendation from Miguel. I’m really looking forward to working with Xamarin; it’s an awesome team of skilled hackers.

Second, I’ve been accepted into university, starting February 1 (at the UCN). The Danish name for the course I’m taking is “datamatiker”. They call it computer science in English, but it’s actually much more practically oriented than that (i.e. actual software development rather than theory). I was accepted based on my ability in the software development field, as I don’t fulfill the formal requirements (yet). This means I’ll probably quit high school, as I’d rather pursue something software-related than natural sciences.

In other (less life-related) news, I’ve also been putting some time into writing a compiler infrastructure library. I’ll be posting more about the goals and directions of that later.

First of all, the project is not completely done. I had anticipated this early on, and discussed with JB which parts I should focus on for the deadline. I did get a fair bit done for the deadline, but I wouldn’t quite say it’s ready for production use yet. I did, however, pass the final evaluation, as JB agreed to let me finish the project outside of GSoC (which I fully intend to do).

The project actually ended up being more involved than originally planned. The idea was to simply swap the PEAPI back end with Cecil, but I quickly realized that this was not quite as trivial as it seemed back then. I ended up ripping out the old code generation completely, emptying the code of every last parser production, updating the parser to .NET 4.0, and finally, adding the Cecil code generation back end in. You might think that all of this would have taken most of the entire GSoC period, but a lot of time was spent on spec reading too, not to mention figuring out weird quirks and corner cases in the Microsoft implementation of ILAsm. It’s scary just how much of the ILAsm language is poorly documented or not documented at all.

I’d like to extend my thanks to everyone in #cecil @ irc.gnome.org for helping me understand ILAsm, CIL, and the CLI in general.

I’ve also made some progress on the ILDasm front. It’s still missing a lot of features, but it does disassemble types and methods decently at this point. I’ll work more on it later; ILAsm comes first.

I recently switched ILAsm to a deferred type resolution model. This was JB’s idea, as a way to solve the generic parameter issue I blogged about earlier, and it’s worked nicely so far. This was probably the last major issue in the new ILAsm, so I expect to be doing more ILDasm work soon (there are only a couple of known ILAsm issues left).

During GSoC I also wrote a command line interface to the Mono soft debugger (SDB). It’s available here. I wrote it primarily because using MonoDevelop (which was the only existing interface to SDB at the time) as a debugger for a command line application was not very convenient. This debugger also has a few cool features such as decompilation of code with no source code (through ICSharpCode.Decompiler).

Overall, it’s been a very fun and educational program for me. I now know a lot of things about the CLI that I didn’t have the slightest clue about before, and I finally managed to contribute something really significant to Mono. Working with JB and the Mono community has been an awesome experience, as everyone’s very helpful and easily approachable. I would definitely do this again if I get the chance.

I’ll probably make a final blog post whenever I get this entire project merged into mainline Mono.

You can program D perfectly fine with the latest Digital Mars D (DMD) release, but I ended up having to contribute some patches to the standard libraries, and figured I might as well build everything from source.

First, you’re going to need Git, as everything is hosted on GitHub.

Next, do the following clones:

• /usr/src/dmd -> https://github.com/D-Programming-Language/dmd.git
• /usr/src/druntime -> https://github.com/D-Programming-Language/druntime.git
• /usr/src/phobos -> https://github.com/D-Programming-Language/phobos.git
• /usr/src/dtools -> https://github.com/D-Programming-Language/tools.git

In all of the makefiles, the MODEL variable specifies what bitness to build for. Here, I’m going to build for x86-64, so I’ll use 64. To build for x86, just use 32.

To build DMD, do:

cd /usr/src/dmd/src;
gmake -f posix.mak MODEL=64;

Everything should Just Work in the compile process. I’m using gmake because the makefiles are designed for GNU’s Make, and some systems don’t map make to that by default.

Unfortunately, DMD (and related repositories) don’t have any standard install target that we can use, so we’re just going to copy things into the file system. I’m using /usr/local/bin, /usr/local/lib, and /usr/local/include:

cp dmd /usr/local/bin;
cp dmd.conf /etc;

Verify that you have a DMD matching your system’s bitness:

dmd;

This should print:

DMD64 D Compiler v2.055

DMD32 should be printed for a 32-bit installation.

Now to build the core runtime libraries:

cd /usr/src/druntime;
gmake -f posix.mak MODEL=64;

If the build succeeds, we can copy the resulting interface files to our file system:

mkdir /usr/local/include/d2;
cp -r import/* /usr/local/include/d2;

Next, we build Phobos, which is the standard library containing facilities for concurrency, regular expressions, I/O, signals, math, text manipulation, and so on:

cp /usr/src/phobos;
gmake -f posix.mak MODEL=64;

Again, we copy library and interface files:

cp generated/linux/release/64/libphobos2.a /usr/local/lib;
cp -r std /usr/local/include/d2;

In the copy command for libphobos2.a, replace 64 with 32 if you built for x86.

With these in place, we should be able to build the additional D utilities:

cd /usr/src/dtools;
dmd catdoc.d;
dmd rdmd.d;
cp catdoc /usr/local/bin;
cp rdmd /usr/local/bin;

You should now have a fully functional installation of DMD! Feel free to post here if you run into trouble, or have some tip you’d like added to this tutorial.

I’d like to extend a thank you to everyone on #d @ irc.freenode.net for putting up with my questions. You guys are awesome. :)

First of all, we have generic parameters. In MS.NET’s ILAsm, !0 and !!0 are simply emitted as VAR 0 and MVAR 0 in the metadata, meaning that you could do something like:

.method public static !!0 Foo(!!1 bar)
{
    ldarg bar
    ret
}

Even though !!1 refers to a generic parameter that does not exist, MS.NET’s ILAsm will gladly emit it, and not even warn you. Due to an API “limitation” in Cecil, Mono’s ILAsm cannot do this. All GenericParameter objects must have an owner (i.e. the type or method definition/reference) and must be contained in that owner’s GenericParameters collection at the correct index. Therefore, Mono’s ILAsm would throw an error for the above code, since !!0 is out of bounds. This is a slightly annoying incompatibility, but there’s not much I can do about it other than patching Cecil. That’s only going to happen, though, if it won’t cause API breaks for version 1.0.

Second, properties like .file alignment, .imagebase, and .stackreserve aren’t fully supported. Cecil currently has no way of setting these on a module. ILAsm will check the values and error if they’re invalid, though.

Third, several native and variant types are not supported in marshal signatures. These include variant, void, syschar, decimal, date, objectref, nested struct and null, void, int64, uint64, unsigned int64, lpstr, lpwstr, safearray, hresult, carray, userdefined, record, filetime, blob, stream, storage streamed_object, stored_object, blob_object, cf, clsid, as well as pointers/references/vectors of these. The reason is that Cecil doesn’t expose any way to set these types. Either way, most of these are deprecated nowadays. All other native/variant types are supported.

Fourth, support for .data declarations is very limited (we actually only do a weak attempt at emulating them). Cecil has no way of emitting data constants, and they’re implementation-defined anyway. Currently, the only thing we do with them is copy them to the InitialValue property of field definitions where appropriate. This is far from correct, but it’s the best we can do. Either way, you should avoid using these declarations. They don’t make a whole lot of sense in managed land.

Fifth, we have no support for parsing System.Reflection-notation strings (yet). This means that things like custom marshalers aren’t supported (the syntax will simply be ignored). A type parsing API will supposedly be exposed in Cecil 1.0.

Sixth, support for declarative security syntax is limited. While we support the full syntax specified by the standard, we don’t support all of the syntax MS.NET’s ILAsm does (specifically, the syntax resembling verbal custom attribute initialization).

Seventh, exported types are unfinished because Cecil lacks a way to manipulate the File table directly, and because of the lack of a type parser.

Eighth, we can’t emit custom attributes on manifest resources and assembly references. This is a Cecil limitation which should be relatively easy to fix.

Lastly, the .vtable/.vtfixup/.vtentry directives are unsupported. Again, these will be implemented once Cecil has support for them.

There are some other incompatibilities, but they’re very subtle and you’re unlikely to ever encounter them. And even if you do, ILAsm will warn you.

Generally, the differences between the two implementations will only be encountered in obscure features that most users are highly unlikely to be using, or when attempting to use nonstandard syntax. Rule of thumb: Stay away from these things and you’re safe.

The XML file provided there is not complete, however. I’ve added the ContractInvariantMethodAttribute and ContractClassForAttribute attributes to the annotations so that ReSharper won’t claim that your contract classes and invariant methods are unused.

The XML is here.

Just drop this into C:\Program Files (x86)\JetBrains\ReSharper\v5.1\Bin\ExternalAnnotations\mscorlib as Microsoft.Contracts.xml or so.

A good while back, I blogged about how bytecode is evil. This post will explain what I intend to do with the AST idea.

The project I’m working on consists of five major components:

• A new executable format consisting of binary-form ASTs
• A managed JIT compiler
• A new assembly-like intermediate language
• An abstract bare-bones type system
• An AST/assembly optimization layer

The new executable format will be similar to the CIL format, but without all the PE bloat, and with one major difference: Code is stored as binary-serialized ASTs rather than bytecode. This allows easy introspection of compiled code, opening the doors for many interesting scenarios, such as translation from one language to another at runtime. I’ve chosen to call the format CAST (Compiled Abstract Syntax Trees).

To make the AST format actually interesting, a new JIT compiler needs to be implemented. This JIT compiler will understand the AST and operate on it for things such as optimization. For compatibility reasons, the AST will have a sort of RawCodeNode that contains plain assembly language (something similar to CIL but with registers). This is useful when one wants to emit non-AST code such as CIL/ILAsm, Java bytecode, Erlang BEAM, etc. I’ve chosen to call this JIT compiler Mono.Jit.

The new assembly-like intermediate language will be similar in nature to CIL. However, unlike CIL, this language will be based on registers and a stack, rather than locals and a stack. While the CIL approach is generally much prettier here, it’s less abstract and in some cases it is impossible to translate code to CIL (such as x86 machine code). I’m not sure what to call this yet, so for now, I’ll refer to it as IAL (Intermediate Assembly Language).

For CAST, I’ll eventually need to implement a compiler/decompiler, but for now, plans are to create a fluent API to create CAST code from .NET land. For IAL, I’ll just make a simple assembler/disassembler.

Now, in order for the AST to have any sort of meaning, a type system is needed. The type system in Mono.Jit is extremely simple; it supports only fields and methods on types. The JIT doesn’t need to know anything else in order to emit code. I do, however, plan to make an interface that allows giving the JIT optimization hints (such as “this type is completely immutable”, “this method makes no modification to members”, etc.). The JIT’s type system currently supports generics (types and methods), but I’m strongly contemplating delegating this to code loaders unless there is a compelling reason for the JIT to be aware of them.

Lastly, an optimization layer needs to be written. This layer will primarily concern itself with the AST input to the JIT, because that’s the entire point of this project. As mentioned, IAL is just a fallback for when you don’t have an AST to feed to the JIT. I don’t have any particular plans for optimization of IAL other than simple stuff like constant folding/propagation.

That’s it for now. More details will come as I write more code.

This post will be about the ILAsm standard (specified in ECMA 335) and the Microsoft implementation. There are several annoying differences and incompatibilities between the two. As part of figuring them out, I’ve been reading Expert .NET 2.0 IL Assembler and the Microsoft-annotated ECMA 335 standard.

The first difference is that MS.NET’s ILAsm does not respect the unsigned modifier on integer types. This means that an unsigned int32 will in effect be treated as an int32 (signed). I’ve chosen to break compatibility with Microsoft and correctly emit an unsigned integer, primarily because I think this is an outright stupid bug. Note that you can still get an unsigned integer by using uint32 with MS.NET. The latter form is preferred, anyway.

Second, MS.NET does not have the platformapi keyword. Instead, you have to use the winapi keyword, which makes your ILAsm source code unportable. In Mono’s ILAsm, we support both, in order to aid portability.

Third, MS.NET allows #line instead of .line for specifying source line information. This seems to be for compatibility reasons. Mono’s ILAsm supports both notations.

Fourth, MS.NET has a bunch of directives that just don’t exist in the standard. Specifically .file alignment, .imagebase, .language, and .namespace. We support all of them, though the first two currently have no actual effect. Note that use of .namespace is considered bad practice.

Fifth, MS.NET’s ILAsm doesn’t use .culture for specifying culture information, but rather .locale. This is directly against the standard, and the assembler won’t even recognize .culture. Mono’s ILAsm supports both notations.

Sixth, the MS.NET ILAsm doesn’t require the .hash directive; if it’s not specified, the hash will automatically be computed.

Seventh, the MS.NET ILAsm allows using “value class” in place of “valuetype” for indicating value types. This is relatively easy to handle, and Mono’s ILAsm does so. It is, however, considered bad notation.

Lastly, the MS.NET ILAsm allows specifying things like calling convention, type attributes, method attributes, field attributes, parameter attributes, and so on using flags(int32) notation. I’m sure this was done for a reason, but it seems like a great way to give developers the opportunity to make their programs unportable or impossible to run at all.

There are a lot of other incompatibilities, but they’re less annoying and easier to work around/implement.

That’s it for the this post. Next time: Implementation details of Mono’s ILAsm.