In early 2013, I wrote an article about how we chose .NET/Mono as the platform for our engine’s scripting system. Today, the scripting system has become one of the most important parts of the engine.

This article gives an overview of how we implemented the scripting system to achieve a simple yet powerful environment for game development. Our system is based on .NET, but several things in this article (especially the API parts) might be also useful for other frameworks such as Java or Python.

Although we call it “scripting system”, it is not only used for scripting of game events or AI patterns. Instead, it is the main programming environment when writing games using the engine. This is a code snippet that creates a world object from a C# script:

World testWorld = Universe.CreateWorld("Test World");
testWorld.SetTerrainGenerator(someTerrainGenerator);

So, nothing spectacular here, just a simple engine interface. We will get back to this example later to see what happens in the background.

The .NET Runtime

In case you are not familiar with it: Both Microsoft .NET and Mono are implementations of the Common Language Infrastructure (CLI), a standardized runtime environment for executing “managed” programs. Microsoft originally introduced and standardized the CLI, so you can also think of Mono as an open source compatible clone of the .NET framework.

For Windows, we use Microsoft’s .NET runtime (embedded using mixed mode C++/CLR). .NET is only available for Windows, so we use Mono on other platforms. At first, we tried to use Mono on Windows as well. However, Microsoft .NET is so much faster and more stable that we decided it is worth the additional effort to integrate both implementations. Mono has become more stable since we made that decision, but the current .NET implementation is still faster and integrates better with the Visual Studio debugging tools.

For simplicity, I will refer to the whole .NET/Mono/CLI environment as .NET in the following.

.NET uses a bytecode format called Common Intermediate Language (CIL). Similar to Java, CIL code can be executed on different platforms. CIL Compilers exist for a large number of languages, which means you can write .NET programs in any supported programming language, including C#, C++, Lua, Python, Java, PHP, and many others. Also, all .NET programs are (with some minor restrictions) compatible to each other. For example, you can directly use classes written in C++ from Java, without the need for any wrappers, as long as both programs are compiled to CIL code.

Scripting with .NET and Mono

We chose .NET as our scripting environment for several reasons:

• .NET and Mono have some of the fastest JIT compilers, so performance is not an issue for most use cases. Interacting with native C libraries works seamlessly if you need more performance.
• The whole framework is mature enough to be used for more complex software. The native framework libraries support threading, GUI, IO, networking, database interaction, image processing, serialization and many other things that would require additional libraries in a more simple scripting platform such as Lua.
• Many different languages are supported, lowering the barriers for programmers interested in using our engine. .NET libraries are compatible to each other, regardless of the programming language used.
• With Visual Studio and Monodevelop, very sophisticated programming and debugging tools exist to make development easier.

However, .NET is not designed as a scripting environment. Dedicated scripting languages like Lua load and execute source code at runtime. .NET can’t do this, so we needed to set up this part on our own. Our solution is quite simple: We have a cache of compiled assemblies that are loaded when you start a game. If you provide the source code to these assemblies in our resource system, the engine checks if the assemblies are up to date and recompiles them before loading if necessary.

Compilation times are quite fast, so even for larger sets of code you probably will not notice the difference in the startup time to a purely interpreted language.

The engine API

When writing a game using our engine, you have access to an API which wraps all the engine’s functions (which are written in C++) into a nice native .NET interface. Most of the API is generated automatically by our powerful ApiGen tool, which extracts the engine’s interface and creates wrapper functions in C and C#.

Let’s have a look at the example C# code from the beginning of this article that uses the API to create a new World object (the equivalent to what’s called a scene or level in other engines):

World testWorld = Universe.CreateWorld("Test World");
testWorld.SetTerrainGenerator(someTerrainGenerator);

So, from here everything looks like a normal .NET interface. You can create objects and use them like regular .NET objects. Internally, everything is piped through a native C interface, which wraps the object oriented C++ interface of the engine into an interface that can be imported as a dynamic library interface from .NET. Then, we have some C# code that wraps those flat C functions back into a nice object oriented API.

Let’s have a look at the Universe.CreateWorld call to understand what is going on in the background. This is the C# code of the CreateWorld method:

public static World CreateWorld(string name)
{
  if (name == null) throw new ArgumentNullException("name");
  return ObjectResolver<World>.FromGCHandlePtr(apiFunctionCreateWorldString(name));
}

This method calls the native C function apiFunctionCreateWorldString, which returns a new instance of World wrapped in a GCHandle pointer, which is then unwrapped into a normal .NET object by ObjectResolver.

The object returned by CreateWorld is an instance of Engine.Universe.World, which is an automatically generated wrapper class that holds a pointer to the respective C++ object and pipes method calls through the C API to finally reach the actual C++ member functions. For example, this is the implementation of World.SetTerrainGenerator:

public void SetTerrainGenerator(TerrainGenerator generator)
{
  if (generator == null) throw new ArgumentNullException("generator");
  apiFunctionSetTerrainGenerator(this.UnmanagedObject, generator);
}

Here, we pass the C++ pointer (UnmanagedObject) of the World instance and the given TerrainGenerator to apiFunctionSetTerrainGenerator, which is an automatically generated C function that calls the respective member function of the given C++ World object.

As you can see, all these API calls add some overhead compared to a simple method call. However, most of the C# API wrappers are so simple that the JIT compilers can inline them, reducing the overhead to an additional C call, which usually takes less than 0.01µs.

.NET could directly wrap C++ classes, but this would require the use of pointers in user code, which is something you want to avoid in .NET. Also, this layer of indirection gives us more control over the actual calls. This comes in handy to integrate our API objects with .NET’s garbage collector.

Object lifetime: Messing around with the garbage collector

.NET is a garbage collected language, which made the implementation of our API objects (like Engine.Universe.World) a bit tricky. We had to make sure that C++ objects are not deleted while a .NET API object is still referencing it. At the same time, we don’t want to create any memory leaks, so C++ objects should be deleted if the last .NET object referencing it is deleted and nothing else in the engine is referencing it.

Our solution was to derive all API-relevant C++ objects from a common base class that knows how to interact with their respective .NET object. This base class (UpvoidObject) holds a weak GCHandle to its .NET counterparts, so we can create references to the .NET object as long as it still exists, but the garbage collector can delete the object if it wants. UpvoidObjects can be asked to return their .NET wrapper object, so there always exists at most one wrapper object for each UpvoidObject because the UpvoidObject always returns the same .NET object.

When a .NET object is deleted, the wrapped UpvoidObject is notified and will create a new wrapper object when it is asked to return it.

Now, how do we delete C++ objects? For this, we decided to use C++ shared pointers for all UpvoidObjects, introducing a simple reference counting. When creating a wrapper objects, the UpvoidObject creates a shared pointer referencing itself. When the wrapper object is deleted, this shared pointer is released, resulting in the deletion of the UpvoidObject when no other shared pointer referencing it exists. Shared pointers introduce some overhead when creating new references, but if you use them in the correct way, this overhead is not too painful.

Conclusion: It’s cool, but it’s a lot of work

As you might have noticed, .NET is not the kind of scripting framework you integrate in two hours. While Microsoft .NET is well documented, the relevant documentation of Mono is outdated and incomplete, and incorrect in some critical parts, which made the integration of Mono quite painful.

.NET was a good choice for our engine. However, we are a team of six people, so I was able to dedicate several months to the scripting system. If you are developing a small game on your own, you might want to use something more lightweight. Lua or Angelscript are probably a better choice if you just want to integrate some kind of scripting without investing too much time.