Why D

In my free time, I’ve been trying to work with cpp lately, but have found many aspects of the process frustrating (dependency management, and compile time in particular). I was working though the backlog of the CppCast and came across the D episode, and Andrei Alexandrescu made a convincing pitch. D seemed like a good opportunity to explore some of the ideas I’m interested in, without long compile times, and difficulties of including libraries.

Why JSON (de)serialization

The xml and json libraries are among the first things I check when looking at a language. Most standard libraries offer a primitive library that produces a map that is a key value parsed from output to a string. Serializing and deserializing objects with these is tedious and can lead to frustrating runtime errors. Go’s standard library is the only compiled option I’ve seen that handles serialization satisfactorily.

There wasn’t an existing library for D that behaves this way. And D’s compile time reflection makes this an interesting project. I’m excited to see if could perform better than the runtime reflection that Go offers (benchmarks to come). The current state (as of writing) can be seen here.

##Using Templates and compile time logic

Working with templates is much easier in D than in Cpp, because the error messages are much more helpful. It might simply be terseness. There is still a stack off messages, but they are easier to read (as a relative novice).

This was a challenge for me though. Because much of the logic was being performe at compile time. Between templates and compile time expressions (enum) and reflection, it was rather different than most coding I do at my day job (Javascript & Ruby). Good thing D compiles quickly!

Some interesting code can be see in the first few lines of the Unmarshall (deserialize) method:

void Unmarshall(T)(ref T obj, in JSONValue json){
  enum auto isArray = isDynamicArray!T;

  // handle arrays
  static if(isArray){
    foreach(el ; json.array){
      obj.length++;
      auto child = new ElementType!T;
      JSONInflater.Unmarshall(child, el);
      obj[$ - 1] = child;
    }

  }

	//.... parse other types
}

The first check is if the passed object is an array, this arrayness is evaluated at compile time via enum. If obj is an array then a new element is created for each member of the json array. Then that child is unmarshalled and stored in the newly create array.

Handling Objects and their Attributes

else {
    // handle signular elements
    enum auto fnt = FieldNameTuple!T;

		// create map from name to type
    foreach(k; fnt){
      if(!(k in json)) continue;

      alias fieldType = typeof(__traits(getMember, T, k));
      enum auto isBasic = isBasicType!fieldType || isNarrowString!fieldType;
      enum auto isSArray = isStaticArray!fieldType;
      enum auto isDArray = isDynamicArray!fieldType;

      auto jsonField = json[k];

      static if(isBasic){

        switch(jsonField.type){
          case JSON_TYPE.INTEGER:
            __traits(getMember, obj, k) = to!(fieldType)(jsonField.integer);
            break;
          case JSON_TYPE.FLOAT:
            __traits(getMember, obj, k) = to!(fieldType)(jsonField.floating);
            break;
          case JSON_TYPE.STRING:
            __traits(getMember, obj, k) = to!(fieldType)(jsonField.str);
            break;
          default:
            writefln("Don't know how to handle: %s", jsonField.type);
        }

      } else static if(isSArray){
        std.stdio.stderr.writeln("Don't support static arrays");

      } else static if(isDArray){
        JSONInflater.Unmarshall(__traits(getMember, obj, k), jsonField);

      } else {
        auto child = new fieldType;
        JSONInflater.Unmarshall(child, jsonField);
        __traits(getMember, obj, k) = child;
      }
	}
}

This is a bit trickier. Firstly each field in the type is visited after fetching them with FieldNameTuple. Then the corresponding json field is visited, and the json field type is used to pull info out. The value is then converted to the fieldType using to.

There are two recursive cases involving dynamic arrays and… everything else. Currently I don’t have checking for objects (structs, unions, classes) working. (edit. was able to check using isAggregateType!). If it isn’t a basic type, or an array, then Unmarshaller assumes its an object and recurses to create it. isNested didn’t seem to work, will have to revisit

Serialization (Marshalling) is pretty similar, only a bit simpler since only one instruction is needed to cover all the basic types. See here.

Ease of use

So far this library is as simple to use (for the base case) as the go lib. See the example code here. Simply defining the class and feeding it to the unmarshall method seems to do the trick.

Conclusion

Compile time reflection seems perfect for this problem. All the information needed to structure the serialization should be available during compile time, including for additional features like serialized field name customization, or checking camel/underscore/hyphen seperation. I’m going to push compile time logic as far as possible, add attributes to allow for field name customization, and benchmark the app against Go with some large json files.

I’m excited to take this further. It’s still very early, but D is fun to program in so far. I like the simplicity of package management, the speed of compilation, and the familiarity of (most) syntax. After tests and benchmarks I’ll get the package ready to publish!