Behind the scenes of the planning DSL

Last week I blogged about an embedded DSL for creating different calendar components like events and to do-tasks. For this post I'll let you in on some of the "secrets" of embedded DSL development in C#. Before we get started, let me refresh your memory with an example of how the DSL can be used:

ToDoComponent planningTask =
    Plan.ToDo("Plan project X").
        StartingNow.
        MustBeCompletedBy("2007.08.17").
        ClassifyAs("Public");
planningTask.Save();

EventComponent planningMeeting =
    Plan.Event("Project planning meeting").
        RelatedTo(planningTask).
        WithPriority(1).
        At("Head office").
        OrganizedBy("jane@megacorp.com", "Jane Doe").
        StartingAt("12:00").Lasting(45).Minutes.
        Attendants(
            "peter@megacorp.com",
            "paul@megacorp.com",
            "mary@contractor.com").AreRequired.
        Attendant("john@megacorp.com").IsOptional.
        Resource("Projector").IsRequired.
        ClassifyAs("Public").
        CategorizeAs("Businees", "Development").
        Recurring.Until(2008).EverySingle.Week.On(Day.Thursday).
        Except.Each.Year.In(Month.July | Month.August);

planningMeeting.SendInvitations();

Writing DSLs is a little different from the regular object oriented programming style. You might have noticed that the Plan class has a verb for its name rather than the usual noun. This allows us to have a natural starting point for writing out the "sentence" explaining our intention.

public class Plan
{
    private Plan() {}
    public static ToDoDescriptor ToDo(string description)
    {
        ToDoDescriptor descriptor=new ToDoDescriptor();
        descriptor.Description= description;
        return descriptor;
    }
    public static EventDescriptor Event(string description)
    {
        EventDescriptor descriptor=new EventDescriptor();
        descriptor.Description = description;
        return descriptor;
    }
}

Rather than returning an instance of the type we want to create, the operations on the Plan class return descriptors which we will use to collect the information needed to create these types. These descriptor types is the backbone of the DSL. The DSL is basically a fluent interface. With some exceptions, every method or property in the fluent interface returns an instance of the class it is defined on allowing operations to be chained together. Take a look at the following excerpt from the EventDescriptor class.

public class EventDescriptor
{
    // Lots of stuff eluded for brevity...
    public EventDescriptor StartingAt(string time)
    {
        return StartingAt(Convert.ToDateTime(time));
    }
    public EventDescriptor StartingAt(DateTime time)
    {
        this.startTime = time;
        return this;
    }        
    public EventDescriptor ShownAsBusy
    {
        get
        {
            timeTransparency = TimeTransparency.Opaque;
            return this;
        }
    }
    public EventDescriptor ShownAsFreeTime
    {
        get
        {
            timeTransparency = TimeTransparency.Transparent;
            return this;
        }
    } 
}

Notice that every property and method returns "this" to the caller. Another abnormality is that the getters actually change the state of the EventDescriptor class. This allows our language to be more natural than if we'd followed the common design guidelines.
Another takeaway is that there are convenient overloads for some of the methods. For instance the StartingAt method accepts a string in addition to an actual DateTime make the client code easier to read.

Plan.Event("Meeting").
    StartingAt("2007.07.10 12:00");
//... is easier to read than ...
Plan.Event("Meeting").
    StartingAt(new DateTime(
        2007,07,10,12,00,00
    ));

A common way to write "regular" object oriented code is to accept the required arguments in the constructor and allow the user to set any additional properties afterwards through the setters. For instance the EventComponent class (which is written adhering to the rules of the OO schoolbook) allows this.

EventComponent meeting=new EventComponent("Meeting");
meeting.Organizer=new Organizer("jane@megacorp.com");
meeting.Organizer.CommonName="Jane Doe";

In the DSL you'll use the OrganizedBy operation to specify who the meeting organizer is, and since there is a lot of different optional properties to set on the Organizer class the OrganizedBy method has quite a few overloads to allow the user to specify these.

public EventDescriptor OrganizedBy(CalAddress organizer)
{
    this.organizer = new Organizer(organizer);
    Attendee chair = new Attendee(organizer);
    chair.Role = ParticipationRole.Chair;
    attendees.Add(chair);
    return this;
}
public EventDescriptor OrganizedBy(CalAddress organizer, string organizerName)
{
    OrganizedBy(organizer);
    this.organizer.CommonName = new CommonName(organizerName);
    return this;
}
public EventDescriptor OrganizedBy(CalAddress organizer, string organizerName, string directory)
{
    OrganizedBy(organizer,organizerName);
    this.organizer.Directory = new Directory(directory);
    return this;
}

Another thing you should notice is that the topmost overload also adds the organizer as an attendee to to the meeting. This saves the user from typing at the same time as it improves the quality of the event component produced in the end. This shows how we can use the DSL interface to provide an more expressive language on top of the underlying API.

In the example shown at the top of this post, the DSL is really a special case of the Factory design pattern. We therefore need to terminate any "sentence" with an operation that returns an EventComponent instance. The common way of doing this is to have an interface like this:

MyObject o=The.Last().Operation().Returns.The.Instance();

Where the Instance() method would return an instance of MyObject. With smaller, well defined DSLs this is usually not a problem because you can live with a strict grammar. E.g. every sentence consists of the same words.
However, in an advanced scenario like our event planning DSL we need less strict grammar to cover all the use cases. For instance you don't have to set a priority, location or similar for an event, neither do you have to make it recursive and even if it is recursive, it doesn't need to have exceptions. This implies that all of the following "sentences" should be allowed.

EventComponent thing;
thing=
   Plan.Event("Dinner").
       At("Dino's Diner").
       OrganizedBy("jane@megacorp.com", "Jane Doe").
       StartingAt("19:00").EndingAt("21:00").
       Attendant("john@megacorp.com").IsOptional.
       ClassifyAs(Classfication.Private);

thing=
    Plan.Event("Time to think").
        WithPriority(Priority.Cua.A3).
        ShownAsFreeTime.
        StartingAt("12:00").Lasting(45).Minutes.
        Resouce("Clear mind").IsRequired.
        CategorizeAs("Development").
        Recurring.UntilForver.EveryOther.Week.
        Except.Once.EverySingle.Year.In(Month.December);

thing =
    Plan.Event("Summer vacation").
        StartingAt("2007.08.06").Lasting(3).Weeks.
        ClassifyAs("Private").
        Recurring.Times(4).EverySingle.Year;

Even if there is a clear structure of the grammar, there are no final terminating keyword at the end of each sentence, so ClassifyAs(), In() and Year must all return a valid EventComponent. However, they can also have additional keywords following them. For instance the ClassifyAs() operation can be followed by any of these keywords:

• At()
• Attendant()
• CategorizeAs()
• ClassifyAs()
• EndingAt()
• Lasting()
• OrganizedBy()
• RelatedTo()
• Resource()
• Resources()
• StartingAt()
• WithPriority()
• Recurring
• ShownAsBusy
• ShownAsFreeTime

In other words the method returns an EventDescriptor instance. To allow ClassifyAs (or any of the other keywords in the above list) to be a terminating keyword we create an implicit cast operator on the EventDescriptor type allowing to to be casted to an EventComponent.

public static implicit operator EventComponent(EventDescriptor descriptor)
{
    return CreateEvent(descriptor);
}

internal static EventComponent CreateEvent(EventDescriptor descriptor)
{
    EventComponent component = new EventComponent();
    component.Attendees = descriptor.attendees;
    component.Categories = descriptor.categories;
    component.Classfication = descriptor.classification;
    component.Created = DateTime.Now;
    component.Description = descriptor.description;
    component.TimeStamp = DateTime.Now;
    component.StartTime = descriptor.startTime;
    component.ExceptionRule = descriptor.exceptionRule;
    component.GeographicalPosition = descriptor.geographicalPosition;
    component.LastMod = DateTime.Now;
    component.Location = descriptor.location;
    component.Organizer = descriptor.organizer;
    component.Priority = descriptor.priority;
    component.RecurrenceRule = descriptor.RecurenceRule;
    component.Summary = descriptor.summary;
    component.Transparency = descriptor.timeTransparency;
    return component;
}

The ability to create custom casting operators is a huge benefit when you develop fluent interfaces with C#.

Astute readers might wonder why the creation of the actual EventComponent is delegated to a separate method. Before we get to that, we need to look at some other parts of the language.
Consider the following snippet:

Plan.Event("Summer vacation").
    StartingAt("2007.08.06").Lasting(3);

If every keyword had returned an EventDescriptor, this would have been a legal sentence in the language. How would we know how long the event lasted? It lasts for three, but what is three? To force the user to specify this the Lasting() operation returns a DurationDescriptor rather than an EventDescriptor. The DurationDescritor doesn't have an implicit cast operator that allows it to be casted to an EventComponent, so the snippet above won't compile.

public class DurationDescriptor
{
    private EventDescriptor eventDescriptor;
    private int units;
    internal DurationDescriptor(EventDescriptor descriptor)
    {
        this.eventDescriptor = descriptor;
    }
    internal int Units
    {
        get { return units; }
        set { units = value; }
    }

    public EventDescriptor Seconds
    {
        get
        {
            eventDescriptor.Length = new TimeSpan(0, 0, 0, units);
            return eventDescriptor;
        }
    }

    public EventDescriptor Minutes
    {
        get
        {
            eventDescriptor.Length = new TimeSpan(0, 0, units, 0);
            return eventDescriptor;
        }
    }

    public EventDescriptor Hours
    {
        get
        {
            eventDescriptor.Length = new TimeSpan(0, units, 0, 0);
            return eventDescriptor;
        }
    }
    // Other properties like Days and Weeks go here...
}

As you can see from the snippet above, well get our EventDescriptor back when we choose one of the properties on the DurationDecriptor. Another difference from plain old OO is that the getters on the DurationDescriptor change the state of the event descriptor. Again this diversion from the rules of OO-design allows us to make our grammar flow nicely.
Creating branches within our grammar using different descriptor objects like this is a good way of restricting the choices the user has, and it helps keep the language consistent. The downside is that we have to write many different descriptor classes when the scenario gets more complex than with the DurationDescriptor. A total of twenty six different descriptor classes are used to control the grammar and flow of the recurrence and exception DSL.
You might have noticed that a sentence could end with a partial recurrence or exception rule, this implies that some of these descriptors can be casted to EventComponents as well. The EventDescriptor exposed the CreateEvent so that any descriptor can create an EventComponent from its current description when this is needed.

Martin Fowler pays some attention to the price of fluency in his article on fluent interfaces. It is more difficult to write a fluent interface than a plain old API, and I agree with Martin that it takes quite a lot of though to get the grammar right. I've found that it is best to start with a test case and just write the "sentences" before starting to implement the actual language. Not only does this help you to come up with a good language, it also helps create all of the classes, methods and properties that are needed - at least as long as you're using ReSharper. An believe me, there will be a lot of code when you're done. I'm almost there with this DSL, and at the time of writing is consists of 58 classes not including the API and tests.