Photo by Darren Hester

If you’ve ever used HttpWebRequest in .NET you’ve probably thought to yourself: man, there’s got to be an easier way! Sure, for the most simplistic uses, you can always fall back on WebClient, but now you have two different models for Web requests which doesn’t improve your code’s maintability, nor your sanity. And forget about testing.

Plug, XUri and XDoc

Since MindTouch Dream–the foundation for MindTouch 2009–has a strong REST services focus, one of the central requirements was making http handling clean and painless. To this end Dream includes a class called Plug (think Socket) to wrap HttpWebRequest (et al.). Two additional pain points with doing web requests are building Uri’s without resorting to string parsing and concatenation and building and processing Xml for request and response payloads. Dream offers, respectively, XUri and XDoc, to assist Plug.

All three provide fluent interfaces for construction and manipulation. In addition, XUri and Plug are immutable, meaning that the return value of each fluent interface call returns a new instance with the requested modification without changing the original. Simply this means that you can do things like this:

XUri serviceBase = new XUri("http://foo.com/api");

XUri authUri = serviceBase.At("authenticate");

XUri profileUri = serviceBase.At("users",userId,"profile");

XDoc on the other hand does modify an underlying document, so a modification does alter the document. To create a new document, the original has to be explicitly cloned. The main benefit of XDoc is simple creation and querying of Xml:

XDoc doc = new XDoc("user")
  .Attr("id",123)
  .Attr("href","http://foo/api/users/123")
  .Elem("name","joe")
  .Start("groups")
    .Elem("group","admin")
    .Elem("group","contributor")
  .End();

// iterate over groups in doc using XPath
foreach(XDoc group in doc["groups"]) {
  Console.Writeline("group: {0}",group.AsText);
}

Fluently making WebRequests

Now that we can build Uris and documents and know how to examine request documents, the only thing left is making the request, which is as simple as:

XDoc users = Plug.New(usersUri)
  .With("limit",10)
  .WithHeader("X-Auth","asdfg")
  .Get()
  .AsDocument();

This is an example of synchronously retrieving the Uri userUri with a query argument of limit=10 and an authentication header of Xauth: asdfg (Dream also allows the same to be done asynchronously). Up to Get(), we’re constructing the Plug. After that we are actually dealing with a response object of type DreamMessage which contains both headers and body.  The AsDocument() method returns the DreamMessage body as an XDoc instance which we can then inspect using XPath.

And that’s all there is to web requests using Plug. The class contains a number of extra methods for doing GET, PUT, POST, DELETE, etc., synchronous and asynchronous calling conventions, header and query arg manipulation, etc. Even doing a application/x-www-form-urlencoded POST is as simple as stringing together the name value pairs:

formPlug.With("key1",value1").With("key2","Value").PostQuery();

Since  application/x-www-form-urlencoded is just a list of key/value pairs, similar to the Uri querystring, Plug provides PostQuery() as a method to convert the query args into a Post body and submit them. Of course, you can still use With() for query args and the regular Post() method with a manually specified POST body, if you need both query and post key/value pairs.

Testing your Service Consumer

Now that web requests are simple, you can concentrate on your application code, rather than having to worry about constructing and dealing with HttpWebRequest. But what if you want to make sure that your application code is correct without having to rely on the service you call?

Testing against a live web service is always problematic, especially if its not a service you control:

• You can’t control whether it’s up and responsive
• You may need some setup before you can make the call you want to test, such as authentication or creating a test resource, adding noise and unrelated sources of failure to your test
• When a failure occurs you don’t know for sure whether you did something wrong or if the service had a hicup
• Unless the service you are consuming offers a test server, you will be mixing test and live data
• If the service provides data to you that you didn’t create (such as an RSS feed), testing whether you can handle that data relies on it being available to you at test time

And the list goes on…

You could always write a wrapper around your service calls, and if you’ve been using HttpWebRequest you’ve probably already done this, both to factor its complexity into a simpler re-usable component and for the ability to mock it out. But taking that approach means you have to dummy down the ease and expressive power of Plug to a mockable interface. Mocking Plug itself isn’t really an option, but even if it was, you’d basically be re-implementing it to get the full api.

This complexity of mocking is commonly perceived as a problem of fluent interfaces. However, most fluent interfaces themselves are not business logic but an access abstraction to an underlying resource. In our case the underlying resource is an http request. So what we really wish to mock is the request, not Plug, allowing us to use Plug directly in our business logic without sacrificing testability.

AutoMockPlug

Dream contains a simple Arrange, Act and Assert mocking harness for Plug. It allows you to set up an interceptor for a base Uri, arrange expectations for calls, and assert that the expectations were met after the desired actions were taken.

Let’s assume we have some repository in our API to retrieve users from the REST service and the repository will automatically authenticate if a call fails as Unauthorized. A test of our repository’s auto-authenticate might look like this:

[Test]
public void GetUser_reauthenticates() {

First we set up  the AutoMockPlug expectations and responses, i.e. the calls we believe our code should make across the wires and what it expects to get back. This is done with the mock.Expect() fluent interface:

    // -- Arrange --
    AutoMockPlug mock = MockPlug.Register(baseUri);
    // expect a call to retrieve user xml, but respond with unauthorized
    mock.Expect().Verb("GET").Uri(baseUri.At("users", userName))
        .Response(new DreamMessage(DreamStatus.Unauthorized, null));
    // expect a call to authenticate for user/pass and return an auth token
    mock.Expect().Verb("POST").Uri(baseUri.At("authenticate"))
        .RequestDocument(new XDoc("authenticate")
            .Elem("user", userName)
            .Elem("password", password))
        .ResponseHeader("X-Auth", authToken);
    // expect another call to retrieve user xml, make sure the authtoken was passed
    // and return user doc (abbreviated here for display purposes)
    mock.Expect().Verb("GET").Uri(baseUri.At("users", userName))
        .RequestHeader("X-Auth",authToken)
        .Response(DreamMessage.Ok(new XDoc("user").Elem("username",userName)));

Next we call the function on our object that we want to test, which in turn should be making the expected Plug calls:

    // -- Act --
    IUser user = repository.GetUser(userName);

Finally, we assert the final state of our test to make sure everything ran correctly. Since Plug calls may be asynchronous, AutoMockPlug uses a reset event signaled method to wait for all expectations (which also has the effect that extra calls via Plug are caught as expectation violations as well):

    // -- Assert --
    Assert.IsTrue(mock.WaitAndVerify(TimeSpan.FromSeconds(1)),
        mock.VerificationFailure);
    Assert.AreEqual(userName,user.Name);
}

In general, AutoMockPlug is created by registering a base Uri for interception, after which all child Uris are routed to the mock Plug. It works on the basis of ordered call expectations, checking both that the call occurred and that it meets the specified criteria. Specifying criteria and return values can be done either by the fluent interface shown in the above example, or by providing a request delegate that provides more finegrained control over the expected behavior.

Consuming the web

There’s an ever growning number of RESTful and other web services available to be consumed by your application. For example, virtually anything you can do inside of MindTouch 2009, you can also access and manipulate using Http web requests. We use Plug extensively internally to communicate between our own services and external services and generally each service call with Plug comes down to a single line of code.

The great thing about Plug is that while it’s an internal piece of plumbing for inter-service communication, it works just as simply on its own for anyone who needs to deal with web resources. And with the addition of mocking, you also get the ability to easily verify that your logic is sound before ever hitting the wire.

Plug and everything it relies on are part of the mindtouch.dream.dll and is released under the Apache 2.0 license. You can find as part of your MindTouch 2009 install or can get separately from sourceforge.  For the fluent interface version of AutoMockPlug, you currently have to get trunk from svn (which does include built binaries). It will be part of the Dream 1.6.1 maintence release.