10.1.3 is here!   This minor release includes various bug fixes and a few new features.  Please see the list below for some notable fixes: 

• Improved editor load time
• Corrected an issue when trying to navigate cursor within search field
• Corrected an issue causing page view count to incorrectly increment (his addresses future increments, though is not retroactive).
• Corrected an issue causing links to fail when referencing multiple tags that have define pages.
• Addressed a rare issue where performing an advanced search would redirect the user to the home page.
• Resolved a cosmetic issue where a user's configuration within the Control Panel was not displaying group membership.

New Features

In addition, 10.1.3 adds the following new features!

• Support for HTML5 content elements
• Pagination for @api/deki/pages

To upgrade* your installation, please visit our upgrade documentation for further details.  In addition, you can download the following packages:

• prepackaged Debian VM
• Windows Server 2003  MSI
• Windows Server 2008 MSI

*IMPORTANT:
Within Linux environments, upgrading to MindTouch 10.1+ from any previous version will require a Mono upgrade to Mono 2.10.2.  Failure to do so will result in the API crashing; this latest version of MindTouch does require Mono 2.10.2.  Please refer to the upgrade documentation referenced above for further details on upgrading your Mono installation within Linux environments.

Are you excited?  We are.  10.1 is coming soon, and we will update you again when it has been released.

In the meantime, there is one important note with 10.1. On Linux installations, an upgrade of Mono 2.10.1 is required (note that Mono 2.10.2 is compatible as well). Mono 2.10.1 is backwards compatible with MindTouch 10.x, so in preparation for our upcoming release you can get a head start by upgrading your Mono version.  That way, you’ll be all ready for the latest and greatest from MindTouch on launch day.

To upgrade Mono on CentOS/RHEL simply follow this documentation.

To upgrade Mono on Debian/Ubuntu (use this for your pre-packaged MindTouch VM) simply follow this documentation.

…stay tuned for more details on MindTouch 10.1.

In my previous post, "Tasked to get Results", I covered the use of DReAM's Result and the TPL's Task as waithandles for asynchronously completed work. Except that all examples dealt with blocking behavior to await the completion of the other task, which made the async illustration rather academic.

Blocking for async work does have its applications. The canonical example being a controlling thread firing off a number of workers and waiting for all of them to complete to gather the results. For implementing this example, Task and Result can be thought of as convenient alternatives to WaitHandle.WaitAll() or Thread.Join() or EndInvoke() calls:

// Summing up Results
var results = new List>();
for(var i = 0; i < 100; i++) {
    results.Add(SomeAsyncWork(i));
}
var total = results.Sum(x => x.Wait());

// Summing up Tasks
var tasks = new List>();
for(var i = 0; i < 100; i++) {
    tasks.Add(SomeMoreAsyncWork(i));
}
var total = tasks.Sum(x => x.Result);

Both rely on implicit blocking behavior to perform the summation. We could just as easily have explicitly waited on all the handles with C#5.0 AsyncCtp's TaskEx.WhenAll() or DReAM's Result.Join(), and then summed the results. One interesting feature to keep in mind about the join constructs in TPL and DReAM is that both return synchronization handles themselves. I'll explain why this is useful in the next article. Either way though, our main thread was blocked until the workers completed.

Finding opportunities for asynchrony

The parallel worker example is a great way to show the power of Task and Result for managing many workers, but in itself is not asynchronous. Asynchrony implies that we do not block waiting for work to complete, whereas the above example explicilty blocks the main thread. Asynchronous behavior may be combined with parallelism to manage resources better, but asynchronous behavior does not mean that we are performing the work in parallel. Quite the opposite, the primary goal of asynchrony is to suspend the caller until the asynchronous operation has completed. This means that most asynchronous workflows are inherently serial even if the steps executed may happen on different threads.

To properly show the benefits of asynchrony let's use something that we deal with every day and is inherently asynchronous: I/O – Whenver we are reading from or writing to a file, making a database call or calling a webservice we are making an I/O request and waiting for the response. We don't think of I/O as asynchronous because most I/O APIs have exposed these operations as synchronous method calls, i.e. they block the thread while waiting for the out-of-context operation to complete:

var command = new SqlCommand(
    "SELECT CategoryID, CategoryName FROM Categories;",
    connection);

// Blocking!
connection.Open();

//Blocking!
var reader = command.ExecuteReader();

// Blocking!
while(reader.Read()) {
    // read row
}
reader.Close();

Most of time, we don't even interact with a DB at this low level, but that doesn't change the fact that in order to get data back from the database, we have to:

1. Open a socket to the DB and wait for the connection to be established
2. Send our SQL and wait to for a cursor to become readable
3. Iterate over the cursor and wait for data to be sent

Each operation sends data over the wire and waits on a response. That waiting time is blocking the current thread.

Why so slow, I/O?

Most I/O operations are quite fast, and much of our optimization work relates to reducing the times of database queries, file reads and web service calls to the single digit milliseconds. However, compared to in-process memory access, which is measured in microseconds, even the best optimized I/O operation is orders of magnitude slower. Every time we treat I/O as synchronous we rely on thread scheduling to utilize the CPU time not utilized by the blocking thread, which in turn costs us in thread context switching and memory footprint.

But we've been doing this for ages and it hasn't been a problem, right? What's changed?

For one thing, it has actually been a problem and for the most part, we've just been throwing larger and larger machines at our server clusters to make up for the resource wastes committed. But all the optimization of I/O aside, in the real world we will always encounter the occasional slow DB query, file read/write blocked by a lock and web service request over a congested network.

For another, on top of those real world concerns, we've received a wake-up call in the form of evented I/O: Suddenly node.js and python's tornado web server have shown us that a single server could handle thousands of simultaneous connections, which would make traditional servers fall over and die. This incredible capacity is not due to some magic optimization or that python and javascript have secretly gotten faster than other languages. What they are doing is deal with I/O as non-blocking. It is easy to keep thousand's of sockets open at the same time when each isn't tied to a process or thread. When node.js boasts 100k+ connections at once, it still is only processing one request at a time, but every time a I/O operation is required, it schedules a callback for completion rather than blocking until the I/O has completed. That this one at a time model ends up with more requests/second shows how much overhead we incur when we rely on thread management. But there's nothing about node.js performance that couldn't be replicated in C# (and Manos de Mono is doing exactly that), as long as we break our addiction to blocking.

…and after you are done with that, could you do this?

Using an asynchronous approach provides a way to mitigate this in-process vs. I/O operation speed differential and stop blocking our threads. What we want is to be notified when the asynchronous operation has completed. While most I/O APIs provide some type of asynchronous calling convention (ADO.NET only provides it on the Execute* members), Task and Result provide a unifying interface for all operations and a way to handle the continuation of work in a non-blocking form in a much simpler way:

// Result
connection.Open();
Async.From(command.BeginExecuteReader,
           command.EndExecuteReader,
           null,
           new Result())
    .WhenDone(
        result => {
            var reader = result.Value;
            while(reader.Read()) {
                // read row
            }
            reader.Close();
        }
    );

// Task
connection.Open();
Task.Factory.FromAsync(command.BeginExecuteReader,
                                      command.EndExecuteReader,null)
    .ContinueWith(
        task => {
            var reader = task.Result;
            while(reader.Read()) {
                // read row
            }
            reader.Close();
        }
    );

The above introduces the continuation constructs Result.WhenDone and Task.ContinueWith, which allow us to chain operations to execute in the context of the asynchronous method's callback. Both Result and Task provide a way to convert the standard Begin*/End* pattern from any API into Result/Task, which we can then attach a continuation to. I should note that in the C#5.0 AsyncCtp Microsoft has added Extension Methods for virtually all Microsoft async patterns to further simplify invocation.

Next time, we'll take a look at a more complex workflow using the continuation passing style possible with Result.WhenDone and Task.ContinueWith and what complications it may introduce.

Photo by Shereen M.

This week, at PDC 2010, Anders Hejlsberg introduced what is to become C# 5.0 and it’s major new feature: Making Async easy. The main facility for achieving this goal is the language keyword pair or async and await. Together they allow synchronous methods to be called in an asynchronous way without having to jump through all sorts of hoops.

This is very exciting stuff for us here at MindTouch, since we’ve built almost exactly that facility with our Asynchronous Method Pattern and Coroutine infrastructure in DReAM. But the one thing that has always bothered us what the additional manual code and sometimes unintuitive syntax required.

The Result and Task based asynchronous patterns

Steve Bjorg introduced Result<T> with DReAM on top of the .NET 2.0 and mono 1.1.16 back in 2006. You can find a quick overview of it here. I've also covered it extensively in my monoconf talk. It's is similar in purpose to the Task class in the Task Parallel Library introduced by Microsoft in .NET 4.0. Both of them are a kind of future, a handle for a value that may or may not have been produced yet. Both provide asynchrony patterns for easily calling a method that promises to produce a result eventually.

The MindTouch  Asynchronous Method Pattern (AMP) takes this form:

Result<T> MethodName(...input paramaters..., Result<T> result)

where the trailing Result<T> is used to set up conditions on the return value, such as Timeout.

The Task-based Asynchronous Pattern (TAP) looks like this:

Task<T> MethodNameAsync(...input paramaters...)

Both are easy enough to call in blocking form, but that defeats their purpose. Both could also be set up with continuation handles, which get executed once a value T is available, but that takes the return code out of the current flow. And as completions call other asynchronous methods, your code just wanders off the right-hand side of the screen. This is common problem with continuations and is both a aesthetic and readability problem, in that your logically linear (if non-blocking) execution flow becomes increasingly difficult to follow.

Simplifying async flow with MindTouch Coroutines

What would be really nice is being able to call an asynchronous method and write the code that deals with the result on the next line, while simultaneously having your code suspend itself, freeing up your thread while the asynchronous code completes.

One way to approach this is with coroutines, i.e. a method that can exit and resume multiple times, rather having only one entry point. A coroutine can yield control to the async method and resume after it completes. In .NET 2.0 the yield operator was added to C# for the iterator pattern. If you turn the iterator pattern upside down, you end up with a method that yields not iteration results but continuations to some engine executing the control flow, i.e. MindTouch Coroutines:

Result<XDoc> r;
yield return r = Download(uri, new Result<XDoc>);
var doc = r.Value;

This provided us with the ability to write methods in synchronous style that could call to potentially long running asynchronous processes but continue their linear flow without ever blocking the thread. For a more detailed look at how coroutines are implemented check this article. However, the cost of achieving this came with some syntactic artifacts.

As illustrated above, we can't declare and assign the result type T in the yield return, hence the pre-declaration of Result<XDoc> r and the trailing var doc = r.Value. Three lines instead of one. In C# 3.0, using Lambdas, we reduced this to two lines:

XDoc doc;
yield return doc = DownloadAndProcess(uri, new Result<XDoc>).Set(v => doc = v);

Another artifact is that a coroutine has to return an enumerator. We use  IYield, which Result implements and coroutines require the return type IEnumerator<IYield> and by convention use a Result<T> as their last argument. That means coroutines can yield AMPs but are not AMPs themselves. As I stated above, a coroutine is an iterator turned upside down, yielding control to some executing engine. This means that every AMP that uses a coroutine can be written using this convention:

Result<XDoc> DownloadAndProcess(XUri uri, Result<XDoc> result) {
  return Coroutine.Invoke(DownloadAndProcess_Coroutine, uri, result);
}
IEnumerator<IYield> DownloadAndProcess_Coroutine(XUri uri, Result<XDoc> result) {
  XDoc doc;
  yield return Plug.New(uri).Get(new Result<DreamMessage>)
    .Set(v => doc = v.ToDocument());
  // do some more async work
  result.Return(doc);
}

Note: I snuck in Plug here, which is our fluent interface wrapper around HttpWebRequest and implements the AMP, internally using the async interface of HttpWebRequest, i.e. we can yield a Plug.Get and it will resume execution once the request completes.

C# 5.0 async/wait continuation flow

With C# vNext, Microsoft introduces two new language keywords that basically let you create the same flow we provide via the AMP and Coroutines, but using Task and leaving the syntactic cost to the compiler instead. This means that the TAP can be used without having to define a custom method with a different signature to execute the async work flow:

async Task<XDoc> DownloadAndProcess(XUri uri) {
  var doc = await Plug.New(uri).Get();
  // do some more async work
  return doc;
}

The async keyword marks the method as one that the compiler needs to rewrite a sequence of continuations. The await keyword tells the compiler that a TAP method is being called and to squirrel the rest of the method body into a continuation that is executed once the Task<XDoc> has a result. Here the compiler takes on the work of letting you use a single method signature to call either as a regular method or as source of a continuation and rewrites a TAP call to yield execution flow and resume after completion. Using async/await certainly reduces ceremony and makes it all more readable. We wish we could have had that luxury when we embarked on this pattern 4 years ago. Note: The above code pre-supposes a version of Plug that implements the TAP instead of the AMP.

yield vs. await vs. something else

Shortly after Anders' PDC talk Eric Lippert posed the question whether await is the best name for this new facility in the post Asynchronous Programming in C# 5.0 part two: Whence await? on his blog. The comment thread is a good read of arguments for and against various keywords. Personally being partial to yield, I read Jon Skeet's case against it in his post C# 5 async and choosing terminology: why I'm against "yield" with interest. His point is that "[w]hen the action completes very quickly and synchronously, it isn't yielding at all." My personal feeling is that whether or not the call returns very quickly is an implementation detail invisible to the caller. The caller is yielding control to the callee, who makes the decision whether to return immediately or suspend the caller until completion.

However await does seem like the least appropriate of the options, since we're really not waiting at all. Syntax like continue after, continue with and continue when certainly seem more appropriate and descriptive of the action taking place. While yield return perfectly describes to me what happens, maybe that is a sign the terminology reveals the implementation detail rather than the intent. The caller doesn't care about the yielding of control. The caller just wants the result from the call and get on with its business. So from a clarity of code perspective, I have to side with the continue (after|with|when) camp.

When can I use async/await?

The CTP is available now. Of course, there is no release date, taking a gaming industry "when it's done!" stance. My bet is that async/await will be usable in the next mono release (2.10? 3.0?) before C# 5.0 ships. Any way you look at it, though, it's going to be a while before you can rely on it existing on a platform you didn't personally configure.

In the meantime, DReAM coroutines are battle tested, powering all REST services in the MindTouch product and any other product using DReAM, and you can be sure that as C#5 approaches we will be tracking its progress closely to evaluate using the TAP in DReAM. We certainly would love to take advantage of the simplified syntax, while staying compatible with our existing syntax.

If you write .NET code consumed by others, you are undoubtedly familiar with the inline XML documentation markup available. It’s a fairly simple embedded documentation syntax for which Visual Studio even provides some Intellisense and compile time checking. It keeps documentation and signature meta-data that can be derived from the code separate to avoid drift between code and documentation as much as possible.

Most developers initially start using it to provide Intellisense support for their code. But sooner or later most will want to publish the documentation as a reference document for their APIs. This would seem like the ideal use case of the mark-up format, but once you starting looking at the available tools, most developers find them lacking. Originally, NDoc provided the equivalent documentation generation that javadoc provides to developers on the Java side. Unfortunately NDoc was discontinued before .NET 2.0. In its place there exist Microsoft’s Sandcastle and mono’s monodoc.  And now mindtouch.doc has been added to that list.

The goal of mindtouch.doc was to generate hierarchical reference documentation that could easily be imported into an MindTouch install. Unlike the standalone documentation generated by other tools, documentation imported into MindTouch can be easily extended by the extensive authoring and curation capabilities of MindTouch, such as IDF. Instead of just statically generated reference documentation, documentation in MindTouch can be tagged, linked, mashed-up with all your other documentation, tutorials, etc., including the ability to involve your community in the further growth of your documentation.

Authoring

We created mindtouch.doc to generate documentation for the DReAM Framework to be hosted on developer.mindtouch.com. The first task was to document all public classes in the framework using the .NET Xml comment docs:

With this in place, Visual Studio generates an XML document containing the meta-data from those comments. Using mindtouch.doc.exe, the code Assemblies along with their Documentation XML were converted into MindTouch import archives and imported into the site to produce documentation like this:

You can take a look at the MindTouch DReAM reference documentation to explore the entire generated documentation.

In addition to generating individual pages for each class, separate pages with Constructor, Property, Method and Event details are generates as child pages, along with Table of Content pages by Namespace and Assembly.

mindtouch.doc can generate documentation for an unlimited number of Assemblies at once. When the code to be documented is analyzed any references to any any class or member mentioned in the same or another assembly generates links between the relevant pages.

Since all content is imported as pages into MindTouch, you also get full revision history of past documentation imports (if the content has changed), so that documentation of newly added classes just requires a fresh import on top of the existing documentation.

Importing .NET documentation

If you want to try out the process, but don’t have documented code handy, you can import the DReAM documentation into your own MindTouch instance with the following command:

mindtouch.doc.exe -h mydomain.com -I ref/Dream mindtouch.dream.dll mindtouch.core.dll

This will prompt you for your user and password and import the documentation in ref/Dream.

The generated documentation has an external dependency on a template called DocToc. The purpose of this template is to allow simple Table of content customization using DekiScript and to make it easy to include custom css for documentation pages. The generated documentation HTML uses minimal markup along with classes to provide the greatest customization flexibility. The version of DocToc we use on developer.mindtouch.com simply looks like this:

<table style="background-color:#F9F9F9; border:1px solid #AAAAAA; font-size:95%; padding:5px;">
  <tbody>
    <tr>
      <td>
        <div align="center"><strong>Table of Contents</strong></div>
        {{ page.toc }}
      </td>
    </tr>
 </tbody>
</table>

Shipping your documentation

mindtouch.doc allows to directly import the generated documentation or to save the generated output as a MindTouch archive (.mtarc) which you can provide to customers to import into their own MindTouch installs. This is a regular MindTouch archive, the same that you would use to package up other documentation, scripts and templates authored in MindTouch. It can be imported via the Desktop Adapter or mindtouch.import.exe.

Documentation generation tool details

Running mindtouch.doc.exe without any arguments produces the following help output:

MindTouch Documentation, Copyright (c) 2010 MindTouch Inc.

USAGE: mindtouch.doc.exe [options] [assembly1 .. assemblyN]

Options:
 (Options must be prefixed by a '-' or '/')

 General:
 ?|usage                  - display this message
 h|host <host>            - MindTouch host (assumes standard API location)
 u|uri <api-uri>          - specify the full uri to the API (instead of using <host>)
 I|importreltopath <path> - relative uri path for import
 importrelto <id>         - relative page id for import (alternative to importreltopath)
 o|output                 - output path for generated documentation (will skip import)
 R|retries                - Maximum number of retries on import/export item failures (default: 3)

 Authentication:
 (if no authentication is provided, the program will prompt for user and password interactively)
 A|authtoken <token>      - authtoken to use for user authentication
 U|user <username>        - username for authentication (requires password option
 P|password <password>    - password for username authentcation

The official documentation can be found on developer.mindtouch.com.

mindtouch.doc offers an easy way to take your documented .NET APIs and publish them with the power of MindTouch, no separate runtime or site required.

Photo Credit

I just got through an elaborate debug session trying to find a sneaky parsing problem with incoming Http POST requests that happened:

• intermittently (every 500-2000 iterations), and
• even in intermittent form was only reproducible (outside of the client production install) on a VM running on Corey’s laptop

Copying the VM to my machine or another VMware server would stop it from occuring at all, no matter how many iterations. At least this ruled out a bad binary, runtime version and environmental config issues.

I narrowed the problem down to occasional premature end of stream issues, i.e. less bytes were read than sent. The fun thing with debugging Streams is that by the time you find out a problem occurred, the debugger is useless. So I wrote a Stream wrapper that recorded each bytes as it came in.

This identified our TrimmingTextReader‘s Read() implementation as sometimes returning -1 before the end of the stream. The TrimmingTextReader is a wrapper around a normal TextReader that trims leading and trailing whitespace as it reads the stream without having to allocate the full string first.

Double-checking MSDN, i confirmed that TextReader.Read() should return “[t]he next character from the input stream, or -1 if no more characters are available“. Our implemenation of Read() had an optimization of using Peak() to check the next byte for whitespace and read it into our whitespace buffer, otherwise return the next character from the whitespace buffer instead:

// accumulate whitespace characters until we determine
// if we have reached the end of the reader or not
for( ch = _original.Peek();
     (ch >= 0) && char.IsWhiteSpace((char)ch);
     ch = _original.Peek()
) {
  _buffer.Append((char)_original.Read());
}

The documentation for Peak() states that it returns “[a]n integer representing the next character to be read, or -1 if no more characters are available“. Essentially the same wording, except further down there was this caveat: “The Peek method returns an integer value in order to determine whether the end of the file, or another error has occurred.” Get that? End of file, or another error! But if i just did Read() it happily returned the next character, so what was this error that Peak() seemed to be detecting?

StreamReader reads bytes from the underlying stream in chunks:

var read = int Read(int[] buffer, int offset, int count)

In the failed request scenario, the Stream.Read call had returned less read bytes than requested in count, a perfectly valid response. Only if read is 0 has the end of stream been reached. But Peek() does not trigger another read from the Stream, while Read() will. This means that, Peek() will report -1 if the next byte it looks at isn’t in current buffer, since it’s at an index larger than read.

So, aside from being ill-documented behavior of two seemingly analogous calls, this behavior is also incredibly useless: You can keep calling Peek() over and over and keep getting -1 and never know whether it’s really the end of the stream or just some error. Only a Read() will tell the difference between the two scenarios. Of course, this is simply a lesson in “don’t assume anything“, but I hope that my debugging session will help someone avoid the tedious path of discovery to this bit of knowledge.

Last week, on October 28th, Steve and I had the pleasure of presenting at the Monospace conference. I picked a topic close to the API Team’s heart, Concurrency, and specifically covered our use of the asynchronous method pattern and our coroutine framework. We’ve covered a lot of the motivation and reasoning for this a number of times on the Concurrent Podcast. I captured the talk as a screencast  because i think it’s a useful primer on concurrency in MindTouch 2009 and our Dream framework and its especially useful as a hands-on companion for Concurrent Podcast 3: Coroutines:

I’d like to thank Scott Bellware for putting on a great conference, as well as everyone from the Mono team and the Mono/.NET community who attended. Monospace was incredibly insightful and I met a lot of very smart people there. Let’s hope this becomes a regular conference.

Photo by ToniVC

Concurrent Podcast 03: Coroutines

One topic we’ve mentioned in both previous episodes is Coroutines. We use them extensively at MindTouch and the asynchronous programming model provided by the Dream framework fundamentally relies on them. Since we’re likely to keep bringing them up in the future, we thought it best to cover them sooner rather than later.

For some background reading on Coroutines in C# check out my previous article corporate blog or the same content as a tutorial on the developer site (better code formatting).

As usual, you can find future topic listed here, and we always welcome suggestions on what should take precedence or what other topics to cover. If you want to subscribe to just the podcasts, you can find the feed here.

Since we don’t have a new Concurrent Podcast ready, I thought I’d go into a little detail about the change away from IBlockingQueue I mentioned in Episode 1 of the Concurrent Podcast.

A little history

When I first wrote the dispatcher for the PubSubService in Dream, I was hoping to dispatch the work against the .NET Threadpool. Early testing showed that the standard threadpool was near useless for dispatching lots of small work items quickly. Load tests seemed to indicate that there was a lock contention scheduling new work that caused short lived workers to execute in a near sequential manner. I.e. a new worker wouldn’t get dispatched until the the previous one was done.

Another problem with using the standard threadpool for dispatch was that i couldn’t easily limit how many workers would be used in isolation from other users of the threadpool, which would lead to worker starvation.

Blocking on work

Rather than fight with the threadpool, I opted for a classic producer/consumer pattern in which all work was queued while dedicated worker threads would pull from the queue and dispatch the items. Since .NET queues are not threadsafe, I implemented a blocking queue:

    public interface IBlockingQueue<T> : IEnumerable<T> {
        bool TryDequeue(TimeSpan timeout, out T item);
        T Dequeue();
        void Enqueue(T data);
        void Close();
        bool IsClosed { get; }
        int Count { get; }
    }

I also made it IEnumerable, so that the worker thread could simply run the dispatch as a foreach loop. This way any thread can enqueue an item to process, while n dedicated worker threads can process the work like this:

   foreach(var data in queue) {
      // process data
   }

The thread automatically exits when someone closes the queue, dropping each processor out of its foreach loop.

This pattern works great if you have a steady flow of work and can determine the optimum number of worker threads to spawn. Unfortunately, many scenarios revolve around inconsistent producers of work, leading to alternately workers sitting idle and the queue backing up. Since each thread has a significant memory footprint having them sit idle isn’t desirable, and neither is work backing up because  you didn’t spawn enough workers.

PubSub is one of these scenarios where the flow of events is generally feast or famine. In addition, we use a number of chained pubsub services, leading to a number of dedicated threads sitting around. It was clear that further use of this pattern was only going to exasperate the problem.

Enter the ElasticThreadPool

With the existing .NET Threadpool being a known problem and the Work Stealing Threadpool of ParallelFX not arriving until C# 4.0, we had a need for our own threadpool to more efficiently dispatch asynchronous work in Dream. Steve set out to build a lock-free work stealing threadpool and after a couple of iterations arrived at the work dispatch abstraction, IDispatchQueue:

    public interface IDispatchQueue {
        void QueueWorkItem(VoidHandler callback);
        bool TryQueueWorkItem(VoidHandler callback);
    }

one of the implementors of which is ElasticThreadPool(int minReservedThreads, int maxParallelThreads), which takes the number of reserved (as in dedicated) and maximum threads to use for dispatch. An ElasticThreadPool(n,n) would function identically to an IBlockingQueue<VoidHandler> with n dedicated threads spawned to pull items from it. Where  the ElasticThreadPool gets interesting is when maxParallelThreads is larger than minReservedThreads — now the threadpool will get a thread from a global dispatch threadpool to grow or shrink the number of workers according to the amount of work to dispatch.

One of the drawbacks of the built-in threadpool is that it uses a single queue that all workers feed off, creating lock contention for acquiring work. Rather than using raw threads as the workers, the ElasticThreadPool uses a wrapper called DispatchThread, each of which has its own work queue, so that work can be queued on a worker directly and pulled without contention. In addition ElasticThreadPool employs a stealing algorithm, so that an idle worker can steal queued work from another worker. This is done via a lock-free double-ended queue, so that stealing does not re-introduce lock contention in work acquisition. The combination of these techniques means that the ElasticThreadPool is very efficient in dispatching work and dynamically scaling the number of workers required.

Dispatching work is an implementation detail

With IDispatchQueue Dream has abstracted the implementation of how processing of work items happens, so rolling your own consumer with something like IBlockingQueue is no longer required. Should the scenario of fixed dedicated threads with a single blocking queue still appears to be more advantageous, the dispatch interface makes it easy to create an implementation to use instead of the ElasticThreadPool. And while the best fit for most work queuing scenarios is generally the  ElasticThreadPool, Dream already includes several alternative IDispatchQueue implementations, including one that wraps the standard ThreadPool.

One of the goals of the Dream framework is to make parallel programming simpler. For this purpose it provides such helpers as Result<T> for coordination, guidance on asynchronous message dispatch, our coroutine framework, etc. We use the ElasticThreadPool, as the underlying threading model for all our own operations and believe that it is an efficient and simple tool for handling producer/consumer scenarios. Let us know if you find it useful or have suggestions for improvement.

Note: The interface IDocStore and its implementations are preview code. We’ve only started playing around with it for production use and will likely introduce changes before we ship any features of MindTouch depending on it. Your feedback is welcome and appreciated.

As part of Dream 1.7 (which is used by MindTouch 9.08), we’ve introduced a schema-less Xml storage inspired by Bret Taylor’s blog post “How FriendFeed uses MySQL to store schema-less data” about a schema-less storage system for JSON blobs used by FriendFeed. Since MindTouch is very much Xml centric and we like the richness of XPath, we decided to adapt the idea to using Xml as the document format.

Why “schema-less” and what does that mean?

In a traditional database, data is split into columns stored in rows in a table. For data more complex than a flat set, other tables are used and referenced from the the first table. That means that to add a new field to your record, you have to create a new column, an operation that can have significant performance implications for that table. In addition, if you choose optimize query performance, you may choose to index that new column, again affecting the the table’s performance during the creation of the index.

But what if you just serialized your record and stuffed that blob into a single column? I mean aside from having just given your DBA an ulcer… Well, you would have the data stored, but in terms of a relational database, you’ve just removed all benefits typically bestowed on a row. How are you going to get this data back out? Pulling each record out and examining it for a match is not a realistic option. Ok, you create an id for the object, and now you have a key/value store on top of an RDBMS… Still lost the ability to query on any other part of your record.

Chances are that you generally only need to retrieve records based on some other field in that record. Adding more columns and populating each for fields to be queried lands you right back where you started. How about creating a self-contained “index” by creating key/value pair tables, one for each field, with values in the rows being taken from your record and linking back to the main storage table. Since each of these index tables is a table of its own index, creation and maintenance no longer affects the main table. You can add an index and populate it without affecting the use of any other key or the data itself.

Introducing IDocStore

This is the storage model that IDocStore follows. The blob record format used is xml, allowing you to store simple fields, or complex hierarchies. Fields to index are defined by using XPath expressions, which makes it possible to not only index single values in the Xml document, but even lists of values. For example, given the following xml:

<user id="123" >
  <name>bob</name>
  <groups xmlns:g="http://foo/groups">
    <group>admin</group>
    <group>contributor</group>
  </groups>
</user>

You could define an index xpath of g:groups/g:group and be able to find bob with a query for either admin or contributor.

The interface of IDocStore is simply this:

public interface IDocStore {
  bool Put(XDoc doc, bool force);
  void Delete(string docId);
  XDoc Get(string docId);
  IList<XDoc> Get(string keyName, string keyValue);
}

We don’t differentiate between Create and Update, simply using Put. We also limit Get to querying on either the primary document id returning at most a single document, or an xpath index, which may return more than one document.

In it’s current incarnation, IDocStore requires the docId already exist in in the document at a configured XPath (default is @id). Lacking a natural key, a Guid could always be used to define the id. Another consideration is to add auto-key generation.

Dream contains two implementations of the IDocStore, MysqlDocStore and SqliteDocStore.

Sqlite Implementation

Starting with Dream 1.7.0 (used by MindTouch 9.08), every service is provided a DataCatalog backed by its own sqlite database (although this can be swapped out for any other database provider via configuration). This can be used as the backend for the SqliteDocStore.

The sqlite implementation, SqliteDocStore, is the simpler of the two. It does not yet include optimistic locking, meaning that Put will force an overwrite of existing data, regardless what force is set to. Also, since the DB file that backs the database cannot be shared by multiple processes, an instance of SqliteDocStore assumes that is the authoritative owner of the storage path, and therefore does index definition and maintenance at instantiation time:

IDocStore store = new SqliteDocStore(_catalog, new XDoc("config")
  .Elem("name", "user")
  .Elem("id-xpath","@id")
  .Start("namespaces")
    .Start("namespace").Attr("prefix","g").Attr("urn","http://foo/groups").End()
  .End()
  .Start("indicies")
    .Start("index").Attr("name","group").Attr("xpath","g:groups/g:group").End()
  .End());

This initialization will either use or create a new store named user in the specified _catalog. It will then perform indicies, adding/removing indicies so that only the index is group.

Mysql Implementation

The mysql implementation, MysqlDocStore,, by virtue of using a remote resource, removes index maintenance from the IDocStore implementation, and adds MysqlDocStoreManager for index manipulation, including updating an index. For this reason, an instance MysqlDocStore requires an instance of MysqlDocStoreManager (really, an instance of an IMysqlDocStoreIndexer) so that it can update the indicies when the documents change.

MysqlDocStore also implements optimistic locking, meaning that the store will attach a revision attribute to any document returned and use that revision to halt an update should the revision in the database be newer (or force is set to true on Put).

Bringing IDocStore to production use

Although an RDBMS is used as the underlying storage, the schema-less store is not ACID compliant. A request based on an index may not result in all the documents that match that index, since the index is maintained asynchronously from the update action. However, the result set it pruned to at least never return a false positive. This is analogous to authoring a page in MindTouch and waiting for the search index to pick up the changes. For most document storage usages, this should be fine.

Schema-less storage is useful when the data to be stored may have unknown or frequently changing structure and simple access patterns, i.e. primary or single key queries, are sufficient. Since MindTouch services often deal with this type of storage and complex DekiScript templates could also benefit from arbitrary storage, we’re experimenting with this as backing store for future extensions, and maybe even expose the storage as a service via an extension so that DekiScript could read from it and JEM could write to it.

If you have use cases for this storage facility, we’d love to hear from you as we test out and mature the design for production. The code is available for preview purposes now in the trunk and 1.7 branches on our source control server, and is part of the binary distribution going out with mindtouch.dream.dll in MindTouch 9.08