In my previous post, "Tasked to get Results", I covered the use of DReAM's Result and the TPL's Task as waithandles for asynchronously completed work. Except that all examples dealt with blocking behavior to await the completion of the other task, which made the async illustration rather academic.

Blocking for async work does have its applications. The canonical example being a controlling thread firing off a number of workers and waiting for all of them to complete to gather the results. For implementing this example, Task and Result can be thought of as convenient alternatives to WaitHandle.WaitAll() or Thread.Join() or EndInvoke() calls:

// Summing up Results
var results = new List>();
for(var i = 0; i < 100; i++) {
    results.Add(SomeAsyncWork(i));
}
var total = results.Sum(x => x.Wait());

// Summing up Tasks
var tasks = new List>();
for(var i = 0; i < 100; i++) {
    tasks.Add(SomeMoreAsyncWork(i));
}
var total = tasks.Sum(x => x.Result);

Both rely on implicit blocking behavior to perform the summation. We could just as easily have explicitly waited on all the handles with C#5.0 AsyncCtp's TaskEx.WhenAll() or DReAM's Result.Join(), and then summed the results. One interesting feature to keep in mind about the join constructs in TPL and DReAM is that both return synchronization handles themselves. I'll explain why this is useful in the next article. Either way though, our main thread was blocked until the workers completed.

Finding opportunities for asynchrony

The parallel worker example is a great way to show the power of Task and Result for managing many workers, but in itself is not asynchronous. Asynchrony implies that we do not block waiting for work to complete, whereas the above example explicilty blocks the main thread. Asynchronous behavior may be combined with parallelism to manage resources better, but asynchronous behavior does not mean that we are performing the work in parallel. Quite the opposite, the primary goal of asynchrony is to suspend the caller until the asynchronous operation has completed. This means that most asynchronous workflows are inherently serial even if the steps executed may happen on different threads.

To properly show the benefits of asynchrony let's use something that we deal with every day and is inherently asynchronous: I/O – Whenver we are reading from or writing to a file, making a database call or calling a webservice we are making an I/O request and waiting for the response. We don't think of I/O as asynchronous because most I/O APIs have exposed these operations as synchronous method calls, i.e. they block the thread while waiting for the out-of-context operation to complete:

var command = new SqlCommand(
    "SELECT CategoryID, CategoryName FROM Categories;",
    connection);

// Blocking!
connection.Open();

//Blocking!
var reader = command.ExecuteReader();

// Blocking!
while(reader.Read()) {
    // read row
}
reader.Close();

Most of time, we don't even interact with a DB at this low level, but that doesn't change the fact that in order to get data back from the database, we have to:

1. Open a socket to the DB and wait for the connection to be established
2. Send our SQL and wait to for a cursor to become readable
3. Iterate over the cursor and wait for data to be sent

Each operation sends data over the wire and waits on a response. That waiting time is blocking the current thread.

Why so slow, I/O?

Most I/O operations are quite fast, and much of our optimization work relates to reducing the times of database queries, file reads and web service calls to the single digit milliseconds. However, compared to in-process memory access, which is measured in microseconds, even the best optimized I/O operation is orders of magnitude slower. Every time we treat I/O as synchronous we rely on thread scheduling to utilize the CPU time not utilized by the blocking thread, which in turn costs us in thread context switching and memory footprint.

But we've been doing this for ages and it hasn't been a problem, right? What's changed?

For one thing, it has actually been a problem and for the most part, we've just been throwing larger and larger machines at our server clusters to make up for the resource wastes committed. But all the optimization of I/O aside, in the real world we will always encounter the occasional slow DB query, file read/write blocked by a lock and web service request over a congested network.

For another, on top of those real world concerns, we've received a wake-up call in the form of evented I/O: Suddenly node.js and python's tornado web server have shown us that a single server could handle thousands of simultaneous connections, which would make traditional servers fall over and die. This incredible capacity is not due to some magic optimization or that python and javascript have secretly gotten faster than other languages. What they are doing is deal with I/O as non-blocking. It is easy to keep thousand's of sockets open at the same time when each isn't tied to a process or thread. When node.js boasts 100k+ connections at once, it still is only processing one request at a time, but every time a I/O operation is required, it schedules a callback for completion rather than blocking until the I/O has completed. That this one at a time model ends up with more requests/second shows how much overhead we incur when we rely on thread management. But there's nothing about node.js performance that couldn't be replicated in C# (and Manos de Mono is doing exactly that), as long as we break our addiction to blocking.

…and after you are done with that, could you do this?

Using an asynchronous approach provides a way to mitigate this in-process vs. I/O operation speed differential and stop blocking our threads. What we want is to be notified when the asynchronous operation has completed. While most I/O APIs provide some type of asynchronous calling convention (ADO.NET only provides it on the Execute* members), Task and Result provide a unifying interface for all operations and a way to handle the continuation of work in a non-blocking form in a much simpler way:

// Result
connection.Open();
Async.From(command.BeginExecuteReader,
           command.EndExecuteReader,
           null,
           new Result())
    .WhenDone(
        result => {
            var reader = result.Value;
            while(reader.Read()) {
                // read row
            }
            reader.Close();
        }
    );

// Task
connection.Open();
Task.Factory.FromAsync(command.BeginExecuteReader,
                                      command.EndExecuteReader,null)
    .ContinueWith(
        task => {
            var reader = task.Result;
            while(reader.Read()) {
                // read row
            }
            reader.Close();
        }
    );

The above introduces the continuation constructs Result.WhenDone and Task.ContinueWith, which allow us to chain operations to execute in the context of the asynchronous method's callback. Both Result and Task provide a way to convert the standard Begin*/End* pattern from any API into Result/Task, which we can then attach a continuation to. I should note that in the C#5.0 AsyncCtp Microsoft has added Extension Methods for virtually all Microsoft async patterns to further simplify invocation.

Next time, we'll take a look at a more complex workflow using the continuation passing style possible with Result.WhenDone and Task.ContinueWith and what complications it may introduce.