One of the projects that gets less mentions these days, but is still just as relevant to the ongoing development effort behind MindTouch Core/2009 is MindTouch Dream.  Those who have been following MindTouch’s development for a long time know that Dream is the foundation for our web-oriented architecture.  It’s a open-source framework (Apache 2.0 license) that makes building and interacting with RESTful web-services incredibly simple.  It also includes a portable REST micro-server that enables us to run Core/2009 as a standalone application, a Windows service, or as an IIS handler, either under Windows or Mono.  In short, it covers a lot of ground of what makes MindTouch Core/2009 such a unique and advanced product.

Ok, enough about buttering up Dream, but since it hasn’t had many mentions recently, it deserved a bit of attention.  What’s more important are the cool new features being added that have much broader appeal than just for web-oriented programming, though they certainly help a lot with it as well.  While this work is being done under the banner of Dream Profile 2–or Dream 2.0, the name is still in flux–the features described in this article will be part of Dream 1.7 “Garnet”, which will ship with MindTouch Core/2009 “Minneopa”.

The video below covers some of the new features, including the ElasticThreadPool class and its derived classes.  In it I explain at a high-level how the thread pool works and how multiple instances interact.  The video is less than 20 minutes long. I had to cut out about two minutes to make it fit within that time limit, otherwise Facebook would not allow uploading it.

Click here for the larger version of the video on Facebook.

Elastic Thread Pools

Anyone who has ever had to deal with the built-in .NET thread pool knows that it is an abomination.  It scales poorly, it lacks configurability, and has no provisions for managing concurrent load.  In short, it doesn’t cut it.  This is not news and the situation is going to get a whole lot better in .NET 4.0 (see video on Channel 9 and blog post by Eric Eilebrecht).  Unfortunately, that’s ways out and since Dream is cross-platform, we would still have to wait for Mono to follow suit.

Luckily, there was already a lot of published information about the improvements in .NET 4.0 and it turned out to be a fairly short exercise to create a custom implementation for Dream.  Granted, this implementation may not be as sophisticated, but it’s available today (trunk) and it’s available under the Apache 2.0 open-source license.  So you can borrow and improve upon it at will.

The ElasticThreadPool class in Dream is built using work-stealing queues to prevent threads from starving while others drown in work.  An ElasticThreadPool instance can be configured to have a minimum number of reserved threads and a maximum number of parallel threads. This makes it easier to write applications that need to control the parallel fan-out when generating lots of work-items.  Furthermore, all instances draw their threads from a common pool.  Meaning, threads can be migrate between thread pools without incurring creation-destruction overhead.  Lastly, most of the logic in the ElasticThreadPool is lock-free.  Adding and removing threads from the thread pool is the only place where a lock is used.  As a result, it scales up very nicely as the number of CPUs increase.  Also, stealing work-items across threads will never block progress on another thread.  A crucial property for a good thread pool.  There is also an ElasticPriorityThreadPool that allows assigning priorities to work-items so the can get executed in a different order than they were scheduled.

Lock-Free Data Structures

Part of what makes the new thread pool in Dream so efficient is the use of lock-free data structures.  Dream includes four classes in this category: LockFreeStack<T>, LockFreeQueue<T>, LockFreePriorityQueue<T>, and LockFreeWorkStealingDeque<T>.

I’m particularly happy with our choice of implementation of the work-stealing deque class.  It’s entirely lock-free and has a great low-overhead memory profile.  It’s based on a paper entitled “A dynamic-sized nonblocking work stealing deque” written by Danny Hendler, Yossi Lev, Mark Moir, and Nir Shavit.

We hope to add a lock-free hashtable and double-linked list by year end, but these require fairly complex algorithms and we don’t have use cases that necessitate them currently.  No work has been on either of them apart from selecting relevant research papers.  If solving these kinds of hairy problems appeals to you, let me know and I’ll share what we have so far.

Other Changes Ahead

In this post, I’ve covered two big topics that I’m really excited about, but that’s not all that’s coming down the pipes for Dream Profile 2.  As always, we’re hard at work at improving the performance, lowering the memory requirements, and adding new features that have broad appeal.  As part of this effort, we’re also cleaning up some of mistakes from the past by marking classes and methods as obsolete.  Your applications and services will still compile and run until the official switch to Profile 2, but I recommend being proactive in abandoning the old patterns in favor of the new ones.  All functionality that is being removed has alternative means to accomplish the same goal.  Let us know if you need any help in the transition or if you recommendation on how to make it as painless a possible.