Archive for the ‘Tutorial’ Category
DekiScript Page Profiler
One of the exciting new features in the next major release (code-named Pipestone), is the DekiScript Page Profiler. I’m confident this new feature will delight many DekiScript developers.
In the past, it has been a hassle to figure what part of a complicated, dynamically rendered page requires the most attention. Often, the only method was to comment out sections one by one to determine the offending line or page inclusion. This is now a thing of the past. The page profiler provides a detailed view of every single DekiScript function called and every page included. It’s your backstage pass to see exactly what’s going on!
The profiler output is split into 4 summaries:
- Rendered Content lists all the pages that were rendered and what functions were invoked on each page. Function invocations are grouped together by location to show what each line is doing. Additionally, each function includes an invocation count and total elapsed time.
- Function Summary lists all functions invoked across all pages with their total elapsed time, total invocation count, average time, and max time.
- Page Summary lists all pages with their total rendering time, inclusion count, average time, and max time.
- Data Access Summary lists how many database queries were issued and the performance of the optional cache.
Obtaining the page profiler report requires directly accessing the API, a task that shouldn’t be a hurdleto DekiScript developers.
Here is how you get your page profiler XML document:
- Identify the page ID of the page you want to profile. The easiest is to look at the page source and look for the line that says:
Deki.PageId = 123; - The page meta-data for any page can be accessed by its ID:
http://mysite/@api/deki/pages/123 - Now append
/contents/explainand you’ll get the page profiler report:
http://mysite/@api/deki/pages/123/contents/explain
Depending on how many operations your page performs, the returned XML can become fairly large, but that’s because it contains lots of details. I would recommend looking at the function and page summaries first to get a sense for the potential culprits and use that information to track down the actual call that is causing the slowdown.
Finally, if your site your users report from sporadic slowdowns that you can’t reproduce, you can add this configuration key to have the profiler log its output when a page takes more than 3 seconds to render:
stats/slow-page-render-alert = 3.0
This will make it easier to capture those elusive moments where for some reason a page takes unusually long to render.
And remember when optimizing your pages: the fastest code is the one that never needs to run!
Behind the MindTouch OpenID add-on
First, an introduction. Hello, everybody! I’m Craig. I’ve been a member of the MindTouch community since February 2008, when I was living in Canada and looking for a collaboration tool running on C# for a Windows software shop I was working at. MindTouch actually ran C# on Mono on Linux, which suited me fine, as I am a dual-purpose Windows and Linux guy, and it was good to have a reason to have Debian around at work again! I had a couple of opportunities to travel to southern California for work, and met the MindTouch gang; there are some great ‘enginerds’ working there, and a Christmas card from Roy & friends is a site to behold.
While I’ve since changed jobs (and countries), I’m still a very active member of the MindTouch community, and still enjoy contributing to the platform. Over last Christmas, I built an add-on for MindTouch to support OpenID, with an view to using it for an application in my new role at the Symbian Foundation. While we didn’t end up finding an opportunity to use MindTouch at Symbian, as an open source foundation we are happy to contribute work like this to to the greater community.
First, some terminology. An OP is an OpenID Provider: a site which provides an authentication servicew using the OpenID protocol. This is Site A in my other post. Your MindTouch site, Site B becomes an RP, also known as a Relying Party, formerly known as a consumer.
Anatomy of a MindTouch add-on
You could implement an OpenID extension entirely in the front-end (using PHP) if you wished, and an earlier attempt at OpenID support worked exactly this way. However, the service approach was more in line with the MindTouch model, and I like C# more than I like PHP!
Back-end service
It was a foregone conclusion that I should use DotNetOpenAuth as the engine for this project. It is the OpenID library that powers StackOverflow, and one of the primary authors, Andrew Arnott, is very active in supporting the library. Unfortunately, the 3.x version uses some new whizz-bang functionality in .NET which means it doesn’t work on Mono; rather than try and randomly disable code in a security library, I’ve stayed with using the 2.x series, from when it was known as DotNetOpenId.
The back-end only has two methods, both of which are called solely by the front-end:
- authenticate: given an OpenID string, discover the endpoint for that string, and build a URL to redirect the user to; or throw an exception if that endpoint is invalid or unreachable.
- validate: when given a signed response from an OP, check that it is correct, and return the details provided (full name, e-mail address etc), or throw an exception if it is somehow invalid.
Using trusted authentication
MindTouch’s trusted authentication is what makes this all so easy. A call to the API (with the site’s secret API key) will let you create a user of your choice; this means we can use front-end code to create users. If the user passes an API call and has a good OpenID, we can create a MindTouch user for them.
Front-end extension
We need to inject ourselves into the login process, so that the user knows that OpenID is an option; after that, there needs to be a page to display the OpenID selector, and handle the interaction with the user. That calls for a special page, using hooks to insert a link to our page on the UserLogin page.
The documentation on PHP special pages is a bit sparse, but you can gleam all you need to know from the examples, and looking at the existing pages. Ultimately, they’re just PHP, with handy functions for displaying the page chrome.
Following suggestions from Max and Guerric, I settled on this approach:
- A user browses to /Special:OpenIDLogin and gets the endpoint selector.
- User selects or enters their endpoint and hits Submit.
- The selected ID is POSTed to /Special:OpenIDLogin.
- If the /Special:OpenIDLogin is called as a POST, it sends a secondary POST to the OpenID API service, calling the ‘authenticate’ method. This validates the endpoint, and responds with either an error message, or a URI to redirect the user to.
- If the response was an error, it is displayed to the user; if it is good, the front-end redirects the user to the OpenID provider with /Special:OpenIDLogin as the return URI.
- The user logs in to their OpenID provider, or authorise your site to use your OpenID, and is redirected to /Special:OpenIDLogin again with signed OpenID parameters.
- When /Special:OpenIDLogin is called with signed OpenID parameters, via POST or GET (both are supported by the OpenID spec), the page then POSTs the query parameters to the OpenID API service, calling the ‘validate’ method:
- If the response was an error or failure, it is displayed to the user and the process ends.
- The front-end queries the site properties store to see if your OpenID is associated with a MindTouch user already.
- If your OpenID is found, the user has logged in previously, and we have stored their username; log that username in using trusted authentication, and return to the page they wanted to see.
- If no property is found, they are redirected to Special:OpenIDCreateUser, with a hashed secret. On that page, they are asked to enter a username (which is validated by AJAX using jQuery, to indicate if the username is acceptable/in use). That username is stored in the property against their OpenID, and they are then logged in.
All that boils down to three pages:
- special_userlogin_openid, which injects “Click here to log in with OpenID” to the main login page
- special_openidlogin, the meat of the extension, which communicates with the API service
- special_openidcreateuser, which allows a user with a valid OpenID, but no MindTouch username, to create one.
The front-end makes use of another open source project; the jQuery OpenID selector by Reflejo.
Pitfalls and abandoned paths
I wanted to be able to use an OpenID URL as a username, but that introduces two problems: first, you may already have a user account; second, MindTouch doesn’t take nicely to usernames containing colons and slashes, especially as everyone gets a User: page. It was tried but ultimately abandoned.
Future additions
A wealth of possibilities for adding to this extension exist: here are but a few.
- When you’re asked to create a new account, you should be able to pick an existing account to associate with, by username and password.
- Conversely, you should be able to add OpenIDs, and verify yourself, from your user preferences page.
- The add-on allows you to restrict the OpenIDs that are accepted (to, say, only your Google Apps domain). If you know this, you should also be able to make the front-end skip asking you for a provider and just redirect you to one.
- Make a MindTouch installation an OpenID provider. Why not?
Getting the add-on
As this is an open source contribution to the platform released under the GPL, it’s available in source for you to improve. MindTouch Inc. have kindly integrated it into their build process so it’s available pre-built in the SVN tree. Thanks, guys!
To learn how to install this add-on into MindTouch, check out its page at the MindTouch App Catalog.
PHP documentation is go!
PHP: Powering more than just your blog
A new initiative for the MindTouch Community Portal is to flesh out the PHP documentation. For a long time, content has been scattered between user pages and deep hierarchies, but now PHP has found a home on the wiki, The PHP Documentation Home.
In the coming months, this section will start filling up with great tutorials and helpful reference documents. One of the newest installments is the first chapter in Accessing the API with PHP Plug. This tutorial covers the two different types of Plug classes available in PHP: DreamPlug and DekiPlug. Power users who have shied away from developing PHP scripts, to interact with the API, now have a great launching point.
Another new addition to the PHP documentation family is the PHP Hooks reference. Since the Lyons release, MindTouch’s UI has had a growing list of integration points available. Each of these points allows a developer to tie in new functionality or augment the existing processes.
PHP is one of the fastest ways to begin development on the MindTouch platform and, going forward, it is my hope that this documentation will enable other developers to build and share compelling MindTouch driven applications. Code on!
Building a Dashboard 101
Google chartsIn this post, I will go over how I approached building a dashboard for a Corporate Intranet. For those who have read some of my earlier posts, they will know that I have been working on a system to capture and display MindTouch download statistics.
As somebody who has programmed primarily for terminals and auto-graders, the process of creating a dashboard was a challenge. My approach was to first break down the dashboard into smaller, more manageable pieces. I have come up with these three steps:
Displaying the Data – Trying to dump massive data sets and expecting people to look through them is asking for trouble. It is usually much better to display the data through charts and pretty tables. MindTouch includes some features to help speed this process along such as Google Charts, Visifire Charts, and dynamic tables.
Finding the Dimensions – The purpose of this step is to figure out how many permutations of the data there are. By knowing this, it makes it easier to create a dashboard that does not overwhelm the user, but still contains all the desired information. A dimension is essentially an option that when altered will change the returned data. The simplest example of this is time. Using the project that I have been working on as an example, if I wanted to chart the number of downloads, I need to know when the data should start and when the data should stop. So in my case, time is a dimension that can be removed. This step is especially important for more complex data sets, taking a look at the download statistics:
3 charting choices X 3 editions X 3 os X 11 flavors X 4 platforms X 2 servers X 5 versions = 11880 different graphs!
Laying out the Information – After removing the extra dimensions from the data, what should be left is a base set of data that can serve as a foundation for the rest of the data’s permutations. This basic set of data will be representative of the possible data so it is a good placeholder for designing a layout to present the data. When designing the layout, it is important to remember that there will likely need to be a space left open for an option bar (I discuss this further below) and that everything should fit on one page without too much scrolling. The goal of the dashboard is to present a simplified and user friendly interface where the end user can efficiently and easily get the information they are looking for. If the data set just calls for more then one page because it is too long to fit on one page or it just makes more sense on separate page, MindTouch has some useful features to help merge multiple pages into a single one. Look into things such as tabs, accordions, and web.toggle.
Tabs used for multiple pages.
Adding Functionability – In this step, the goal is to add back the dimensions that were removed earlier. Now that the layout is done, we are going to add the ability for the end user to access the dimensions that were removed earlier. There are some fancy options out there, but for my project, I used a simple post form, but slicker menus are also available. MindTouch offers some excellent features for handling query parameters with DekiScript (See DekiScript globals). I used these as parameters in the Download Statistics extension which allowed me to, through the use of a simple form, manipulate which dimension in the data set was returned.
Using these steps, I created a dashboard which harness the power of the Download Statistics extension without burdening the end user with memorizing method calls. The final product is clean, simple, and most importantly, presents the multitude of data in small, digestible doses.
The final product:
Download Statistics Dashboard
