Archive for the ‘Documentation’ Category
DekiScript Page Profiler
One of the exciting new features in the next major release (code-named Pipestone), is the DekiScript Page Profiler. I’m confident this new feature will delight many DekiScript developers.
In the past, it has been a hassle to figure what part of a complicated, dynamically rendered page requires the most attention. Often, the only method was to comment out sections one by one to determine the offending line or page inclusion. This is now a thing of the past. The page profiler provides a detailed view of every single DekiScript function called and every page included. It’s your backstage pass to see exactly what’s going on!
The profiler output is split into 4 summaries:
- Rendered Content lists all the pages that were rendered and what functions were invoked on each page. Function invocations are grouped together by location to show what each line is doing. Additionally, each function includes an invocation count and total elapsed time.
- Function Summary lists all functions invoked across all pages with their total elapsed time, total invocation count, average time, and max time.
- Page Summary lists all pages with their total rendering time, inclusion count, average time, and max time.
- Data Access Summary lists how many database queries were issued and the performance of the optional cache.
Obtaining the page profiler report requires directly accessing the API, a task that shouldn’t be a hurdleto DekiScript developers.
Here is how you get your page profiler XML document:
- Identify the page ID of the page you want to profile. The easiest is to look at the page source and look for the line that says:
Deki.PageId = 123; - The page meta-data for any page can be accessed by its ID:
http://mysite/@api/deki/pages/123 - Now append
/contents/explainand you’ll get the page profiler report:
http://mysite/@api/deki/pages/123/contents/explain
Depending on how many operations your page performs, the returned XML can become fairly large, but that’s because it contains lots of details. I would recommend looking at the function and page summaries first to get a sense for the potential culprits and use that information to track down the actual call that is causing the slowdown.
Finally, if your site your users report from sporadic slowdowns that you can’t reproduce, you can add this configuration key to have the profiler log its output when a page takes more than 3 seconds to render:
stats/slow-page-render-alert = 3.0
This will make it easier to capture those elusive moments where for some reason a page takes unusually long to render.
And remember when optimizing your pages: the fastest code is the one that never needs to run!
New Bug Tracker
Over the past week, BrianH has been busy with migrating the issues, bugs, and tasks from our existing Mantis bug tracker to the new YouTrack installation. All of the open tickets have been migrated and even their issue numbers have remained the same. In short, the migration was a success and BrianH will follow-up with a technical post on how he did it.
If you ever filed a bug or commented on one, you should have an account already created for you. However, you will need to reset your password. Just head over to the YouTrack login page and request the reset email.
Most of you suspect you’ve never heard of YouTrack and may wonder why we decided to switch to it. When I looked at all available options—and I looked at a lot of options—I realized what we needed the most was a tool that would make it super easy to cut through the data. YouTrack’s query language is amazingly powerful and is consistently available for searching, reporting, and creating release notes. It’s simply well thought out.
Aside from the query language, YouTrack has a convenient keyboard-friendly design that enables power users to navigate through search results quickly and update, annotate, tag, and comment all at the same time with minimal keystrokes. In short, it’s really efficient to work with.
Next, YouTrack also provides a voting feature. Yes, I said voting. Finally, you can participate in prioritizing what issues need the most urgent fixing. Please take advantage of this as it really helps us figuring out what you think is in need of immediate attention. To vote, you’ll just need to create an account. That’s it.
As with Mantis, YouTrack also has commenting and watching issues, but it works better by sending immediate emails when status changes happen or new comments are added. A big plus when we need to get more information and there is no other means to communicate with the reporter of the issue. In Mantis, we had to close quite a few bugs, because those who reported them never checked back in to read our questions. That was unfortunate.
Lastly, YouTrack has a very handy issue reporting bookmarklet (accessible from the issue submission form). You can drag it into your browser bar. When you need to file an issue for your installation or on developer.mindtouch.com, you click it and it immediately opens an overlay window, which allows you to file the issue without losing context. Both the bookmarklet and issue submission form also make it easy to attach a screenshot thanks to a bundled java applet.
My hope is that YouTrack will enable much better communication and management of submitted issues. We’ll be porting our Mantis DekiScript Templates next week, which should be the last piece in the migration.
Hope you enjoy our new bug tracker and keep filing away!
Generate .NET documentation with MindTouch 2010
If you write .NET code consumed by others, you are undoubtedly familiar with the inline XML documentation markup available. It’s a fairly simple embedded documentation syntax for which Visual Studio even provides some Intellisense and compile time checking. It keeps documentation and signature meta-data that can be derived from the code separate to avoid drift between code and documentation as much as possible.
Most developers initially start using it to provide Intellisense support for their code. But sooner or later most will want to publish the documentation as a reference document for their APIs. This would seem like the ideal use case of the mark-up format, but once you starting looking at the available tools, most developers find them lacking. Originally, NDoc provided the equivalent documentation generation that javadoc provides to developers on the Java side. Unfortunately NDoc was discontinued before .NET 2.0. In its place there exist Microsoft’s Sandcastle and mono’s monodoc. And now mindtouch.doc has been added to that list.
The goal of mindtouch.doc was to generate hierarchical reference documentation that could easily be imported into an MindTouch install. Unlike the standalone documentation generated by other tools, documentation imported into MindTouch can be easily extended by the extensive authoring and curation capabilities of MindTouch, such as IDF. Instead of just statically generated reference documentation, documentation in MindTouch can be tagged, linked, mashed-up with all your other documentation, tutorials, etc., including the ability to involve your community in the further growth of your documentation.
Authoring
We created mindtouch.doc to generate documentation for the DReAM Framework to be hosted on developer.mindtouch.com. The first task was to document all public classes in the framework using the .NET Xml comment docs:
With this in place, Visual Studio generates an XML document containing the meta-data from those comments. Using mindtouch.doc.exe, the code Assemblies along with their Documentation XML were converted into MindTouch import archives and imported into the site to produce documentation like this:
You can take a look at the MindTouch DReAM reference documentation to explore the entire generated documentation.
In addition to generating individual pages for each class, separate pages with Constructor, Property, Method and Event details are generates as child pages, along with Table of Content pages by Namespace and Assembly.
mindtouch.doc can generate documentation for an unlimited number of Assemblies at once. When the code to be documented is analyzed any references to any any class or member mentioned in the same or another assembly generates links between the relevant pages.
Since all content is imported as pages into MindTouch, you also get full revision history of past documentation imports (if the content has changed), so that documentation of newly added classes just requires a fresh import on top of the existing documentation.
Importing .NET documentation
If you want to try out the process, but don’t have documented code handy, you can import the DReAM documentation into your own MindTouch instance with the following command:
mindtouch.doc.exe -h mydomain.com -I ref/Dream mindtouch.dream.dll mindtouch.core.dll
This will prompt you for your user and password and import the documentation in ref/Dream.
The generated documentation has an external dependency on a template called DocToc. The purpose of this template is to allow simple Table of content customization using DekiScript and to make it easy to include custom css for documentation pages. The generated documentation HTML uses minimal markup along with classes to provide the greatest customization flexibility. The version of DocToc we use on developer.mindtouch.com simply looks like this:
<table style="background-color:#F9F9F9; border:1px solid #AAAAAA; font-size:95%; padding:5px;">
<tbody>
<tr>
<td>
<div align="center"><strong>Table of Contents</strong></div>
{{ page.toc }}
</td>
</tr>
</tbody>
</table>
Shipping your documentation
mindtouch.doc allows to directly import the generated documentation or to save the generated output as a MindTouch archive (.mtarc) which you can provide to customers to import into their own MindTouch installs. This is a regular MindTouch archive, the same that you would use to package up other documentation, scripts and templates authored in MindTouch. It can be imported via the Desktop Adapter or mindtouch.import.exe.
Documentation generation tool details
Running mindtouch.doc.exe without any arguments produces the following help output:
MindTouch Documentation, Copyright (c) 2010 MindTouch Inc. USAGE: mindtouch.doc.exe [options] [assembly1 .. assemblyN] Options: (Options must be prefixed by a '-' or '/') General: ?|usage - display this message h|host <host> - MindTouch host (assumes standard API location) u|uri <api-uri> - specify the full uri to the API (instead of using <host>) I|importreltopath <path> - relative uri path for import importrelto <id> - relative page id for import (alternative to importreltopath) o|output - output path for generated documentation (will skip import) R|retries - Maximum number of retries on import/export item failures (default: 3) Authentication: (if no authentication is provided, the program will prompt for user and password interactively) A|authtoken <token> - authtoken to use for user authentication U|user <username> - username for authentication (requires password option P|password <password> - password for username authentcation
The official documentation can be found on developer.mindtouch.com.
mindtouch.doc offers an easy way to take your documented .NET APIs and publish them with the power of MindTouch, no separate runtime or site required.
Behind the MindTouch OpenID add-on
First, an introduction. Hello, everybody! I’m Craig. I’ve been a member of the MindTouch community since February 2008, when I was living in Canada and looking for a collaboration tool running on C# for a Windows software shop I was working at. MindTouch actually ran C# on Mono on Linux, which suited me fine, as I am a dual-purpose Windows and Linux guy, and it was good to have a reason to have Debian around at work again! I had a couple of opportunities to travel to southern California for work, and met the MindTouch gang; there are some great ‘enginerds’ working there, and a Christmas card from Roy & friends is a site to behold.
While I’ve since changed jobs (and countries), I’m still a very active member of the MindTouch community, and still enjoy contributing to the platform. Over last Christmas, I built an add-on for MindTouch to support OpenID, with an view to using it for an application in my new role at the Symbian Foundation. While we didn’t end up finding an opportunity to use MindTouch at Symbian, as an open source foundation we are happy to contribute work like this to to the greater community.
First, some terminology. An OP is an OpenID Provider: a site which provides an authentication servicew using the OpenID protocol. This is Site A in my other post. Your MindTouch site, Site B becomes an RP, also known as a Relying Party, formerly known as a consumer.
Anatomy of a MindTouch add-on
You could implement an OpenID extension entirely in the front-end (using PHP) if you wished, and an earlier attempt at OpenID support worked exactly this way. However, the service approach was more in line with the MindTouch model, and I like C# more than I like PHP!
Back-end service
It was a foregone conclusion that I should use DotNetOpenAuth as the engine for this project. It is the OpenID library that powers StackOverflow, and one of the primary authors, Andrew Arnott, is very active in supporting the library. Unfortunately, the 3.x version uses some new whizz-bang functionality in .NET which means it doesn’t work on Mono; rather than try and randomly disable code in a security library, I’ve stayed with using the 2.x series, from when it was known as DotNetOpenId.
The back-end only has two methods, both of which are called solely by the front-end:
- authenticate: given an OpenID string, discover the endpoint for that string, and build a URL to redirect the user to; or throw an exception if that endpoint is invalid or unreachable.
- validate: when given a signed response from an OP, check that it is correct, and return the details provided (full name, e-mail address etc), or throw an exception if it is somehow invalid.
Using trusted authentication
MindTouch’s trusted authentication is what makes this all so easy. A call to the API (with the site’s secret API key) will let you create a user of your choice; this means we can use front-end code to create users. If the user passes an API call and has a good OpenID, we can create a MindTouch user for them.
Front-end extension
We need to inject ourselves into the login process, so that the user knows that OpenID is an option; after that, there needs to be a page to display the OpenID selector, and handle the interaction with the user. That calls for a special page, using hooks to insert a link to our page on the UserLogin page.
The documentation on PHP special pages is a bit sparse, but you can gleam all you need to know from the examples, and looking at the existing pages. Ultimately, they’re just PHP, with handy functions for displaying the page chrome.
Following suggestions from Max and Guerric, I settled on this approach:
- A user browses to /Special:OpenIDLogin and gets the endpoint selector.
- User selects or enters their endpoint and hits Submit.
- The selected ID is POSTed to /Special:OpenIDLogin.
- If the /Special:OpenIDLogin is called as a POST, it sends a secondary POST to the OpenID API service, calling the ‘authenticate’ method. This validates the endpoint, and responds with either an error message, or a URI to redirect the user to.
- If the response was an error, it is displayed to the user; if it is good, the front-end redirects the user to the OpenID provider with /Special:OpenIDLogin as the return URI.
- The user logs in to their OpenID provider, or authorise your site to use your OpenID, and is redirected to /Special:OpenIDLogin again with signed OpenID parameters.
- When /Special:OpenIDLogin is called with signed OpenID parameters, via POST or GET (both are supported by the OpenID spec), the page then POSTs the query parameters to the OpenID API service, calling the ‘validate’ method:
- If the response was an error or failure, it is displayed to the user and the process ends.
- The front-end queries the site properties store to see if your OpenID is associated with a MindTouch user already.
- If your OpenID is found, the user has logged in previously, and we have stored their username; log that username in using trusted authentication, and return to the page they wanted to see.
- If no property is found, they are redirected to Special:OpenIDCreateUser, with a hashed secret. On that page, they are asked to enter a username (which is validated by AJAX using jQuery, to indicate if the username is acceptable/in use). That username is stored in the property against their OpenID, and they are then logged in.
All that boils down to three pages:
- special_userlogin_openid, which injects “Click here to log in with OpenID” to the main login page
- special_openidlogin, the meat of the extension, which communicates with the API service
- special_openidcreateuser, which allows a user with a valid OpenID, but no MindTouch username, to create one.
The front-end makes use of another open source project; the jQuery OpenID selector by Reflejo.
Pitfalls and abandoned paths
I wanted to be able to use an OpenID URL as a username, but that introduces two problems: first, you may already have a user account; second, MindTouch doesn’t take nicely to usernames containing colons and slashes, especially as everyone gets a User: page. It was tried but ultimately abandoned.
Future additions
A wealth of possibilities for adding to this extension exist: here are but a few.
- When you’re asked to create a new account, you should be able to pick an existing account to associate with, by username and password.
- Conversely, you should be able to add OpenIDs, and verify yourself, from your user preferences page.
- The add-on allows you to restrict the OpenIDs that are accepted (to, say, only your Google Apps domain). If you know this, you should also be able to make the front-end skip asking you for a provider and just redirect you to one.
- Make a MindTouch installation an OpenID provider. Why not?
Getting the add-on
As this is an open source contribution to the platform released under the GPL, it’s available in source for you to improve. MindTouch Inc. have kindly integrated it into their build process so it’s available pre-built in the SVN tree. Thanks, guys!
To learn how to install this add-on into MindTouch, check out its page at the MindTouch App Catalog.
Traceroute
Let’s face it, from time-to-time circumstances can arise where you’re not able to reach a specific site from your Internet connection. In the event that you can’t reach your MindTouch installation, one key aspect to note is whether or not you’re having network issues. Can you reach other websites? If you can, it’s easy to jump the gun and think your network connection is working fine. The truth is, you still could have a network issue.
The best way to find out? Traceroute. No rhyme intended.
Traceroute will provide you with ping times (speed) between hops (routers along the path of your Internet connection). While you may be able to reach other websites, your Internet Service Provider could have an issue with the way it routes traffic to a specific domain. If you see (*) asterisk symbols within a traceroute report you may be having network routing issues.
How do you use traceroute? It’s simple. Pull up a command line interface and run “traceroute domain.com” (Linux / Mac) or “tracert domain.com” (Windows), substituting ‘domain.com’ for the domain you are trying to reach. A more detailed tutorial can be found at the link below:
http://www.mediacollege.com/internet/troubleshooter/traceroute.html
Having trouble understanding your traceroute output? Contact a MindTouch Support Agent today and we’ll be happy to point you in the right direction.
