Author Archive

18 Sep 09

Using tags to make powerful apps

Over the course of the last eight months or so I’ve been working quite diligently on the DekiScript “modules” that power WhoRunsGov.com.  I have learned quite a lot about implementing largly DekiScript powered projects and I am going to share some examples of how I used tags to develop very efficient reusable DekiScript templates.

The Use Case

WhoRunsGov.com is a user generated site focused on political profiles from all facets of the government.  With well over 700 profiles they are rapidly growing their content base and have successfully utilized DekiScript to offer a fluid and intuitive form of navigation.  Their solution includes a large number of “browse by” pages that display a list political profiles based on specified tags.   To make this more challenging there are about 150 different browse by pages so  I needed to come up with a solution that was both flexible and manageable.

The Solution

To solve this issue I created one centralized DekiScript template called ProfileList.  The ProfileList template is quite complex and is responsible for:

  • selecting the profiles
  • retrieving the profile content
  • identifying the most recently uploaded profile image
  • resizing it to fit the profile

In addition to these functions I also gave the template a number of different parameters.

  • $searchquery
  • $sortby
  • $limit

Template DekiScript

By adding these parameters I was able to make this template flexible enough to be called from all 150 browse by pages.  You can see that the code below uses the parameters to build a uri for the search API.   Using the search API is the fastest and most resource friendly approach to retrieving a set of pages.  Also, because lucene is so powerful you can filter the results with extreme control.

<h1>Template:ProfileList</h1>
{{
var qsortby = $sortby ?? '-tag:id-*';
var qlimit = $limit ?? 100; //

var results = web.xml(uri.build(site.api, [ 'site', 'search' ], { limit: qlimit, sortby: qsortby, format: 'search', q: $searchquery }), _, _, 900);
}}

‘Browse By’ DekiScript

When calling this template I simply passed in the $searchquery and the page displayed the appropriate profiles.

{{
profilelist{searchquery:'tag:"Governor" AND tag:id-*'}
}}

As you can see this searchquery is identifying pages that are tagged with ‘Governor’ and pages that are tagged with a tag that starts with ‘id-’.    Here are some more examples of how I used the ProfileList template for more advanced searches.

{{
profilelist{searchquery:'(tag:"administration official" OR tag:"obama administration official") AND tag:[id-a* TO id-i*]'}
}}

This query returns all pages tagged with ‘administration official’  or ‘obama administration official’ that are between A and I.

{{
profilelist{searchquery:'tag:"House member" AND tag:democrat* AND tag:congress AND tag:id-*'}
}}

This query returns all profiles that are tagged with ‘House member’, ‘congress’ and that also have a tag that starts with ‘democrat’.

Thanks for reading,

Damien Howley
@DamienH

11 Sep 09

Writing ajax search suggestions using the MindTouch API

After writing a recent blog post about using the search API to offer efficient and widespread navigation, I started thinking about another neat use for the search API, search suggestions.  This may be a misnomer so please allow me to explain.  Search suggestions refer to the dropdown menus under a search box that use ajax to offer realtime search results and commonly eliminate the need to travel to a search results page, ultimately making the search experience faster and generally much more impressive.  Here are some examples.

Apple

The Apple website offers what I consider the most elegant search suggestion solution.  They have combined design and function to offer a tool that is easy to understand and thorough enough to cover multiple “departments”.

apple-search-suggest1

Google

The Google search suggestion solution was actually launched as a full Google product and was kept separate from the standard Google search until recently. While it is not as elegant as the Apple offering it does offer more search results (which is obviously expected) and does eliminate the need to visit the search results page.  They’ve also included relevant advertisements which may be a testament to its success.

google-search-suggest

Building a MindTouch Search Suggest

Search API

To get started the first thing you need to familiarize yourself with is MindTouch Search API.  Basically all you need to know is how to construct the url.

www.yourmindtouch.com/@api/deki/site/search?limit=5&q=help

Using this url you can quickly access the search API for different terms and limit your result set accordingly.

HTML

In order to get our search suggest working we need to layout some basic HTML.  This HTML includes the search input and the dropdown menu that will later get populated by jQuery.  You must add this HTML to a MindTouch page in source mode. 

<h1>Search Suggestion</h1>
<input type="text" id="search-input">

<div id="search-suggest">
	<a href="#" class="close">x</a>
	<div class="search-pages">
		<strong>pages</strong>
	</div>
	<div class="search-files">
		<strong>files</strong>
	</div>
	<div class="search-comments">
		<strong>comments</strong>
	</div>
</div>

JQuery

Now that you know how to access the API and you have your base HTML you need to connect all the parts.   Again this needs to be entered in source mode .  I’ve added some highlighted comments to walk you through the jquery.

<script type="text/javascript">
	$("body").ready( function() {

                /*Hide the dropdown menu when a user clicks the close button*/
		$("#search-suggest a.close").click( function() {
			$("#search-suggest").hide();
			return false;
		});

                /*Run this function each time the uses pushes a key in the search box*/
  		$("#search-input").keyup( function() {
			if ($(this).val()==''){
				$("#search-suggest").hide();
				return false;
			}
                        /*set q = to the search term*/
			var q = $(this).val();

                    /*Delay the function from executing for 400ms so it doesn't run on every keyup*/
 		    clearTimeout($.data(this, "timer"));
		    var ms = 400; //milliseconds
		    var wait = setTimeout(function() {

                      /*Run the search function with search string q*/
		      loadsearch(q);
		    }, ms);

		    $.data(this, "timer", wait);
		});
	});

function loadsearch(q) {
        /*API url*/
	var qurl="http://developer.mindtouch.com/@api/deki/site/search?limit=10&q="+q;
        /*Count variable to keep track of how many pages, files and comments*/
	var cntpage=0;
	var cntfile=0;
	var cntcomments=0;
        /*Load the search using ajax, return the results in xml variable*/
	jQuery.get(qurl, function(xml) {
                /*Remove the existing links so you can add new content*/
		$("#search-suggest .search-pages a").remove();
		$("#search-suggest .search-files a").remove();
		$("#search-suggest .search-comments a").remove();
		$("#search-suggest .search-comments span").remove();
                /*Use Jquery selectors with XML to find the necessary nodes - Path, Title*/
		$(xml).find('search > page').each(function(){
			var qpath = $(this).find(" > path").text();
			var qtitle = $(this).find("> title").text();
                        /*Add the new search result to the search-pages dropdown*/
		        $("#search-suggest .search-pages ").append('<a class="page" href="/' + qpath + '">' + qtitle.substr(0,30) + '</a>');
			cntpage++;
		});
                /*Use Jquery selectors with XML to find the necessary nodes - Contents, Filename*/
		$(xml).find('search > file').each(function(){
			var quri = $(this).find(" > contents").attr("href");
			var qtitle = $(this).find(" > filename").text();
			$("#search-suggest .search-files").append('<a class="file" href="' + quri + '">' + qtitle + '</a>');
			cntfile++;
		});
                /*Use Jquery selectors with XML to find the necessary nodes - Username, Comment, URI*/
		$(xml).find('search > comment').each(function(){
			var quser = $(this).find("username").text();
			var qcomment = $(this).find(" > content").text();
			var quri = $(this).find("path").text();
			$("#search-suggest .search-comments").append('<a href="/' + quri + '" class="comment-user">' + quser + '</a><span class="comment-text">' + qcomment.substr(0,40) + '...</span>');
			cntcomments++;
		});
                /*If there are no pages, hide the page dropdown*/
		if(cntpage>0){
			$("#search-suggest .search-pages").show();
		}else {
			$("#search-suggest .search-pages").hide();
		}
                /*If there are no files, hide the file dropdown*/
		if(cntfile>0){
			$("#search-suggest .search-files").show();
		}else {
			$("#search-suggest .search-files").hide();
		}
                /*If there are no comments, hide the comment dropdown*/
		if(cntcomments>0){
			$("#search-suggest .search-comments").show();
		}else {
			$("#search-suggest .search-comments").hide();
		}
                /*If there are any results show the appropriate dropdowns*/
		if(cntpage>0 || cntfile>0 || cntcomments>0){
			$("#search-suggest").show();
		}else {
			$("#search-suggest").hide();
		}
	});
}
</script>

CSS

Lastly, once you’ve got it working it needs to look good, otherwise it might not be legible at all.   I’ve put some simple CSS together but feel free customize the look & feel.

<div>
<style>
#search-suggest strong{
	margin:0 0 10px 0;
	color:#999;
	display:block;
}
#search-suggest {
	border:1px solid #ccc;
	display:none;
	padding:0;
	position:absolute;
	overflow:hidden;
	background:#fff;
	font-size:11px;
	min-width:150px;
	z-index:50;
}
#search-suggest .search-pages{
	float:left;
	padding:5px 20px;
	max-width:200px;
}
#search-suggest .search-files{
	float:left;
	padding:5px 20px;
	max-width:200px;
}
#search-suggest .search-comments{
	float:left;
	padding:5px 20px;
	max-width:200px;
}
#search-suggest a {
	display:block;
	padding:0 0 0 20px;
	height:16px;
	overflow:hidden;
	margin:0 0 5px 0;
	background-image:url(/skins/common/icons/icons.gif);
	background-repeat:no-repeat;
}
#search-suggest .search-pages a {
	background-position:0 -768px;

}
#search-suggest .search-files a {
	background-position:0 -1040px;
}
#search-suggest .search-comments .comment-user {
	display:block;
	font-weight:bold;
	margin-bottom:0px;
	padding:0;
}
#search-suggest .search-comments .comment-text {
	display:block;
	font-weight:100;
	margin-bottom:10px;
	color:#999;
}
.no-show {
	display:none;
}
#search-suggest a.close{
	float:right;
	display:block;
	font-size:14px;
	width:15px;
	padding:0 0 1px 0;
	line-height:1;
	-moz-border-radius:7px;
	border:2px solid #fff;
	font-weight:bold;
	text-align:center;
	background:#ccc;
	color:#fff;
	text-decoration:none;
}
#search-suggest .close:hover{
	background:#9f1313;
	text-decoration:none;
	color:#fff;
}
</style>
</div>

MindTouch

The MindTouch search suggestion tool cleanly separates pages, files and comments to show you a wide variety of search results.  Additionally it is legible and does eliminate the need to travel to the search results page.

mindtouch-search-suggest1

I hope you enjoyed this tutorial

Damien Howley
@DamienH

02 Sep 09

Writing a more flexible MindTouch theme

Download this Theme

Theme File Hierarchy

It is important to understand the difference between a MindTouch  “theme” and “skin”.

Goals & Objectives

Recently I have been exploring how to make the process of skinning MindTouch a little easier.  It’s not that skinning Ace, Fiesta or Deuce is hard but there are some improvements that could make customizing MindTouch faster and more flexible.  So, I created a new theme designed specifically to meet the following goals.

  1. Provide much more granular control over the layout
  2. Make PHP skinning functions understandable
  3. Provide a more bare-bones HTML layout for designers to customize as they see fit
  4. Require designers to modify just one CSS file
  5. Make navigating the theme file system more intuitive

Implemented Changes

As of now this is still experimental and I have no immediate plans to ship it with a MindTouch release.  I figured I could at least document it in the meantime to provide some insight into our skinning ideas.  Here are some of the steps I have taken so far.

  1. Separated most of the PHP & HTML
    Formerly only one markup file existed for each theme and its name matched that of the theme, ace.php, fiesta.php, deuce.php.  Each of the these files were loaded with markup, PHP and JavaScript, much of which was not important to a designer.  Additionally because these files were so cluttered visually parsing them was difficult, even for the trained eye.  To solve this problem I moved the HTML content from the markup file to a new file called HTML.php.  The new separated file is intended to provide designers with a more accurate and manageable skinning experience.  I also located HTML.php inside the skin folder to provide more granular control over each skin.  With this change now each skin can have custom markup that is independent of other skins.
  2. Restructured the theme files
    When creating this new theme I wanted to make accessing its files more intuitive.  With the other themes there are multiple files in numerous folders that are inserted into the skin.  Custom CSS modifications were always stacked on top of a large underlying layer of MindTouch prescribed CSS.  This approach limited flexibility and forced a very sculpted look & feel upon designers.  In order to solve this problem I removed the base CSS files (_common.css, _style.css) in exchange for one CSS file, _style.css, which resides in the skin folder.  I also shifted the _print.css, html.php and all images to the skin level.  Again, this provides much more granular control over the look & feel of each individual skin and also eliminates any MindTouch defined CSS dependencies.
  3. Renamed all the PHP skinning functions
    When I set out to change the theming process I immediately noticed that the PHP skinning functions in each markup file were inconsistent and could be challenging to a non-developer.  For instance there were references to wfMsg(), $this->html(), $this->haveData(), $wgTitle, $wgUser and many more.  While programmatically these functions all make sense they add an unnecessary burden on a designer.  As an alternative I gave these functions new pseudo names that follow a simple pattern and consistent naming schema.  For instance:

    1. $this->SiteLogo();
    2. $this->PageTags();
    3. $this->PageEdit();
    4. $this->PageAdd();
    5. $this->PageTitle();
  4. Consolidated CSS files
    Initially when we built MindTouch themes we planned for CSS modifications to be made as an addition to the existing CSS.  While this enabled very rapid theme development it also limited the flexibility of the design and greatly raised the skill level required to completely customize MindTouch.  Additionally it also made many assumptions about how MindTouch should look and generally limited the scope of customization to aesthetic CSS such as colors, fonts, etc.  As a solution I decided to completely remove the base CSS files (_common.css and _style.css) and only reference the _style.css in the skin folder.  I will add, however, that _content.css will remain, as content styling has proven to be the most challenging portion of creating a new MindTouch skin.  The _content.css file is pre-populated with default content CSS but can be modified on a per skin basis.
  5. Eliminated unnecessary structure
    In order to make theming more flexible there were some deeper consideration required for the PHP skinning functions.  Whereas the older PHP skinning functions contained a large amount of predefined HTML I opted to remove most of this HTML and place it directly in HTML.php.  This may seem fairly useless but it does shift the control of the HTML into the hands of the designer.  For instance I removed extra wrapper <div>, <li> and <span> elements that were added for CSS identification or to meet some assumed function.  Ultimately now you can decide if your logout button is in a dropdown list, wrapped in <div class=”logout-wrapper”> or just by itself.

Theme File Structure

  • Theme (called beech as an example)

    • _reset.css – Resets CSS
    • beech.php – The base php file, formerly contained all markup and php
    • head.php – Contains all <head> included files:  CSS, JS, favicon, etc
    • pale/ – Skin folder (called pale as an example)
      • _content.css – Contains editor/content CSS
      • _print.css – Contains CSS for MindTouch print preview
      • _style.css – Contains all chrome/layout CSS
      • css.php – Consolidates css files
      • html.php – Contains the markup for your skin

Conclusion

As I continue to develop this theme I will do my best to keep the community informed via my documentation.  Seeing as this is currently experimental there are no immediate plans to releases.  Please let me know if you have any thoughts or suggestions regarding my approach.

I will continue to document this theme at http://developer.mindtouch.com/Deki/Skinning/Beech

Thanks for your time

Damien Howley
@DamienH

30 Jun 09

Animated Ajax Pageview Ticker

In my infinite quest to make everything better I found myself quite frustrated with the use of “plaintext” to display the number of pageviews on a MindTouch page .  Plaintext was clearly way too simple and I saw an opportunity to make it way more complex, quite possibly even confusing.

Using plain numbers is boring, they don’t move, shake, glide or slide.  Plaintext is not flashy enough and it doesn’t update.   If another ten billion people visit the page you’ll never know it’s true popularity, at least not until you refresh (so 90′s).  I know, all very pointless points but this is what I have to tell myself to justify such an endeavour.

counter_ticker_bg

To combat my frustration I decided to build an automated number ticker that updates using ajax to display the REAL-TIME pageviews of a MindTouch page.  The ajax utilizes the MindTouch API to retrieve the latest pageview data.  I used CSS to make the nice numbers which referenced this image.

The Number Ticker currently hits the API every 10 seconds.  This can be reduced to savor server resources and should be a major consideration when deciding to use this script or not.

Lastly, I consolidated it all into one MindTouch template so it can be included into multiple pages.  To create this template:

  1. copy the code below
  2. go to your template namespace (yourwiki.com/template:)
  3. create a new page
  4. view source
  5. paste and save.

Template Code

<h1>Template:AjaxCounterTicker</h1>
<input type="hidden" id="pgapi" value="{{page.api}}">
<div class="counter-wrap">
    <div class="counter-number">
        &nbsp;
    </div>
</div>
<div>
<style>
.counter-wrap {
    height:18px;
    overflow:hidden;
}
.counter-number {
    height:198px;
    width:12px;
    position:relative;
    background-image:url(http://developer.mindtouch.com/@api/deki/files/4548/=counter_ticker_bg.gif);
    float:left;
}
</style>
</div>
<script type="text/javascript">
Deki.$("body").ready( function() {
    getviewcount();
    setInterval("getviewcount()", 10000);
});
function getviewcount() {
    var pgapi = Deki.$("#pgapi").val();
    Deki.$.ajax({
        type: "GET",
        url: pgapi,
        async: false,
        dataType: "xml",
        success: function(xml) {
            var pgviewnum = Deki.$(xml).find("metric\\.views").text();
            loadticker(add_commas(pgviewnum));
        }
    });
}
Deki.$(".counter-number").each( function(i) {
    Deki.$(this).attr('id','num'+i);
});
function loadinput() {
    var newval = Deki.$("#numgo").val();
    loadticker(newval);
}
function loadticker(ticnum) {
    var numheight=18;
    addticker(ticnum);
    if (ticnum && ticnum != 0) {
        var s = String(ticnum);
        for (i=s.length;i>=0; i--)
        {
            var onum=s.charAt(i);
            Deki.$("#num"+i).attr('value',onum);
        }
        Deki.$(".counter-number").each( function() {
            var nval=Deki.$(this).attr("value");
            if (!isNaN(nval)) {
                var nheight = Number(nval)*numheight*-1;
                Deki.$(this).animate({ top: nheight+'px'}, 1500 );
            }
            if (nval==','){
                Deki.$(this).animate({ top: '-180px'}, 1500 );
            }
          });
    }
}
function addticker(newnum) {
    var digitcnt = Deki.$(".counter-number").size();
    var nnum = String(newnum).length;
    var digitdiff = Number(nnum - Number(digitcnt));
    if (digitdiff <0) {
        var ltdig = (Number(nnum)-1);
        Deki.$(".counter-number:gt(" + ltdig + ")").remove();
    }
    for(i=1;i<=digitdiff;i++) {
        Deki.$(".counter-wrap").append('<div class="counter-number" id="num' + (Number(digitcnt+i-1)) + '">&nbsp;</div>');
    }
}
function add_commas(nStr) {
    nStr += '';
    x = nStr.split('.');
    x1 = x[0];
    x2 = x.length > 1 ? '.' + x[1] : '';
    var rgx = /(\d+)(\d{3})/;
    while (rgx.test(x1)) {
        x1 = x1.replace(rgx, '$1' + ',' + '$2');
    }
    return x1 + x2;
}
</script>

How to call the Template

{{AjaxCounterTicker();}}

If you have questions please post them as comments and I’ll be sure to respond.  Thank you for your time.

Damien Howley
@DamienH

13 Mar 09

Threaded discussions with Jquery and the Deki API

discussion-forum-threaded-dI’ve been on a fairly large jQuery kick since I realized what it is truly capable of and my latest effort is no different.  By combing the jQuery ajax library with the MindTouch Deki API I was able to create a threaded discussion tool that uses a Deki page for content storage and asynchronously submits and retrieves discussion contributions.

Benefits

  1. No extension installation required – just create a Deki template and paste
  2. Deki permissioning also works for the threaded discussion
  3. Embed DekiScript!
  4. Easily retrieve statistics about the forum via DekiScript

How it works

I had never really developed using this approach before so I was able to learn quite a lot during my development.  I started off by using Jquery to post simple data to another page like so:

DekiWiki.$.ajax({
type: “POST”,
url: “/@api/deki/pages/=FOOFOOFOO/contents?abort=never”,
data: “This is my contribution to this page”,
success: function(msg){
alert(  msg );
}
});

Initially Deki creates a page (lets call it the “data page”) if there is no page available.  This process is fairly straight forward, however, appending data to the Data Page  is a little more complicated.  For starters Deki needs to know what revision you’re are basing your modification from so it requires the edit time of the version that you’re currently seeing.  For edit time you will have to access the API of the Data Page and retrieve the edit time.  Once you have the edit time you can post your data (along with edit time) to the Data Page using the Jquery Ajax library.  Naturally Deki will overwrite all existing data in the Data Page unless you append “?section=0″ to the url, which I did so.  So after it is all said and done here’s what happens:

  1. GET – access the API for edit time (also used to tell me if the page exists or not)
  2. POST – submit my form to the API with edit time
  3. GET – retrieve the newly updated content and display it

Now there is a lot more involved in this process.  For starters I needed to be able to read back the data that I posted to the Data Page so I had to give it some sort of a structure with unique identifiers.  Additionally the sorting, styling and displaying process is fairly complex as the order and threads come in chronological order, not thread order.  JQuery is excessively useful for this type of process.  It deals with HTML and XML very well and made my life much easier.   For instance after retrieving the XML I was able to use Xpaths as follows:

var datetimeraw = DekiWiki.$(“date\\.edited”, xml).text();

Jquery also has very powerful selectors which makes identifying all of the threaded comments very easy.  Take a look at the following loop with a selector:

DekiWiki.$(“#” + formid + ” > :input”).each(function(i){
ddata += “<div class=’” + DekiWiki.$(this).attr(‘name’) +”‘>” + DekiWiki.$(this).val()+ “<\/div>”;
});

Well, you get the idea.  There’s a lot more I could write about this script but I’m sure I’ll address in it another post or via comments.  Please use the following links as a reference:

  1. Live Demo
  2. Code

Thanks for your time

Damien
@DamienH

Copyright © 2011 MindTouch, Inc. Powered by