Python: Django on IIS

Because I only have easy access to IIS servers, installing open source software is often an exercise in total masochism. PHP is invariably an unnecessary amount of pain. Anything that then uses PHP then adds more pain (MySQL is surprisingly pain free actually - it just works, even more so than SQL Server).

because I am now used to the level of pain that PHP on IIS creates, I decided to up the ante and try Django out.

(The trick with PHP is FastCGI. I had lots of legacy PHP sites using the deprecated ISAPI extension; with FastCGI everything just works.)

Back to Django. Frankly, I don't know anything about it. However, I have been playing with Python, and really liking it, so I thought I needed a web framework as well. And a challenge.

The technology / versions I am using are:

• Windows Server 2003 / IIS 6.
• Python 2.6 (more on this later).
• PyISAPIe 1.1.0-rc4 as the ISAPI magic connect to Django doobrie.
• Django 1.2.3.
• SQL Server 2005 (this is not yet set up with Django - that will be a later post).
• Test site installed as new web site NOT a virtual directory in IIS (set up on a new port rather than with a host header, but that shouldn't matter)

Most of the installation can be done by following the PyISAPIe instructions, but there are a few gotchas, and a few things that glosses over, so here they are, hopefully in a Google friendly fashion, because I found nothing on most of them.

1. Using Python 2.7 with PyISAPIe

You can't. At least not with the available binaries, as far as I can tell. When I installed Python, I chose the 2.x branch as the most widely supported, and 2.7 as the most recent version. After following the steps in the link above, up to the bit about running Info.py as a test, I went to the website and got the error Specified module could not be loaded.

To cut a long story short, this is a Windows error; it cannot load a DLL referenced by the ISAPI filter. This is because the RC4 release of PyISAPIe looks for python26.dll, and Python 2.7 has, you guessed it, python27.dll.

The solution for this is to install Python 2.6 instead; after that Info.py works perfectly.

There are a couple of posts out there that allude to using PyISAPIe with Python 2.7; unless I am missing something then you cannot do this with the binaries; you would have to download and compile the Visual Studio project, which was more than I was willing to do for a test environment.

2. Installing PyISAPIe

There is an import omission in the instructions, that if you do not know about it will result in the test script working fine, but every Django request later returning Virtual Directory listing denied.

In the section Setting Up Files, it says "Go to c:\pyisapie\source\PyISAPIe\Python\ and copy the entire Http folder to c:\python25\lib\site-packages". What you ACTUALLY want to do is either MOVE the folder, or copy it then rename the original; the reason being is that the Python module Http supplied will be loaded by preference from the same folder as the PyISAPIe DLL, meaning that if you have copied everything nicely into the Python directory and made configuration changes there, sod all will happen.

3. Installing Django

This is pretty straightforward; a few things to mention that aren't immediately apparent if you are used to the Microsoft way:

• The tarred Django directory should be placed in a sensible place BEFORE you run python setup.py install; you still need it after the install script has run - it is not analogous to an .msi file.
• The PyISAPIe instructions set up a test project in C:\test - it might bear mentioning that this will become the Django project directory so I put mine in a more sensible place, with all the other web applications on the server. Note however that you want to be careful with permissions here - you don't want the web server to have direct access to the source files, so don't put it in a subdirectory of an Inetpub or similar folder.

4. Getting Django working with PyISAPIe

First of all, ensure that you have followed the PyISAPIE instructions, and also that the original Http folder in the PyISAPIe folder is renamed / deleted.

The next thing to do is ignore the Examples\Django folder completely; this is implied, but I don't think it was implied strongly enough. Or maybe I'm just dense. What you want is the Examples\WSGI folder; copy Isapi.py out of that into the Http folder in your Python install.

Next, add the sys.path related lines and alter the os.environ["DJANGO_SETTINGS_MODULE"] as in the instructions. Note that the sys.path line points to the parent folder of your Django project, NOT actually the project folder.

This then gave various weird errors:

Could not initialize interpreter

Traceback (most recent call last):
  File "C:\Python26\Lib\site-packages\Http\Isapi.py", line 29, in 
    from md5 import md5
  File "C:\Python26\Lib\md5.py", line 8, in 
    DeprecationWarning, 2)
  File "C:\Python26\Lib\warnings.py", line 29, in _show_warning
    file.write(formatwarning(message, category, filename, lineno, line))
Exception: Not currently processing a request

So far, I have fixed this by commented out the from md5 import md5 line; I am not using the RunScript stuff, so it is fine.

This then gave me:

Could not initialize interpreter

Traceback (most recent call last):
  File "C:\Python26\Lib\site-packages\Http\Isapi.py", line 54, in 
    from trac.web.main import dispatch_request as TracHandler
ImportError: No module named trac.web.main

I'm definitely not using Trac, so I commented out the Trac related lines.

This then gave me Hello World! and a working Django install.

Now I just have to figure out what the hell it is, what the hell it does, and how to make it do it. Oh, and connect it to SQL Server.

Just to recap, here are the major fuzzy bits I found in the PyISAPIe documentation:

• Make sure your PyISAPIe version matches your Python version.
• If you copy the Http folder into your Python install as suggested, make sure you rename the original.
• Make sure you use the PyISAPIe Isapi.py from Examples\WSGI, NOT Examples\Django.
• Remove references in Isapi.py to anything you don't have, and either fix or comment out the MD5 import so you don't get the deprecation warning.

Or I suppose you could run it on Apache, but where's the fun in that?

Javascript: JQuery DatePicker, Chrome and MAXLENGTH

Having just spent 2 hours swearing violently at my computer, I thought I'd share this little bug tracking story...

I am writing a simple data entry page that contains a couple of date fields, for which I'm using the very nice JQuery UI datepickers.

I have used them all over the place with no problems at all, however on this page I was getting some weird behaviour when the submit button was clicked:

1. If the date fields were empty, the form submitted fine.
2. But if a date had been selected, when you clicked submit, the date picker for the first populated date field popped up and the form did not submit.

Some testing revealed this only happened in Chrome - my version of which is the v6 beta release.

An update to the JQuery UI did not fix the problem.

Some debugging code revealed that on submit, the datepicker fields were being focussed - so as far as the datepicker code was concerned, it was working correctly.

However, the object inspector in Chrome showed no unwanted event listeners that could be doing this.

Some more swearing, a minimal test case and a comparison with a working page later revealed the problem - the MAXLENGTH attribute had been set, in a copy/paste debacle, to 8 - clearly too short for a dd/mm/yyyy date.

So, presumably, Chrome allows you to set content longer than the MAXLENGTH from script, but then will not let you submit the form, instead focussing the offending field - it makes perfect sense when you think about it, really.

Firefox and IE both allow you to submit the form with the longer data - although you can make an argument for that making sense too. I don't know what the W3C say on this issue.

C#: Doing stuff with Google using OAuth - an idiot's guide

I have been playing around with OAuth and connecting to Google apps. After wading through the OAuth standards and Google API docs I decided a library was the way forward, so downloaded DotNetOpenAuth.

As often seems to be the case with freely provided software, it is awesome and well documented at the reference level, but not very good at handholding you through a basic scenario; although that may be due to my preferred way of starting up a blank project and attempting to get a simple application working rather than starting up a sample project and modifying it :o)

I eventually found a good example of what I wanted to do in Samples\OAuthConsumerWpf, but having done hardly any WPF and being too old and grumpy to use lambda functions, it still took a while to get my head round it and get a simple Windows Forms app going.

This is what I did:

1. Created a new Windows Forms application in VS2008
2. Added a reference to the DotNetOpenAuth assembly.
3. Copied the files GoogleConsumer.cs and Util.cs from \Samples\DotNetOpenAuth.ApplicationBlock to my application.

4. You also need an object implementing IConsumerTokenManager. The samples use the InMemoryTokenManager, which is fine for testing - obviously you will need to re-authorise the application on each run, and a useful application would have some kind of backing store.

   The InMemoryTokenManager is located in \Samples\DotNetOpenAuth.ApplicationBlock\InMemoryTokenManager.cs but is disabled outside the samples using compiler directives; I copied it into my new project and removed the compiler directives because I'm too lazy to change the DEFINEs - and again, I prefer to start with nothing and work up to a working sample app in order to get my head round things.

Once you have all these files, you can talk to Google from a Windows Forms app with a tiny amount of code, most of which is palmed off to the GoogleConsumer object in GoogleConsumer.cs.

If you, like me, are new to OAuth programming, the sequence is as follows:

1. Create a DesktopConsumer that will talk to Google, using the GoogleConsumer.ServiceDescription and a new IConsumerTokenManager for the consumer key anonymous and secret anonymous.

   The consumer key and secret identify your application - it will not be registered with Google, so you use anonymous - see http://code.google.com/apis/accounts/docs/OAuth_ref.html#SigningOAuth.

2. Call GoogleConsumer.RequestAuthorization with this consumer and flags specifying the Google apps you want access to. This returns the URL of a page that will ask the user to authorise your application, and a request token which is used later in the process.
3. So far, you have just told Google you want to access application data; at this point, no specific user has been mentioned. You now need to start up a web browser to the URL returned by GoogleConsumer.RequestAuthorization. This will allow a user to grant access to an application, and give them a verification key. The easiest way to achieve this is to start off the browser with System.Diagnostics.Process.Start(url) and show a modal window asking for the verification key to be pasted when it is retrieved.
4. You now use the ProcessUserAuthorization of the DeskTopConsumer instance to get an access token, passing in the request token returned in step 2 and the verification code pasted in by the user.
5. Once you have the access token, you can use that and the consumer to start interacting with Google apps!

Assuming all of the required DotNetOpenAuth files have been referenced or copied in as above, here is the (very simple) code to post a blog entry on Google:

// You will need the following using directives:
using DotNetOpenAuth.OpenId.Extensions.OAuth;
using DotNetOpenAuth.OAuth;
using DotNetOpenAuth.OAuth.ChannelElements;
using DotNetOpenAuth.OAuth.Messages;

// Needed for GoogleConsumer
using DotNetOpenAuth.ApplicationBlock;

// Set up consumer
DesktopConsumer consumer = new DesktopConsumer(GoogleConsumer.ServiceDescription, new InMemoryTokenManager("anonymous", "anonymous"));

// Get the request token and user access authorisation URL
String requesttoken;
Uri authuri = GoogleConsumer.RequestAuthorization(consumer, GoogleConsumer.Applications.Blogger, out requesttoken);

// Start up a browser for the user to grant access and be given the verification key
System.Diagnostics.Process.Start(authuri.AbsoluteUri);

// I have not documented my FormGetKey, but it is a simple form with one textbox and an Ok button on 
// it that just sits there being modal until the user pastes the verification key in and clicks Ok.
FormGetKey form = new FormGetKey();
form.ShowDialog();

// Oh yes, and the form has a public property which is simply { get { return textboxVerificationKey.Text; } } :o)
String vercode = form.VerificationCode;
//

// Get the access token given the request token and the verification code
AuthorizedTokenResponse grantedAccess = consumer.ProcessUserAuthorization(requesttoken, vercode);
String accesstoken = grantedAccess.AccessToken;
//

// Now we are authorized, we have an access token, and we can do stuff
GoogleConsumer.PostBlogEntry(consumer, accesstoken, "http://mylovelyblog.blogspot.com", "OAuth test", XElement.Parse("<p>OAuth test content.</p>"));
//

The next step in the process is providing a more useful implementation of IConsumerTokenManager with a backing store and using the tokens from that. I will post a similarly lift-up-flaps level post on that when I have tried it out, and again with a web app based system.

Looking at the code now, it must be just the fact that it's Friday that I found this complicated...

J2ME: Quickie release of "lightmeter" code

I don't seem to be ever getting around to doing this and now a Java developer with a Nokia has asked me for it, so here is the full source code to the phone EXIFy lightmeter.

Well, I say "full"; it's very simple! If you have seen Java before and have the J2ME SDK you should be able to do something with it.

I will try to do a more informative post on it soon. You will obviously need Java installed and also the J2ME 3.0 SDK.

Quickly, though: /_support contains the Javascript files I ported the EXIF reader from (see this post), /src/akme/mobile/util contains maths routines missing from the mobile edition (check the source for URLs and credits), /src/expinfo contains the hastily hacked together and very horrible code for the MIDlet itself, /src/rext/graphics contains a routine for generating thumbnails that seems to exist across many blog and forum posts, and /src/rext/simpleexif is the main codebase, containing the EXIF reader port and the code for moving around EV values.

I stopped developing the code as soon as it worked for the first time, so it is a fairly parlous state :o)

ExposureInfo.zip

If you have found this post and are not a developer but just want to try the app, the Flickr group has more useful info.

XSL: xsl:element and short tags

I have had problems with XSL and short tags before when using identity transforms, but it can also be an issue doing simple element creation. Obviously as far as a parser is concerned, it makes no difference whatsoever whether you have short tags (e.g. <fish/>) or the complete form (<fish></fish>) for you empty elements, but it makes it much fatter and far less human readable - especially as the .NET System.Xml.Xsl.XslCompiledTransform at least puts closing tags on a new line for empty elements as well as ones with children.

So, if you generate a completely empty element with <xsl:element>, then the short form is used:

<xsl:template match="fish">
 <xsl:element name="{@type}"/>
</xsl:template>

However, if you add any attributes, then the complete form will be used, even though there are still no child elements:

<xsl:template match="fish">
 <xsl:element name="{@type}">
  <xsl:attribute name="type">Fish</xsl:attribute>
 </xsl:element>
</xsl:template>

The trick is to generate your element into an <xsl:variable> and then use <xsl:copy-of> to output it, which results in the short form being maintained:

<xsl:template match="fish">
 <xsl:variable name="el">
  <xsl:element name="{@type}">
   <xsl:attribute name="type">Fish</xsl:attribute>
  </xsl:element>
 </xsl:variable>
 <xsl:copy-of select="$el"/>
</xsl:template>

Easy when you know how. I couldn't find this anywhere on Google, so it's either an obvious thing that everyone but me already knows, my searching sucks, or I'm the first person to have this problem.

Following a comment by Martin Honnen on my original MSDN query about this, it seems that it is a bit more complicated; the basic problem is that XSLT does not define an output format for these situations, so it is entirely up to the processor / serialization combination.

The processor I was using to run these tests was on v2.0 of the framework, and was the deprecated System.Xml.Xsl.XslTransform object, which requires the workaround. The newer System.Xml.Xsl.XslCompiledTransform uses the short form for empty tags and so does not need the workaround.

Annoyingly, the target application uses XslCompiledTransform and therefore would have worked fine, it is only the scratchpad I use for knocking up stylesheets that uses the legacy code that caused the issue! And a quick change of typename for one object and removal of one parameter that was null anyway, and it's bloody working there too. Hurrah for wasted time.

Ah well, if someone is still using the deprecated code then maybe this will help.

J2ME: EXIF and lightmetering Part 1

I am a bit of a camera obsessive. I got into photography with a digital P&S, progressed to a DSLR, and then got into film. Since then, my camera collection has taken over several surfaces in the flat. I don't have that many, but then I don't have that many surfaces.

Most of the collection consists of old cameras; my theory is that if you want to take crisp "perfect" images, a DSLR is the way to go. This is a theory based mainly on laziness, but it works for me.

In general, the film cameras have been chosen for looks at least as much as quality. The Rolleiflex has both in spades, of course.

Most of the old cameras do not have a light meter, and many with the old selenium cell meters simply do not work any more.

I do own a couple of lightmeters, but I find they are a pain to use, and new ones are basically extremely expensive flash looking versions with exactly the same functionality.

What I have been doing it using the dSLR to take a preview shot, then transferring the settings over to whatever camera I am currently playing with. However, this becomes a bit tedious after a while, and also having, say, the Rolleiflex and the dSLR round my neck results in severe neck strain.

So, ideally, I want a very small camera than can take a preview shot with the ability to tell it the ISO and, usually, the aperture I want. I can then check the preview looks okay, and transfer the settings over with any modifications required. However, P&S cameras with these manual settings are not cheap and often have big limitations on the ISO and f/stop they allow you to set, so an alternative solution is to take the preview with whatever settings the camera wants, then allow me to easily convert them into ones I can use.

This is where my mobile comes in. It is extremely small, I always have it on me, and it has a camera with a full auto mode that takes JPEGs stuffed full with the EXIF data for the exposure. Perfect. Except that, on the phone, there is no way of actually viewing the EXIF.

However, the phone, a Sony Ericsson C903, has mobile Java, and I've seen some Java before, so I decided to give writing an app for it a shot.

It was an interesting exercise, and along the way I discovered several fun things. For example, mobile Java is one of the most frustratingly NON-device independent environments it's been my misfortune to use, the certification requirements are beyond absurd, and my phone has one of the most annoying bugs ever to have existed - when your app works perfectly and is extremely useful, having the phone randomly lose and/or corrupt the memory card it's sitting on is rather irritating.

The first bit of the code, which we have finally come to, is something that will read EXIF data out of JPEGs. I had a fish round for some libraries and found lots of great code that was either extremely big, extremely non J2ME compatible, or both.

Then I discovered this Javascript library, which I have to say, is brilliant. It's a use for client side Javascript and AJAX that simply would never have occurred to me in a million years. It is perfect for my uses because being entirely client side Javascript it uses very basic language features and is very small, ideal for conversion to Java by someone who hasn't used Java for about 8 years :o)

The code was written in the J2ME 3.0 SDK editor, which seems to have mangled the indentation for any other editor, but source code formatters are not exactly hard to find. The basic EXIF when compiled has not J2ME specific imports, so can be linked into any Java project; this is very handy for testing, obviously.

I made a couple of changes; I added X_FNumberAsString as a new data item for laziness, and created a Fraction class to help with maintaining the normal form of shutter speeds when manipulating them (a later post).

The ByteArrayWrapper class duplicates the required functionality from binaryajax.js.

EXIFReader contains the main functionality, and EXIFTags the tag names / descriptions and offsets, and also the list constants for various tags.

The classes live in the package rext.simpleexif.

It is pretty easy to use; you simply load in an image file into a byte[], then create the reader:

EXIFReader reader = new EXIFReader();
  
boolean ok = false;

try
{
 ok = reader.readEXIFFromByteArry(data);
}
catch(Exception e)
{
}

The reader object has three Hashtables for the various tags, EXIF, TIFF and GPS, and a Vector for debug messages called LOG.

Print out all EXIF tags to the console:

Enumeration exifk = reader.EXIF.keys();

while(exifk.hasMoreElements())
{
 String s = (String)exifk.nextElement();
 System.out.println(s + ": " + String.valueOf(reader.EXIF.get(s)));
}

You can get the code from the link below, including a small test console app that will print off the debug info then the EXIF data for a JPEG file.

exif.zip

C#: The Gödel at the Starbow's End

The Gold at the Starbow's End is a short story collection by one of my favourite sci-fi authors, Frederick Pohl, author of the superb Heechee Saga, which still to this day makes me extremely annoyed that we haven't found a fleet of faster than light alien spaceships anywhere in the solar system yet. Still, apparently Phobos is probably hollow, so there's fresh hope!

The eponymous story in the collection is an extremely bleak story I find quite unsettling, like so many Cold War era novels. The basic premise is that a bunch of astronauts are sent off on a long boring journey with nothing to do but play around with maths in the hope they will derive some completely new physics. I can assure you that the story and indeed the others in the book are a good 85,000 times better than my primary school summary.

At one point in the book, the astronauts, having developed theories and modes of thought way beyond the poor suckers left on a socially disintegrating Earth, send back their research, including the theory of easy fusion power, in what is called a Gödelized form.

Maths is not my strong suit, so this was new to me. Basically, it encodes arbitrary information in the form of a single number. A big number. A very very big number. A really quite stupendously large number. A number so big ... well, you get the idea.

How it works is this:

1. List the prime numbers, one for each of the characters in your message.
2. Raise each prime number to the power of your character encoding - e.g. a = 1, b = 2, ...
3. Multiply them all together.

So, to encode the word "whelk", you would calculate:

(2^23) * (3^8) * (5^5) * (7^12) * (11^11) =
679,212,979,662,721,270,806,215,943,782,400,000

To decode, you simply factor the number for each of the primes in the list. So, factoring for 2 gives you the first character, for 3 the 2nd, 5 the 3rd etc. etc.

Note: If you are like me a maths muppet, then this is how you factor a number, using the first prime, 2 as as example:

1. Divide the number by 2 and get the remainder
2. If the remainder is 0, add one to a counter and go back to the first step with the halved value
3. Stop when the remainder is not 0

The counter gives you the factors of two, and hence your first character - e.g. if it were 26, it would be "z".

As you can see, this results in numbers of a size I may have already indicated as being quite large, and decoding requires huge amounts of factoring. Plus of course you need to generate lots and lots of primes in the first place.

In the story, the data size is "the contents of a standard unabridged dictionary". The number is transmitted as a sum of bases and exponents in a highl compact format that fits on one line, like this: (73^3489 + 342^43 + 67^12 + 56^4) - 78. (This is not the actual number, I will update that later when I have the book in front of me).

I really have no idea if it is possible to reduce the numbers to sums of bases and exponents like this and still maintain the information in it; it seems to me to result in such a ludicrously high compression ratio that there is no way it could work - but as I said, maths is not my strong suit. And if it is possible, I don't have a clue how to go about it. In the story, IBM require a 25 year development time before they will even bid on building a machine capable of dealing with the whole number.

So, as I don't know how to reduce the size of the numbers, it is basically a scheme which takes very short strings and turns them into very long ones. Brilliant! How incredibly useful!

However, I am a sucker for the useless, so I decided to write a class which did it.

The C# built in types are not sufficient; using a double any string over about 3 characters runs out of precision and results in garbage, so I downloaded the excellent freeware BigNum library from http://www.fractal-landscapes.co.uk/bigint.html.

The code itself is pretty simple, I was just messing around so it is arbitrarily limited to lowercase alphabetic characters only and a maximum input length of 25 characters. The input length limit is because I lazily copied the first 25 primes from the Wikipedia entry. The precision in the BigInt stuff is arbitrary too; it just needs to be big. I didn't really spend a lot of time on this code :o)

// You will need the BigNum assembly on your application path and "using BigNum" in the source file

public class Godel
{
    public PrecisionSpec Precision { get { return precision; } set { precision = value; } }

    public String Chars { get { return chars; } set { chars = value; } }
    public uint[] Primes { get { return primes; } set { primes = value; } }

    // ------------------------------------------------------------------------------------------------------------------------------------------------------------
    public BigInt Encode(String s)
    {        
        if (s != null && s.Length <= primes.Length)
        {
            BigInt res = new BigInt(1, precision);

            s = s.ToLower();

            int idx;
            for (int i = 0; i < s.Length; i++)
            {
                idx = chars.IndexOf(s.Substring(i, 1));
                idx = idx >= 0 ? idx : chars.Length - 1;

                BigInt curr = new BigInt(primes[i], precision);

                // Exponents should start at 1, then we can use zero occurrences to check for end of data
                BigInt pow = new BigInt(idx + 1, precision);

                curr.Power(pow);

                res.Mul(curr);
            }

            return res;
        }
        else if (s.Length > primes.Length)
            throw new ArgumentException("String cannot be longer than " + primes.Length.ToString() + " characters");
        else
            return new BigInt(0, precision);
    }

    public String Decode(BigInt value)
    {
        String res = "";
        for (int primeidx = 0; primeidx < primes.Length; primeidx++)
        {
            int exponent = Factor(value, primes[primeidx]);

            if (exponent <= 0)
                break;

            int cidx = exponent - 1;

            if (cidx < chars.Length)
                res += chars[cidx];
            else
                res += "[" + cidx.ToString() + "]";
        }

        return res;
    }

    // ------------------------------------------------------------------------------------------------------------------------------------------------------------
    private int Factor(BigInt value, uint prime)
    {
        int res = 0;
        int sanity = 10000;
        
        BigInt remaining = new BigInt(value);
        BigInt mod;

        BigInt.DivMod(remaining, prime, out remaining, out mod);

        while(mod.IsZero() && sanity-- > 0)
        {
            BigInt.DivMod(remaining, prime, out remaining, out mod);
        
            res++;
        }

        if (sanity <= 0)
            throw new InvalidOperationException("Sanity check reached");

        return res;
    }

    // ------------------------------------------------------------------------------------------------------------------------------------------------------------
    private uint[] primes = new uint[] { 2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97 };
    private String chars = "abcdefghijklmnopqrstuvwxyz_";
    private PrecisionSpec precision = new PrecisionSpec(10000, PrecisionSpec.BaseType.DEC);
}

Pretty easy to use:

Godel godel = new Godel();

BigInt encoded = godel.Encode("test");

String originalstring = godel.Decode(encoded);

In the highly unlikely event you find a use for this, you can set the Primes property for longer strings, and the Chars to encode more than just lowercase strings. If decoding doesn't give you back the original string, you may need to increase the precision.

C#: Exceptions are slllooooowwwww

Exceptions, eh? I've always avoided them in high performace code, because everyone knows they're slow. I have never actually bothered testing this though, relying on cleverer and more dedicated people than me doing the grunt work.

Currently, I am writing a high throughput application that performs actions based on a passed in "verb".

Normally, I deal with this by setting up the verbs as an enum and using Enum.Parse() to return the verb and check it is valid at the same time.

I had no idea how quick Enum.Parse() actually is, so decided to test it.

The testing code uses the handy System.Diagnostics.Stopwatch class. It uses an enum containing 3 names, and creates a List<String> of 500,000 of each of those three values; so 1,500,000 strings in total.

The code the loops through the array and converts the strings to the enum values using either Enum.Parse() or hardcoded if / else constructs using String.Compare(String, String, StringComparison).

I also tossed a switch statement in there; usually I avoid them because I find the syntax ugly and opaque, but I read on an MS blog that switches on strings are converted to a hash comparison, so I though if that was true it may be worth checking for speed against the if / else. The switch used String.ToUpper() in the checking loop to provide case-insensitivity.

I then repeated the tests, but this time with every string an invalid enum value. Obviously with the Enum.Parse() this requires catching an exception, and with the other two just an extra condition, in all cases just setting a private member bool flag.

Here are the results:

DoTheEnumThing()

Enum.Parse
00:00:08.64

if/else
00:00:03.48

switch
00:00:02.53

DoTheEnumThingWithErrors()

Enum.Parse
00:05:15.11

if/else
00:00:05.63

switch
00:00:02.74

I think that pretty conclusively demonstrates that exception handling is slow. You certainly wouldn't want to use it in a high throughput situation where you were expecting a lot of invalid values to be passed. The switch is appreciably faster than the if / else, so this hardcoded solution, though ugly, is definitely the speediest way to go.

JS: Client side templating

I originally created this for an autocomplete widget I needed for a project ages ago; eventually it became too annoying to write all the code for the millions of conditions so I just used the YUI one instead, which is a pain in the arse to set up but works perfectly when you've done it.

Anyway, while looking around at existing solutions, they all seemed to have the problem that the markup was either fixed and styled with CSS, or generated in a callback. Neither of these solutions was ideal; the first results in a widget usable most of the time but very uncustomisable for more unusual uses, and the second is very designer unfriendly if you need to separate your coding and design teams.

So, my idea was to be able to write your HTML as normal, then mark an element as a template container. This template would contain databound variables. It would be removed from the DOM on page load, stored, then repeatedly generated for each item in a provided dataset.

Basically, it is inspired by the ASP.NET repeater control, .NET being my primary development environment.

I recently refactored the code to be more generally useful, and created a jQuery plugin wrapper that enables it to be used using the jQuery selectors and AJAX functionality.

It has uses beyond autocomplete, of course; any AJAXian updatable areas could use it.

Pros and cons are:

• Keeps the template HTML in the same file as everything else; this can be a pro if the area is specific to a certain page or you are using a "master page" or "header / footer includes" style design, and a con if the template is to be used in several places. In that case you are probably better off with a server-side templating system to generate the HTML and dumping that in using the jQuery native functions.
• Allows lightweight data transfer; instead of returning all of the HTML you can just return a JSON dataset.

The basic template code and the jQuery wrapper can be downloaded from the links below:

tiny_template_base.js
jquery.htemplate.js

Usage:

First of all, you need a datasource, which will be an array of objects, for example:

[
 {id:1, name:'Alice'},
 {id:2, name:'Bob'},
 {id:3, name:'Carol'}
]

Then, you need to set up the template:

<ul id="testTemplate">
    <li class="@ItemClass">@Text</li>
</ul>

Note the SQL style variable placeholders. When the template is created with the ID of the <ul> element the <li> will be cleared. It is then reproduced for every item in the data array.

Note: EVERYTHING contained within the template element is treated as the data row, so you can have very complex repeating templates; as a very simple example you could have two list items for every data item should you want to. This is more useful for creating table based templates.

The actual call to populate the template looks like this:

// Call using local data and the simple unwrappered template code
var template = new HTemplates.Template('testTemplate');
template.populate(data);

// Call using the jQuery wrapper and an AJAX data source
$('#testTemplate').hTemplate();     
$('#testTemplate').hTemplate().populate(data);

You can download an example page plus an example JSON datasource from the links below:

template_example_ajax.html
template_example.html

Note: You will need the jQuery framework; it has been tested with 1.4. You will also need to change the script paths to match your setup.

The jQuery plugin is the first time I have looked at jQuery; I'm not 100% sure this is the best way to create a plugin; the hTemplate().populate() call looks odd to me. Also there is probably a better way of JSON parsing that is less fragile than my check-for-open-bracket-then-eval() method.