Karl Seguin

ASP.NET Performance Framework

karl — Fri, 19 Mar 2010 15:10:00 GMT

At the start of the year, I finished a 5 part series on ASP.NET performance - focusing on largely generic ways to improve website performance rather than specific ASP.NET performance tricks. The series focused on a number of topics, including merging and shrinking files, using modules to remove unecessary headers and setting caching headers, enabling cache busting and automatically generating cache busted referneces in css, as well as an introduction to nginx.

Yesterday I managed to put a number of those things together into a framework which I hope will make it easier for developer to leverage these best practices. The project is called Metsys.WebOp, and you can download it from github. It comes with a sample application, which not only shows how to use the framework, but also has all the documentation you'll hopefully need.

The first part of the framework is a console application which is meant to be run as part of your build process. This is driven by a text file which supports 4 commands - merging files together, shrinking them, generating cache busting hashes and pre-zipping files. Here's what the sample app's command file looks like:

#combine JS files
combine: js\all.js: js\jquery.js, js\jquery.rollover.js, js\web.js
 
#combine css files
combine: css\merged.css: css\reset.css, css\main.css
 
#shrink the generated files
shrink: js\all.js, css\merged.css
 
#generate cache-busting resource hashes
busting: hashes.dat
 
#pre-zip our files
zip: js\all.js, css\merged.css

The next part is meant to be used from within a MVC application (it wouldn't take too much effort for someone to make it work with WebForms) - first by allowing you to configure the runtime component, and then by providing extension methods to HtmlHelper. Essentially this gives you 4 methods, Html.IncludeJs, Html.IncludeCss, Html.Image and Html.ImageOver. You can also toggle debug mode, which'll make all of this transparent during development (nothing worse than dealing with merged and shrank files in development).

The last part are a couple HttpModule which make everthing possible. The Zip module will returned the pre-zipped files (generated by the build process) should the browser accept zip files. The WebOp module will remove unecessary headers and add caching headers to js, css and images - only really practical if you are also using the cache busting featuers.

You can download the project from http://github.com/kseg/Metsys.WebOp.

You might also be interested in checking out the mscd project, which does a lot of the same stuff, but is probably more mature.

WebForms vs MVC (again)

karl — Thu, 11 Mar 2010 16:12:00 GMT

There's a new video up on www.asp.net which aims to help developers pick between ASP.NET WebForms and ASP.NET MVC. The video boils down to 5 benefits per technology which Microsoft thinks you should consider.

Let's go over the points, shall we? First, ASP.NET WebForms:

1 - Familiar control and event base programming model

The claim here is that the ASP.NET model is comfortable for WinForm programmers (thankfully this unbiased analysis left out who it's more familiar for). This is largely accurate, but disingenuous. The differences between web and desktop cannot be overstated nor can one overstate how bad ASP.NET's (or any other framework) is at hiding the difference. "Familiar" is probably the right word to use so long as you recognize that, in this case, at best it means: superficial; at worst: a serious pain in the ass. Your knowledge of building a VB6 app will allow you to write a "Hello World" web application - great.

Familiarity can be a liability when it tries to force a square peg into a round hole.

It also largely relies on your inability (or unwillingness) to learn. Today, next month or even next year may not be the right time for you to learn something new - that's fine. Eventually, sticking with what you know, only because you know it, will kill your career and possibility part of your spirit.

2 - Controls encapsulate HTML, JS and CSS

It's true that in ASP.NET WebForms controls can, and frequently do, encapsulate HTML, JS and CSS. How this ads "value" is beyond me. You can't, and shouldn't, be trying to build website without a solid command of HTML, JS and CSS. Whatever programming language and framework you use, the ultimate output of any website is HTML, CSS and JavaScript. Your server code essentially generates a stream of characters, which a browser loads and renders. To suggest, or think, that generating HTML, CSS or JavaScript in C# has any advantage is insane. It'll be more complicated to learn, do and maintain - and the end result will be inferior. Its like saying we should write C# in VB.NET; or drive cars by bolting planes to the roof and getting in the cockpit.

3 - Rich UI controls included - datagrids, charts, AJAX.

Point 3 is a different perspective on point 2, which is a different way of saying point 1. However, it is the most interesting and important perspective. Fancy tables and charts, as well as client-side behavior, shouldn't be a server-side concern. This is fundamental to what we all know about good and bad design. Classic ASP was a mess because it intermingled presentation code with server side code. The value of WebForms is that presentation logic is now a server side concern. Do you really believe this? Would you consider generating your HTML from a stored procedure?

The claim also implies that by using ASP.NET MVC, you won't be able to have a rich UI. In truth, you won't only have access to a wider range of controls; you'll also avoid a bunch of poor abstractions, and generate JavaScript by writing JavaScript, css by writing css and html by writing html.

4 - Browser differences handled for you

I'm guessing that the claim is that some of the controls mentioned in point 3 might render different HTML based on the requesting browser. Guess what, most jQuery (or any other js framework) plug-ins are fully compatible with all relevant browsers because they too can generate different HTML. In fact, doing this on the client side is almost always better - since you can tell the exact capabilities of a browser.

Also, it would probably be better if you generated correct HTML, CSS and JS in the first place - something you can't normally control using ASP.NET WebForms. So not only is this really just a benefit because IE is a pain, but its only worth mentioning because points #1, #2 and #3 mean that you've lost complete control over doing it right in the first place.

5 - SharePoint builds on WebForms

Yes, if you use SharePoint, you'll have to use WebForms.

Now on to ASP.NET MVC:

1 - Feels comfortable for many traditional web developers

ASP.NET WebForms is familiar while ASP.NET MVC is comfortable - that's helpful. Nonetheless, when I see "traditional" I think "not-modern". A more honest counterpoint to the WebForms claim would be: "A more natural way to build web applications". WebForms tried to help WinForm developer's transition to the web. ASP.NET MVC is a model that better reflect the realities of programming on the web. It's more than just comfort, and has nothing to do with tradition.

2 - Total control of HTML markup

HTML, JS and CSS are yours to command. That doesn't mean you can't use controls to speed up development and improve your applications. The way this is worded sure makes it sound like MVC is a lot more work than WebForms though. It isn't.

3 - Supports Unit Testing, TDD and Agile methodologies

I'm not sure what the technology stack has to do with the development methodology, so we'll just ignore the last part. That aside, its true that MVC makes it possible to unit test your code. The counter point to that is that WebForms is essentially impossible to unit test. This also understates the architectural superiority and design of MVC - its doesn't only allow you to leverage a number of best practices, it itself is actually built around those same practices. Code that can be unit tested, regardless of whether it is or not, is almost always superior to code that cannot.

4 - Encourages more prescriptive applications

So if ASP.NET MVC lets you build you application the way you should be building it, should you infer that ASP.NET WebForm forces you to build applications the wrong way? Yes, you should.

5 - Extremely flexible and extensible

Both frameworks share this value - but ASP.NET MVC is more about building on top of existing code, while ASP.NET WebForms is more about hacking things until they work. If you think this means that ASP.NET MVC can only be useful once you've extended it, then you are wrong. It works great out of the box is is feature rich.

Other Stuff

The video goes on to make weird assertions, like the possibility of turning back and picking a different stack if you feel you've made the wrong choice because of how similar and how much infrastructure they share. The better solution is to pick the right technology because going back months or years into your project doesn't sound like good advice to me.

It also mentions that it's common to have some pages handled by MVC and others by WebForms. It's good to know that you can do this - especially since it's a good way to upgrade from WebForms to MVC. However, I'd hardly call it common or even recommended. It's a useful transitional tool which you should aim to get out of as quickly as possible.

Ultimately, the first 4 values of WebForms all boil down to the same thing: there's a System.Web.UI namespace which represents the wrong way to build a web app. There are good reasons to pick WebForms - but they all come down to time and practicalities of learning new things (and SharePoint). I won't tell you that you have to learn MVC, because that may not be practical for you. I'll repeat what I've said before, ASP.NET MVC and WebForms DO NOT serve different purposes and one is not better suited for a particular type of application than the other(except SharePoint). They are completely overlapping technologies, and ASP.NET MVC is superior to WebForms.

The 8th Phase

karl — Tue, 09 Mar 2010 15:27:00 GMT

I once posted a semi-serious post entitled The 7 Phases of Unit Testing. The phases are:

Refuse to unit test because "you don't have enough time"
Start unit testing and immediately start blogging about unit testing and TDD and how great they are and how everyone should do it
Unit test everything - make private methods internal and abuse the InternalsVisibleTo attribute. Test getters and setters or else you won't get 100% code coverage
Get fed with how brittle your unit tests are and start writing integration tests without realizing it.
Discover a mocking framework and make heavy use of strict semantics
Mock absolutely everything that can possibly be mocked
Start writing effective unit tests

I think the cycle I went through is extremely healthy - unit testing is something best learnt from practice and is something you refine over time. Judging by the comments on the original post, a lot of you agree.

Recently though, I've felt like adding another stage:

8 . Sometimes the best tests aren't unit tests

After awhile, it becomes obvious that some tests are significantly more meaningful when you expand your scope - say to include hitting an actual database. Narrow unit tests and wider integration tests can always work together; but I've found that, in some cases, more comprehensive tests can replace corresponding unit tests. This may not be a proper, but it is practical.

When I say unit test, I mean the smallest possible unit of code - generally a behavior. Most methods are made up of 1 or more behavior. I'd say that you shouldn't have too many methods with more than 6 behaviors (as a rough goal). As an obvious example, in NoRM this method helps identify the type of the items in a collection:

public static Type GetListItemType(Type enumerableType)
{
    if (enumerableType.IsArray)
    {
        return enumerableType.GetElementType();
    }
    if (enumerableType.IsGenericType)
    {
        return enumerableType.GetGenericArguments()[0];
    }
    return typeof(object);
}

Clearly, this method is a good candidate for 3 or 4 unit tests (one when the type is an array, one when it's a generic, one when its something else, and maybe one when its null).

As your code moves closer to the boundaries of 3rd party components, the value of unit testing may suffer. You'll still get the benefits of flushing out coupling and enabling safe refactoring (which shouldn't be underestimated), but you'll likely miss out on making sure things will work like they should in production. The solution can be to expand the scope of your tests to include the 3rd party component.

The most common example is database code. Testing a Save method by mocking the underlying layer might work, but there's value in making sure that the object actually does get saved. That isn't to say that a single test that hits the database is good enough - your Save method might be made up of multiple behaviors, some which are better validated with one form of testing than another.

Really, that's one of the key things to remember as you walk down this path - don't think that just because your method is actually saving an object that your job is done. There are likely other behaviors that aren't being tested at all. It's easy to abuse these types of tests and get a false sense of security. The other key is to make sure that it runs like a unit test - namely, that its fast, doesn't require any manual setup, and isn't dependent or doesn't break any other test.

Lately, I've seen interest in using in-memory databases for this type of thing. The benefit is that they are super fast and don't leave stale data. They also don't require special setup. On the downside you still aren't truly testing the most fundamental behavior of your method - that an object will be saved to the database in production. Even with the best O/R tool I've seen code work against one database but not work against another - due to a bug on my part. Writing a script that can automatically and quickly setup and teardown against the final database, and having your team members set up a local database, may or may not work for you (it'll depend on the nature of your team and your system).

Ultimately, the most important thing is that you have automated tests which aren't a nightmare to setup, maintain or run. Integration tests have more dependency and thus are more fragile, but can be an efficient way to verify correctness.

BSON Serialization

karl — Fri, 05 Mar 2010 18:56:00 GMT

BSON is a binary-encoded serialization of JSON-like documents, which essentially means its an efficient way of transfering information. Part of my work on the MongoDB NoRM drivers, discussed in more details by Rob Conery, is to write an efficient and maintainable BSON serializer and deserializer. The goal of the serializer is that you give it a .NET object and you get a byte array out of it which represents valid BSON. The deserializer does the opposite - give it a byte array and out pops your object. Of course, there are limits to what they can do - they are mostly meant to be used against POCO/domain entities.

Grammar
The first thing to understand when building serializers is how to read grammar. In programming languages, grammar is a way to express the valid keywords and values a parser might run into. Both the JSON and BSON grammars are great to learn, given how simplistic yet powerful they are. The JSON grammar, available on the homepage of json.org gives a nice representation of what valid JSON should look like. The BSON grammar, available at bsonspec.org under the specification button, follows a more traditional dialect. Essentially, you have symbols on the left and expressions on the right. The expressions can, and often will be, made up of additional symbols and or actual values. Eventually though, you'll end up with a symbol which is only made up of values - which means you can stop going down the rabbit hole. Its also very common for a child symbol to reference a parent symbol - but eventually something breaks this cycle.

An Example
So, say we wanted to serialize the following json:

{"valid": true}

Everything in BSON starts with a document. From the bson specification, we can see that a document is made up of a 32bit integer (representing the total size of the document, including the integer itself), another symbol called an e_list, and finally a termination character. As a start, we'd have something like:

Now, an e_list itself is made up of a symbol called an element followed by another e_list or an blank string. An element is made up of a single byte type (with \x08 representing a boolean), a symbol called e_name and a byte value for true or false. So now we have:

The only thing missing now is our e_name (which represents the word "valid" in the original JSON). An e_name is really just a cstring which is our value UTF8 encoded into an array of bytes with a trailing byte of \x00:

Our final byte array looks something like:

Serializing a single bool value might be the simplest of cases, but once you understand that, you're well on your way to being able to serialize anything. Sure, serializing an array might be a bit trickier, since each element within the array is its own document - but the challenge is mostly implementation versus conceptual.

What's the Length?
It may surprise you at first, but the most difficult part to implement is actually determining the length of a document (or various other symbols which have a length). The problem is that we don't know the length until after we've serialized it. Some implementations will essentially serialize the object graph twice, first to calculate lengths, then to write out the array. In NoRM we do things more efficiently. We keep a linked list of documents, and a pointer to the current document. A document is a very simple object - it keeps track of where it started, who its parent is (null in the case of the root document) and how much data was written. When a new document is needed - say when we start serialization, or when the grammar dictates that we need a new document (arrays or nested objects), we mark where we are and write out a placeholder length. Then, when the document ends, we seek to our placeholder, and write out the length. The relevant code looks like:

  private void NewDocument()
  {
      var old = _current;
      //we start the Written at 4 because of the length itself
      _current = new Document { Parent = old, Start = (int)_writer.BaseStream.Position, Written  = 4};      
      _writer.Write(0); //length placeholder
  }
  private void EndDocument(bool includeEeo)
  {
      var old = _current;
      if (includeEeo)
      {
          Written(1);
          _writer.Write((byte)0);
      }            
      _writer.Seek(_current.Start, SeekOrigin.Begin);
      _writer.Write(_current.Written); //override the document length placeholder
      _writer.Seek(0, SeekOrigin.End); //back to the end
      _current = _current.Parent;
      if (_current != null)
      {
          Written(old.Written);
      }

  }
  private void Written(int length)
  {
      _current.Written += length;
  }

EndDocument is pretty interesting. Since the length of a nested document contributes to the length of the parent document, we need to make sure to update the parent (now current) document with the length of the nested one.

Conclusion
Everything else is pretty straightforward in terms of serialization - largely reliant on reflection and reflection helpers. We use Jon Skeets reflection to delegate approach to make things even faster (something I truthfully don't fully understand). Currently our implementation has some coupling to other NoRM components. Hopefully one day the BSON stuff will be stand-alone. If you can't wait, you can either use another library like JSON.NET (which more mature anyways), or spend a few minutes (it shouldn't take more than that) pulling out our serializer/deserializer.

I don't like MSDN

karl — Thu, 11 Feb 2010 20:53:00 GMT

I've long held that Microsoft could do better with its developer-focused sites. It isn't a topic I'm likely to let up on, but today I wanted to look at a specific example.

I was reading the PostgreSQL documentation on the ALTER INDEX command, and was generally pleased with everything about it. Its easy to read, informative, quick to load, and usable. Here's a screenshot of the page as it first appears (or you can go there yourself at http://www.postgresql.org/docs/8.1/interactive/sql-alterindex.html):

The MSDN page for ALTER INDEX loads up looking like (or you can go there yourself at http://msdn.microsoft.com/en-us/library/ms188388.aspx):

Maybe its just me (is it?) but for me, the difference is night and day. I can put aside the poor color choice, and weird (and branded) search bar, and the overwhelming amount of navigation (actually, I find this quite annoying), but there are some fundamental things that make MSDN a pain to use.

Some observations:

The MSDN breadcrumb is 12 levels deep, two lines, and each item brings up its own popup navigation. The PostgreSQL one is 4 levels deep and is plain.
The MSDN has a Printer Friendly Version, the PostgreSQL one IS printer friendly.
MSDN's left-hand navigation is useless - it's too big, too nested and uses up too much room
The MSDN page took 46 requests, 226.5KB and took 5.44seconds to load for me. The PostgreSQL page took 16 requests, 21.3KB and took 1.46seconds to load
The MSDN page hits 8 domains (including webtrends). The PostgreSQL page hits 2 domains (including google analytics)
The MSDN page has a cryptic URL. The PostgreSQL has a human-readable url
The MSDN page allows you to lookup other versions via a box in the middle. The PostgreSQL lets you change the URL.
The MSDN page scores 67 on Yslow, PostgreSQL scores 76.
The MSDN page is inconsistent about cross linking other topics, sometimes CREATE TABLE is linked and sometimes it isn't. CREATE INDEX though is never linked. The PostgreSQL documentation is more consistent.
Searching "Create Table" on MSDN brings up the SQL Server 2000 documentation (with no way to get to 2008). Searching "Create Table" on PostgreSQL brings up exactly what you want. (Both sites display search results poorly)
MSDN's Titles don't include the version, so the 2005 and 2008 titles are the same (which means your bookmarks will be the same). PostgreSQL has stupid long titles, but at least the version is there
The MSDN text is small and tight and ends up looking like a wall of text. The PostgreSQL looks like its meant for a human.

A note about the version selection. The MSDN approach may be more intuitive, but it appears limited to going from SQL Server 2008 to 2005, there's no way to go the other way around, and no way to pick an older database. With PostgreSQL you can change the url for any minor version starting at 7.1.

Of course, when everything's said and done, despite all the small things (like # of dns lookups), the end result is that one is a mess, while the other isn't.

MSDN's Lightweight and ScriptFree are definitely a step in the right direction - but this choice clearly falls in Barry Schwartz's Choice is Bad For Us point of view - something a lot of people at Microsoft need to desperately be made aware of. I actually like ScriptFree, and it addresses a number of the above problems, but not enough and in the end it still lags behind PostgreSQL's usability. I mean, why is the ridiculous syntax the first thing even displayed? Why do I still see a wall of text, and who actually thinks that because I'm reading up on ALTER INDEX, I'm interested in a bunch of truncated links to other ALTER statements?

An introduction to iptables

karl — Wed, 03 Feb 2010 14:36:00 GMT

iptables is a linux application that allows you to configure the built-in linux firewall. There are abstraction layers built on top of it, including nice GUIs, but I recently had some very basic rules I wanted to implement, and figured I'd learn how to do it directly in iptables. It turned out to be easy to configure, and I thought I'd give a very basic introduction.

First though, I relied on the ubuntu documentation to get up and running (https://help.ubuntu.com/community/IptablesHowTo). I'll pretty much just regurgitate what's in there - I've said it before, stop believing that linux documentation is written only for people who already know linux. Also, depending on your setup, most of these commands might require elevation (sudo).

By default, iptables defines three rule chains: INPUT, FORWARD and OUTPUT. We'll only concern ourselves with INPUT - namely the chain used for incoming packets directed at this particular machine. At any point, you can list the rules by entering iptables -L. If listing the rules turns out to be really slow, try using iptables -L -n. The -n option means that the display will output IP addresses and ports in numeric format, without trying to do a DNS lookup. If the -n option makes it fast, then you're system likely has incorrect DNS settings (you can check out http://www.cyberciti.biz/tips/linux-how-to-setup-as-dns-client.html for help with that).

In my particular case, the server is has two network interfaces - a public facing one (on eth0) and a private one (on eth1). For my purposes, I want to remotely (eth0) access SSH from my static IP, and locally (eth1) access postgresql on port 5432.

To accomplish this we simply append two rules to the INPUT chain:

iptables -A INPUT -p tcp --dport ssh -s 205.38.19.199 -i eth0 -j ACCEPT
iptables -A INPUT -p tcp --dport 5432 -i eth1 -j ACCEPT

The first rule appends itself to the INPUT chain. Its limited to the TCP protocol targeting the SSH port (22) with a source originating from 205.38.19.199 on the eth0 network interface. It jumps to an ACCEPT status (all other processing stops). The next rule is similar, except it allows any source so long as they are connected to eth0 and targeting port 5432 (over tcp).

We can now add a third rule to block all other access:

iptables -A INPUT -j DROP

if incoming traffic doesn't mean the first two criteria, we'll hit this third rule and the firewall will reject the connection.

We'll run into 1 issue with our above configuration though - we've blocked access to the loopback interface (127.0.0.1). We need to add a special rule to allow loopback connections (since a lot of programs rely on it). However, we can't simply append it as it'll never be reached. Instead of using -a to append, we'll use -i to insert:

iptables -I INPUT 1 -i lo -j ACCEPT

Here we are inserting our rule in the 1st position, and accepting all connections made on the special lo (loopback) interface.

A really neat thing about iptables is that they aren't persisted by default. So, if you happen to mess up and cut yourself off from your server, a simple reboot will reset everything (hopefully you have access to do a remote reboot from a web interface, or maybe you can just walk up to the machine and reboot it).

At some point though, you'll want to make sure your rules are persisted. Achieving this is a two step process. First, you save your rules to a file by using the iptables-save and redirecting STDOUT to a file:

iptables-save > /etc/iptables.rules

Second, and this depends on which distro you have but on debian/ubuntu, you make sure your rules are loaded on startup. Simply edit /etc/network/interfaces, find your interface, and add:

pre-up iptables-restore < /etc/iptables.rules

to the list, so that it looks something like:

auto eth0
iface eth0 inet static
address xxx
netmask yyy
gateway ddd
pre-up iptables-restore < /etc/iptables.rules

That's it!

Beyond web.config

karl — Thu, 28 Jan 2010 14:43:00 GMT

I recently needed a configuration mechanism which would detect changes without requiring an application domain restart. I also wanted to move away from XML. This is what I came up with (and hopefully I'll get some helpful feedback).

First, we declare a ConfigurationData object which holds our actual configuration values:

public interface IConfigurationData
{
   bool LogAll{ get; }
   string CdnUrl{ get; }
}

public class ConfigurationData : IConfigurationData
{
   private bool _logAll;
   private int _cdnUrl;
   
   public bool LogAll
   {
      get { return _logAll; }
      set{ _logAll = value; }
   }
   public string CdnUrl
   {
      get { return _cdnUrl; }
      set { _cdnUrl = value; }
   }
}

There isn't much to explain here, so let's move on to the class that does the heavy lifting:

public static class Configuration
{
   private static readonly string _applicationPath = HttpRuntime.AppDomainAppPath;
   private static ConfigurationData _instance = LoadInitialConfiguration();

   public static IConfigurationData GetInstance
   {
      get { return _instance; }
   }

   private static ConfigurationData LoadInitialConfiguration()
   {
      var watcher = new FileSystemWatcher(_applicationPath, "settings.config");
      watcher.NotifyFilter = NotifyFilters.LastWrite;
      watcher.Changed += (s, e) => _instance = LoadConfiguration();
      watcher.EnableRaisingEvents = true;
      return LoadConfiguration();
   }
   private static ConfigurationData LoadConfiguration()
   {
      return Converter.DeserializeFromFile<ConfigurationData>(_applicationPath + "settings.config", "_");
   }        
}

The class essentially loads the ~settings.config file, and sets up a watch to reload it whenever it changes. Here I'm using JSON (and my own JSON library) to read the file, but you could use anything, such as yaml or xml. The file might look something like:

{
   "logAll": true,
   "_cdnUrl": "http://cdn.mysite.com/"
}

If you are using a DI framework, such as ninject, you can hook it via:

Bind<IConfigurationData>().ToMethod(c => Configuration.GetInstance);

otherwise, you can call it directly from code with:

if (Configuration.GetInstance.LogAll) {...}

There are two things you'll need to be careful with. First you'll want to make sure to avoid caching values within a class. For example, if you did something like:

public static class HtmlExtensions
{
   private static readonly string _cdnUrl = Configuration.GetInstance.CdnUrl;
}

then a copy of CdnUrl would be stored in _cdnUrl and changes to your file wouldn't be available to this class.

Secondly, the Changed event is known to fire twice. This shouldn't cause any threading issues, but if your LoadConfiguration is particularly intensive, or if you happen to be debugging something, then you'll at least know to expect it.

Don't Use Try/Catch

karl — Mon, 25 Jan 2010 15:02:00 GMT

Exception handling has been talked about a lot over the last decade and a half. However, despite a general consensus on how to properly handle exceptions, a divide on usage continues to exist. Improper exception handling is easy to spot, easy to avoid, and is a simple code (and developer) quality metric. I know absolute rules come off as close minded or exaggerated, but as a general rule you shouldn't be using try/catch.

If an exception happens, you need to know about it. If a truly unexpected exception happens, you're better off (most of the time) crashing than letting the application continue. This is particularly true in a web application where you'll only crash 1 thread for 1 user while your site remains accessible. The best way to achieve both is let the exception go unhandled and log the exception in a global exception handler. Too often I see developer writing code which masks errors and make it impossible to build a quality system. Users complain about system instability, and developers are left scratching their heads with no leads on what the problems are.

What's even more frustrating is when framework code swallows exceptions. Take a look at this example, written by Microsoft:

private void LoadImage()
{
    if (this.image != null)
    {
        this.image.Dispose();              
    }
    

    if (source != null)
    {                
        try
        {                  
            source = Path.Combine(Canvas.ApplicationPath, source);                    
            image = new Bitmap(source);                  

           IImagingFactory factory = ImagingFactory.GetImaging();
           factory.CreateImageFromFile(source, out imagingImage);
        }
        catch{}                                                      
    }
}

This is a fundamental piece of code which is part of Microsoft's UI Framework for .NET Compact Framework. The above code is buried within their framework, but through various key APIs is exposed and used extensively. This is simply horrible code. Why would they swallow any exceptions that come out of this code? The above code means that if you try to load an image "images/critical_warning.png" which doesn't exist, the application will continue to work normally. There may be cases where this is the behavior that you want, but those would be edge cases, and they should be decided at the application level, not within the framework.

The only time you should swallow exceptions (catch{..} or catch(Exception){...}) is in your global exception handler to avoid truly fundamental issues (and recursive global exception handling). You should catch specific exception (catch(XmlException){...}) when an exception isn't unexpected and you know that you can safely proceed.

A system that swallows and masks exceptions isn't a healthy system.

The WebForms Rant

karl — Fri, 22 Jan 2010 19:15:00 GMT

Ever since ASP.NET MVC was announced/released, there's been a lot of talk about it with respect to ASP.NET WebForms. People want to know which technology they should use, and are likely confused by the fact that there isn't a consensus amongst respected developers with respect to which of the two is better.

Ideally every developer would take the time to truly understand ASP.NET MVC and compare that with their WebForms experience to come up with their own opion (and even more ideally we'd all largely agree on the pros and cons). The reality though is that some developers don't have the time or opportunity to take the necessary time to learn a new framework. If we are being completely honest, many developers don't have the necessary skill to accurately judge two competing technologies - maybe they are casual developers, maybe they are new to developing; maybe they come from a different technology. There's nothing shameful about that - in fact, that's how most of us learn.

With that said, should you be using ASP.NET MVC instead of WebForms? The short is: continue using WebForms for any existing projects that you have that are working well (you can define for yourself what "working well" means). Use ASP.NET MVC for any other project.

The long answer is the same, except it uses the word "stupid" and "idiots" a lot more. Here goes...

So-called experts who claim that ASP.NET MVC and WebForms serve two different purposes are stupid idiots. Seriously. They are both web frameworks, capable of producing the exact same systems. You could build a digg, facebook, twitter, joomla or any other website using either technology. What makes one (ASP.NET MVC) better than the other (ASP.NET WebForms) isn't where you can go, but rather how you get there. ASP.NET WebForms is an ugly and messy framework that complicates an otherwise simple thing. ViewState, codebehind, postback, page lifecycle and databinding are things that you have to constantly program against. They don't help you; they just slow you down, slow your site down, and make it harder to change things in the future.

I've known programmers who thought that complex frameworks were required to build complex systems. I also know programmers who think you can just duct tape anything together. The truth is so much simpler, and so much better, that both those groups of programmers are idiots, and in fact can't really program. Simplicity is the only way to build simple or complex systems in a clean and timely manner. Build a system too complex, and it'll cost a fortune and be too rigid. Build something too quickly and shitty, and it'll break and won't be fixable. ASP.NET WebForms, along with ASP.NET Ajax, was clearly built by people who believe that complexity can solve all problems. Postback with server side events is such a poor solution to what’s a really simple problem. ViewState is a solution to a problem that doesn't even exist (which makes it infinitely complex).

When WebForms first came out, developers were overjoyed that their tangled code and html were a thing of the past. We were so stupid not to see what mess we were getting involved with (I'm sure some did). Get it through your head people, having ASPX pages loaded with non-html or javascript is an equally horrible situation to be in. Take your most complicated ASPX page and give it to someone who knows HTML (but not ASP.NET) - if they can't understand it (which they won't) or think it looks ridiculous (which they will) then you're doing something wrong. Just because you are coding with angle brackets doesn't make it clean code.

Pro-WebFormers like to use a scare tactic to make sure developers don't switch to ASP.NET MVC, they claim that ASP.NET MVC is suited for Test Driven Developer (TDD) specifically, and unit testing in general. This is a scare tactic that relies on the growing divide between those who do and those who don't unit tests. You can use ASP.NET MVC without writing a single unit tests, or knowing how to write a unit test, or knowing what a unit test is. It is true that ASP.NET MVC is easier to unit test than ASP.NET WebForms (which is next to impossible). Even if you don't write unit tests, or know how to write unit tests, or know what a unit test is, believe me that the simple fact that ASP.NET MVC can be unit tested and WebForms can't, is a very compelling reason to forget about WebForms. It means that ASP.NET MVC is better designed, is more flexible, has less coupling, and less magic shit which will break.

If someone tells you that ASP.NET MVC is more complicated, requires more heavy lifting or is less productive, they are trying to tell you a lie. Of course there'll be the lost productivity of having to learn something new. But I promise you that mastering ASP.NET MVC will take a fraction of the time its taken you to master the clusterfuck known as ASP.NET WebForms (just like learning jQuery will take you less time than learning ASP.NET Ajax). Maybe you don't have time to learn something new, that's fine. Just don't believe what you've been reading. ASP.NET MVC is simpler - unless you're the type of developer who doesn't like to learn new things.

Now, I have my problems with ASP.NET MVC, which I've shared in the past. But those are problems with ASP.NET MVC with respect to other MVC frameworks.

A framework that accepts that HTTP is stateless will always be simpler, cleaner and more powerful that a framework that doesn't. A framework that allows unit testing, regardless of whehter you are going to unit tests, is better than one that doesn't. WebForms is nasty.

ASP.NET Performance - Part 3a - Better Cache Busting

karl — Fri, 15 Jan 2010 14:53:00 GMT

There have been some good, informative, comments to this series. One in particular, from Dan Sargeant, deserves to be addressed directly - in large part because its something I overlooked but need to implement, so I might as well share the code.

Dan points out that my cache busting technique has a fatal flaw - it doesn't address images referenced from stylesheet. He's right, and it's a pretty important issue to fix for two reasons. First, a lot of images are references from stylesheets, secondly, we're returning a long expiry headers for all of those assets, so we really need a way to bust the cache.

The solution is simple, we'll take the code introduced in Part 3 which calculates the hashes and generates our data file, and add a 3rd step which relies on regular expressions to update images referenced in stylesheets. First we define our regular expressions:

private static readonly Regex _stylesheetPattern = new Regex("'\\.\\./images/(.*?)(\\?.*?)?'", RegexOptions.Compiled);

This assumes that your structure looks something like:

assets/css/main.css
assets/images/background.gif

meaning that your image references will look something like:

#main{background:url('../images/background.gif');}

Next we run the code which'll find all .css files and do a search and replace:

private static void ProcessStyleSheets(string root)
{
   foreach(var directory in Directory.GetDirectories(root))
   {
      //recurse
      ProcessStyleSheets(directory);
   }
   foreach(var file in Directory.GetFiles(root, "*.css"))
   {
      var content = File.ReadAllText(file);
      content = _stylesheetPattern.Replace(content, new MatchEvaluator(ApplyHashToMatchedStyle));
      File.WriteAllText(file, content);
   }
}
 
private static string ApplyHashToMatchedStyle(Match match)
{
   var key = string.Concat("/assets/images/", match.Groups[1].Value);
   if (_hashes.ContainsKey(key))
   {
      return string.Format("'../images/{0}?{1}'", match.Groups[1].Value, _hashes[key]);
   }
   return match.Groups[0].Value;
}

Finally we change our main program to call the ProcessStyleSheets method:

...
_hashes = new Dictionary<string, string>();  
ProcessDirectory(root, assetPath);  
WriteFile(root + outputFile);  
ProcessStyleSheets(root + assetPath);

I also want to mention that Herman, who had a number of valuable comments, pointed out that, by default, some proxies won't cache based on querystring (Squid being the best known). Instead, we need to make our hash part of filename. This is something I'll leave to you to play with, but I will point out that this is something Nginx's rewriteurl module would be very well suited for.

You should know Linux

karl — Thu, 14 Jan 2010 13:00:00 GMT

Its hard to get everyone to agree on what core competencies developers should have. I've long believed that knowledge of C is fundamental to becoming a great developer. Actually, C in general, memory management, pointer arithmetic, and basic hardware (interupts, scheduling, ...) more specifically. Up until recently, when someone asked me what they should learn to be a better programmers, I'd always suggest C. As someone else put it "Not 'getting' algebra is not acceptable for a mathematician, as not 'getting' pointers is not acceptable for programmers. Too fundamental."

I'm now feeling pretty comfortable in including Linux as a must-know skills. I'd go as far as saying that for certain type of work, like web development, knowing Linux is more important than knowing C.

I'm not arguing that you need to move your development to Linux, I'm saying that you need to be comfortable enough with Linux to be able to leverage it. Linux is the platform of choice for a massive list of tools that you should have in your toolbox. Not having access to those tools puts you and your company at a disadvantage. It isn't just about having access to programs either, its about having knowledge about alternative ways to do things. I speak to a lot of ASP.NET developers who don't know what in-memory distributed caching is or who don't understand non-relational persistence options. This ignorance is a major liability which will be hard to address as long as developer remain uncertain and fearful of Linux.

The other nice thing about Linux is that its dead easy to setup and configure. The documentation no longer solely consists of cryptic man pages, and the community is broader than ever. If you can't get LAMP installed, then you either picked an uncommon distro, or you haven't tried very hard.

If you are really serious about diving in, I offer you some choices. First, convert an old PC to a linux server. An single core P4 with 1gb of RAM is more than enough to run a linux server. You can also get a Mac, since Mac OS X is based on BSD (close enough to Linux for our purposes), and has great documentation for developers to install, configure and learn all types of useful things. You can get a nettop. I'm currently running an ASRock 330 (non-ION or blu-ray - unless you need/want them), which has a dual core Atom 330 + 4gb of ram (added afterwards by me). Finally, you can rent an unmanaged VPS. Probably the most popular provider, due to its performance, price, and the fact that you have full root access, is Linode - however there's plenty of choice.

Which option you pick should largely be based on your needs. If you'll want something that you can actually use to host something, then a VPS is clearly the way to go. Otherwise, reuse an old machine if you can, or pick between a cheap low-end dedicated nettop or a more expensive (but also more useful) mac.

As for distributions, I'd recommend Ubuntu - in large part because of the excellent documentation and community (though CentOS is a commone server of choice in production). You can pick the Desktop or Server edition. I picked the server since I only ever SSH into the machine and have no need for a GUI.

ASP.NET Performance - Part 5 - Nginx

karl — Wed, 13 Jan 2010 13:48:00 GMT

The previous parts of this series focused on small pieces of code we could use to maximize real-world performance - largely based off of YSlow recommendations. In this final installment we'll think outside the box (literally) by placing a reverse proxy infront of IIS to get the most bang possible. Our technology of choice is Nginx running on Linux (while Nginx runs on windows, it doesn't seem to support shared memory - which is critical to a number of features).

Nginx (pronounced engine-x) is an event-driven asynchronous server. Such servers are specialized at handling extreme loads while consuming limited resources. On top of this foundations are a number of modules which enable various feature. Such features include load balancing, compression, reverse proxy (which is mostly what we'll look at), fastcgi, memcached, url rewriting and so on. Nginx is used by a number of high profile sites, including Wordpress, GitHub and SourceForge. Oh, and its completely free. (Another equally popular and equally free alternative is lighttpd).

Nginx is prettty easy to configure and the configuration file is pretty easy to read. However, if you prefer to learn by doing but don't have linux setup, then you can use the windows version (that's how I started). You'll be able to get a feel for it and do most of the things we'll be looking at.

For the most part we'll focus on Nginx's reverse proxy module. A reverse proxy [typically] sits infront of your web server, accepting all connections and deciding on a course of action. In its simplest form, our nginx.conf file will have the following reverse proxy settings:

user www-data;
worker_processes  5;

error_log  /var/log/nginx/error.log;
pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
}

http {
    server {
        listen       80;
        server_name  oursite.com www.oursite.com;
        location / {
            proxy_set_header  x-real-IP        $remote_addr;
            proxy_set_header  x-forwarded-for  $proxy_add_x_forwarded_for;
            proxy_set_header  host             $http_host;
    		proxy_pass        http://10.10.1.1/;
        }	
    }
}

We've configured Nginx to listen on port 80 for connections with a host header of oursite.com or www.oursite.com. When Nginx receives a request, it will look for a matching server, and a matching location (here we are matching all locations), and then issue a request to our imaginary windows machine at 10.10.1.1. Since Nginx is actually issueing the request to our web server, we are adding headers to the request so that the web server has all necessary informaton.

The above configuration gets us up and running, but doesn't really do anything (except for add another point of failure). However, by adding just two more lines of code, things can really get awesome:

#define a shared memory named cache (don't think this will work in windows)
proxy_cache_path /usr/local/nginx/proxy_temp/ levels=1:2 keys_zone=cache:10m inactive=10m max_size=1000M;

server {
    listen       80;
    server_name  oursite.com www.oursite.com;
    location / {
        proxy_set_header  x-real-IP        $remote_addr;
        proxy_set_header  x-forwarded-for  $proxy_add_x_forwarded_for;
        proxy_set_header  host             $http_host;
        proxy_pass http://10.10.1.1/;
        proxy_cache cache #enables the cache
    }	
}

First we've defined a zone where cached data is to be stored (called cache), as well as how much room to allocate in memory (for cache lookup metadata) and on disk (for the actual cached data). Then, we've modified our location and simply told it to allow caching using the zone called cache. This means that once oursite.com/assets/images/logon.png?23188a4 has been requested, Nginx will keep a copy on disk and use its local copy instead of forwarding the request. We are still serving the same total number of requests, but the load for our static/plain files has been moved from our Windows machine which should focus on executing our code (and which is suceptible to the C10K problem) to a highly specialized, low-resource system. There are actually a lot of benchmarks (as well as real life case studies) which show Nginx's destroying Apache (which is thread-based like IIS) when it comes to high concurrency, low resources and response time.

Note that we aren't just limited to caching requests for images, css and javascript file - Nginx will cache any response that has the appropriate headers. You can take your homepage, add an OutputCache (directive or MVC attribute) with the location to Downstream and watch your your site fly.

We can even add a new location to our server (locations support regular expression matching too, incase you need it), and do something like:

location /favicon.ico {
    proxy_pass	http://10.10.1.1/favicon.ico;
    proxy_cache cache;	
    expires 30d;
}

This showcases Nginx's expires directive which can be used to set an expiry header.

Moving on, we can also use Nginx's Gzip module to compress responses. Again, this is something IIS can do, but there's something reassuring about letting our application machine worry exclusively on running our code. With the following lines, applied globally (so that it relates to any server definition), nginx will compress all responses greater than 500 bytes for a number of different filetypes, providing the browser isn't < IE 7:

gzip  on;
gzip_buffers 16 8k;
gzip_comp_level 6;
gzip_proxied any;
gzip_types text/css text/plain application/x-javascript image/x-icon;
gzip_min_length 500;
gzip_disable "MSIE [1-6]\."

All of this is just the tip of the iceberg of what's possible. Other interesting modules protect against denial of service attacks, help harden the system, support load balancing (with sticky sessions), logging and memcached integration. We can also set headers - removing the need for some of the code we wrote in Part 2. Best of all, the Nginx documentation is quite good, and the community is thriving.

ASP.NET Performance - Part 4 - CDN

karl — Tue, 12 Jan 2010 13:31:00 GMT

In part 3 we created three helper functions to use when including css, js and images in our pages. As you might expect, YSlow has something to say about how to maximize the performance of such files beyond caching. Specifically, YSlow makes the following three recommendations:

Use a Content Delivery Network (CDN),
Use cookie-free domains, and
Split Components Across Domains

All three are super-easy to implement, and are all "fixed" by the same code (we won't actually use a CDN, but we'll make it so that its easy to use if the need ever arises). To be honest, I hadn't even considered the possibility of using separate domains to minimize the overhead associated with sending cookie information. Some may say that this is a microoptimization, but it takes less than five minutes to implement, so why not? (if you aren't using cookies, you won't see any benefits of this, except for the parallel downloading and flexibility of using a cdn in the future).

If you recall, the initial version of our helper functions, without our cache busting, looked like:

public static string IncludeCss(this HtmlHelper html, string name)
{            
    return string.Format("<link href=\"/assets/styles/{0}.css\" rel=\"stylesheet\" type=\"text/css\"></link>", name);
}
public static string IncludeJs(this HtmlHelper html, string name)
{
    return string.Format("<script src=\"/assets/js/{0}.js\" type=\"text/javascript\"></script>", name);
}
public static string Image(this HtmlHelper html, string name, int width, int height, string alt)
{    
    return string.Format("<img src=\"/assets/mages/{0}\" width=\"{1}\" height=\"{2}\" alt=\"{3}\" />", name, width, height, alt);
}

(I'm doing this against the initial version just to keep things simple, but integrating our Part 3 and Part 4 end-results should be trivial).

All we need to do is a little change:

private static readonly string _assetUrl = ConfigurationManager.AppSettings["assetUrl"];
public static string IncludeCss(this HtmlHelper html, string name)
{            
    return string.Format("<link href=\"{0}assets/styles/{1}.css\" rel=\"stylesheet\" type=\"text/css\"></link>", _assetUrl, name);
}
//...similar changes the for js and image method.

and change our web.config to include the value:

<appSettings>
    <add key="assetUrl" value="http://assets.mydomain.com/"/>
</appSettings>

Finally you'll need to setup the subdomain.

While we aren't using a CDN, we've made it very easy to start using a CDN by making the root URL path for our assets configurable. We've also ensured that cookies defined for mydomain.com aren't going to add overhead when requesting assets sitting on assets.mydomain.com.

For the curious, YSlow doesn't know whether the domain is a CDN or not, you have to tell it that it is by adding it to a list.

A final word of caution, YSlow also warns against too many DNS lookups - so while its important to use a cdn, use non-cookie domains and split up our content to allow for greater parallel downloads, don't over do it. They generally recommend less than 4, but their own website uses 6. We've added 1, which, in my opinion is worth it, but try not to add too many (consider that you'll likely add one for google analytics and possible for ads...).

Part 5 will be our last part in the series, and will be very different than what we've looked at so far.

ASP.NET Performance - Part 3 - Cache Busting

karl — Mon, 11 Jan 2010 13:51:00 GMT

In part 2 of the series we looked at ways to tweak our headers to maximize performance. One of those tweaks was the addition of far-reaching expiry headers for static files (images, css, js). What happens though when you need to change those files? We can make a change to the file on the server, but clients that got the previous version won't request the new file since the headers that came with the initial one told them not to.

You may be thinking that a simple solution would be to set a shorter expiration time. But that only masks the problem while negating the benefits we're after. What we're after is a solution that'll let us maximize caching yet allow us to update content on demand.

HTTP caching works off the requested URL. The browser makes a request for a URL, and gets a response, which contains headers as well as the body. Any subsequent request to the same URL should follow any caching headers that were returned in the original request. To make matters better (or worse depending on your point of view), any proxy between you and the client can also follow this same logic. Thus, a user that visits your site might get original content, but subsequent visits by him or anyone else behind the same proxy requesting the same file, will get a cached version from the proxy. This is great because a far reaching expiry can really turn tends of thousands of hits into 1, but also means that we really do need to come up with a good cache-busting solution.

(As a side note, since the cache is based off of the requested URL, you should really consider leveraging things like google's javascript hosting. Instead of linking to jquery.js in your /js folder, you can link to http://ajax.googleapis.com/ajax/libs/jquery/1.3.2/jquery.min.js. By doing so, if a user has previously browsed a site which linked to the google version, then it'll be in their cache when they visit your site and get faster loading. Google sets the expiry header to 1 year. On the down site, it decreases your sites availability by introducing another system that can break - google).

You may be thinking that a good solution is to simply change the name of our file. For example, if you're currently linking to logo.png and need to make some changes, you could switch it to logo2.png. This would work, but its a very manual and thus risky. What if you forget that a particular page is linking to your logo (like that HTML email template that gets sent out every now and again)?

Instead, what's typically done, and what we'll do here, is to include some type of version in the querystring. And, instead of incrementing integers (which can be tricky to manage, even if we automate it), we'll use a hash of the files content. So, instead of linking to /images/logo.png, we'll actually link to /images/logo.png?asdaslkj3918.

To achieve this, we'll need to do two things. First we'll have to generate a file hash for all of our assets and make that available to our system. Second we'll need to read the list of hashes and use it whenever we create a link to an asset.

Exactly how you implement the first part is up to you. What I'll show here is doing it as part of the build process - much like we saw in Part 1. The exact output of our first step is simply a text file that looks something like:

/Content/css/main.css|a5feab8f
/Content/images/logo.png|fa80c7bc
/Content/images/modal_close.png|a5722737
/Content/images/modal_close_o.png|0ddd517d
/Content/js/dd_roundies.js|955e9bc5

Our program will take three arguments, the root directory of our site, the asset folder, and the output file name. So, given the following parameters:

c:\projects\coolsite\ assets\ assethash.dat

our application will recurse through the c:\projects\coolsite\assets\ folder and generate a file in the above format in c:\projects\coolsite\assethash.dat

Here's the code:

internal class Program
    {
        private static IDictionary<string, string> _hashes;
        private static void Main(string[] args)
        {
            var root = args[0];
            var assetPath = args[1];
            var outputFile = args[2];

            _hashes = new Dictionary<string, string>();
            ProcessDirectory(root, assetPath);
            WriteFile(root + outputFile);
        }

        private static void ProcessDirectory(string root, string assetContainer)
        {            
            foreach (var directory in Directory.GetDirectories(root + assetContainer))
            {
                if (directory.Contains(".svn"))  { continue; }
                ProcessDirectory(root, directory.Remove(0, root.Length));
            }
            foreach (var file in Directory.GetFiles(root + assetContainer))
            {
                var assetName = file.Remove(0, root.Length - 1).Replace('\\', '/');
                _hashes.Add(assetName, CreateSignature(file));
            }            
        }

        private static string CreateSignature(string file)
        {
            byte[] bytes;
            using (var hash = new Crc32())
            {
                bytes = hash.ComputeHash(Encoding.ASCII.GetBytes(File.ReadAllText(file)));
            }

            var data = new StringBuilder();
            Array.ForEach(bytes, b => data.Append(b.ToString("x2")));
            return data.ToString();
        }

        private static void WriteFile(string path)
        {
            using (var sw = new StreamWriter(path))
            {
                foreach (var kvp in _hashes)
                {
                    sw.WriteLine("{0}|{1}", kvp.Key, kvp.Value);
                }
            }            
        }
    }

First we build up a dictionary of paths (relative to the root of our asset folder) and hash values. For my "hash" I'm using a CRC32, specifically Damien Guard's C# implementation. The nice thing about CRC32 is that we end up with pretty small values. The downside is that collisions are far more likely (but still rather unlikely - I haven't really tested this tough, so maybe using a true-collision resistant hash, like md5 (which can collide but only in extremely rare occasions) makes more sense).

Next we dump the dictionary to our file - nothing much to explain there.

This is our postbuild line:

$(SolutionDir)..\Tools\ContentSigner\CodeBetter.Web.ContentSigner.exe $(ProjectDir) assets\ assethash.dat

Now we need to use the generated file. First we'll start off with three vanilla helper functions:

public static class HtmlAssetExtensions
{
    public static string IncludeCss(this HtmlHelper html, string name)
    {            
        return string.Format("<link href=\"/assets/styles/{0}.css\" rel=\"stylesheet\" type=\"text/css\"></link>", name);
    }
    public static string IncludeJs(this HtmlHelper html, string name)
    {
        return string.Format("<script src=\"/assets/js/{0}.js\" type=\"text/javascript\"></script>", name);
    }
    public static string Image(this HtmlHelper html, string name, int width, int height, string alt)
    {    
        return string.Format("<img src=\"/assets/mages/{0}\" width=\"{1}\" height=\"{2}\" alt=\"{3}\" />", name, width, height, alt);
    }
}

First we'll modify our helper to load the text file back into a dictionary (this could be handled in a separate class, but for simplicity, we'll just stuff all the logic in here):

public static class HtmlAssetExtensions
{
    private static IDictionary<string, string> _assetHashes = LoadHashes();

    private static IDictionary<string, string> LoadHashes()
    {
        var hashes = new Dictionary<string, string>(StringComparer.InvariantCultureIgnoreCase);
        using (var sr = new StreamReader(HttpRuntime.AppDomainAppPath + "assethash.dat"))
        {
            while (sr.Peek() >= 0)
            {
                var parts = sr.ReadLine().Split('|');
                hashes.Add(parts[0], parts[1]);
            }
        }
        return hashes;
    }

    ....
}

Next we need to use the data:

public static string IncludeCss(this HtmlHelper html, string name)
{
    name = GetVersionedName(string.Format("/assets/styles/{0}.css", name));
    return string.Format("<link href=\"{0}\" rel=\"stylesheet\" type=\"text/css\"></link>", name);
}
//...similar changes the for js and image method.

private static string GetVersionedName(string name)
{
    string hash;
    if (!_assetHashes.TryGetValue(name, out hash))
    {
        return name;
    }
    return string.Concat(name, '?', hash);
}

And that, my dear reader, is all it takes. There is one problem with our implementation, changing our assethash.dat isn't going to noticed by our application. That's something I leave to you to figure out - though it isn't too hard to implement (hint: you could just store the dictionary in the HttpRuntime.Cache with a file dependency).

In the next [very short] part, we'll make a single small tweak to our helper functions that'll help address three other YSlow recommendations. Stay tuned.

ASP.NET Performance - Part 2 - YSlow

karl — Fri, 08 Jan 2010 15:10:00 GMT

In part 1 we looked at integrating a javascript compressor as part of our build process to shrink javascript files for production (at the time, I didn't realize that it would lead to a series). Its worth mentioning that if you go down that path, you should really also merge all your files together so that they can be pulled down from a single request - but that's up to you to figure out how (come on, merging x files together isn't hard now!)

Today we'll write a simple HttpModule that'll help us meet a number of YSlow recommendations. YSlow, created by Yahoo, is a Firebug addon (which itself is a [must have] addon to Firefox)) which identifies potential performance issues with your site. Here's a screenshot that shows the sort of things YSlow can help identify.

With just a few lines of code, 2 of those F's can be solved - removing ETags from the server response and adding expiry headers.

public namespace CodeBetter.Web
{
    public class YSlowHttpModule : IHttpModule
    {
        private static readonly List<string> _headersToRemove = new List<string> { "X-AspNet-Version", "X-AspNetMvc-Version", "Etag", "Server",}; 
        private static readonly List<string> _longCacheExtensions = new List<string> {".js", ".css", ".png", ".jpg", ".gif",};
     
        public void Init(HttpApplication context)
        {
            context.EndRequest += new EventHandler(context_EndRequest);
        }
     
        private void context_EndRequest(object sender, EventArgs e)
        {
            var context = HttpContext.Current;
            _headersToRemove.ForEach(h => context.Response.Headers.Remove(h));
            var extension = Path.GetExtension(context.Request.Url.AbsolutePath);
            if (_longCacheExtensions.Contains(extension))
            {
                context.Response.CacheControl = "Public";
                context.Response.Expires = 44000; //slightly over 1 month
            }
        } 
        public void Dispose() {}
    }
}

Simply register the module in your web.config and you're good to go:

<add name="YSlowHttpModule" type="CodeBetter.Web.YSlowHttpModule, YourAssemglyName"/>

Its worth pointing out that this module will only work with IIS 7.x's integrated pipeline.

The code does two things. First, it strips out extra headers - we only really need to strip out the Etag header from YSlow's perspective, but I like to remove the other useless stuff as well. The second is that it adds a very long cache control policy to our assets.

You may be wondering why ETags should be removed beyond mindlessly following YSlow's recommendation. ETags are inferior to a long lasting cache control policy, and as such, it does nothing more than add overhead to each request. Unlike far-reaching expiry headers, ETags still require an open connection to your server to see if the content has changed.

A problem with our YSlowHttpModule is that we've told anyone in the request chain that they can cache our static files for a month. What if we need to change an image, or update our css? In part 3 we'll solve that issue by placing file hashes in our querystrings.