Automatic ETag Management with Web API Message Handlers

One of the great things about the fact that Http is a full fledged application protocol is that it has built in semantics for doing all sorts of application types of things.  Two of these things which I’ll talk about together are caching and optimistic concurrency.  In the world of Http-based applications, these need to be talked about together, since they balance each other out.  More specifically, caching is great because it lets intermediaries return cached Http responses to clients – but it also means that clients may be working with stale content, and hence we need concurrency to ensure that we’re not updating a resource with a representation that is older than the current state of the resource.

Http provides the tools to manage both caching and concurrency.  For caching, the server can set headers such as cache-control to set the time that a representation can live in a cache along with other control settings (such as where the representation can be cached – local, intermediaries, both).  For concurrency, Http provides 2 high level approaches – a timestamp approach which uses the last-modified response headers and if-modified-since and if-unmodified-since request headers.  It also provides a more generic approach called entity tags.  I say “more generic” here since the value of an entity tag doesn’t shouldn’t have any meaning to your client other than it represents the state of a resource at a point in time.  It’s important to reiterate that the entity tag should not have any meaning to your client – in fact, as we’ll see in a second, it doesn’t necessarily need to have any meaning to your server either.  For dealing in entity tags, the Http headers to care about are the etag response header and the and the if-match, if-none-match request headers.  As far as I can tell, the decision of whether to use entity tags vs. the timestamp approach really just comes down to whether or not you expect your resources to change state more frequently than 1 second (the format for last-modified goes down to only the second).  Given that both serve the same purpose, and that I don’t know how the usage patterns around my services will evolve, I tend to go the entity tag route up front and spare myself and my clients from potential changes down the road.

I talk a lot more about the interplay of caching and concurrency in my TechEd talk, but the basic workflow looks like this:

1) Client makes a GET request to a resource.  Server responds with a 200 status code, some cache settings, and an etag

2) When the cache time window expires, client sends a conditional GET request to the server using the if-none-match header and the etag value supplied in step 1.

3) Server checks to see whether the etag value supplied in the conditional GET request matches the latest etag value for the requested resource.  If there’s a match, the resource state hasn’t changed and the server can return a 304-not modified.  If there’s not a match, allow the request to flow up into the service, process as normal, and return the response along with the updated etag.

4) When the client wants to update the resource, it sends a conditional PUT request to the resource URI.  This means that it includes with the PUT request a if-match header with the value of the last known etag value the client received for that resource.

5) The server compares the value of the if-match request header to the value that represents the latest state of the resource.  If they match, then the update operation is allowed to proceed.  If not, the server responds with a 412-precondition failed status code.  Generally, the pattern is for the client to GET the latest representation of the resource (which includes the latest etag value) and then try updating the resource again using the latest etag value.

So getting the background out of the way, one thing that I hope has surfaced is that managing etags doesn’t need to be tied to the actual model that sits behind your resources (e.g. you don’t necessarily need to generate your etags from your model).  What follows is an example of using Web API’s message handlers to automatically manage etags for a set of resource URIs.  One thing to point out: this example doesn’t deal with updating etags for resource hierarchies (e.g. scenarios where updating some related resource should change the state of the resource).

The key thing I want to show is how message handlers work in Web API – particularly the use of task-based async.  I’ll stop now and just show the code.

The handler uses a state store (which in this simple implementation is just a dictionary of resource URIs and etag values).  When there’s a GET request, the handler looks for the if-none-match header and tries to match the value of the if-none-match header to a value in the dictionary (whose key is the resource URI).  If there’s a match, the handler returns a 304 without invoking the service operation.  Otherwise, it lets the call continue up to the service operation and then appends the etag value as the response flows back down to the wire.  When there’s a PUT or POST request, the handler looks for the if-match header and tries to match the value of the header to a value in the dictionary.  If there’s a match, the call is allowed to proceed up to the service operation.  Additionally, because we know (from the semantics of PUT/POST) that this operation likely changed the state of the resource, we generate a new etag value which we store and return with the response.

I will point out that this code is making a fundamental assumption that the Http uniform interface is being followed by the server – and based on that assumption, it can manage etags in a manner which is completely agnostic to the underlying service classes.

The other thing about message handlers which I think is both cool and yet not the most intuitively obvious thing is the use of Task<T> (at least it wasn’t intuitively obvious to me at first glance).  Here are the big ideas that you need to keep in mind for using Tasks with async message handlers:

  • When you’re processing the request side and you want to send the message onward towards the service operation, call base.SendAsync(..) – this translates into calling innerChannel.SendAsync(..).
  • When you’re processing the request side and you want to stop the message flow towards the service operation (e.g. short circuit the processing), return your own Task<T> without calling base.SendAsync.
  • When you want to process the response side, go ahead and call base.SendAsync(..) to allow the flow to continue, but append a .continueWith statement.  Your continuation will get called as the message is flowing back to the wire.

The async model here is really powerful in that it frees up your WCF threads to handle other requests when your handler is blocked (note that your request will still be blocked, you’re not blocking a WCF thread in addition).  The Task model is also a lot better than previous async methods – it still didn’t feel completely natural to me, so hopefully this will help if it doesn’t feel completely natural to you either.

About Howard Dierking

I like technology...a lot...
This entry was posted in HTTP, WCF, Web API. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • Anonymous

    Generally, I would create bookmarkable child resources for each format

  • Smolesen

    Hi Howard

    What if the resource returned depends on the MediaType supplied in Accept header ??

    Lets say that the same resource can have two different representations Json and XML
    If the first request ask for XML and the second ask for JSon, wouldn’t your ETagMessageHandler wrongly return 304 for the second call, telling the client that it can use the cached XML representation??

    How would you solve that problem?

  • http://tugberk-ugurlu.myopenid.com/ tugberk

    This looks great but I found this a little incompatible on WCF Web API Preview 6. For example, did HttpMessageChannel renamed? also, where is the IEtagStore is coming from?

  • http://twitter.com/jcrespoalvez Javi Crespo

    Hi Howard,

    Great post! I created a similar sample in my post. My sample takes a different approach: it gets  the ETag from the resource object by convention and it handles the storage to the ETag cache automatically. I also split up the functionality in several smaller delegation channels with fewer responsibilities.
    If it sounds interesting:

    http://javiercrespoalvez.com/2011/06/etags-with-wcf-web-apis-preview-4.html
    http://javiercrespoalvez.com/2011/06/etags-and-optimistic-concurrency.html

    And the original post from Jose Romaniello
    http://joseoncode.com/2011/04/27/rest-and-the-http-cache-with-wcf-web-api/