Node.exe version requirement lowered to raise compatibility. This should enable using our SDK on Heroku and should make Juan Pablo Garcia and Matias pretty happy.
Storage Services UT run against a mock server which should make tests run much faster. (just “npm test”)
Service Bus
The Service Bus provides secure messaging and relay capabilities that enable building distributed and loosely-coupled applications in the cloud, as well hybrid application across both private and public clouds. It supports multiple messaging protocols and patterns and handles delivery assurance, reliable messaging and scale for your applications. The Service Bus is a managed service that is operated by Microsoft and has a 99.9% monthly SLA. more here
In our latest update, you can now access Service Bus right from within your node.js applications! Below are snippets indicating usage.
Here’s an example of using the apis to create a queue
Here is how you send a message with a specific topic
And finally this is how you poll the queue for a message.
Be sure to also check out our unit tests for more samples. We’ll also have content on the dev portal for Service Bus very shortly.
We would like to thank the community and our advisors for the feedback and suggestions so far, which we have tried to accommodate as much possible for this release.
We’d also like to thank the community contributors that participated (Einar in particular a big thanks) in this release. We are looking forward to more contributions in the future
As to the road ahead, well here’s a little teaser…..
Service runtime –> Allows you to obtain and control the information on the actual machine your app is running in.
Service management and deployment –> Manage and deploy services on any platform using node cli tools.
I rarely review books however the past few years there’s a few gems I’ve found in my REST travels that I have felt compelled to review. Mike Amundsen’s new book on hypermedia is one such work.
Note: This review is a slightly modified version of what I posted at Amazon.com.
If you are a hypermedia developer you should read this book. If you are building web apis to reach multiple devices then you should read this book. If you are building web apis that will be around for a long time and evolve over many versions, then read this book. Hypermedia systems have been around since the dawn of the web, however they have mainly been HTML pages offered up to a browser agent. It is extremely rare that you see the usage of hypermedia in web apis / machine to machine scenarios.
Hypermedia is basically a black art that very few folks actually understand. I experienced this first hand when I was the PM for WCF Web API as hypermedia was something that we wanted to be able to support. I immediately discovered that there was very little literature on the concept, there was no definitive reference to turn to. The guidance was mainly spread through a set of articles and through thousands of mail threads. Mike is one of the few who actually gets it having many years of experience building such systems. He is one of the foremost experts on the topic. Just head to his blog or the REST discussion list and you’ll see this is the case.
This book is important because with it Mike lifts the veil of mystery of how to build such systems, taking it from a black art to something broadly available to developers. Mike goes into depth defining what hypermedia is and the different ways it can manifest itself. He describes the different types of hypermedia controls and how best to use them. Mike also pays particular attention to using hypermedia in javascript agents running in the browser, something that is extremely relevant today considering the rise of rich ajax style applications using frameworks like jQuery and extjs. The book is not a book of theory, from start to finish it uses real world examples to illustrate the concepts.
The one thing I would have liked to have seen Mike elaborate on a bit more was why hypermedia itself was important. Throughout the book it does become more self-evident however for one who doesn’t understand what it is I felt that the book could have laid more of a foundation / done a bit more selling on the value. Mike does plenty of that in his blog though.
If you are building RESTful systems particularly in node (though the guidance is general applicable) then I highly recommend this book.
Preamble: The intent of this post is to educate on how HTTP was designed to be used , clarify misconceptions and to give folks food for thought on different ways they can design a system. It is not to ignite a war against RPC. There are tradeoffs to consider when your design your system. If you are not designing a system that is to be consumed by many clients over a long time and where the server needs to evolve independently of the client, then using HTTP for RPC may be absolutely fine.
Recently there was a question on the forums asking why we encourage usage of HttpRequestMessage<T> / HttpResponseMessage<T> in the signature of a web api implementation. The point made in the post is that if you have an ICalculator contract which your API implements, then it’s violating SOC / inappropriate to have those messages as params and in the contract. The argument is valid when looking at HTTP from the standpoint of an RPC mechanism, which is actually a quite common view.
Not of the HTTP authors though. If you read Roy Fielding’s dissertation you will see very clearly that:
HTTP is not a transport protocol and HTTP is not RPC.
You might be surprised, but it is true. There are real quantifiable reasons for this that underlie the foundations of how and why the web infrastructure was built. Roy explains it much better than I can, though I still continually attempt to do it I recommend reading what Roy has to say on the subject here in sections 6.5.2 and 6.5.3.
Back to our HTTP messages. This has deep implications in your API design. Once you accept that the thing that you are exposing is an HTTP resource which clients interact with via the uniform interface and not a class with standard OOP methods, you design them differently. Your APIs become a gateway where HTTP concerns are addressed and then work is delegated off to the business domain rather than being part of the business domain. Once they do, you can safely use HttpRequestMessage<T> and HttpResponseMessage<T> without fear as they are specifically for addressing HTTP concerns.
Mike Amundsen has a really nice post where he emphasizes this last point and why it is just better for us to embrace HTTP in our code and stop fighting it:
“the better way to deal with HTTP is to embrace it. let HTTP lead system designers in the direction of lossy, chunky, state-less designs. accept HTTP as the stable rarely-changing foundation for your implementations. learn to ‘think’ and ‘speak’ HTTP so that all your dist-net designs reflect the power, stability, and reliability of HTTP itself. and don’t fuss with it.”
You can read more about the design implications from a Web API perspective in my reply on the thread.
Although I left WCF Web API, I am very passionate about continuing on my journey toward understanding REST and how it fits with Web API. For that reason you will likely continue to see posts on REST and Web API here along with my new found love node.js.
If you head over to our codeplex site, one question commonly asked is “Does Web API support hypermedia?” To understand this question you have to first understand what hypermedia is. I am not going to cover that here, but there are many articles about it, a ton of discussions in the REST Discuss Yahoo group and a new book by Mike Amundsen which is well worth reading. (I need to write a review for the book at some point). I gave a quick primer which doesn’t do justice, but will get you through this post here.
The answer to the question depends on your notion of support:
If by support you mean, “Can I embed links within my responses which will indicate to my client available resources / state transitions?” then the answer is YES.
If it means, “Will Web API automatically embed links for related resources out of the box?” the answer is no”.
Web API was designed to be highly pluggable, especially as it relates to how a request is handled, or how the content is processed. You can take advantage of this for plugging in your own custom code for generating hypermedia.
In the rest of this post I will discuss various strategies for how as well as point you to several samples.
Why formatters are a good choice
There are several different ways you can technically attach hypermedia including formatters, message handlers, and operation handlers. Formatters are the way we would recommend as they are specifically responsible for taking the result of an operation (could be anything) and transforming it into some representation. That representation can contain hypermedia.
How conneg and formatters work in Web API
Each Web API host has a collection of formatters. Each formatter specifies information about the media types it supports. On any incoming request, if content is sent, Web API matches up the incoming request’s Content-Type against the available formatters to choose one to appropriately handle processing the content in the request. It invokes the matched formatter passing in the content of the request. The formatter then outputs a CLR object which is passed to the operation.
Once the operation has completed, Web API performs server side content-negotiation to determine the response representation. It looks at the accept header supplied by the client in the request and matches up against the available formatters. The algorithm will look for a best match based on the specified media type / media type ranges in the accept along with the q-values. Once a formatter is identified it will be invoked passing in the return value of the operation. The formatter takes the the incoming value and writes out the representation.
As the formatter is writing out the response, it can also embed hypermedia controls i.e. links and forms. If you read the REST in Practice book they show a media type “application/vnd-restbucks+xml” which contains hypermedia. You can create a custom RestBucksFormatter that handles that media type. If you check the RestBucks sample for Web API on codeplex, you’ll see an example of this.
Url / Link generation
In a hypermedia based system, links are generated based on a state machine. Depending on the state of the resource, different links apply. As soon as you start writing the formatter you will have to deal with how to handle the generation of links. There’s a several approaches you can use.
Generate links within the formatter.
With this method, your formatter contains the state machine logic for generating links. In the case of an Order management API, the API implementation for GET can return an order. The formatter then comes a long and grabs the order and adds appropriate links based on the state of the order. I used to be a big fan of this approach as it keeps links as separate concern from the API code itself. I’ve since changed that opinion, as I don’t view links as a separate concern
The upside of this approach is everything is nicely encapsulated within the formatter. The downside is that it makes it harder for one to follow the logic of the system if they are reading the API code. They must also read the formatter. It also means that the link code is duplicated if you support different variants like xml and json that require the hypermedia.
Example: Hypermedia formatter for contacts
A while ago, Mike Amundsen visited Microsoft and I was fortunate to have the chance to brainstorm with him around Media Type design. We took the ContactManager example and spiked on how it “could” look if it used hypermedia. The implementation used the approach I just described. Below is a snapshot of how the media type itself looks. The “.” for the uri is a placeholder as the uri doesn’t matter, it is opaque to the client.
One thing to note here if you are familiar with hypermedia, is that it is not using the standard link element with “rel” and “href” attributes that you are probably accustomed to seeing. Instead a domain specific format is used to to represent links and forms. In this example, Tweets has an href indicating where to go to get tweets. DM is a form for sending a direct message. The query named “query-search” includes a form with fields for specifying search criteria. This type of approach of using domain specific hypermedia deserves it’s own post. It has several advantages in that it is more intention revealing, offers richer semantics than just being limited to a rel and href , and also doesn’t abuse the rel tag.
As to the implementation, here you can see the operation within the ContactsApi. Then in a custom formatter all the hypermedia affordances are attached.
You can find all the code for Mike’s example at his github repo here
Generate links close to the operation.
I mentioned above how my perspective changed on this. My current understanding is that links relate completely to the core business logic of the system as opposed to being a cross-cutting concern like logging. They are NOT a separate concern. Think of your Web API as a bridge between HTTP and your business domain, not the domain itself. Thus the model being returned from your API being tailored to HTTP is completely valid. This is not about DTOs.
With this approach links will get generated by the operation. Your operation returns a model that has links on it. The formatter’s responsibility is then to take the model and encode the links correctly within the representation. This allows you to add new formatters easily without replicating the linking logic. You could have an ILinkable<T> type which your formatter understands for attaching links. This type is not sufficient for collections as in that case you have links nested at multiple levels. For that you can have an ILinkableCollection<T> that contains items of ILinkable<T>. Thus you can return an ILinkableCollection<Order> that contains within ILinkable<Order> instances each with their own links attached.
The link generation code itself does not have to sit within the operation itself. Introduce a separate service such as IOrderLinkService which has that specific responsibility and have it injected via an IoC container. The service exposes an AddLinks method which accepts ILinkable<T> or an ILinkableCollection<T>. Although the link logic does not exist within the service, it is easy to step through the code to debug the logic.
Example: Domain specific media type for Orders
Here’s a snippet of the Order API.
The link service is injected into the API. The Get method then invokes it passing the order.
Once the service is called it looks at the state of the order and adds links appropriately. Above you can see that when the order is in the created state, a link is added for an approval resource.
The code can definitely be refactored / made cleaner, but it illustrates the basic idea. As an example of something more sophisticated, the RestBucks port for Web API contains a ResourceLinker service that generates links via accepting a lambda expression.
Lastly there is the formatter itself. It will take the linked order and write out the appropriate links. Below is a skeleton of such a formatter.
Example: HAL formatter
HAL is a media type in draft at IANA that is designed specifically for expressing resources and links. Steve Michelotti has created a formatter for HAL. The way it works is your operations return a HalResource model. HalResource can recursively contain other resources thus you can have a very deep resource map. You can use it both on the client and the server. The formatter then takes that model and writes it out appropriately as HAL, it also do the reverse taking a HAL response and turning it back into a HAL model. You can read more about how to use it in Steve’s post. Hopefully we’ll see this on nuget soon!
Example: XHTML using the plain old XML formatter
Mike Amundsen talks quite a bit about XHTML in his new book. What makes XHTML interesting from a hypermedia perspective is that it contains native support for hypermedia affordances, and browsers naturally understand it. You can express links very easily using an <a href=””> construct. You can embed forms as well as it supports them natively. It also offers annotations like name, id and style which we can use in order to attach domain specifics to the elements. For an rich example of what you can achieve with XHTML check Mike’s ALPS micro-blogging specification.
Web API supports the ability for you to return XML dom objects directly, for example you can return XmlDocument/XDocument instead of a model. The beauty of this is that you can easily represent any XHTML document using the XML dom. Thus in your operation you simply create an XmlDocument, populate it appropriately and return it. The formatter will very happily then write it out to XHTML.
Here’s a skeletal snippet illustrating what I mean
If you do use this approach one thing you will need to do is add the media type to the default XML formatter. You can do that off of the configuration object’s Formatters collection using code similar to the following:
Example: XHTML formatter example using Razor
Recently I delivered a talk at Tulsa TechFest where I demonstrated RestBugs. A sample app written by Howard Dierking. It contains a web API for bug tracking which and supports XHTML via a custom formatter. The app also includes a console hypermedia client which I hope to cover in another post. You can watch the video from that talk here. If you jump to the last 30 minutes that’s where I get into RestBugs where I cover the API, formatter and the hypermedia client.
One interesting thing about the app is the formatter that is used. Instead of manually writing out the XML, Howard uses a custom Razor formatter. The formatter looks for a custom content header which specifies the template. It then uses ASP.NET’s Razor engine to take the model passed in along with the template to generate the XHTML. To illustrate here is the template for the bug list.
I really like this approach because it gives you absolute freedom to format things exactly the way you want them without jumping through a 1000 hoops in code. I know from discussions with Mike that using templates is really his only way to fly.
You don’t have to wait for that future post I eluded to (which might never happen) to try the code in the talk as I’ve put it on Dropbox here. You’ll find RestBugs including the Razor formatter and the client.
Generate links through middleware.
This approach is more of a hybrid of the other two. Instead of generating links in the formatter or the operation, you use an operation handler. The operation handler takes the model that is returned like ILinkable<Order> and then adds the appropriate links.
Taking the above code, you can simply move the logic to an operation handler to achieve the same.
The pros of this is that it still allows the link logic to be used cross formatters, and it makes the code within your operation much tighter. The downside, and why I no longer prefer it is that it makes the code harder to follow. You have to know that a handler was registered which applies the links and you have to find that handler to understand the link logic.
How to generate the uris themselves
Aside from the strategy to embed links there is the question of how best to generate uris. Here’s a couple of recommendations on how you can do this.
Relative uris
If you can get away with relative uris do it. What I mean by that is this, let’s say you have a link to an Order approval resource for order 5. You can generate a link for “/order/approval/5”. The client will take the uri and append it to the host url in order to navigate. This alleviates the need for you having to get access to the base url for the site when you generate urls.
Absolute uris
If you have multiple domains where your resources live, or if you want clients to always get the full uri so that they can simply copy/paste, use absolute uris.
For absolute uris, create a service which is configured with a map of all the available uris and exposes methods for retrieving them.
OrdersBaseUri returns “http://contoso.orders.com” the address of the order system, while InventoryBaseUri returns “http://constoso.inventory.com” for the inventory system. UspsBaseUri is a the address for a third party site for shipping .
The uri service can then get injected into the link uri service which can use it to generate absolute uris.
Your mileage may vary
As you can see from this post, there are many different ways to approach implementing a hypermedia system with WCF Web API. As building hypermedia systems for web apis is still a pretty young topic, I don’t think there’s a clear winner yet on which hypermedia design approach to use. By extension there is also no clear winner on implementation. Over time and with more people building hypermedia based systems, hopefully this will change and there will be more accepted standards.
What has worked for you? Do you have a particular approach you’d like to see better support for in the box?
If you’ve my last few posts around the new Windows Azure SDK for Node.js and you have not been following me on twitter, then you might be suspicious. Let me remove the suspicion, I’ve moved on from the Web API team about 5 months ago to help with our Node.js on Azure story.
My ride with Web API
I’ve had a thrillingly fantastic ride being with the Web API team. When I first joined WCF after leaving the MEF team, I had a ton of folks pinging me on twitter asking me what in the world I was doing / why would I go to WCF. I told them there was an opportunity for WCF to do the right thing for customers building HTTP/RESTful APIS. I also said that we would not do it alone and we would work with the community. Sebastian Lambla (@serialseb) can attest to this as he and I chatted before I even agreed to take the job. Seb was supportive and agreed to help us along with a bunch of other folks.
Since then we think we’ve delivered on that promise. At PDC last year we released our first drop of Web API on Codeplex and published our Mercurial repo, since then we’ve been releasing as frequently as we could including on Nuget.
So far, we’ve heard that people like likewhat we are doing. We’ve been developing a library that really helps you take full advantage of HTTP and REST (if you desire) to support an evolving set of clients. Not only that, but it’s complexity bar is low and the simplicity bar is high. Also it plays much better with good principles of software design like separation-of-concerns, and offers good compatibility with good software practices like test-driven-development.
We’ve had a core set of advisors from the community helping us at every stage to build something useful in the real world, telling us what to do and more importantly what NOT to do. We’ve taken a ton of feedback from the broader community and driven it back to the product. Finally we’ve seen some great efforts from the community both ineducationonhow to use it and in extensions to the core product.
<mushy-stuff-follows>
I am indebted deeply to the people I worked with on the Web API team. We accomplished a lot with the odds stacked against us. I am also indebted to our advisors who invested a lot of personal energy and time to helps guide us. I feel privileged to have worked with such stellar people, there’s too many of you to name, but you know who you are!
As for my part well, I felt like the phony on the team with all the great minds that were around me. I did my part, but the real work was of the folks I mentioned.
Web API IS still moving and it will ship
Since my last few posts and throughout my tweeting over the past few months, I’ve received a lot of questions from folks asking me “What about Web API”. Is it another dead carcass left in the framework graveyard? Is it another sacred text to be stored in the gigantic warehouse of Raiders of the Lost Ark*? The answer is an emphatic no!
* – Yes I am delusional
Yes, I’ve moved on, but Web API is moving on as well to bigger and brighter places. The team is pumped up and the product is moving like a high speed train going at 200 mph! You’ll have to take my word for it, but I believe it is headed in the best possible direction than I could have ever hoped for. I also believe it is the best outcome for you, our developers. I can’t talk about it now, but you will be hearing about it shortly in a big announcement. I am sure many of you can imagine what that will be. You can find a big teaser in this discussion thread.
The meta point here is if you are investing in Web API, you’ve made a good decision. And if you haven’t, well you should probably start There’s much more goodness to come!
Keep watching ScottGu’s and Howard Dierking’s blog for details on the extremely bright future of Web API!
Why Node.js?
I have been an avid mostly observer of Node.js for the past two years. I first learned about it at Øredev. It offers a simplified model for building highly scalable web applications using server side JavaScript. I talked a bit about my own perspectives on Why Node in my interview here. Since it’s inception we’ve seen an explosion of interest in Node across the industry. Recently we’ve also seen a lot of interest around making Node.js work well for Windows. We’ve worked with Joyent to make that a reality, and we’ve also look at how to better integrate Node with Windows with efforts like iisnode.
A few months back I was asked if I’d be interested in helping to take that support for Node.s to Windows Azure. Seeing the excitement around Node.js, the opportunity to make Azure a great place for developing Node.js applications, the opportunity to make Azure a truly open platform, and the bonus of doing it in ScottGu’s org, I jumped at the chance.
And so, if you look for me now going forward, you’ll find me on the Azure SDK team! Going forward you’ll see much more about the work we are doing in this space on my blog.
<more-mushy-stuff>
Thank you all for your support! I am truly grateful for the opportunities you (and Microsoft) gives me to help you build better software. See you in the cloud!