Hypermedia hangout: Collection+JSON, HAL, and building .NET hypermedia APIs with Nancy and ASP.NET

Yes that title is a mouthful. Recently I had the pleasure of doing a google hangout with Jonathan Channon of Nancy fame and Phil Cleveland on a topic you’ve heard me be slightly passionate about, Hypermedia.

You can find the full hangout which was published on air here, which was recorded for your enjoyment and displeasure :-)

My reasoning for getting in the convo was a bit selfish. I maintain a library for Collection+JSON APIs (which I’ve never blogged about) which I want to be used with Nancy.

Originally the library was built strictly for ASP.NET Web API, however after getting feedback from Andreas, that I should decouple it so NancyFx users could use it. I took his advice and did that! I refactored into 3 packages two of which can easily be used in Nancy. However, no one is doing it. My hopeful result for this call with Jonathan is that would change as he is deep into Nancy and interested in Hypermedia. It looks like I reached that goal :-)

In the hangout we start off (well mostly I was talking) talking about Hypermedia itself and different formats, namely HAL and Collection+JSON. Then we went into looking at a few sample APIs (including out issue tracker example from the book) using Collection+JSON, my CJ library, as well as how to implement a hypermedia API / useful patterns. Along the way we also delved into Profiles and the role they play for hypermedia APIs. Finally we ended the discussion talking about what this means for Nancy and what we could do to enable CJ in Nancy. That led to the creation of this github project! Jonathan and I will be iterating the next few weeks on bringing CJ to a Nancy near you.

Let the hypermedia games begin!

Posted in collectionjson, HTTP, Hypermedia | 1 Comment

Our new book and my personal journey toward REST and Hypermedia

It’s hard to believe, but it has been 6 months since our new book, “Designing Evolvable Web APIs with ASP.NET” shipped! So far the feedback that we’ve received has been really positive and we’re excited to see the momentum.

book

In the first part of the book we focus on educating about the fundamentals of the web architecture and HTTP, and APIs. Next we focus on the designing of a hypermedia API, and a TDD/BDD driven implementation of a CollectionJson API using ASP.NET Web API. We also give you several coding patterns that you can use to get started building hypermedia APIs.

In the second part of the book, we focus on ASP.NET Web API itself and the underlying architecture. We do this in the context of the first part, with the explicit goal of helping you further toward building out an evolvable API. We delve deeply into the nuts and bolts of Web API including hosting, routing and model binding. We cover comprehensively how to secure your API, and the security internals. And we cover other topics like TDD and using Inversion of Control.

You can buy it online now here. We also have a Github repo with all the code, which includes a working hypermedia API.

The people that made this book a reality.

This book has been a collaborative effort with four other amazing folks that I’ve had the privilege to work with. It combines the knowledge of the Web API team itself as well as key advisors who were with us every step of the way as we built it. It was a collaborative effort across 4 time zones and countries:

  • Howard Dierking, my co-conspirator on the Web API team. An amazing individual who is helping educate the world on REST and hypermedia through his Pluralsight work.
  • Darrel Miller, a long time proponent of REST and hypermedia in the .NET world before many of us at Microsoft had a clue of what that means. Darrel has been building real hypermedia systems for a long time. Darrel was one of our first handfuls of advisors on Web API.
  • Pedro Felix, an expert on Web API security and the ASP.NET Web API architecture. His comprehensive contribution in the book on the security aspects of Web API is unparalleled in the .NET world.
  • Pablo Cibraro, former CTO of Tellago and a consultant who has implemented many API solutions. Pablo is an expert in Agile and TDD, and he wrote some deep chapters testability and IOC.

We also had a fantastic set of reviewers and advisors.

Why write this book?

This book is part of a very personal journey, a  journey that started several years ago when I joined the WCF team at Microsoft. At that time I had the pop culture understanding of REST, that is a lightweight API exposed over HTTP that doesn’t use SOAP. I joined the team to help build a better experience in the platform for these types of APIs. Little did I know the journey that awaited me, as I would delve deeper and deeper into the REST community.

It was a journey of learning.  I was fortunate to have wonderful teachers who had already been treading the path, chief of which (ordered last name first) were: Jan Algermissen, Mike Amundsen Alan Dean, Mike Kelly, Sebastien Lambla, Darrel Miller, Henrik Nielsen, Ian Robinson, Aaron Skonnard, and Jim Webber. I am deeply indebted for their help. The is book was built really on the shoulders of these giants!

During that period, I learned about HTTP and REST, that REST is much more than building an RPC JSON API, and about how to make APIs more robust. This included learning about fundamental aspects of the web architecture like caching, ETAGS and content negotiation. The learning also included the hidden jewel of REST APIs, hypermedia, and new media types like HAL, CollectionJson and Siren that were designed specifically for hypermedia-based systems.

It was clear looking at our existing Microsoft frameworks that they did need meet the requirements for building APIs that really leveraged HTTP and took advantage of the web architecture. Looking within the .NET OSS community, solutions like Open Rasta, however were explicitly designed with this in mind. There were also a ton of other options outside the .NET world in Java, Ruby, Python and PHP, and more recently in node.js.

After soaking this all in, my team at Microsoft, and a team of fantastic advisors from the community, worked together to create a new framework that was designed from the get-go to fully embrace HTTP, to enable, but not force building a RESTful system.

As part of this we had an explicit goal from day one to ensure this framework would also enable building a fully hypermedia based system. Another explicit goal was to make the framework easier to test, and to allow it to work well with agile development. This framework ultimately became what you now know as ASP.NET Web API.

Although ASP.NET Web API had the foundations in place to enable such systems, you have work to do. I am not going to say that it naturally leads you to build a hypermedia system. This was not an accident. We deliberately did not want to force people down a specific path, we wanted to make sure though if you wanted to build such a system, it wouldn’t fight you.

We saw a lot of folks jump on this, even internally, folks like the Office Lync Team that built probably one of the largest pure hypermedia APIs in existence using Web API.  Many of these early adopters had the advantage of working directly with our team. They worked with me, and Henrik Nielsen (My Web API architect, and one of the creators of HTTP) to help guide them down the path. We did a lot of educating on HTTP, API fundamentals and what hypermedia is and how to build systems that support it.

On the outside we also saw a lot of energy. Folks like our advisory board jumped in and started doing the same in the wild. They built real systems, and they guided customers.

All of this work (including shipping Web API) definitely succeeded in help raising the bar around modern API development and hypermedia in the industry in general. More and more folks including some of the largest companies started coming out of the woodwork and saying, we want to build these types of systems.

However, many folks, in particular in the .NET community, have not crossed this chasm yet and do not understand how or why to build such systems. In particular building a hypermedia-based system is a sticking point. Many are scared by the term and don’t understand what it is. Others dismiss it as purely academic. Even those that do grasp the concepts often do not know where to start to implement them. This last point makes total sense, taking into consideration the points I mentioned earlier that ASP.NET Web API does not lead you toward building such APIs, it enables it.

With this book we wanted to change that. We wanted to bring these API techniques to the masses. We wanted to show you why and how to actually build these APIs with ASP.NET Web API. We wanted to show you how to make the underlying architecture of the framework to work with you in achieving that goal.

I believe we’ve done that. We look forward to your experiences with the book as you embark on API development with ASP.NET.

Posted in ASP.NET, ASP.NET Web API, Hypermedia, patterns | 3 Comments

Debugging xUnit tests in Xamarin Studio and Mono Develop using The Debugging Trick™

Update: This post has been completely revamped, as it turns out there is much easier way, kudos to Michael for pointing this out. I will lament my image editing skills for a small second :-)

As I mentioned in my recent post on the Splunk blog, we’ve been developing a new awesome portable C# SDK that will support Xamarin/Mono.  We ran into a bunch of issues running on Linux with tests failing due to file paths and supporting pct-encoded URIs, so I decided to try to debug in Xamarin Studio.

I had really hoped this would be easy. It wasn’t as easy I hoped, but it turns out after a bunch of trial and error that there is a pretty simple path.  Xamarin Studio does have a “Debug Application” option, but running that all I could do was run the xUnit console runner, but I couldn’t pass it any parameters. I tried various things for a while and came up empty.

And then I decided to tweet the always helpful Michael Hutchinson to see if he had any ideas, and he did!

tweet

Update: It turns out you don’t need to do what is suggested above, and which the first version of this post recommended. 

Read on my friends to find out what you can do…..

First open Xamarin Studio to your test project. You can see I’ve done this below and I’ve added a breakpoint on the completely ridiculous, ficticious test we want to debug.

test project1

Now right click on the project, then select Options->Run->Custom Commands. Next a select Execute in the drop down. And put the following into the Command box.

“/Users/glennblock/xunit/xunit.console.clr4.exe ${TargetFile}”

dialog

The first part points to the location of the xunit console runner. The second is an expansion which will get replaced with the test dll.

Now all you have to do is run!

debug

Volia!

And that my friends is “The Debugging Trick™“!

Side note, I don’t think anyone should have to do this, though this version is far less painful than my first post. I just found out from Michael great news, that there is proper IDE support for xUnit in the works.

Screen Shot 2014-05-14 at 11.29.01 PM

This is very good news! In the meanwhile you have the trick :-).

As a side note this technique is general useful for beyond unit tests, you can use it to debug any managed exe where you need to pass arguments.

Thank you Michael Hutchinson for the help!

Posted in mono, xamarin | 1 Comment

Fluent 2014, Splunk for JavaScript developers, and a dash of Hypermedia

Recently a bunch of my fellow Splunkers and I had the pleasure of attending Fluent 2014 in San Francisco where we held down our Splunk booth. It was a great event!

While at the booth I had the privilege of being interviewed on what Splunk offers of JavaScript developers. It turns out we offer quite a lot. Check out the video to learn more:

It was great to talk to customers that are using or interested in Splunk. Anyone who used it loved it, point blank! Working the both was also a chance to educate developers on what you can do with our developer platform, which our CTO Todd Papaioannou did a great job of in his talk on Big Data and Splunk.

As to the talks, I didn’t get to see many but did see a bunch of keynotes of which here are my favorites (with links where appropriate)

  • Scott Hanselman’s talk entitled “Virtual Machines, Javascript and Assembler“. He did a fantastic and highly entertaining talk (in a way that only Scott can do) on how web (in particular JavaScript and cloud development has radically changed the way we build software, and how it has deeply impacted the Microsoft platform.
  • Lea Verou’s keynote on the “Humble Border-Radius“. I had seen this talk online, but there was something magical about watching it live. It was kind of a comedy show through code, and it had some great information about the design of Border-Radius. Lea is an excellent and engaging speaker.

After the event I managed to sneak in a last minute treat before jumping on a plane. I went to see Mike Amundsen‘s (co-author of RESTful Web APIS) talk on building Hypermedia systems at the API-Craft meetup at Heroku,  Mike speaks with authority, and experience. He discussed the problems hypermedia is trying to solve and then walked through how to build a hypermedia client and server all in JavaScript. There was great conversation after the talk a combination of folks sharing their experiences as well as their skepticism ;-)

And as a bonus I got to see some great friends in the community like Ward Bell (author of Breeze.js), Chris Patterson (author of MassTransit), and Steve Klabnik (author of Designing Hypermedia APIs). I also got to meet some people in person for the first time, namely Mark Foster who has been very active in the ALPS and Blueprint space and Emmanuel Parasakis, the organizer of the meetup.

Shot of Chris, Mike, Chromatic Ward, Steve, Mark, and Emmanuel:

IMG_1616

A great way to end a great trip :-)

Posted in splunkdev, Uncategorized | Leave a comment

Extending Splunk’s search language with custom search commands.

I recently blogged on a feature which I worked on, custom search commands for Splunk in our Python SDK. Custom search commands allow you to extend Splunk’s search language with new commands that can do things like apply custom filtering, perform complex mathematical calculations not in the box, or generate events dynamically from an external data source like an external API. Currently we only support them in Python but we’ll be adding more languages in the future.

In the first post of the series I talk about how to build a simple generating command.

Screen-Shot-2014-04-14-at-3.52.49-PM

http://blogs.splunk.com/2014/04/14/building-custom-search-commands-in-python-part-i-a-simple-generating-command/

Posted in splunkdev | Leave a comment