A Gentle Reintroduction to the Reactive Extensions for JavaScript Part 1

In the previous post, I briefly covered that yes, the Reactive Extensions for JavaScript is alive, with two releases to show for it.  Before going into the depths of what’s in the box for V1 and V2, I want to step back and cover some of the basics and why you might care about this library.  It’s great to talk about APIs and general capabilities, but let’s first cover asynchronous and event-based programming in JavaScript today.

Callbacks as a Way of Life

The life of a JavaScript programmer is filled with callbacks whether it is listening to events, creating Ajax requests or even simple coordination using setTimeout, there is no getting around this.  For many who hang out on the Node.js mailing list, this continues to be a pain spot as they dip their toes into large scale application development in JavaScript. 

In any given day in the browser, you might be using

  • DOM and Custom Events
  • Timeouts and Intervals (setTimeout, setInterval)
  • Ajax Requests

And with the introduction of HTML5, we have even more possibilities for rich interactions adding in, each with their own way of expressing asynchronous/evented behavior.

  • Scheduling (requestAnimationFrame)
  • Geolocation
  • EventSource
  • File API
  • IndexedDB
  • Web Sockets
  • Web Workers

Node.js also embraces JavaScript to the core by using callbacks everywhere from the file and network APIs down to event emitters.

Making the Case for Composition

With me highlighting all of the areas of callbacks within a JavaScript developer’s daily routine, is there anything wrong with it?  By themselves, no, meaning if I just want to handle a single click event on a DOM element and then take result from an input and query an Ajax service via a Promise API, and populate the results, standard callbacks work just fine, whether nested or named functions.

But, what if we want to go a little deeper than that?

  • What if we want to throttle data such as key up events?
  • What if we want to filter the data before querying another service?
  • What about composition?  Can I combine events together?
  • The list goes on…

When we start addressing those concerns, then we quickly lose sight of the actual logic required for simply taking text and querying a service.  It goes from a pretty simple solution to something a bit harder to understand.

Instead, we should be focusing on our actual behavior that we want, such as the distinct values after an allotted period of time.  What if we could design an API where we can add compositional behavior via a Fluent API to solve this issue?  The Reactive Extensions for JavaScript provides a mechanism to do exactly that via a well known pattern of the Subject/Observer pattern.

This is just a simple example, but the idea is that we can combine together both events which yield zero to many values, with single return methods such as Ajax requests, in a single API.  Patterns such as Promises go only so far with assurances of composition in a single way of dealing with completion, progress and errors, but it doesn’t address the very core issue of event composition.

Conclusion

In the coming posts, I’ll go over in detail, reintroducing the Reactive Extensions for JavaScript, why it’s useful and where you can take advantage of it.  Check us out on GitHub, download us via NuGet or the MSDN Data Developer Center, and give feedback on the forum!

Posted in JavaScript, Reactive Framework | Tagged , | Leave a comment

A Gentle Reintroduction to the Reactive Extensions for JavaScript

One of the things that I’ve been doing since I last blogged has been working on the Reactive Extensions for JavaScript (RxJS).  Since I last blogged, we have had two releases, a version 1.0.10621 SP1 which we released back in December of last year and version 2.0 Beta which was just released a couple of weeks ago.  You can check out the release notes for each version, v1.0.10621 SP1 and v2.0.20327 Beta respectively.  Along with these releases, we’ve also put the packages up on NuGet, create Visual Studio style documentation/intellisense, and put the bindings for third-party libraries and runtimes such as Node.js, jQuery, Dojo, ExtJS, MooTools and others on GitHub.

With the release of version 1.1, it was a complete re-engineering effort to not only maintain parity with the .NET version, which had changed quite a bit since the Reactive Extensions for JavaScript was first released.  Not only did we maintain parity with the .NET APIs, we also made an effort to make it feel more like JavaScript, but camel-casing the names to be more inline with JavaScript itself. For example, the old way of doing things with version 1.0 with C# style API calls:

Becomes camel cased to fit more in line with JavaScript norms.  This move made us rename some of the methods which collide with JavaScript reserved words such as Return => returnvalue, Catch => catchException.

The Reactive Extensions for JavaScript V2 Beta once again tries for parity with the Reactive Extensions for .NET V2 Beta in terms of APIs, but differs as with the current release, we still support the old schedulers of Immediate, CurrentThread and Timeout.  We’ve also added new experimental operators such as imperative operators as if, case, for, while and do while, as well as others such as fork join.  I’ll go over more in detail in subsequent posts.

Conclusion

In the coming posts, I’ll go over in detail, reintroducing the Reactive Extensions for JavaScript, why it’s useful and where you can take advantage of it.  Check us out on GitHub, download us via NuGet or the MSDN Data Developer Center, and give feedback on the forum!

Posted in JavaScript, Reactive Framework | Tagged , , | Leave a comment

The Reactive Extensions Extensions Now Live on MSDN Data Developer Center

As you may have noticed, the Reactive Extensions for .NET and JavaScript have a new home, moving from MSDN Dev Labs to the MSDN Data Developer Center.  The new home will provide you with not only how to get them, but also a beginner’s guide and other additional resources including hands on labs, videos, tutorials, forums and much more.  

First Off, How Do I Get Them?

There are several ways to get your hands on the Reactive Extensions (Rx) either by installing them via the Reactive Extensions for .NET Home or via NuGet which we’ll cover below.  The Reactive Extensions is available on multiple platforms such as:

  • .NET 3.5 SP1
  • .NET 4
  • Silverlight 3
  • Silverlight 4
  • Windows Phone 7
  • XBox and Zune via XNA
  • JavaScript

The great news about getting the Windows Phone 7 is that you get the Reactive Extensions out of the box as it shipped on the ROM itself.  Since it shipped with the phone, there have been updates and you can get the latest always by visiting the site or via NuGet.

Now About NuGet…

Besides a direct download on the Rx Home Page, you can also get them via NuGet.  We have several packages to choose from based upon your needs.  First, open up the NuGet window by adding a Library Package Reference.

image

Next, in the search window, type Rx and hit enter to search.  You’ll see that we ship quite a few packages already and have since NuGet was first introduced.

image

As you can see from the picture, we have the following packages available.  We’ll briefly describe each although some will need their own blog post to cover.

  • Rx-Core
    Contains the libraries shared by all Rx libraries
  • Rx-Main
    The main library for the Reactive Extensions which contains the Observable LINQ operators
    Depends on Rx-Core
  • Rx-Interactive
    Contains extensions for IEnumerable<T> to bring parity to Observable operators
    Depends on Rx-Core
  • Rx-ClientProfile
    Brings the Rx extension methods to the .NET Client Profile API
    Depends on Rx-Core, Rx-Main
  • Rx-ExtenededProfile
    Bring the Rx extension methods to the .NET Extended Profile API
    Depends on Rx-Core, Rx-Main, Rx-ClientProfile
  • Rx-Testing
    A library for testing the Reactive Extensions which integrates into your unit testing framework of choice
    Depends on Rx-Core, Rx-Main
  • Rx-AsyncLINQ
    Contains an asynchronous version of the Enumerable pattern
    Depends on Rx-Core
  • Rx-All
    Gives you all of the above

The last few in this list including the Client Profile, Extended Profile, Testing and AsyncLINQ will need to be broken down into its own post. 

Let’s say we want just Rx-Main for now which will give us the basic Reactive Extensions for .NET that we need to get moving.  We click the install button next to it and then we’re prompted for the license, which you must accept:

image

Once accepted, we then have System.CoreEx.dll and System.Reactive added to my .NET 4 project along with my packages.config.

image

On the file system, it will bring down the .NET 3.5 SP1, .NET 4 and Silverlight 4 libraries for future reference.  This includes not only the libraries themselves, but also the documentation in the form of CHM files.

image

And you’re ready to go!

image

Conclusion

With the move from MSDN Dev Labs to the MSDN Data Developer Center, we now have a better home for now only how to get the Reactive Extensions, but also how to get started by watching videos, tutorials and asking questions via the forum.  So, with that, enjoy and we’d love your feedback!

Posted in Uncategorized | Leave a comment

Getting Started with Node.js on Windows

UPDATE: New packages and further items

What a difference a year makes.  Last year at the first conference solely dedicated to JavaScript, JSConf 2009, there was a big focus on the uses of JavaScript on the client side.  This year, there was a marked change with server-side JavaScript taking up about half the sessions at JSConf 2010 this past April.  The rise of server-side JavaScript has been in large part to two platforms, Narwahl and node.js.  Node in particular has garnered a bit of attention lately, especially with the node knockout which happened very recently.  The winners in the node knockout competition alone should give you some clue as to its potential.

Why Node?  Simply put, the purpose of node.js is “To provide a purely evented, non-blocking infrastructure to script highly concurrent programs.”  This stands in contrast to many other concurrency models where threads and synchronization blocks are used, which can cause any number of issues.  Because of its lock free nature, and a bigger freedom from deadlocks, it allows for a more approachable way to concurrency.  The node.js site itself has a great amount of resources here to get started.

In this series, I’m going to focus a little bit on getting started on node.js and how you can mix the Reactive Extensions for JavaScript into your node solutions.

Getting Started

One downside has been in the past for me, being a developer who uses Windows frequently (given that I’m employed by Microsoft), has been the lack of native support for Windows.  Fortunately, with the release of node.js version 0.2, there is now support for Windows, albeit through Cygwin.  Following the instructions here, I was able to get it to install on any number of machines, from XP, to Windows 7 to Windows Server 2008 R2.   

When installing Cygwin, I chose the following packages:

  • Dev (install all works just fine for all of them if space is not an issue, and I’d recommend that)
    • gcc g++ C++ compiler
    • gcc mingw-g++
    • gcc4-4++ G++ subpackage
    • git
    • make
    • openssl
    • pkg-config
    • zlib-devel
  • Python – install
  • Web
    • wget

Once that installs, before you’ll be able to successfully build anything, you should do a rebaseall by doing the following steps:

  1. Close any instances of Cygwin
  2. Launch a command prompt
  3. Execute ash.exe (commonly found in C:\Cygwin\bin)
  4. Execute ./rebaseall –v

You should see results similar to the following:

/usr/lib/abc.dll new base = 1234, new size = 1234

Now it’s time for us to pull down a fresh copy of node.js.  You can either do it from wget and unpack it as the following:

$ wget http://nodejs.org/dist/node-v0.2.0.tar.gz
$ tar xvf node-v0.2.0.tar.gz

Or using Git, we can pull that down by the following command, if you installed Git of course like I told you to:

$ git clone http://github.com/ry/node.git
$ cd node
$ git fetch --all
$ git tag
$ git checkout v0.2.0

If you’re using a proxy server, like I was in several installs, you can get around that by setting the http_proxy environment variable:

$ export http_proxy=http://someproxy:1234/

Then you can start to configure and build such as the following:

$ cd node
$ ./configure
$ make
$ make install

Let this chug for a little bit and it should build just fine.  You can check for success by displaying the version of node such as:

$ node --version
v0.2.0

Another thing to notice is that we need to configure the DNS.  Although Cygwin itself relies upon the Windows DNS for its queries, node.js uses the c-ares library that relies on /etc/resolv.conf file.  Edit this file (if found, else create it) and add the entries to the Google public DNS entries:

nameserver 8.8.8.8
nameserver 8.8.4.4

Now, it’s time to set up our first file, called example.js and put it in the root of your node install, just for simplicity sake at this point.  We’ll create the typical web server example where we write our header and a body containing Hello World from Cygwin.

var http = require('http');

http.createServer(function (request, response) {
  response.writeHead(200, {'Content-Type': 'text/html'});
  response.end('<b>Hello World from Cygwin</b>');
}).listen(8124);

console.log('Server running at http://127.0.0.1:8124/');

And then we can start the server by the following:

$ node example.js
Server running at http://127.0.0.1:8124/

And now we can see the fruits of our labor by going to the address in our browser:

image

And there you have it, ready to roll with node.js on Windows (via Cygwin).

Conclusion

Here we’ve just scratched the surface of what we can do with node, now that it can be accessible to the many Windows developers out there.  Node, with its engaging community, is giving developers options for creating interesting scalable applications using JavaScript (yes, JavaScript).  In this series, I hope to dive a little bit into it as well as how the Reactive Extensions for JavaScript can play a part.

Posted in JavaScript, node, Reactive Framework, ssjs | 25 Comments

Reactive Extensions for .NET – Event-based Async Operations

With the many posts that I’ve done on the Reactive Extensions for both JavaScript and .NET, I’ve covered a wide variety of the basics as well as some of the deeper stuff.  This time, I’d like to get back to the basics that I do when I give a talk on Rx in person, and this time explaining some of the problems we face today with reactive programming in general, asynchronous and event-based.

Why Do We Need It?

Let’s go into an example of using the event-based asynchronous programming methods that became a bit more prevalent when .NET 2.0 came around, especially with the introduction of WCF in the .NET 3.0 timeframe.  This was an alternative to the other methods of asynchronous programming which included the Begin/End Pattern and using WaitHandles.  One distinct advantage it did have was for the ability to report progress, report incremental results or state changes that the others could not provide.  But with it, comes many downsides as well which we’ll go into and where the Reactive Extensions can help. 

Typically, we’d have some form of asynchronous method which has no return value and takes in any number of parameters, and a given user token which would allow us to correlate our token with what comes in the user state to make sure we’re looking at the right result. 

public void DoSomethingAsync(object token) 
{
    // Kick off something async
}

And we’d also have some form of EventArgs which would encompass our result and exception should one occur and optionally a cancelled flag should we need to do so and be notified that it happened.  Then we’d also have our EventHandler which is then called when our asynchronous method is complete.

public class DoSomethingCompletedEventArgs : EventArgs 
{
    public string Result { get; set; }
    public bool Cancelled { get; set; }
    public Exception Error { get; set; }
}

public event EventHandler<DoSomethingCompletedEventArgs>
    DoSomethingCompleted;

To give you an idea of some of the problems we face with this pattern, let’s go over an example using the System.Net.WebClient class and downloading a string asynchronously via the DownloadStringAsync method.  Let’s enumerate some of the challenges in the code below.

var uri = new Uri("http://search.twitter.com/search.json?q=4sq.com");
var client = new WebClient();

client.DownloadStringCompleted += (sender, args)
{
    // TODO: Get the only one we want?
    // TODO: Check for cancel?
    // TODO: Check for exception?
    // TODO: Check for result?
    
    // TODO: What about more composition?
};

client.DownloadStringAsync(uri);

// TODO: How to unsubscribe
client.DownloadStringCompleted -= ???

The first challenge is how to make sure we get the value we want.  After all, if multiple calls are made to this WebClient instance, we’ll need to correlate the user state with our token.  Next, we need to check for cancellation and do something when that happens.  After checking for cancellation, we might also want to check for exceptions and handle them appropriately as well.  And finally we get our result, now what do we do with it?  Better yet, now that I have the result, how can I compose it with another asynchronous call?  Finally, when using lambdas for subscribing to events, I can’t unsubscribe our inline lambda, so that could cause an issue as well.  Making some of those changes to handle our cases properly, we end up with code that looks like the following:

var uri = new Uri("http://search.twitter.com/search.json?q=4sq.com");
var client = new WebClient();
var token = new object();

DownloadStringCompletedEventHandler h = (sender, args)
{
    // Check if it's ours
    if (token != args.UserState) return;

    if (args.Cancelled)
    {
        // Do something to say we stopped
    }
    else if (args.Error != null)
    {
        // Do something with that error
    }
    else
    {
        // Well, now what?
    }
};

client.DownloadStringCompleted += h;
client.DownloadStringAsync(uri, token);

// When we're ready to clean up
client.DownloadStringCompleted -= h;

We solved a good majority of the problems with a lot of boilerplate.  But within all this boilerplate, we’re losing sight of our core goal here, which is to get our result and do something with it, especially when that next call happens to be to another asynchronous call.  For example, what if we want to download a string from somewhere, transform it into JSON, modify the results and then upload somewhere else?  Our code might look like the following should we use the Reactive Extensions for .NET to solve it.

var client = new WebClient();
var searchUri = new Uri("http://search.twitter.com/search.json?q=4sq.com");
var uploadUri = new Uri("http://localhost:8080/uploads");

IObservable<Unit> query =
    from result in client.DownloadStringAsObservable(searchUri, new object())
    let transformed = TransformResult(result)
    from upload in client.UploadStringAsObservable(
        uploadUri,
        "POST",
        transformed,
        new object())
    select upload;
    
var subscription = query.Subscribe(
    x => {}, // Nothing to do
    exn => 
    {
        // Do something with the exception
    }, 
    () => 
    {
        // Indicate we're finished
    });

What we’re able to do is take operations that would have required quite a bit of code in terms of boilerplate, and allow for composition through LINQ via the Reactive Extensions for .NET.  But, how do we get there? 

How Do We Get There?

What we need to do at this point is to transform our asynchronous call from above and turn it into an IObservable<T> instance.  Using the FromEvent which we’ve covered earlier, won’t cut it because ultimately, we want to combine the asynchronous call with the event handler which is invoked upon completion, which FromEvent cannot provider.  Instead, we want to focus our effort into the Observable.Create and Observable.CreateWithDisposable methods to help us.  This allows us to create IObservable<T> instances from the subscribe implementation.  We can either choose the CreateWithDisposable method which gives us an IObserver<T> instance and we return a concrete IDisposable instance, or we call Create in which are given the same IObserver<T> instance, but we return an Action instead of the IDisposable instance.  Let’s look at those signatures below:

public static IObservable<TSource> CreateWithDisposable<TSource>(
    Func<IObserver<TSource>, IDisposable> subscribe
)

public static IObservable<TSource> Create<TSource>(
    Func<IObserver<TSource>, Action> subscribe
)

Using the Observable.Create method, let’s create a method called DownloadStringAsObservable extension method to handle the asynchronous request.  Inside the Observable.Create, we’ll create our DownloadStringCompletedEventHandler where we can check for our user token, handle cancellation by calling observer.OnCompleted, handle exceptions via observer.OnError and finally handle our result by calling observer.OnNext and observer.OnCompleted to indicate we are finished.  We then take this handler and listen to the DownloadStringCompleted event.  We want to be able to catch any errors in our DownloadStringAsync should it throw an exception, so we put that in a try/catch block and send the Exception to observer.OnError.  Finally, Being the good citizens we are, we want to clean up after ourselves by removing our handler from the DownloadStringCompleted event when we dispose of our subscription.

public static IObservable<string> DownloadStringAsObservable(
    this WebClient client, 
    Uri address,
    object userToken)
{
    return Observable.Create<string>(observer =>
    {
        DownloadStringCompletedEventHandler handler = (sender, args) =>
        {
            if (args.UserState != userToken) return;
    
            if (args.Cancelled)
                observer.OnCompleted();
            else if(args.Error != null)
                observer.OnError(args.Error);
            else
            {
                observer.OnNext(args.Result);
                observer.OnCompleted();
            }
        };
    
        client.DownloadStringCompleted += handler;
        
        try
        {
            client.DownloadStringAsync(address, token);
        }
        catch (Exception ex)
        {
            observer.OnError(ex);
        }
        
        return () => client.DownloadStringCompleted -= handler;
    });
}

The sample logic applies to uploading a string as well by wrapping the UploadStringAsync method as we do below.

public static IObservable<Unit> UploadStringAsObservable(
    this WebClient client, 
    Uri address, 
    string method, 
    string data,
    object userToken)
{
    return Observable.Create<Unit>(observer =>
    {
        UploadStringCompletedEventHandler handler = (sender, args) =>
        {
            if (args.UserState != userToken) return;
    
            if (args.Cancelled)
                observer.OnCompleted();
            else if (args.Error != null)
                observer.OnError(args.Error);
            else
            {
                observer.OnNext(new Unit());
                observer.OnCompleted();
            }
        };
    
        client.UploadStringCompleted += handler;
        
        try
        {
            client.UploadStringAsync(address, data, method, token);
        }
        catch(Exception ex)
        {
            observer.OnError(ex);
        }
        
        return () => client.UploadStringCompleted -= handler;
    });          
}

By using these tokens as we do above, we could easily wrap the DownloadProgressChanged event as well filtering for our token as well.

public static IObservable<DownloadProgressChangedEventArgs> 
    DownloadProgressChangedAsObservable(
        this WebClient client,
        object userToken)
{
    return Observable.FromEvent<DownloadProgressChangedEventArgs>(
            client,
            "DownloadProgressChanged")
        .Where(x => x.EventArgs.UserState == userToken)
        .Select(x => x.EventArgs)
}

What we have done here is taken a less than optimal event-based asynchronous programming model and moved it to use Reactive Extensions for .NET so that we can take care of uniform cancellation checking, error handling and allow for composition.

Conclusion

Asynchronous programming is something that’s creeping up more and more to .NET developers, especially with Silverlight, Windows Phone and so forth.  Unfortunately, dealing with asynchronous programming has been and continues to be hard.  By using the Reactive Extensions for .NET, we have ways to build bridges from the old asynchronous programming models, to a more composable form where we can uniformly handle exceptions, cancellation and so forth.

So with that, download it, and give the team feedback!

Posted in C#, Event-based Porgramming, Reactive Framework | 2 Comments