RestBugs Ported to Node

Posted by Howard Dierking on June 9, 2012

I just merged my NodeJS port of the RestBugs sample application and pushed it up to github. In case you’re not 100% sure what RestBugs is, it’s an example of a hypermedia-driven bug management system (system including both client and service) that I originally created as a part of my REST Fundamentals course for Pluralsight.

The Node version is, for the most part, identical to the original ASP.NET Web API version. The only major difference is that the Node version uses a MongoDB database rather than a static collection running in memory. This was primarily driven simply by my desire to play with MongoDB.

My boss asked me this morning what my overall thoughts were comparing the 2 different frameworks and approaches. My comments went something like this: Because there’s less hand-holding by tools coupled with the fact that there’s 10 million ways to do any specific thing, Node can *feel* like it has a bit of steep “trial and error-like” learning curve. However, for those that get over that curve, I can already see that it will be freaking amazing and it will be tough to have patience for platforms that require more work in the name of safety.

That’s my 2 cents for tonight – take it for what it’s worth.

Now, a favor to ask.

For all of you folks who are further down this road then I am, I want to learn from you. If you have a couple minutes to spare, take a look at my < 200 lines and let me know how I can improve my code – and thereby my skills. There are a couple specific areas where I’m particularly interested in improving at the moment:

  • best patterns for error handling – particularly when I have all those nested closures
  • patterns for organizing my code (both modules and object design) – at the moment, everything’s in one file and my “domain logic” is module-scoped. This is workable because the sample isn’t that big, but I can see it becoming a problem for anything non-trivial. I had originally organized things into a “3-layer” type of design, with a Bug JavaScript object – but found it really painful constantly transforming JSON objects in and out. Would especially love to get your thoughts here.
  • good use of middleware – I was thinking that particularly for those post functions where I’m being sent an ID and I’m looking it up and working with a JSON object, it would be handy to push the lookup logic into a middleware function. Good idea? How does this work with errors or no value found?
  • other stuff jump out where I can improve?

I am confident that this code (as well as my Node skills) is a shadow of what it can be – but I wanted to put something tangible up there and then get pointers on where to look next.

So, thanks in advance!

About Howard Dierking

I like technology...a lot...
This entry was posted in Uncategorized. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • http://blogs.msdn.com/gblock Glenn Block

    One other feedback on your listening port. Instead of listening on a fixed port like 9200, consider doing app.listen(process.env.PORT || 9200). This will allow the app to work when deployed to cloud hosts i.e. heroku/azure etc which set the PORT environment variable for apps to listen on as you can’t choose the port. It will still continue to function locally as well as if the PORT variable is not set, it will default to 9200. This is a very common pattern.

  • http://blogs.msdn.com/gblock Glenn Block

    +1 on including a package.json. You can also include a shrinkwrap.json (generated by doing npm shrinkwrap) which includes locked down versions of all the modules as package.json can often have a very liberal versioning policy.

  • http://blogs.msdn.com/gblock Glenn Block

    Nice job Howard

    Closures:

    On closures my main feedback is don’t nest to multiple levels. Instead used named functions which you can pass in rather than inlining a callback. As you mentioned the code gets very hairy to debug.

    For example, here is a case where we do something similar for retrieving / updating task information against table storage.

    https://github.com/WindowsAzure/azure-sdk-for-node/blob/master/examples/tasklist/home.js 

    If you look at the markCompleted function we have a series of local named functions within which end up getting invoked as part of the invocation. In this case we chose to use a process object which we attached the functions to, but you don’t have to.

    Another option is to use the event emitter syntax. Rob Conery has a nice post on that: http://wekeroad.com/2012/04/05/cleaning-up-deep-callback-nesting-with-nodes-eventemitter

    Async:

    Furthermore if you are running a series of steps that are async, there are some helpful modules out there like  ”async” (https://github.com/caolan/async). It simplifies calling a series of callbacks in succession without having the spaghetti code. It also has error handling such that if any of the callbacks fail it will automatically exit out and not continue processing the callbacks.

    In our example we chose not to use async to keep things closer to the metal however if I was doing it for a production app, I would probably use async.

    Factoring:

    I generally recommend not keeping all the business logic int the entry js  i.e. app.js. App.js should be mainly for wiring things up and should not have to change that often. It’s easier to test if you have the code factored into a separate file. For example in the link I have above we moved all the handling logic to a home.js. Home is using protoype and designed with Dependency Injection so that it can be easily tested passing in a fake client.

  • Caro Quintero

    I found a lot of useful information about this subject in this website http://freelancer.peggylopez.com/

  • Wayne M

    Quick question being a total newb to Node.js but really liking the idea (IMO it’s as close to the “holy grail” as we have right now of a single unified platform).  You have everything going off of one HTML file, and have everything defined in the app.js file.  Is that because this is a demo/sample or is that the way a small Node.js app is meant to be when using the Express framework? I mean I suppose it would be trivial to give each grouping its own view if necessary, but just for my own sanity I want to verify how things are “supposed” to be done so I don’t tread down the wrong path when I start to play more with Node.

  • Pingback: RestBugs Ported to Node | Code to Preload

  • http://twitter.com/NotMyself Bobby Johnson

    One thing that I noticed, you have committed your dependencies to your git repo. You might want to look into adding a package.json instead. See here for an example https://github.com/Benvie/Node.js-Ultra-REPL/blob/master/package.json

  • Anonymous

    Now make a version with meteor! :) http://www.meteor.com/

    Not sure if CoffeeScript would help with the node version clarity, though.

  • Jos de Jong

    You are right, it requires some effort to get familiar with Node.js. Javascript does not force you to structure your code, and does not force you do do things in a certain way. Nice about that is that you have all freedom to do it your way.
    You need some discipline to structure your code yourself.

    NESTED CLOSURES

    If you put your callbacks inline as a function parameter, your code gets deeply nested very quickly, which gives unreadable and hard to read code. Just split the callbacks from the function calls. Instead of:

        db.bugs.save(
            newbug(req.body.title, req.body.description), 
            function(err, savedDoc) {
                db.bugs.find( {status: ‘Backlog’}, function(err, docs) {
                res.render(‘bugs-all.html’, {
                    title: ‘Backlog’,
                    model: docs
                });
            });
        });

    you can write:

        var onFoundCallback = function(err, docs) {
            res.render(‘bugs-all.html’, {
                title: ‘Backlog’,
                model: docs
            });
        }

        var onSavedCallback = function(err, savedDoc) {
            db.bugs.find( {status: ‘Backlog’}, onFoundCallback);
        }

        db.bugs.save(newbug(req.body.title, req.body.description), onSavedCallback);

    ERROR HANDLING

    I personally very much dislike the way callbacks and errors are handled by most
    Node.js applications and libaries: when you use callbacks with two arguments (error, result), you need to check if there is an error in every callback. It is much nicer to provide callback functions: callback(res), and errback(err). You can just propagate the errback function without need to check for success and failure every time

    The callback mechanism with a single callback method with two parameters:

      function add (a, b, callback) {
          if ((typeof a != ‘number’) || (typeof b != ‘number’)) {
              callback(‘Error: invalid params’, undefined);
          }
          callback(undefined, a + b);
      }

      function eval(callback) {
          add(2, 3, function (error, res) {
              // in each callback we MUST to check for errors
              if (error) {
                  callback(error, undefined);
              }
              else {
                  add(res, 4, callback);
              }
          });
      }

    The callback mechanism with two callback methods with each a single parameter:

      function add (a, b, callback, errback) {
          if ((typeof a != ‘number’) || (typeof b != ‘number’)) {
              errback(‘Error: invalid params’);
          }
          callback(a + b);
      }

      function eval(callback, errback) {
          var addNext = function (result) {
              // the callback will only be executed when there is no error,
              // so no need for error handling here!
              add(res, 4, callback, errback);
          }
          add(2, 3, addNext, errback);
      }

    NAMESPACES, MODULES

    You can just use namespacing and create prototypes “classes” like you do in other languages:

        // create a namespaces
        var restbugs = {};
        restbugs.services = {}; 
        
        // constructor
        restbugs.services.HtmlFormatter = function (options) {
            this.options = options;
        }
        restbugs.services.HtmlFormatter.prototype.writeStream = function (value, stream) {
            // … write stream
        }
        
        // usage:
        var formatter = new restbugs.services.HtmlFormatter();
        formatter.writeStream(value, stream);

    IDE

    It helps a lot to use a helpful IDE. I use WebStorm since some time now, which is really amazing. It gives you a code outline, and when you properly comment your code with jsdoc annotations, it does typechecking for you (!!!). WebStorm has Node.js and Github integrated, which makes development really fun.

    (I hope the code indentation in my post will not be lost when submitting. If so, sorry for that)

  • http://kenegozi.com/blog Ken Egozi

    pretty good for someone claiming to be new to the techs.

    a couple of things though:

    as far as mongo related stuff – you’d want to use send an update that does the change server side in an atomic update, instead of “load, edit, save”. right now it is open to concurrency problems

    you are also not showing how the db is set – specifically the bugs collections’ indexes.
    you could  potentially drop the bugs collection instead of remove()ing the docs, then ensureIndex()s if relevant (probably is – an index on Status will make sense for e.g.)
     
    as far as web app stuff = the POST actions should not generally be rendering html as a response, but instead redirect to a GET that will render. This will eliminate the problem with a user doing F5 (or back later on) and asked by the browser to re-send the POST

    as far as node moduling – this example is simple enough to fit nicely in a single file.
    you could break the logic calls (like updateStatus, close, etc) into a separate module file, which would also potentially encapsulate db access. you could also use modules to declare group of routes (think mvc Controllers as a unit of grouping Actions) which in turn will delegate work to (a / some) logic modules. Since express is a lower-level fw than the likes of Rails/MVC/etc and is more similar in nature to Sinatra or Nancy, you enjoy the light-ness, but lose the guidance. which leaves you in common sense.