Brendan Tompkins

Ducksboard API for .Net

Posted on February 11, 2013 by Brendan Tompkins

Last time, I wrote about how neat I thought the Ducksboard app was. I found a nice little project over on github that wrapped up the update components of the api pretty well. So I could easily create a dashboard, and configure my application to push data to Ducksboard using a .NET component with nuget support and coding techniques I’m used to.

I started playing and noticed that the Dashboard API (used to create Ducksboard dashboards, not update them) wasn’t implemented, and I kind of needed it. We have a product that could use a dashboard, and I wanted the dashboard functionality to go something like this:

App starts up, checks to see if there’s a Ducksboard dashboard for client x. If not, create it, populate it with widgets, and then go on doing work stuff, occasionally updating the Ducksboard dashboard with metrics, etc.

All this creation stuff happens at ducksboard with their Dashboard API.

The Dashboard API can be used to manipulate all objects found in Ducksboard: dashboards, widgets, external accounts, public access tokens and so on.

I contacted the author of the Ducksboard API for .NET to see if he was working on this api, and he wasn’t, so I coded up what I needed trying to stick with the original authors api design as much as possible, submitted a pull request, and now it’s part of the project. What the heck did we do before OSS?

Anyhow, there’s some awkwardness in the code there (mostly due to my limited knowledge of json serialization, but also due to some constraints of their api), but it is working pretty nicely if you know what you want to do with their api. For example, the following few lines of code creates a dashboard with a leader board component :

    
      
        
        
          var dashboardClient = new DashboardClient("** Your API Key Here **");
var dashboard = new Dashboard()
            {
                Name = dashboardName,
                Background = "dark wood",
            };
 
dashboard = dashboardClient.Create(dashboard);
 
var leaderboard = client.Create(new Widget()
            {
                WidgetProperties = new WidgetProperties()
                {
                    Dashboard = dashboard.Slug,
                    Kind = "custom_textual_status_leaderboard",
                    Sound = true,
                    Title = "My Leaderboard",
                    Width = 1,
                    Height = 2
                },
                Slots = new Slot
                {
                    Slot1 = new SlotData
                        {
                            Subtitle1 = "Thing",
                            Color1 = Color.DarkSlateBlue.ToDucksboardColor(),
                            Subtitle2 = "Status",
                            Color2 = Color.Black.ToDucksboardColor()
                        }
                }
            });
 
 
leaderboard.Slots.Slot1.Label = "MySlotLabel";
 
dashboardCient.Update(leaderboard, leaderboard.Slug);

        
      

  

          view raw
          gistfile1.cs
          This Gist brought to you by GitHub.
        

One thing I’m grappling with is the long term viability of using Ducksboard for corporate dashboards. I’m exploring the idea, but am waiting to see if our clients will go for a super nice, accessible-from-anywhere dashboard, with reduced complexity on in house servers, and all of the benefit that that brings, or if they think it’s too risky to send out data to a third party… For now, I’m happy to have it there as an option for those that want to use it.

Posted in .NET, Cloud Hosting | 2 Comments

A Log4Net Ducksboard Appender

Posted on January 26, 2013 by Brendan Tompkins

Ducksboard, is absolutely the most fun I’ve had on a web api in a long time.. “Monitor your metrics with Ducksboard, a real time visualization of your key performance indicators..” Basically, you make dashboards with it. It’s got built in pugins for tons of other apps (basecamp, github, twitter, pingdom etc. etc.) and a really nicely done restful api where you can manage your dashboards.

I’m just playing at this point, but I’m thinking that I’d like to use it to create some customer dashboards to show real-time metrics that are applicable to our apps. One thing I was thinking was an uptime dashboard that could show the health of our services and devices, sort of like this.

One way I can easily post to this board whenever there is an error in any of our apps is to create a Ducksboard Appender for log4net. I used a nice little .net wrapper for Ducksboard and made the following appender :

    
      
        
        
          using Ducksboard;
using Ducksboard.Objects;
using log4net.Appender;
using log4net.Core;
 
namespace Logging.DucksboardLog4NetLogger
{
    public class DucksboardAppender : AppenderSkeleton
    {
        private static DucksboardClient _ducksboardClient;
        public string ApiKey { get; set; }
        public string WidgetId { get; set; }
 
        protected override void Append(LoggingEvent loggingEvent)
        {
 
            if (_ducksboardClient == null)
                _ducksboardClient = new DucksboardClient(ApiKey);
 
            var timeline = new Timeline
                {
                    Value =
                        {
                            Image = GetImage(loggingEvent.Level),
                            Content = loggingEvent.RenderedMessage,
                            Title = loggingEvent.Level.ToString()
                        }
                };
 
 
            _ducksboardClient.Update(WidgetId, timeline);
        }
 
        private static string GetImage(Level level)
        {
            if (level >= Level.Error)
                return
                    "https://app.ducksboard.com/static/img/timeline/red.gif";
           
            return level == Level.Warn ? 
                "https://app.ducksboard.com/static/img/timeline/orange.gif" :
                "https://app.ducksboard.com/static/img/timeline/green.gif";
        }
    }
} 

        
      

  

          view raw
          gistfile1.cs
          This Gist brought to you by GitHub.
        

Configure it like this :

    
      
        
        
           <log4net>
    <appender name="DucksboardAppender" type="DucksboardLog4NetLogger.DucksboardAppender">
      <apiKey value="[your api key]" />
      <widgetId value="[your widget id]" /> 
      <layout type="log4net.Layout.PatternLayout">
        <conversionPattern value="%date | %-5level | %message~%newline" />
      </layout>
    </appender>
    <logger name="[your logger name]">
      <level value="ALL" />
      <appender-ref ref="DucksboardAppender" />
    </logger>
  </log4net>

        
      

  

view raw gistfile1.txt This Gist brought to you by GitHub.

I’m hoping to contribute to the Ducksboard api for .NET and add the dashboard API, which allows you to create dashboards in code. A new installation of one of our services could create a new monitoring dashboard, add all of the relevant graph and informational displays, then start updating it.

Disclaimer : I recently asked Ducksboard to become “Friends” of CodeBetter.Com so you’ll see them linked at the bottom of the site, but this post is unrelated.

Posted in .NET | Leave a comment

Fun with the Nancy Self Host!

Posted on January 15, 2013 by Brendan Tompkins

Nancy Logo

If you’re not familiar with the Nancy framework for hosting HTTP services in .net, you owe it to yourself to check it out.

From the docs:

Nancy is a lightweight, low-ceremony, framework for building HTTP based services on .Net and Mono. The goal of the framework is to stay out of the way as much as possible and provide a super-duper-happy-path to all interactions.

For a nice audio introduction, check out this recent episode of Hanselminutes Understanding NancyFx With Richard Cirerol.

One of the things I like best about Nancy, other than it’s super fun and easy to stand up and build HTTP services, is the Self-Host.

At work, one thing we make are some networked edge-devices for things like kiosks that are normally deployed into the field at our client sites. One of the things I find super helpful for configuration, debugging and troubleshooting, is a simple HTTP endpoint.

Imagine a problem with an edge device out in the field – like a kiosk, camera, DIO controller, etc. One of the first things a sysop may do is browse to the device’s HTTP endpoint, pull up a web page, check configuration and logging. There are certainly more sophisticated ways of going about management and troubleshooting, but for simple cases an administration web page works well. It often can save a sysop a trip out into a hazardous environment if a quick fix can be done remotely, even better using a secure browser session.

So what are your options for doing this? If you already have an existing .NET application running on your device, with state, etc. spinning up IIS and a web site isn’t a great option.. You will have to tackle issues of sharing data, installation of the web site every time you setup a new device, the overhead of running IIS, IIS security, just to name a few. What if there was a simple way of generating an HTTP endpoint? – Enter Nancy Self Hosting.

With the Nancy Self-Host, you can easily make your HTTP endpoint part of your existing application, which means you can share state, installation process, security, authentication etc. And you can do it all on the “Super Duper Happy Path.” Here’s how easy it is to host Nancy in a console app:

    
          namespace Nancy.Demo.Hosting.Self
{
    using System;
    using System.Diagnostics;
 
    using Nancy.Hosting.Self;
 
    class Program
    {
        static void Main()
        {
            var nancyHost = new NancyHost(new Uri("http://localhost:8888/nancy/"), new Uri("http://127.0.0.1:8888/nancy/"), new Uri("http://localhost:8889/nancytoo/"));
            nancyHost.Start();
 
            Console.WriteLine("Nancy now listening - navigating to http://localhost:8888/nancy/. Press enter to stop");
            Process.Start("http://localhost:8888/nancy/");
            Console.ReadKey();
 
            nancyHost.Stop();
 
            Console.WriteLine("Stopped. Good bye!");
        }
    }
}

view raw gistfile1.txt This Gist brought to you by GitHub.

How about from a Service? Here’s how easy it is to host Nancy from within your own TopShelf service.

    
      
        
        
          using System;
using Nancy.Hosting.Self;
using Topshelf;
 
namespace Nancy.Demo.Hosting.TopShelf
{
    public class NancySelfHost
    {
        private NancyHost _nancyHost;
 
        public void Start() 
        {
            _nancyHost = new NancyHost(new Uri("http://localhost:8888/nancy/"));
            _nancyHost.Start();
            Console.WriteLine("Nancy now listening - http://localhost:8888/nancy/. Press ctrl-c to stop");
        }
 
        public void Stop()
        {
            _nancyHost.Stop();
            Console.WriteLine("Stopped. Good bye!");
        }
    }
 
    public class Program
    {
        public static void Main()
        {
            HostFactory.Run(x =>                                    //1
            {
                x.Service<NancySelfHost>(s =>                       //2
                {
                    s.ConstructUsing(name => new NancySelfHost());  //3
                    s.WhenStarted(tc => tc.Start());                //4
                    s.WhenStopped(tc => tc.Stop());                 //5
                });
 
                x.UseLog4Net();
                x.RunAsLocalSystem();                               //6
                x.SetDescription("Sample Topshelf Host");           //7
                x.SetDisplayName("Stuff");                          //8
                x.SetServiceName("stuff");                          //9
              
            });                                                     //10
        }
    }
}

        
      

  

          view raw
          gistfile1.cs
          This Gist brought to you by GitHub.
        

I’ve put a quick repo up on Github as a demo for hosting Nancy under TopShelf.

In short, I personally see a huge sweet spot for Nancy in these cases where you need a simple management endpoint. Next time, I’ll blog about how to do this from NServiceBus when using the NSB host – even sharing the NSB IOC container.

Happy Self Hosting!

Posted in .NET, Uncategorized | Tagged .NET, HTTP, NancyFX, TopShelf | 3 Comments

Setting up Munin with Nginx and Ubuntu 12.10 (Quantal Quetzal)

Posted on December 18, 2012 by Brendan Tompkins

A few years ago, I posted about our over-architected setup here at Codebetter. I had fun working with Rackspace cloud slices, and getting this all to run on ridiculously cheap, 256 M linux hosts. One of the tools I’ve come to depend on for monitoring is Munin.

Munin is a networked resource monitoring tool that can help analyze resource trends and “what just happened to kill our performance?” problems. It is designed to be very plug and play. A default installation provides a lot of graphs with almost no work.

Here’s the graph of CPU usage since yesterday. Munin let’s you drill down into almost any aspect of your system, with plugins to allow you to monitor almost any service you may be running, and even support for monitoring Windows boxes.

Check out this graph from Codebetter’s main app server’s CPU. How nice is that?

I’ve recently migrated to new servers yet again, and I ran into a few gotcha’s for Ubuntu 12.10 users that I’d like to post here. It seems that most apps (WordPress for example) are much easier to configure under Apache than Nginx, and Munin is no different. If you crave the performance and configurability that Nginx brings, you’ll probably be used to taking a few extra steps to get applications to run properly. This page from the Munin docs has some good starting points for setting up under Nginx, but even so I got stuck. There’s one sentence that says the following:

Note: Depending on your installation method, the “munin-*-graph” programs may be in another directory. Check Makefile.config if you installed from source, or your package manager if you used that to install.

This may be obvious to some, but I ran into all sorts of problems until I understood what changes I had to make for everything to work.

1) Correct the path to the munin-cgi-graph and munin-cgi-script perl files, when you create your fast-cgi sockets

    
          spawn-fcgi -s /var/run/munin/fastcgi-graph.sock -U www-data \
  -u munin -g munin /usr/lib/cgi-bin/munin-cgi-graph
 
spawn-fcgi -s /var/run/munin/fastcgi-html.sock  -U www-data \
  -u munin -g munin /usr/lib/cgi-bin/munin-cgi-html

view raw gistfile1.sh This Gist brought to you by GitHub.

2) Correct NGINX configuration, here’s mine. Notice the location /muni-cgi/ changes to /cgi-bin/. Also notice that I’m running Munin at /, not /munin/ on my web server.

    
          location ^~ /cgi-bin/munin-cgi-graph/ {
    fastcgi_split_path_info ^(/cgi-bin/munin-cgi-graph)(.*);
    fastcgi_param PATH_INFO $fastcgi_path_info;
    fastcgi_pass unix:/var/run/munin/fastcgi-graph.sock;
    include fastcgi_params;
}
 
location /munin/static/ {
    alias /etc/munin/static/;
}
 
location / {
    fastcgi_split_path_info ^(/)(.*);
    fastcgi_param PATH_INFO $fastcgi_path_info;
    fastcgi_pass unix:/var/run/munin/fastcgi-html.sock;
    include fastcgi_params;
}

          view raw
          gistfile1.nginxconf
          This Gist brought to you by GitHub.

That’s it! Happy monitoring!

Posted in Monitoring, Munin, Nginx, Ubuntu | 3 Comments

Updates to CodeBetter.Com’s Cloud Setup

Posted on February 22, 2011 by Brendan Tompkins

A few weeks ago I mentioned that we were now hosting CodeBetter.Com in the Rackspace cloud. Overall I was happy with the setup, but one thing kept bugging me. Apache was using a LOT of RAM, and I had to end up scaling the middle tier to two 500M cloud slices in order to keep the site running smoothly. I got one comment that made me cringe since I knew I was having to throw too much hardware at this problem:

Err.. why do you need this much hardware? Surely a blog with 32k subscribers doesn’t need 4 machines, or even decoupled web and database stacks!

I responded that these cloud slices are cheap (and they are @$15/month) and I DO stand behind the decision of having a tiered deployment should we have to scale, but at the same time he was right. A site as small as ours shouldn’t need that much of a footprint. Don’t get me wrong – it’s nice to know that we can scale out should we need to, but I was determined to downsize our current deployment.

I had heard that it was possible to run WordPress under NGINX (the highly performant web server alternative to Apache) but to do it right, you have to jump through some hoops to get it up and running, like building PHP and configuring it to run within the PHP FastCGI Process Manager. After a couple of days of tweaking, we’re happily up and running on ONE NGINX/PHP-FPM middle tier server with 256M ram (behind one NGINX reverse proxy and in front of our MySql server)! It’s probably still overkill, but I like it

I found a few resources which helped me get NGINX/PHP/Wordpress Multisite working, one WordPress support topic and this excellent series of posts by Pranav Rastogi with step by step instructions. If you’re going to do this I suggest checking these out. Also, should you be interested in the gory details of getting this setup, I’ve modified these steps and created a Fabric configuration script to automate this process, which you can find up at my github account.

-Brendan

Posted in Uncategorized | 1 Comment

Ducksboard API for .Net

A Log4Net Ducksboard Appender

Fun with the Nancy Self Host!

Setting up Munin with Nginx and Ubuntu 12.10 (Quantal Quetzal)

Updates to CodeBetter.Com’s Cloud Setup

Archives

Friends of CodeBetter.Com