Dolphin Deploy: Deploying ASP.net applications using IronRuby

Over the years I’ve found that build and deployment is one the most important, yet often overlooked aspects of software development. While NAnt\MSBuild has given the .Net platform automated builds, it’s XML based. People then took this approach and applied it to deploying ASP.net applications. 

I find this really interesting. If you speak to any developer about XML configuration they will run to the hills – yet it’s OK for build\deployment? Without a reliable and flexible way to deploy, the entire development line is blocked and the team will struggle to push new features out.

A while ago, I gave some support to Derick Bailey with albacore to help solve the “build” problem. I then wanted to look at the deployment problem. Having perviously manage near 200 line XML deployment files I could relate directly to the problem. I’ve found it near impossible to change settings or configure new servers when the script is that long and complex. There must be a better way. 

After playing around with some different approaches I’ve created a prototype called Dolphin Deploy (based on the Albacore name) available on github at http://www.github.com/benhall/DolphinDeploy. A simple, yet complete, example of how to deploy an MVC application is at http://github.com/BenHall/DolphinDeployExample.

Dolphin Deploy

Dolphin Deploy is a DSL for defining your deployment scripts.

The aim is to allow you to configure multiple different environments with different servers to deployment ASP.net applications. By supporting different servers in each environment, you can easily configure additional load balanced servers and have flexibility in terms of the file system structure which was one of the key issues we faced with XML.  

For example, below is a very simple script which will deploy an ASP.net MVC application, zipped in a file called MvcExample1.zip in the directory example.

environment do 
  desc “Example MVC Deployment”
  configured_as :mvc
  env :live do
    name “MvcExample1″
    host “MvcExample1.com”
    deploy File.join(Dir.pwd, “/example/MvcExample1.zip”)
    to “webserver0001″, “C:/inetpub/wwwroot/MvcExample1/”, “127.0.0.1″
  end
end

This will deploy the website into C:\inetpub\wwwroot\MvcExample1\, automatically tagging it with a release number to support rollback scenarios. The final directory would be C:\inetpub\wwwroot\MvcExample1\MvcExample1-01, incrementing the tag for each deployment. If you need to rollback, you simply point IIS back to the previous directory – at some point this will be automated. Within IIS, the site will be called MvcExample1 and have the host header MvcExample1.com.

In terms of support, Dolphin Deploy works with both IIS6 and IIS7. The script will detect the version of the server being deployed to and use the implementation appropriate. At the heart of this implementation is IronRuby.

IronRuby

The reason for using IronRuby is very simple. It wouldn’t have been possible to have the dynamic nature of Ruby, combined with interacting with IIS without it. IronRuby allows Dolphin to interact with IIS via existing C# libraries such as DirectoryServices and Microsoft.Web.Administration. By not re-inventing the wheel I was able to iterate more quickly on the important aspects, such as the DSL. While it still needs work, I think it has potential.

The DSL is executed at runtime by IronRuby. This allows Ruby methods, such as File.join and Dir.pwd, to be used directly in the configuration file allowing for more flexibility that allows for more complex examples as shown below. 

Below specifies two environments, local which has a single server while test with two servers which need deploying to. 

Both have a list of subdomains which need to be configured, instead of duplicating the list we can have it as a variable. Within the after block, we can loop around each of the items and call the method extra_header to assign the additional host header on a particular port (9999). 

In a similar way, we can call the method virtual_directory to create a virtual directory, or shell out to the cmd and write to Deployment.txt file.
headers = ['subdomain1', 'subdomain2', 'subdomain3']

environment do 
  desc “Example MVC Deployment”
  configured_as :mvc    

  env :local do
    name “MvcExample1″
    host “MvcExample1.local”
    deploy File.join(Dir.pwd, “/example/MvcExample1.zip”)
    to “localhost”, “C:/inetpub/wwwroot/MvcExample1/”, “127.0.0.1″  
    after do
      headers.each {|entry| extra_header(“9999:#{entry}.local.header”)}   
      virtual_directory “Test”, “Content”  
    end
  end

  env :test do
    name “MvcExample1″
    host “MvcExample1.test”
    deploy File.join(Dir.pwd, “/example/MvcExample1.zip”)
    to “testserver0001″, “C:/inetpub/wwwroot/MvcExample1/”, “127.0.0.1″
    to “testserver0002″, “D:/apps/MvcExample1/”, “127.0.0.1″ 
    after do
      headers.each {|entry| extra_header(“9999:#{entry}.test.header”)}   
      virtual_directory “Test”, “Content”  
      cmd “echo Hello World ” + Time.now.to_s + ” >> C:/temp/deployment.txt”
    end
  end
end

To execute and use the framework, you simple need to download the gem and reference a rake task.

The gem is available from http://rubygems.org/gems/dolphindeploy. Within your rake file, require ‘dolphindeploy’ and require ‘dolphindeploy_rake’, this will then give you the additional tasks.

The syntax, executed via IronRuby, is:

ir -S rake dolphin:deploy[env,server]

For example to deploy to the test environment for server testserver0001, you would need to execute:

ir -S rake dolphin:deploy[test,testserver0001] 

The only thing it expects is that the deploy.conf containing the DSL script above will be in the same directory as your rakefile.

The project includes other functionality, such as dynamically adding sections to the DSL to perform custom tasks, or deleting old builds. Until I document this functionality via blog posts, the best place to look would be the specs.

A minor comment, at the moment the script needs to be executed on the server itself (SSH \ Powershell for example). You also need to be administrator (and have evaluted permissions on Windows 7). After all, this is a prototype, so please don’t use it on production until it’s had a little bit more testing :) 

If you would like to help, please, please do! Grab me on Twitter while the code is available at http://github.com/BenHall/DolphinDeploy

 

Posted in Albacore, DolphinDeploy, ironruby | 5 Comments

Using Pik to switch between local and production Ruby versions

On my other blog, I recently covered how to install Ruby 1.9.2, Rails 3.0 and a framework called Pik. Pik is a Ruby Version Manager (RVM) for Windows, making a huge difference when you want to run multiple different versions of Ruby such as 1.8.6 and 1.9.2. However, I’ve recently found another use for the framework.

While the 7digital code-base (where I currently work) is based around C# and ASP.net, we are using Ruby to help us manage builds, deployments and more importantly our acceptance tests. One of the issues we faced with Ruby was making sure that our dev machines, build agents and servers were all running the same set of gems. Having to manually install\update gems on each environment became too time consuming, new starters spent too long configuring their environment and as a result making changes became hard and our Ruby progress slowed.

As it happens, the team have been distributing generic libraries, MSBuild files and configuration files for a while without any problems via TeamCity. Within subversion we have a directory to store various artifacts with TeamCity waiting for changes. When a change occurs, the directory is automatically checked out on the different environments all managed by TeamCity. It’s a great way to distribute files.

It turns out that this also solved our Ruby problem. Every time we want to update a gem we can upgrade our local machine and commit the changes, which will then automatically be replicated. The problem is, you can only have one version of Ruby in your path at any one point in time. This makes it much harder to hack play work with gems which I don’t want to be shared.

This is where Pik comes in.

While Pik manages different ruby versions, it works in exactly the same way when the Ruby version is the same but in different independent locations. This allows you to have a different set of gems depending on your requirements – such as local and production versions.

After installing, it will detect your default installation (our teamcity version lives in C:\TeamCityBuildTools). However, I can also add a second Ruby location even if it’s the same version:

>pik add C:\ruby\bin
This version appears to exist in another location.
Path:  C:/TeamCityBuildTools/ruby/1.8.6/bin
If you’d still like to add this version, you can.
Enter a unique name to modify the name of this version. (enter to quit)

The unique name allows it to be accessed later – I called mine 186-local. By default, pik will still point to my default original location.

>where ruby
C:\TeamCityBuildTools\ruby\1.8.6\bin\ruby.exe
c:\ruby\bin\ruby.exe

When I list all the installations, it knows about both the default TeamCity distributed runtime together with my own personal installation of 1.8.6.

>pik list
  186-local: ruby 1.8.6 (2007-09-24 patchlevel 111) [i386-mswin32]
  186: ruby 1.8.6 (2007-09-24 patchlevel 111) [i386-mswin32]

However, I can switch to my local installation with a single command.

>pik switch 186-local

This will change my path variable, together with the gems being used and other tools such as rake, cucumber and rails being executed.

>where ruby
C:\ruby\bin\ruby.exe

I can now prototype, play and hack without touching the shared distribution and potentially affecting others on the team. Pik also means I can install 1.9.2 and any other version of Ruby, but when it comes to working on the code-base and pairing with people, I can quickly switch to the standard set of gems and version of Ruby with a single command.

WARNING! HERE BE DRAGONS!

Having said that. If you “accidentally” have the wrong Ruby version set in your pik config, you will have to use the “SVN revert” feature before you completely mess everything up by mistake. Yes, I made this mistake and yes, I’ve used this feature. Luckily I realised, but it’s something to keep in mind.

Aside from that, I really like how Pik allows me to have multiple independent Ruby environments on my same machine. Combine this with the way we are distributing our generic libraries and Ruby environment, allowing for a nice balance of flexibility and reliability.

Posted in pik, ruby | Leave a comment

CodeBetter/Devlicio.us/LosTechies MSDN Ultimate Giveaways

Cross posted with http://devlicio.us/blogs/tuna_toksoz/archive/2010/07/27/codebetter-devlicio-us-msdn-ultimate-giveaways.aspx

UPDATE: Our friends from LosTechies also wanted to participate, and they donated a number of licenses (6 and counting). Thank you guys

In an internal discussion we had, an interesting idea popped up. MVP bloggers we had at CB/DL were generous enough to donate some of their MSDN subscriptions they were given as part of MVP program. Currently we have 7-10 licenses available, and this number may increase if we get outside donations. The subscriptions are for one year.

We decided to give them away to successful/promising OSS projects that are in need of licenses. The rules are simple.

  1. The project has to be opensource
  2. The project has to be in need of MSDN license. We’ll trust your word in this.
  3. The selection will be based on our discretion, we’ll do our best on being fair.
  4. We may change the number of licenses available during the selection progress.

You can send me an email (tehlike (hat) gmail (hot) com) with title starting with [CD/MSDN], or DM me on twitter (@tehlike) if you want to participate. You have to include

  1. Public URL for the project page
  2. Your name and your position in the project

Good luck!

Posted in codebetter, devlicio, msdn | 3 Comments

Improving testability with the Castle Dictionary Adapter

Frequently when reviewing code I see one of my pet hates appear and that’s a direct dependency on the ConfigurationManager.  The ConfigurationManager provides a way to access values in the Web\App.config. Yet, like any dependency, they generally bite you at some point – generally when you attempt to write the test.

Let’s imagine that our web.config has a value like below:

EnableNewsletterSignup = “false”

This value defines if we should hit the live web service. During development\systest we don’t want this to happen, however we do in UAT and Live. As a result, our code will generally look like this:

public bool Signup(string email)
{
    if (!Boolean.Parse(ConfigurationManager.AppSettings["EnableNewsletterSignup"]))
        return false;

    return true; // technically this would go off to an external web service
}

Simple, yet with multiple problems

Firstly, we have a magic string value which relates to the key in the config. If we wanted to change this value we would have to perform an error-prone Search\Replace. Secondly, we have to manually parse the string value to a boolean – again, this is error prone as we’ll need to protect against bad data. This additional logic hides the true intent of what the method is meant to be doing which increases complexity. To make matters worse, we have a major problem when it comes to testability.

The configuration manager will automatically detect the config file based on the executing assembly, this means that your test assembly’s App.config needs to match your implementation’s (web.)config with all the values pre-configured for testing purposes. Having pre-configured values offers you very limited flexibility and in the example above we would be unable to test both paths (without our tests changing the value directly)? If we had multiple possible paths, this would cause us a very real problem.

This week I came across the issue were I required an AppSetting value. Not wanting to face the issues above I looked for help.

Thankfully, help’s available

The Castle Dictionary Adapter removes these problem for us. Given an interface and a dictionary of values, the adapter will create an object with all the properties populated for us. Our interface will match the settings in our config file.

public interface IApplicationConfiguration
{
    bool EnableNewsletterSignup { get; set; }
}

The same implementation mentioned before becomes this, with a dependency on the above interface instead of the concrete ConfigurationManager. Notice our ‘if’ statement now uses a strongly typed property without all the noise associated.

class NewsletterSignupService
{
    private readonly IApplicationConfiguration _configuration;

    public NewsletterSignupService(IApplicationConfiguration configuration)
    {
        _configuration = configuration;
    }

    public bool Signup(string email)
    {
        if (!_configuration.EnableNewsletterSignup)
            return false;

        return true; // technically this would go off to an external web service
    }
}

Testing!

The real advantage arrives when you look at the problem from the testing point of view. Because it’s an interface, we can use Rhino.Mocks to produce a stub, allowing us to test using any possible value.

var stubConfig = MockRepository.GenerateStub<IApplicationConfiguration>();
stubConfig.EnableNewsletterSignup = true;

We also no-longer need to maintain the App.Config as everything is driven by stub config objects, making life easier all round.

The next level comes when you use it with an IoC framework such as Castle Windsor. When an object defines a dependency on IApplicationConfiguration, they will be provided with an object created via the DictionaryAdapterFactory with the values coming from our actual AppSettings.

WindsorContainer container = new WindsorContainer();
container.AddFacility<Castle.Facilities.FactorySupport.FactorySupportFacility>();

container.Register(
    Component.For<IApplicationConfiguration>().UsingFactoryMethod(
        () => new DictionaryAdapterFactory()
             .GetAdapter<IApplicationConfiguration>(ConfigurationManager.AppSettings)));

As a result of implementing the adapter together with it’s use in Windsor we have more control, less complexity and a more maintainable solution going forward.

But it’s not only for AppSettings, the Castle Dictionary Adapter works on a number of different directories and collections meaning you no longer need to index into them using strings. If you want to know more, then CastleCasts has a great screencast on this at http://castlecasts.com/Episodes/3/Show/dictionary-adapter

In order to implement this in your own codebase, Castle Dictionary Adapter is currently a separate single assembly with no external dependencies that you can download from http://www.castleproject.org/castle/download.html

Going forward, it will be part of Castle Windsor 2.5 with some interesting improvement as discussed at http://devlicio.us/blogs/krzysztof_kozmic/archive/2010/07/05/castle-windsor-2-5-the-final-countdown-beta-1-released-and-core-dynamicproxy-dicitionary-adapter.aspx

 

The code for the above example is available at http://gist.github.com/486603

 

Posted in Castle, TDD, Testing | 3 Comments

Testing a WPF UI using Ruby, Cucumber and WiPFlash.dll

Almost two years ago I blogged about Project
White,
an open source project available on CodePlex which provides
an abstraction on top of the UI Automation library built into WPF and
Silverlight.

Fast forwarding and there is a new project available called WiPFlash which allows you
to drive WPF UI’s for testing purposes created by Liz Keogh who is an
agile\lean coach here in the UK. When I saw this framework, one of the first
advantages such stood out was how clean the API is to work with but also how
lightweight and focused the overal framework was.

This got me thinking… I’ve been presenting on how you can
test ASP.net web applications using Ruby, most recently at QCon London which
was an amazing experience. This got me thinking, if Ruby is a great language
for testing web applications then surely the same applies to desktop
applications. Thankfully, by using WiPFlash together with IronRuby you can have
the same advantage!

One of the questions I’m commonly asked is why would you use
Ruby when testing C# applications? For me, one of the major advantages of Ruby
is it’s readable natural language – this is perfect for tests. As Ruby is a
dynamic language you can also adapt your tests and frameworks to fit your test
domain more effectively. As a result, your tests become more readable and more
maintainable which means they stand much better chance of supporting the
development instead of hindering it. Given the amount of Ruby you need to learn
to get started, there is very little overhead for a project as to get started
you use a very focused subject and grow\learn, as such team members should be
able to pick-up everything required in a hour or so. When combined with
frameworks such as Cucumber you can create really powerful acceptance tests which
can cover large parts of your application with very little implementation.

WiPFlash – Start application and find control

After downloading IronRuby
(currently RC3) and WiPFlash,
you can use the REPL (ir.exe) to interact with your WPF application. For
example:

    require ‘lib\WiPFlash.dll’
    include WiPFlash
    include WiPFlash::Components
    a = ApplicationLauncher.new 
    app = a.launch ‘D:/SourceControl/wipflash/ExampleUIs/bin/Debug/ExampleUIs.exe’

The first line adds a reference to the assembly. We can now
start interacting with the objects. The second\third line includes the namespaces
so we can access the object without including the full namespace. Notice how we
use the Ruby syntax of :: for navigating the namespace – IronRuby will take
care of the translation for us.

The fourth line is where the action starts, we first create
the ApplicationLauncher object, this is part of WiPFlash.dll and is used to
launch the application which is what we do on the fifth line. We now have a
running WPF application along with a reference to the app in the variable. The
application is this example looks like this:

The next stage is to gain access the controls and start
interacting with them as a user might. First, obtain the window where the
controls are. This is very similar to how you would using C#.

    window = app.find_window ‘petShopWindow’

The next stage is to obtain a reference to the actual
control.  This is where C# and IronRuby
go in a slight different direction.  The
C# API uses generics, with the syntax such as:

    TextBox name = window.Find<TextBox>(“petNameInput”)

With IronRuby, we have a slightly different syntax, the most
important part is the of() which defines the generic type:

    name =
window.method(:find).of(TextBox).call(“petNameInput”)

petNameInput refers to the WPF control name, which if you are
unsure of, you can use applications like Snoop
to find the name.

Once we have the control, we can set\get the text property.

    name.Text = “Bob”

Fundamentally, we can now use the API in exactly the same way
we could via C# – but why is that interesting?

Extending WiPFlash via IronRuby

But before we get into the interesting Cucumber + WiPFlash,
we have a problem with the API and the generic syntax. Personally, I think it
is too wordy with the impact facts being lost. Thankfully Ruby is a dynamic
language, and we can extend the framework to meet our requirements resulting in
a syntax of:

    text = window.find_TextBox “petNameInput”

Amazingly, this is easy to achieve thanks to the special
method_missing hook in Ruby. This method is called when a message is sent to an
object which doesn’t match a pre-defined method. This means you can to take
action and is how ActiveRecord allows for a flexible query syntax. 

The code below makes the syntax above possible. GetType takes
in the method name we called, and returns the part relating to a WiP component.
This is converted to a class name within get_class before being used within
find_ui_element together with the args originally passed to the method. Simple,
but effective.

  class WiPFlash::Components::Container
    def get_type(name)
        name.to_s.split(‘_’)[1]
    end

    def get_class(name)
       Kernel.const_get(name)
    end

    def
find_ui_element(class_name, *args)
       self.method(:find).of(class_name).call(args[0][0].to_sym)
    end

    def method_missing(m,
*args, &block)
       ui_type =
get_type m
       ui_class =
get_class ui_type
       find_ui_element(ui_class,
args)
    end
end

With our nice clean API allowing us to obtain controls via
window.find_TextBox(“petNameInput”) we can start testing.

WiPFlash + Cucumber

With this in place, we can now plug everything together and
use Cucumber together with IronRuby to test WPF applications. The complete
example can be found at http://github.com/BenHall/wipflash_cucumber_driven_example

To install Cucumber, run the command (ir being the IronRuby
interpreter):

   ir -S gem install cucumber

For example, let’s say we wanted to test the following:

   Scenario: Saving a product by name only
      Given the application has started
      And I enter the name “Product 1″
      When I save the product
      Then “Product 1″ should be available to purchase 

Above is a cucumber scenario. Using the Given, When, Then
(GWT) syntax we define the ‘given’ steps to setup our test context and data,
‘when’ to perform a particular action and ‘then’ to verify it worked
successfully. The most important aspect is the language used. When defining
your scenario, you should use business oriented language to describe the
behaviour of the application instead of 
the actual implementation or UI. There are two (main) reasons for this,
one is that you can use the scenarios as a communication base with customers
and the business to form acceptance criteria about how the application should
work. The second reason is that if your application’s implementation changes
you only need to change the implementation of the test instead of the wording
and aim. This will save you a huge amount of effort during the project. 

The implementation of the test is as follows. Each cucumber
step relates to a ruby code block:

  Given /^the application has started$/ do
    host = ApplicationLauncher.new
    @app = host.launch Dir.pwd + ‘/src/ExampleUIs/bin/Debug/ExampleUIs.exe’
    @main_window = @app.find_window ‘petShopWindow’
  end
  
  Given /^I enter the name “([^\"]*)”$/ do |product_name|
    textbox = @main_window.find_TextBox “petNameInput”
    textbox.text = product_name
  end
  
  When /^I save the product$/ do
    button = @main_window.find_Button “petSaveButton”
    button.click
    sleep(1)
  end
  
  Then /^”([^\"]*)” should be available to purchase$/ do |product_name|
    combo = @main_window.find_ComboBox “basketInput”
    combo.Items.should include(“Pet[#{product_name}]“)
  end
  
  After do |scenario|
    @app.process.kill
  end  

The main different to what we discussed previous is the
‘After’ step which is a hook within Cucumber called after the scenario has
finished. This means we can kill the application launched in the first step and
ensure we have a clean system with no left over processes.

To execute the tests you would run the command:

   ir -S cucumber features

We can now test our WPF applications using Cucumber, taking
advantage of the flexible nature and 
re-useable steps together with an appropriate language for driving
acceptance tests.

 

Posted in cucumber, ironruby, ruby, wipflash, wpf | 4 Comments