The Agile development community has struggled for years with an array of solutions for automated testing solutions for web development. NUnitASP is a good way to unit test server side ASP.Net code, especially now that it doesn't require XHTML compliant pages, but it can't handle client side scripting and AJAX is exploding in popularity. Several tools have used COM (must die) to drive Internet Explorer (IE) with varying degrees of success. My personal experience is that the IE COM API is too byzantine and flat out flaky. Besides the flakiness, Firefox and other browsers are gaining in popularity so the IE only testing might not cut it anymore.
Enter the Selenium project. The developers of Selenium had the brilliant, but in retrospect painfully obvious, idea to use Javascript inside a browser to drive the web testing. Presto, automated testing for web applications that can test client side Javascript and multiple browser engines. In its original incarnation Selenium ran FIT style test tables inside a web browser. Recently, Selenium Remote Control has been released to that allow the core Javascript engine to be driven through API's for .Net, Java, Ruby, or Python. The FIT style table runner is perfectly functional, but for us it's very convenient to use the Selenium RC .Net wrapper. So far I'm pleasantly surprised by how easy it is to use the .Net wrapper.
Sample Test
The API libraries communicate with the Selenium Server, a Java executable that can start and stop any supported browser and send testing commands to the browser. The .Net callable wrapper works by sending HTTP messages to the Selenium Server that controls a browser. The first step is to start up the Selenium Server from a command prompt.
java -jar server\selenium-server.jar -interactive
The next step was to create a simple HTML page with a textbox that changes values when a button is clicked:
<html>
<head>
<script language=javascript src=prototype-1.4.0.js></script>
<title>Selenium Target Page</title>
<script id=clientEventHandlersJS language=javascript>
<!--
function button1_onclick() { $('text1').value = "Goodbye"; // Using the Prototype library}
//-->
</script>
</head>
<body>
<form name="form1">
<input id="button1" type=button onclick="return button1_onclick()"
value="click me"/>
<input id="text1" type=text value="Hello"/>
<select testid="select1" >
<option value="1">North</option>
<option value="2" selected=true>West</option>
<option value="3">South</option>
<option value="4">East</option>
</select>
</form>
</body>
</html>
Next I wrote a little NUnit test fixture class to run tests against the web page. The key object is the DefaultSelenium class that is created in the SetUp() method:
/// <param name="serverHost">the host name on which the
/// Selenium Server resides</param>
/// <param name="serverPort">the port on which the
/// Selenium Server is listening</param>
/// <param name="browserString">the command string used
/// to launch the browser, e.g. "*firefox", "*iexplore"
/// or "c:\\program files\\internet explorer\\iexplore.exe"</param>
/// <param name="browserURL">the starting URL including
/// just a domain name. We'll start the browser pointing at
/// the Selenium resources on this URL,
/// e.g. "http://www.google.com" would send the browser to
/// "http://www.google.com/selenium-server/SeleneseRunner.html"</param>
public DefaultSelenium(String serverHost, int serverPort,
String browserString, String browserURL)
{ this.commandProcessor = new HttpCommandProcessor(serverHost,
serverPort, browserString, browserURL);
}
using System;
using NUnit.Framework;
using Selenium;
namespace SeleniumTarget
{ [TestFixture]
public class WebPageTester
{ DefaultSelenium selenium;
[SetUp]
public void SetUp()
{ // 4444 is the default port for the Selenium Server
selenium = new DefaultSelenium("localhost", 4444, "*iexplore", "http://localhost"); selenium.Start();
}
[TearDown]
public void TearDown()
{ // Make sure the Selenium environment is cleaned up after each test
selenium.Stop();
}
[Test]
public void CheckTheTitle()
{ selenium.Open("http://localhost/SeleniumTarget/TestPage1.htm");
Assert.AreEqual("Selenium Target Page", selenium.GetTitle(),
"Check the title of the browser");
}
[Test]
public void ClickButton1ChangesText1FromHelloToGoodbye()
{ selenium.Open("http://localhost/SeleniumTarget/TestPage1.htm"); Assert.AreEqual("Hello", selenium.GetValue("text1"), "Initial Value");
selenium.Click("button1");
Assert.AreEqual("Goodbye", selenium.GetValue("text1"), "Value after clicking button1");
}
/// <summary>
/// Check that the options of a <select></select> element are
/// as expected. Finds the <select> element by using an xpath expression
/// </summary>
[Test]
public void Select1Values()
{ selenium.Open("http://localhost/SeleniumTarget/TestPage1.htm"); string locator = "xpath=//select[@testid='select1']";
string[] options = selenium.GetSelectOptions(locator);
Assert.AreEqual(new string[] {"North", "West", "South", "East"}, options,
"Values in the select1 dropdown");
}
}
}
It's a trivial example, but it's a start.
What I don't know -
- What's the best way to handle the Selenium Server process? How do you guarantee it's up when you start your tests? I'm thinking some kind of Windows service wrapper
- How do you integrate Selenium with CruiseControl.Net to get it into your Continuous Integration strategy? I can't find much on the web about this yet. You can run Selenium from NUnit for developer testing, but that isn't a very desirable answer from a tester's perspective. For a couple of reasons, our thinking is to wrap the Selenium manipulation inside a FitNesse DoFixture. We already know how to integrate FitNesse tests within a CC.Net build and it's convenient to run all the tests together. We're also hoping that the DoFixture tests will be easier to understand and that we can hide some of the web page details behind the fixture.
Other Tools
We're looking primarily at Selenium, but we're also considering WATIR (Ruby based tool) and Sahi (I don't know much about it, but it looks strong). One of our colleagues is experimenting with a Ruby based DSL for testing another web product using WATIR as the core that looks promising. I've also been playing with the Ruby driver for Selenium with an eye towards creating a testing DSL for our application.