CodeBetter.Com
CodeBetter.Com
RSS 2.0 via Feedburner
           Do you Twitter? Follow us @CodeBetter

Darrell Norton's Blog [MVP]

Fill in description here...

How do you setup NUnit tests in Visual Studio.NET projects?

I received this in an email a while ago, but have been too busy to blog about it until now.  The email:

I've recently started TDD, mostly after reading your blog. I downloaded
NUnit and have begun running a few simple tests. My question is, do you
normally include test classes and more importantly, the NUnit DLL in
your projects? I didn't think it was a good idea, not that I have any
argument for why, so I would write my class and then create a whole new
project for the test files and finally run my tests from new DLL that
VS.NET would create. Should I not be concerned about inlcuding test code
or the the reference to the NUnit DLL in my projects?

As far as I can tell, there are 4 ways to setup NUnit tests in Visual Studio .NET.  And they are:

  1. In the same class – the advantages to this setup are you have full access to everything, including private methods and variables.  However, since you have to make the test methods public, it could be a security issue.  I don’t generally do this except for learning.

  2. Separate class, same project – here you make a new class within the same project (or folder).  Thus you can access all internal methods without worry.  There could still be security issues, but it is somewhat less dangerous this way.  I sometimes use this for smaller projects if requirements allow.

  3. All tests in 1 separate project – here all of an application’s unit tests are in a separate project.  This has the advantage of keeping all unit tests in one place, so it’s easy to fire up NUnit and run all the application’s tests.  But I tend to need a lot of directories to organize my tests, and for larger applications the unit test project becomes huge.  I use this for all but the smallest and largest projects.

  4. All tests for a project go in a separate project – here you have one test project per real project.  For example, the BusinessServices project has a BusinessServicesTest project.  This keeps the unit test projects manageable, but it takes more effort to run all the application’s unit tests at once.  I generally use this for large applications.

For options 1 and 2, perhaps not deploying the NUnit dll is good for security reasons, but anyone can still call those public test methods.  I’m not sure of the security implications, since I generally don’t do either of these or if I do, it’s a small local app.  For options 3 and 4, it doesn’t really matter if you include the NUnit dll in the project since you won’t deploy the unit test project(s).

Which of these methods do you use and why?  Are there other methods out there you are using?  Am I smoking crack?  Leave a comment letting me know!



Comments

Darrell said:

Brian - I hadn't thought of a pre-processor symbol. That's an *excellent* idea. That would certainly ease security concerns, and would more tightly couple the tests to the production code.
# February 16, 2004 3:00 PM

Roy Osherove said:

Usually go the same routs you do, although #4 has been a bit of a problem for me.
Nunit GUI seems to have a big problem with multiple assemblies at the same time. Especially if your tests are reading from a config file for various stuff.
# February 16, 2004 5:50 PM

James Avery said:

I tried #4 once but it got to be too much of a mess, so I switched to #3. I have a project that includes tests for the entire app, and folders in that project for each separate project. (Or just class files if the project is small enough)

The preprocessor option is interesting, I might give that a shot also.

-James
# February 16, 2004 11:48 PM

Udi Dahan - The Software Simplist said:

#4 for me, all the way. This may be because I work on larger systems. A project/assembly is what others will be using, and so it is imperative to test it from the outside, as the client code would. This is important from a TDD perspective, less so from a pure testing perspective.
# February 16, 2004 11:51 PM

Steve said:

I usually go with #3, however, if the project is large enough #4 (very rare though).
# February 17, 2004 12:53 AM

Darrell said:

Roy - the way I get around that is to use XML files versus config files, since you can only have one config file per running application, but no limit on the number of XML files.

I ran into this same problem when I tried to add a config file to an NUnitAsp unit test assembly, but the NUnitAsp runs with the web.config file of the ASP.NET app! So I just created a simple XML file and a per-Assembly config file sample available here (http://dotnetjunkies.com/WebLog/darrell.norton/archive/2004/01/15/5646.aspx) and was good to go.
# February 17, 2004 1:09 AM

Darrell said:

James - maybe #4 is the scenario NUnit was trying to address with the new "Project" feature. I don't know, but it could help. Unfortunately I have no reference app handy to try it out!
# February 17, 2004 1:10 AM

Darrell said:

Udi - another advantage to #4 is the ability to deploy the unit test assembly to wherever the production code is to see if it is running its unit tests properly. I've done that on occasion, and it has helped me figure out some basic configuration issues. Of course, you have to make sure that your unit test code does not affect the existing data in any way! :)
# February 17, 2004 1:14 AM

Steve Maine said:

My preference is for #2 + conditional compilation. Unit tests live in the same project, so they can poke at internal members but are compiled out of production code.

Unfortunately, I end up having to use #4 more often than not. A good reflection library is really handy here (lets you keep internal things internal, but still allows you to test what you need to test).

# February 17, 2004 4:41 AM

Darrell said:

Steve - I have done some internal testing via reflection when using option #3.

What factors end up forcing you to use #4?
# February 17, 2004 4:44 AM

Grant said:

Usually #4 for me; it helps me nail down the interfaces of my program. From time to time, I might use the other options depending on the specific project.
# February 17, 2004 10:40 AM

Darrell said:

Grant - interesting that an option can affect the way you code. I haven't noticed it, but that is not saying it doesn't occur.
# February 17, 2004 1:01 PM

Peter Provost said:

#4 all the way. That way the project and its test harness can be thought of as a single unit. Makes it much easier to reuse things.

My 2¢
# February 19, 2004 7:26 AM

Darrell said:

Peter - thanks for the input, especially coming from Mr. TDD with .NET! :) I hadn't thought of reusability of tests, but I guess they are code just like everything else...
# February 20, 2004 2:58 AM

Anil Kumar Gupta said:

Hi
Can anyone help me to tell 'How to set up Nunit in visual' and 'How to test a sample application'
I am a newbee to this plz explain as steps.
I would be extremely oblized for your help.
Regards...
Anil Gupta
secureguru@hotmail.com
# February 20, 2004 8:47 PM

Darrell said:

Anil - the best tutorial out there is Peter Provost's tutorial available at (http://www.peterprovost.org/wiki/ow.asp?Test%2DDriven%5FDevelopment%5FIn%5F%2ENET).

Then check out Roy Osherove's tutorial here (http://msdnaa.net/Resources/display.aspx?ResID=2365).
# February 21, 2004 6:11 AM

Juanma Cervera said:

Is there a way to setup a separated test-project to test a single assembly exe project.
My problem is that I can´t set the reference in the test project to the tested project, because I only reference dll assemblies.
And I don´t understand why.
Thanks
Juan M. Cervera
jmcervera@gtfadrell.com
# March 3, 2004 7:59 PM

Darrell Norton's Blog said:

Unit Testing a Windows Forms or Console Application
# March 4, 2004 6:17 AM

BJ Allmon said:

I would check out the possibility of leveraging NAnt. I think it's still premature but you should be able to break up the project and use NAnt to automatically run the tests. Atleast you can do this in J2EE. Not sure Ant for the .NET framework... But you can bet that NAnt will continue to grow up. Besides, it's a way to easily manage and deploy a project without VS Studio.
Hope this helps.
# March 25, 2004 3:40 AM

Darrell said:

BJ - yes, NAnt is an excellent tool. But it doesn't answer where to allocate testing code.
# March 25, 2004 3:47 AM

beaudetious said:

I use conditional compilation with my unit test class set as an inner class inside of the class I'm testing. This allows me full access to every routine -- private or public. However, my concern is this....I'm leaning towards Nant and automatic builds and I don't know how my Nunit setup will affect this. I'd like to use Nant to make a build then test it with Nunit but will I have to make the build in my special UnitTest configuration (where the preprocessor UNITTEST is defined) everytime in order to run the tests? Should I build the UnitTest configuration, run the unit tests, then build the Release or Debug configuration if all of the unit tests pass? I'm really grateful that you posted this topic and that so many people have responded. Keep it up.
# March 29, 2004 6:28 AM

Darrell said:

beaudetious - I would have a task that builds in the UnitTest configuration, and if the unit tests pass would then create the debug and release builds.
# March 29, 2004 7:32 AM

Lyle said:

First my vote. Always #4. This way I can just test seperate projects and get some code reuse. Again I have only done this twice in large projects.

I have also used NAnt, though a bit of a pain I am getting better with it. One advantage of it that I like is the ability to run by tests after the build. I can run my testing setup through NAnt also, like creating files and running sql to setup and clean up the testing database. With NAnt I can also have it run different type of builds, like a testing build that goes through all the steps to build, setup and run my tests, as well as deployment deployment builds without unit test. Of course I have only tried this on two projects and the setup has been painful.

On a side note I have had some success in using xslt transforms to convert solution and project files into build files through a master NAnt build process, that are excuted later from that same master build process. This way I don't have to update my build files by hand as much, though as the solution gets more complex new issues with my transform always pops up and I have to go back and modify the files by hand. xslt is not my favorite tool in the world, but it does have some uses.

Darrell, Peter and all thanks for all the great stuff on your blogs. I have enjoyed and used much of it in my TDD and .NET travels, I hope to someday contribute more myself.
# April 27, 2004 5:44 AM

Lyle said:

NAnt and unit tests

First my vote. Always #4. This way I can just test seperate projects
and get some code reuse. Again I have only done this twice in large
projects.

I have also used NAnt, though a bit of a pain I am getting better with
it. One advantage of it that I like is the ability to run by tests
after the build. I can run my testing setup through NAnt also, like
creating files and running sql to setup and clean up the testing database.
With NAnt I can also have it run different type of builds, like a testing
build that goes through all the steps to build, setup and run my tests,
as well as deployment deployment builds without unit test. Of course I
have only tried this on two projects and the setup has been painful.

On a side note I have had some success in using xslt transforms to
convert solution and project files into build files through a master NAnt
build process, that are excuted later from that same master build
process. This way I don't have to update my build files by hand as much,
though as the solution gets more complex new issues with my transform
always pops up and I have to go back and modify the files by hand. xslt is
not my favorite tool in the world, but it does have some uses.

Darrell, Peter and all thanks for all the great stuff on your blogs. I
have enjoyed and used much of it in my TDD and .NET travels, I hope to
someday contribute more myself.
# April 30, 2004 7:01 AM

Bruce Crawford said:

How do folks implement options 3 and 4 when the application is an exe given that:
VS .NET IDE does not support referencing EXE. See:
Q318165 PRB: Can't Add Reference to Project in Project Dialog in VS .NET
http://support.microsoft.com/support/kb/articles/q318/1/65.asp

Thanks for the blog.
# September 18, 2004 12:00 PM

Darrell said:

Bruce - unfortunately you can't do it within Visual Studio, but you can from the command line. Kind of a pain, I know. What I end up doing is have the executable assembly just load forms from another project. We ended up having to do it that way anyway to support our updateable Winforms application. Hope that helps!
# September 19, 2004 1:55 PM

Martí Bayo Alemany said:

I tried renaming the exe file to dll and it seems it works. I can see the code also on the object browser. With a postbuild step on your project you can copy your exe file to dll and link them to the Testing project. I cannot think in any reason why Visual Studio doesn´t allow to include exe files. Any idea ?
# November 26, 2004 3:51 AM

Darrell said:

I don't know why the Visual Studio team would do that. I'm sure they have some reason.
# November 26, 2004 5:37 AM

Martí Bayo Alemany said:

The renaming had despite of all not worked, so there is some reason not to rename ;-)
# February 16, 2005 6:00 PM

Sarika said:

The renaming does work. I have tried it myself. I have a GUI VB.NET application and was faced w/ the problem of NUnit not being able to test EXE projects.

Things to remember while doing the rename:
Do the rename from the command prompt to ensure you rename the file to .dll.

Everytime you compile your project to be tested, you need to rename your exe to dll. Else you end up using the old dll (which corresponds to the older compiled version of your program)

Hope this helps.
# March 7, 2005 9:43 AM

Darrell said:

NUnit *can* test EXE files. Visual Studio cannot reference a project that produces an EXE instead of a DLL. It's a Visual Studio limitation, not an NUnit limitation. Of course if you are doing Test Driven Development using TestDriven.NET from inside Visual Studio, it's a very real limitation.

My suggestion would be to move all logic from the EXE to a DLL. That way you can test all the logic, and the EXE will make a FEW calls (very few) to the DLL. This is what everyone talks about when they say "The untested UI code should only be a very thin layer over your well-tested code."
# March 9, 2005 5:49 AM

Tzvir said:

Inspired by a lecture Ayende gave in a Israeli vb.net user group and by Working Effectively with Legacy

# July 23, 2007 6:45 PM
Check out Devlicio.us!