in Search

Do you Twitter? Follow us @CodeBetter

Brendan Tompkins [MVP]

Blog First. Ask Questions Later.

Automatic NUnit Unit Test Generation?

The other day, I emailed Darrell about SkeletoNUnit.

>Know anything about this?
>
>http://sourceforge.net/projects/skeletonunit/
>
>Looks interesting, I remember you were talking about NUnit addins, and
>got me thinking of an automatic unit-test generator. Should be fairly
>easy to make one, so I thought for sure there's one out there.
>
>I know it's backwards to generate unit tests, but I can see the
>advantages, especially with legacy code.

Anyhow, he didn't know of anything else off-hand, and SkeletoNUnit seems, like it's name suggests, to be a dead project... So, I'm asking if anyone knows of anything else out there that will do this? Basically, I'm looking for something that will automate the generation of NUnit unit tests.

I've tried .TEST from Parasoft, (which does generate NUnit tests) but I couldn't get it to work fully. It would choke on big assemblies, and I couldn't figure a way to generate tests for one class at a time. I also can't justify the price tag, when VS2005 may do much of this stuff and make .TEST obsolete..

-Brendan

Published Sep 17 2004, 10:48 AM by Brendan Tompkins

Comments

Michael Russell said:

Doesn't automatic generation of test cases defeat the purpose of testing?

The best part about TDD is that you really have to think out how to test your components. You need to think of valid inputs, invalid inputs, etc. and test for them.

What's even more important is creating a unit test to test your bug fixes. Take the case which triggered the bug and create a test case for it.

Parasoft's unit testing software is good at coming up with generic test cases, but is abhorrent at coming up with intelligent test cases that are truly useful.

# September 17, 2004 7:09 AM

Rob Gibbens said:

Well, in the simple cases, it could be very useful. Say you have an object with some properties on it. Say one of these properties (FirstName) is a string. An Automatch Test Generator could see that property, see that it's a string, and pop up a box asking 1. Maximum length for this field 2. Is it allowed to contain non-alpha characters, etc. Common things for strings. Another property may be a collection. It should be able to automatically test adding a new object and testing that the object is in the collection. It's obviously not going to be able to generate every test case, but it should be able to generate the simple, common cases, leaving you more time to think about the not-so common cases.

# September 17, 2004 7:13 AM

Brendan Tompkins said:

I agree with both of these comments so far. New development - write unit tests first. But for maintaining/debugging old code, or to write a test for that code you had to churn out (or for the time you cheated on your test-first diet) it'd be really nice to have some tests generated for you.

# September 17, 2004 7:18 AM

Rev said:

Yes, I'm about to do some major refactoring and wholesale rewriting of a project I did before I found the joys of NUnit. I'd rather generate NUnit test that represent the state my project is in now so I can use it to make my changes.

I'll have to flag this conversation in case anyone comes up with a solution.

Until then... I've got a lot of tests to write.

Rev

# September 17, 2004 7:57 AM

Darrell said:

An earlier version of the NUnitAddin to Visual Studio could do this for a class, but not, as far as I know, with a whole assembly. I'm sure it could be done via Reflection.

# September 17, 2004 8:49 AM

Darrell said:

Found something:

http://blog.dotnetwiki.org/archive/2004/06/10/390.aspx

# September 17, 2004 8:52 AM

Brendan Tompkins said:

Darrell,

I had found that too, but I can't make any sense of the posts. Can you? Is there a Reflector plugin?

# September 17, 2004 9:20 AM

Brendan Tompkins said:

Okay.. I got something to work:

First Download Reflector:

Then Download "Reflector.Graph.Addins" :

http://www.dotnetwiki.org/Default.aspx?tabid=52

Run Reflector, select "AddIns..." and add the dlls from the Reflector.Graph.Addins.

Then Load Your dll in Reflector, right click on the class, choose "Code Generators", then Select "Unit Tests" from the ddl, set your namespace in the properties pane, right click below the properties pane, then click "Generate" cross your fingers, sacrafice a small goat, and hopefully you'll get a test case out of it.

I'm not complaining, but there has to be a better way.

-Brendan

# September 17, 2004 9:45 AM

Brendan Tompkins said:

Oops, reflector is here: http://www.aisto.com/roeder/dotnet/

# September 17, 2004 9:46 AM

Rev said:

I'm getting an exception:

System.ArgumentOutOfRangeException

when I try and generate the test. Where is the output? Or would that pop up if I didn't get the exception?

Also... only 4 of the .dlls in the Add-in loaded for me... the rest failed with a message that they did not contain packages.

Rev

# September 17, 2004 10:24 AM

Brendan Tompkins said:

Hi Rev.. Yes. I got this message on some classes too. I'm not sure what the breaking point is or why that's happening.

Me too about the DLLs as well. The others I don't think apply, so I'm pretty sure its okay..

There's an output directory field in the parameters section. You can pick a directory with it, and that's where the classes go when and if they work.

I'm just playing around with it too... Let me know if you get any further.

We should probably post this stuff on Peli's Blog

# September 17, 2004 2:26 PM

Josh Pollard said:

It looks like Peli took it down:

"Reflector.Graph.Addin temporarly removed.."

# September 20, 2004 5:30 AM

Rev said:

We must've been hitting it too hard, mini-/. effect.

I got some of my smaller classes done, but the big guys bombed out. I decided to skip the autogen and do it all by hand. I guess that's the way it was meant to be done in the first place.

I picked up 'TDD in Microsoft .NET' by Newkirk and Vorontsov, as a primer for NUnit. I kind of found that it was more like a quick start pamphlet you often times get in your packaged software.

Has anyone looked at "Extreme Programming Adventures in C#" http://www.amazon.com/exec/obidos/tg/detail/-/0735619492/qid=1095715609/sr=8-1/ref=sr_8_xs_ap_i1_xgl14/104-2632373-7360715?v=glance&s=books&n=507846

Is it worth grabbing?

Rev

# September 20, 2004 10:27 AM

Jonathan de Halleux said:

Hi all,

I'm the author of the Reflector addin that "should" output "empty" unit test cases out of .NET assembly.

First thing I have to say is that: it is EXPERIMENTAL. Second, there are a lot of traps that this tool could fall in:

- The statement graph that I build is not perfect and still suffers some bug
- it does not generate "minimal" path test cases because I'm using a basic depth-first-search approach
- Reflector decompilator is not perfect and sometimes fails

Nobody should hope for miracles with automatic test generation, because it tries to solve an undecidable problem (Hamming theorem).

At last, you should definitely forward this kind of feedback to me.

Cheers,
Jonathan

# October 3, 2004 10:30 PM

Brendan Tompkins said:

Jonathan... Thanks for the post. Good job getting your PhD! Maybe this addin tries to do too much? Perhaps something more useful would be just generating tests that are empty, with System.NotImplementedExceptions being raised.

-Brendan

# October 4, 2004 7:24 AM

Jonathan de Halleux said:

There is an option in the property grid to disable path generation and simply generate 1 test per method.

# October 20, 2004 3:01 AM

edwinyeah said:

Hi all,

I noticed SkeletonNUnit before, but unfortunately it seems to be dead? and I searched the Internet but I found nothing. so I developed one by myself, check

http://www.freewebs.com/edwinyeah/

Features
1. Supports popular languages including C#, VB.Net, JScript, Managed C++ and J#.
2. Options for generate test cases for a whole module, a whole class or a single method.
3. Options for output to files or on-screen.
4. All parameters of methods being tested are assigned with default values; also members of non-primitive type of parameters are assigned with values.
5. Options for generate a "for loop".

# July 21, 2005 2:15 AM

Terry Go said:

Hi,
I also has been thinking of this NUnit test generation tool for a long time and VS2005 team edition shipped this functionality. However, as always, it supports only their own testing framework, which is not what I want, since I invested a lot on NUnit. So, I finally created NUnitGenAddin, VS2005 add-in that you can generate NUnit tests as you code. Check that out here, http://forge.novell.com/modules/xfmod/project/?nunitgenaddin.

# January 19, 2006 9:45 AM

Mario Muja said:

Hi Terry,
you write that NUnitGenAddIn is for VS2005. However, on the download page your comment says that 0.1.0.1 is for VS2003. Should this be also usable for VS2005?

Thanks for clarification,
Mario

# February 18, 2006 3:10 AM

Mario Muja said:

# February 18, 2006 3:11 AM

Terry Go said:

Sorry for the late reply.
I created two different versions of NUnitGenAddIn for both VS.NET and VS2005. Version 0.1.1 for both VSes are now available for download along with class diagram if you are interested. Enjoy~~ :)

# March 20, 2006 12:36 PM

Andrzej Stencel said:

Hi Terry

Your solution works for me, I use VS2005. It's simple and works fine.

One thing I wanted to change was for the test methods to have the word "Test" postfixed rather then prefixed. However, I have achieved this with a simple 'Find and Replace' using regular expressions:
Find: "public void Test{[^(]+}\("
Replace with: "public void \1Test(".

Andrzej

Thanks again, great job!

# March 28, 2006 8:54 AM

san said:

hi terry
as i was googling for the codegenerator i came across this forum. I went through ur link.I downloaded testgenerator for vs 2005 but i could not generate tests i dontknow wht the problem is are there installation directions for this tool bec i dint find any on the site.If u have can u post it.

# May 5, 2006 4:13 PM

Igor Vachshuk said:

No installation required. Just run .reg file

# May 9, 2006 1:13 PM

Terry Go said:

san,

I just posted a lame manual on the project page (http://developer.novell.com/wiki/index.php/Special:Downloads/nunitgenaddin/NUnitGenAddIn-Manual-v0.1.1.doc).

Hope this answers your question.

Thanks for you interests :)

# August 28, 2006 2:28 PM

Nathan said:

I am having problems with the add in for vs2005. It is giving me a null reference exception when I select generate tests. Any idea what might be going on?

# August 31, 2006 4:26 PM

Anonymous said:

Hi,

I am getting an exception while Visual Studio 2003 and 2005 is loading the Addin. Can you please help me out.

Thanks in advance

# May 7, 2007 5:33 AM

Tim Wingfield said:

I just saw another NUnit Test Generator. The price is really reasonable. It works too.

www.kellermansoftware.com/p-30-nunit-test-generator.aspx

# September 16, 2007 12:11 AM

automatic Unit test generation « Amr’s Weblog said:

Pingback from automatic Unit test generation « Amr’s Weblog

# July 2, 2008 2:12 PM

Rejkpolg said:

For NUnitGenAddIn one cause of null reference is lack of namespace in classes, add one or:

replace any occurance of:

string targetNamespace = (targetClass.Parent as CodeNamespace).FullName;

with:

string targetNamespace;

if ((targetClass.Parent as CodeNamespace) == null) targetNamespace = "";

else { targetNamespace = (targetClass.Parent as CodeNamespace).FullName; }

And my short review cause i'm retyping this: I think i'd prefer generation of just empty stubs to all the stuff he tries to generate. The add-on works well tho after that namespace problem.

# August 18, 2008 7:41 PM

Clint said:

Have you looked at Doubler by Jay Flowers?

jayflowers.com/.../index.php

# August 18, 2008 10:38 PM

Name (required)

Your URL (optional)

Comments (required)

Remember Me?

Enter the numbers above:

Add

About Brendan Tompkins

Brendan has been programming with .NET since the first public beta and is owner and operator of Port Technology Services, a consultancy company providing .NET application development services to the Maritime industry. In July, 2007, he was awarded the Microsoft MVP award for ASP.NET. He's also a proud co-founder of failed .COM startup Intrinsigo, and has had a hand in the failure of numerous other businesses. He currently runs CodeBetter.Com and Devlicio.us, and lives in Norfolk, Virgina with his wife Tiara and son Ian.

Our Sponsors

This Blog

Home
Contact

Syndication

News

This work is licensed under a Creative Commons Attribution 3.0 License.

Brendan Tompkins [MVP]

Automatic NUnit Unit Test Generation?

Comments

Michael Russell said:

Rob Gibbens said:

Brendan Tompkins said:

Rev said:

Darrell said:

Darrell said:

Brendan Tompkins said:

Brendan Tompkins said:

Brendan Tompkins said:

Rev said:

Brendan Tompkins said:

Josh Pollard said:

Rev said:

Jonathan de Halleux said:

Brendan Tompkins said:

Jonathan de Halleux said:

edwinyeah said:

Terry Go said:

Mario Muja said:

Mario Muja said:

Terry Go said:

Andrzej Stencel said:

san said:

Igor Vachshuk said:

Terry Go said:

Nathan said:

Anonymous said:

Tim Wingfield said:

automatic Unit test generation « Amr’s Weblog said:

Rejkpolg said:

Clint said:

Leave a Comment

About Brendan Tompkins

Our Sponsors

This Blog

Syndication

Recent Posts

Tags

News

Archives