Recently, we needed to isolate our integration tests from our unit tests. They were mixed in the same test project and we wanted to avoid the cost of executing integration tests while executing our unit tests. Integration tests takes longer to be executed because they are identified as tests that are accessing external systems, such as DB, files, registry, web service…

At first, it looked like a daunting task because we needed to know which tests were triggering file access and we have thousands of tests. More precisely, we needed to know which test method from our test assembly were depending on the class System.IO.File or depending on the method System.Xml.XmlDocument.Load(String). Hopefully, we realized that a simple CQL query would list all integration tests.

SELECT METHODS FROM ASSEMBLIES "NDepend.Test.Unit" WHERE

(IsUsing "System.IO.File" OR IsUsing "System.Xml.XmlDocument.Load(String)")

This CQL query works well because the condition IsUsing XXX matches direct and indirect use. It means that if Test1() is calling a method that is calling a method that … that is calling XmlDocument.Load(String), then Test1() will be identified as an integration test. Notice that if an interface is used to mock a service (such as file or DB access), the IsUsing condition won’t see what’s hidden behind the interface. Also, if our application was using DB, we would have needed some more DB conditions such as

… WHERE … IsUsing "System.Data.Common.DBCommand" OR IsUsing "System.Data.Common.IDBConnection" OR …

We also used the Dependencies Matrix to see which test was using which file method. A blue cell tagged with X means that the test method (in abscissa) is using the file method (in ordinate) with a minimum path depth of X. For example, we can see that the test Test_NoDeclarationForAnonymousNamespace() is using the class System.IO.Path with a depth of 4 (see the generated graph below the matrix):

Because NDepend relies on static analysis (i.e it is analyzing the code structure, not the run-time call graph), we got a few false positives. Indeed, some test methods were statically depending on a file access method without accessing it at run-time. However, we decided to consider such tests as integration tests, this way we can keep our simple criteria to sort between unit and integration tests.