Include IL Offset into production exception StackTrace

Posted on June 27, 2012 by Patrick Smacchia

We are currently doing an intensive pesky minor bugs session. Bugs are logged into our gmail account as a production bug tracking server. For each logged crash, we get a stack trace and plenty of information about the machine environment.

One recurrent problem with exception stack trace in .NET, is that, one needs to release PDB files with assemblies installed in production code. Without PDB files, you still gets the stack trace of methods called, but you loose the exact source file line from where the exception popups. The problem is that usually the PDBs files weight between one and two times the weight of the target assemblies. Hence, releasing PDBs is usually not an option because this consumes so much more memory (both install hard-drive memory, and process memory at run-time).

The exact source file line from where the exception popups is not just a convenient info, it is an essential info. The typical wrong situation is when a method throws a NullReferenceException, but contains several references that can potentially be null: in such case you are stuck because you are missing the information from where exactly the exception popups. Knowing the exact source file line from where the exception popups, it would become easy to identify the faulty null reference.

To fix this situation, Mike Stall, the MSFT debugging guru, wrote a blog post about Converting a managed PDB into a XML file. This blog post shows a trick about how to use some tooling to both, not release the PDB files, and be able to retrieve the exact source code line from a production exception stacktrace. The problem is that often real-world application assemblies are obfuscated. The correspondence between released obfuscated assemblies and their initial PDBs is then completely broken.

Yesterday, I just realized that somehow, it should be possible to retrieve the IL offset in the obfuscated method body, that provokes the production exception. After googling a bit, it seems that sometime stacktrace contains the IL offset, at the end of each frame, after a plus ‘+’ character. However I haven’t been able to determine what should be done to get this extra information.

Hopefully, as always, more googling lead to exactly what I was looking for: in the step 3 of this blog post Getting file and line numbers without deploying the PDB files Tim Stall (another Stall debug guru?) shows how to retrieve all stack trace and IL offset programatically. Using this info, I’ve been able to rebuild completely the stack trace, including IL Offset formatted the same way as Reflector does (hexadecimal with 4 digits prefixed with L_).

Now we get clean stack trace with IL offset. It is pretty straightforward  to infer a source code line from a IL instructions. You just have to know that if the Nth IL instructions  throw the exception, you’ll get the offset of the N-1th IL instruction logged.

Notice that we use Preemptive Dotfuscator for obfuscation, and this tool come with Lucidator to transform a stacktrace made of obfuscated methods, into a stack trace with original methods name.

Find below our source code + associated tests. This code implements another requirement: we remove all localization info from the stacktrace. This way we are able to build a hash code from a stacktrace. Such hash code is pretty useful to group similar crash logs, independently from the underlying Windows machines localization settings.

…and the test code that covers 100% StackTraceHelper:

Posted in Uncategorized | 3 Comments

An Original Algorithm to Find .NET Code Duplicate

Posted on June 12, 2012 by Patrick Smacchia

The upcoming Visual Studio 2012 version comes with tooling to find code duplicate.

However, today, I’d like to talk about another code duplicate tool for .NET code. This tool comes as a NDepend Power-Tool. Power-Tools are a set of open-source tools based on NDepend.API. The source code of Power-Tools can be found in $NDependInstallPath$\ NDepend.PowerTools.SourceCode\ NDepend.PowerTools.sln.

The algorithm underlying this code duplicate Power-Tool is simple, yet powerful in practice. It consists in defining sets of methods that are using the same members, i.e calling the same methods, reading the same fields, writing the same fields. We call these sets, suspect-sets. Suspect-sets are sorted by the number of same members used.

Let’s run this Power-Tool on the NUnit v2.6.0 code base. The screenshot below shows the 3 steps of the algorithm executed, and the first suspect-set. This first suspect-set is, as often from my testing experience, the entry points of the application. Very often an application has several entry points that does almost the same initialization things. Sometime it is pure duplicate, sometime it is slightly changed. Most of the time this initialization code can be factorized in one or several parametrized method.

Notice the option o for Open callers methods decls. The declarations are opened in Visual Studio actually.

The second suspect-set is the following one…

…bingo, we have an exact code duplicate!

The third suspect-set is the following one…

…by looking at the code below, this suspect-set can eventually be considered as a false positive. Personally, I still consider such code as code duplicated. These two methods implement a unique layout algorithm, that could be parametrized. So here I’d vote for a factorization.

I could continue on and on, but I invite you to download the 14-days full featured evaluation of NDepend to see what this algorithm will report on your own code.

Hopefully duplicate results found by this algorithm are pretty relevant. Its main advantage over others algorithms is that it is not disturbed by dirty copy-paste where the pasted code is then sightly modified. Another advantage is that this algorithm can be run on IL code only, source code is not required. Notice that what this current blog post doesn’t show, is that a suspect-set can have more than 2 suspect methods.

On the cons side, sometime a suspect-set can be humanly considered as a false positive. At least it is almost always possible to refactor found duplicates to factorize them into one or several parametrized methods (as explained for the 2 layout methods). The fact is that two or several methods don’t use the same set of members by coincidence.

Also this algorithm is very fast to run, from 10 to 100 times faster than the VS2012 one. Concretely it takes few seconds to be executed on a real-world large code base. This speed is the consequence of the fact that NDepend.API is optimized to browse dependencies very quickly.

Concerning the algorithm itself, it is open-source so if you are interested in browsing and tweaking it you can. This open-source + NDepend.API form makes it particularly suited to be integrated into a CI process.

The algorithm consists in 3 steps:

  1. Investigate each method (including third-party ones) to see if their callers could be considered as suspect (methods called very often like the ones from types in System.Collection.Generics or System.Xml are discarded to limit false positive).
  2. Merge suspect sets obtained from step 1).
  3. Sort suspect-sets from a weigh computed on number of members called.

Hopefully this algorithm can be optimized further, and your feedback is welcomed. Enjoy!

Posted in Uncategorized | 10 Comments

LINQ Performance: Some Case Studies

Posted on June 5, 2012 by Patrick Smacchia

One essential requirement while developing Code Query and Rule through LINQ (CQLinq) has been performance, both performance of query compilation and performance of query execution. The reason is simple: we want hundreds of CQLinq rules to be verified right inside Visual Studio in a few seconds, to let the developer be quickly informed of problems. Hence, we’ve targeted and achieved 100 queries compiled/executed per second against a real-world large code base.

To achieve this goal, a large research effort has been made. In the product itself, a query is compiled and then executed live at edition time. In the screenshot below, we can see the query execution duration measurer at the top right (here 4 millisecond).

This instant measurer has been proven very useful to optimize CQLinq and the 200 default rules. Hopefully we’ve learnt a lot about LINQ performances. Several re-usable patterns are listed in the documentation CQLinq Performance and I’d like here to digress a bit deeper on these findings.

HashSet<T>, Dictionary<K,V>, Lookup<K,V> my dear friends

Often in my past posts, I claimed that hash tables (HashSet<T>, Dictionary<K,V>, Lookup<K,V) are one of the best tool to optimize performances. Of course they played a major role to optimize many code queries. Last year I wrote about LINQ Intersect() 2.7x faster with HashSet and in the v4 RTM, the concrete conclusion of this study is the class NDepend.Helpers.ExtensionMethodsSet of the NDepend.API. For example without the single call to ToHashSet() the query below would be 200 times slower! This is because the call to one overload of Intersect() is very fast, because it takes account of the fact that one of the collections to intersect is a HashSet<T> with O(1) search:

It is disappointing that these simple, common yet efficient optimizations are not proposed by default by the .NET Fx.

Below is the rule Do not hide base class methods source code. It took us significant effort to make it fast enough for our requirement. Here again the key to high performance has been a hash table, more specifically a lookup table. Methods are indexed by their names (including their signature) in the lookup table to let us retrieve quickly the set of methods potentially hidden by a method:

Another case where HashSet<T> was the key to performance, is the rule Avoid naming types and namespaces with the same identifier. Here we want to match types whose name is also a namespace simple name (for example the simple name of the namespace System.Collections.Generic is Generic) . The fastest approach is to first create a hashset containing all namespaces simple name, and then for each type, check if some namespace share the same name:

Tricky Performance Enhancement

Some rules were pretty tricky to write to achieve acceptable performance. One of them is Methods should be declared static if possible. This rule is especially computation intensive since for each instance method we need to check if it is not using any other instance method or field of its parent type, or any of its base class. Here two optimisations were harnessed:

  • First, we filter types that cannot have instance method to turn into static method (interface, static class, delegates…). Then we filter methods of selected types (static, abstract, virtual, access the this reference). Filtering first objects with trivial/fast-to-check criterias is the rule of thumb to optimize LINQ queries.
  • Second, we call the extension method FirstOrDefault() to evaluate as fast as possible if the instance method is calling another instance member. This is a good example where FirstOrDefault() can replace and be faster than a call to Count() or Any(). Many approach were tried here, like first getting all base classes of the parent type and reuse this set accross all members. But the query execution measurer tells us without any doubt that this approach is the fastest one.

LINQ Expression and Performances

Before compiling a CQLinq query with the C# compiler, we’ve included a smart preprocessor that solves in a variety of cases, the classic dilemma readability vs. performance. For example, if you wish to use a regular expression in your query, it is tedious to first create and compile your regular expression object, and then use it. It would be more readable to create, compile and use the regular expression in the same code spot, but it would be terrific for performance if this code spot is in a loop.

This is why when calling methods like NameLike() like for example, in the rule below, Methods name should begin with an Upper character, the CQLinq preprocessor transforms the LINQ expression to:

  1. relieve the user from creating and compiling the regular expressions.
  2. make sure that the regular expression is created and compiled just once.

This particular LINQ expression magic trick has been used extensively to make many rules both more readable and more performant. This is why we’ve created the dedicated namespace NDepend.Reserved.CQLinq that contains these methods whose calls are transformed by the CQLinq preprocessor. Obviously these methods shouldn’t be used from a program that relies on NDepend.API since in this situation, there is no CQLinq preprocessor.

A Word on Time-Out

On a related topic, we’ve also implemented a tunable time-out set to 2 seconds by default for CQLinq query execution. Internally, the time-out manager code is based on this great piece of OSS code that is concise, elegant, efficient and AFAIK bug-free.

Conclusion

Clearly pretty much any realistic performance goal is reachable with a sufficient amount of effort. One more time, with a lot of effort we’ve verified this tenet.

We can observe the same gain on the Visual Studio side. VS2010 was so slower than VS2008 that personally, I never really upgraded to VS2010. But having used VS2012 Beta and RC for months, I can attest that the performance boost is huge. I can’t wait to upgrade all our dev from VS2008 to VS2012 RTM :)

Posted in CQLinq, LINQ, Performance | 2 Comments

Code Query and Rule over LINQ

Posted on May 24, 2012 by Patrick Smacchia

Yesterday, after two years of a relentless development effort, we finally released NDepend v4. Personally, I consider this version as the biggest milestone we’ve ever achieved. The three flagship features are:

  • Code query and rule over LINQ (CQLinq)
  • NDepend.API to let user develop its own static analyzers (+14 OSS Power Tools proposed on top of NDepend.API)
  • VS11 addin support

Developing these features by respecting the following drastic requirements was a real challenge:

  • Features Richness: The code query features set was already rich in prior versions, but we wanted it to be even richer to be able to write easily all the code queries and rules that existing users asked us.
  • Syntax: Queries must be easy to write and to read. Hopefully most NDepend users are aware of the C# LINQ syntax, but the proposed code model API needs to be LINQ-friendly. We started with a three months research effort to define what was the cleanest syntax to express a few dozens of essential code queries. A few natural syntax enhancements were also developed (like the prefix warnif count > 0 to transform a code query into a code rule).
  • Performance: We wanted something like a hundred of queries compiled and executed per second against a large real-world code base, because we want them to be run often in VS without slowing down the IDE.
  • Usability: We wanted query edition to be seamless thanks to code completion, live tooltip documentation and detailed error reporting.  All this without the Roslyn power that is not yet RTM. Also, for v3 users, we’ve developed a CQL to CQLinq automatic converter (and CQL is still supported).

One simple CQLinq code rule that illustrates well all these, is the following one.

Hopefully the syntax is simple enough to convey the underlying meaning to any .NET developer. Only the JustMyCode highlighted word might not be clear. It represents a facility proposed to avoid matching generated code elements, that often are pesky false positive matches for code rules.

After  an instantaneous compilation/execution phase (1 millisecond), the result is displayed with facilities for browsing it and exporting it:

Today, I’d like to focus a bit more on the syntax aspect. Developing for LINQ and with LINQ during the last two years has been a great joy. Everybody agrees that LINQ is very elegant, but it is also a super-extensible technology.

We extended LINQ in several different ways. One way has been to develop a LINQ-friendly fluent API. Often we found convenient to write default code rules with a mix of both Query Expression and Query Operator syntaxes. Takes the following rule for example.

  • First, we define with two fluent sub-queries (expressed with the operator syntax)  the UI layer (types in namespaces using any UI framework) and the DB layer (types in namespaces using any DB framework)
  • Then we use more fluent query operator syntax to check if the UI layer is using the DB layer.
  • Finally the whole query is structured with the query expression syntax.

In a few lines of code, we are expressing fluently a pretty complex and popular requirement, in a generic way adaptable to any situations.

For a few years now, the code metric C.R.A.P (Change Risk Analyzer and Predictor) became increasingly popular in the Java community thanks to the crap4J plugin. The C.R.A.P metric has been exposed by Alberto Savoia in this Artima article dated from October 2007. The C.R.A.P metric is a mathematical formula that helps to determine which piece of code is both complex and poorly covered by tests. Since both code coverage and cyclomatic complexity code metrics are proposed by the NDepend.API code model, it is fairly easy to write a CQLinq code rule to match crappy code:

A popular feature of NDepend is the ability to diff two snapshots of a code base to explore what was changed. This feature being completely integrated with CQLinq, it is now possible to write simple code queries that will match complex evolution requirements. One immediate requirement that comes to my mind, is that a class 100% covered by tests should remain 100% covered by tests, no matter whether it has been touched or not. The following CQLinq code rule detects classes that are not anymore 100% covered by tests (since the predefined base-line), and lists the culprit methods, i.e the method that are not 100% covered anymore:

Hopefully CQLinq is a simple answer to many requirements that formerly demanded significant efforts (imagine the effort to develop the tool crap4J compared to the effort of writing a single CQLinq query). You can download freely and try CQLinq live on your code base, and here you can browse all 200 default code rules.

In future posts I’ll dig into the low-level implementation tricks needed to implement all this.

Posted in Code Query, Code Rule, CQLinq, NDepend, Object oriented programming, VS Integration | 2 Comments

On Writing Unit Tests for C#

Posted on March 19, 2012 by Patrick Smacchia

My recent blog post Non-trivial and real-world feedbacks on writing Unit-Tests has been rewritten in depth with the help of the SimpleTalk team, and published on SimpleTalk as an article: On Writing Unit Tests for C#

This article has been an opportunity for me to dig deeper into several aspects of my experience with real-world unit-testing, especially concerning the Test-First and code coverage sections. The critics and feedbacks I got from the initial post has helped also rewriting this as a more insightful content.

Enjoy!

Posted in Uncategorized | Leave a comment