The only
way to have bug-free code is to mathematically prove the code. Very few
programs in the world are mathematically proved. Some industry can afford the
price of a mathematical proof, especially when bugs would turn into human death,
such as embedded software in planes, trains or cars. Most of us are working on
projects that cannot afford the cost of a mathematical proof. We then rely on
some tricks to maintain our bug rate as low as possible. I classify these
tricks into 6 categories.
- Prioritizing
bugs fix over new features development
- Programming
style and code quality
Contract
The idea of
contracts is to insert correctness information inside the code. When contract
is violated it means that your code contains a bug somewhere. Typically, the
.NET world is very poor in terms of contracts. Our only way to express contract
is to use the System.Diagnostics.Debug.Assert(…)
method. This is very unfortunate because
not only our code is blurred with numerous Debug.Assert(…)
calls, but also the compilers cannot check contracts at compile-time. For
example the Spec# non-nullable types is an excellent form of contract at
language level that help a lot avoiding pesky NullReferenceException. And Spec#
is far from being the only language that propose contract facilities, think
about what Eiffel proposed more than twenty
years ago in terms of contract!
As I
believe in contract, I personally use a lot Debug.Assert(…). This represents around 15% of my code. I will
likely blog more on this because with all the buzz actually made around automatic
tests, I feel that contracts should deserve more attention.
Automatic test
I won’t
enumerate the numerous benefits of automatic tests and high code coverage ratio
here. The important thing to remember is that having a solid battery of
automatic tests is an excellent way to decrease significantly the bug rate on tested
code, but also to avoid new bugs when code gets refactored. As many, my
opinion is that the cost of writing automatic tests is worth the price compared
to the cost of maintaining not automatically tested code.
Empirical approach
What I call
empirical approachis a simple tenet that every seasoned programmer know:
- Most of bugs in a new version are
coming from modified code and brand new code
- Unchanged code that works well for
a long time in production won't likely crash within the next release (what we
call stable code). We don't say that stable code don't contain bug, but
discovering a bug in stable code is rare.
How do you know that some code works
well for a long time in production? Simply by listening to users. If they
didn’t report problems on some features for a long time then you can be
confident that underlying code is stable.
Some might say that I consider here users as testers but don’t take me wrong, we
don’t have the choice. If you have real users, they will complain when they
will find a bug and they will remain silent when they consider that the product
is working fine. You are responsible to deliver correct code that will satisfy
users but why wouldn’t you infer statistics from their feedbacks to asses where
is your stable/unstable code?
I wrote an
article about how we (the NDepend team) use this simple but effective
idea and our own dog-food to avoid regression bugs while adding new features. Basically we focus our code
reviews mainly on code that have been added or code that have been changed
since the last release and of course.
Code review
Code
reviews are good to enhance quality and to educate programmers but I don’t
believe in the efficiency of code reviews to anticipate bugs. Even though some
portions of code can be fascinating to read, the bulk of a code base is tedious
and will make you lose your focus in less than an hour. The problem of code
review comes from the mass of code to read. This is why I advise to only
focus your review on not stable code,
i.e added code and modified code. If you release new versions often, the mass
of code to review before each release will quickly become an epsilon of the size of your entire code
base. Doing so can also be seen as a way to capitalize on code reviews made during previous iterations.
Prioritizing bugs fix over new features development
This popular methodology directly results from the concept of stable code. Prioritizing
bugs fix over new features development can be seen as a way to constantly struggle to maximize the surface of stable code in our code base.
Programming style and code
quality
The recent
buzz around LINQ or F# comes from the fact that object
style programming is more bug-prone than functional style programming. This
fact results from the expressiveness of functional style. In other words,
functional code is easier to read and understand. A major aspect of the
expressiveness of functional programming style is IMHO the concept of
immutability that I described in a previous blog post Immutable types:
understand their benefits and use them. That’s a fact, it is hard to
write, understand and maintain code that mutates states at runtime (for example
this is why global variables are so harmful). And this becomes much harder in
concurrent environment.
Obviously, code
quality has also a direct impact on code correctness. Anti pattern such as methods
with high LOC,
high Cyclomatic Complexity or
high Nesting Depth, entangled components, methods with multiple concerns, classes with multiple responsibilities…
leads to code harder to debug and to test.
Static Analysis Tools
We all
dream of a static analyzer that could pinpoint automatically bugs in our code
by just pushing an Analyze button. Some
tools are already able to detect naïve mistakes, such as calling a method with
a null reference as parameters where there are no tests of parameter nullity
(i.e interprocedural analysis).
But most
bugs are not that easy to pinpoint. By just analyzing the code a tool cannot distinguish
between a feature and a bug because it doesn’t know how the application should
behave. Some heuristic exists but still, to be efficient, a bug finder static analyzer needs to be
feed with more information than just the code. Typically, this extra
information can be found in contracts and unit-tests code. As far as I know,
in the .NET area there are 2 projects revolving around this idea NStatic
and Pex,
and I am really looking forward to use them on my own code.
Conclusion
From what I
understand from the agile trend, having a bug-free product to release shouldn’t
be the goal. Anyway, every program not mathematically proved has bugs. The goal
should then be to tend toward a bug-free product by applying as many correctness
tricks as possible.
Posted
Sun, Mar 30 2008 9:02 PM
by
Patrick Smacchia