Magnus has been off doing some interesting work around integrating MEF with Windows Azure. The first question you might be asking is Why?

In his words, he set out to build a template for Windows Azure templates that:

• enables testability
• abstracts away storage
• is extensible and easy to evolve during development

Last Sunday I took my family to Red Robin (a family style restaurant) after church. I requested a table with a high chair for my  3.5 yr old son. Upon bringing this high chair the hostess simply moved an adult chair to the side and put the high chair in its place. (See Image)

This is where the fun began…

For the next hour or so I watched countless customers, servers and managers limbo their way between these two chairs. A lot of them did it repeatedly, some even carrying trays full of drinks and/or food. Finally I provided evidence to to my family that I am 100% full on geek. I got excited about the insight that this little experience game me (to the point of taking pictures)

I was simply amazed that people would allow this “friction to continue” It was a simple problem in this case. Move the chair!

I find it interesting that it’s built into our DNA incur friction in our lives without really addressing it. In real life its called denial. I know the chair was not that big a deal but it did open my eyes to our basic nature. Our basic nature follows through to software development as well.

The example of the misplaced chair provides some interesting parallels when it comes to technical debt (which is a cause of software development friction) We all have technical debt in our applications and it’s in our nature to shimmy, shuffle, limbo, mambo, and shake around these issues  whenever possible. It takes a constant vigilance to address technical debt. We must be active in our quest to remove technical debt. We must take the time to do two things:

•     Recognize the debt. In our case: The chair.
•     Remove the debt: In our case: Move the chair.

That’s it for this blog post. Just a simple observation brought to you by the staff and management of Red Robin :)

One of my team members was showing me some code in HandleUnknownAction based upon sample code in the online documentation. Per the documentation:

“The following example shows how to render views that do not have a matching ActionResult method. For example, if you have a Details.aspx view but no corresponding method exists that returns an ActionResult instance, the following example displays the Details view when a request to for the Details action is made on the controller. If there is no matching view, the error page displays a message.”

The only problem with this example is that it does not take into account attributes, like the ActionMethodSelectorAttribute, that can decide based on runtime information whether an action can fulfill a request. So, it may not be that the action does not exist, but that it is not supposed to respond to the request based on the context of the request.

For example, if I use the AcceptVerbsAttribute, an example of ActionMethodSelectorAttribute, to do something as simple as this on my HomeController:

The DisplayOnlyViaPost View will not be displayed via the DisplayOnlyViaPost Action when someone types the URL in a browser ( HTTP GET Request ) – http://…/Home/DisplayOnlyViaPost.

Unfortunately, the HandleUnknownAction code above will kick in and notice that indeed the DisplayOnlyViaPost View does exist. Therefore it goes ahead and displays the view, undermining the use of our AcceptVerbsAttribute on the original action. Not good :)

Just something I noticed that you may want to be careful of if you are using similar code in HandleUnknownAction.

David Hayden

The Tampa ASP.NET MVC Developer Group is starting to kick it up a notch with this month’s meeting being about ASP.NET MVC Extensibility. We have concluded our introductory topics and are now focusing on more intermediate topics. This month’s meeting will hit more advanced topics, such as:

• Creating Custom Controller Factories for Dependency Injection
• Using Custom View Engines
• Creating Custom ActionFilters for DataBinding, Validation, etc.
• Using Custom ControllerActionInvokers
• Developing Custom ActionResults for RSS Feeds, etc.
• Building HtmlHelpers for Productivity

If you are interested in building your own opinionated version of the ASP.NET MVC Framework and have it bend to your will, you don’t want to miss this meeting.

In addition, we will also be going over last month’s homework assignment, which was to use the ASP.NET MVC AjaxHelpers to “ajaxify” an ASP.NET MVC Web Application. As a hint, you can find an answer in our last screencast, ASP.NET MVC Ajax Helpers - Ajax.ActionLink and Ajax.BeginForm – Screencast :)

Those completing the homework assignment will be included in a random drawing of Steven Sanderson’s Awesome book, Pro ASP.NET MVC Framework. Absolutely love Steve’s book. Huge thanks to Apress for sending us several copies to give away to our members.

Starting in August we will be working on voluntary group projects to strengthen our team development skills using ASP.NET MVC. Again, huge thanks to two of our sponsors for making this happen: Server Intellect and Unfuddle.

See you on Thursday, July 9 at the Microsoft Office in Tampa!

David Hayden

The links to the videos recorded for NDC 2009 talks are online and linked in the agenda pages here.  My talks are at:

• Lessons Learned from a Long Lived Codebase
• Presentation Patterns for Composite Applications (I'd dearly appreciate feedback from folks on this talk)
• Convention over Configuration applied to .NET
• Software Design and Testability

I have a love it /hate it relationship with reports. In many an application the reports are the main output. They are the prints which get filed or the invoices which have to ensure the ROI of the application. A good looking output rich with information does increase customer satisfaction. The downside is that fine tuning a report is tedious and requires quite different skills than coding. Building a system with a lot of reports can be pretty boring. So the tools better be good. Switching from Crystal Reports to sql server  reporting services (RS) was quite a relief.

But  RS still has two drawbacks. First is that it accesses data using plain sql. My application is built around a domain model using nHibernate to hide all db details. The domain model includes calculated properties which are to be included in the report. The calculations are done in the domain object. It would be repetitive and prone to error to repeat them in sql or reporting expressions. It would be quite a relief to report straight from the domain model itself.

The other drawback is that RS requires an ms sql server instance with installed reporting services. The actual data is just a sqlexpress instance and I don’t want to set up a full blown reporting services server.

New in Visual Studio 2005 were client side reports (rdlc’s). They are very much like RS reports (rdl’s) but can work with any data source and don’t require a sql reporting server instance. To include these reports in an app two client side report viewers are include in the .NET framework, one for WinForms and one for asp.net. This looked like the best of both worlds so I decided to give the winforms version a try.

For a good overview on the reports check the got reportviewer site. In name it supports VS 2008, but (some of ?) the samples are based on VS 2005. Moving to 2008 there has been a change in the namespace for the classes, after fixing the reference the samples run in VS 2008. How to work with a non sql datasource is covered but not that clear. Here I will describe my minimalistic approach to work with a domain model fed by nHibernate.

The domain model

The intent of the application is to print invoices. This class diagram describes the model

An invoice is for a contact, has a number of lines and an enumerated property. That is a number of values which contain a tax rate and have a clear name describing this value. Which gives the model additional value, something which would get lost when viewing the data as raw db data. An invoice has a reference to a contact and many Invoice lines. The (fluent) nHibernate mappings express this.

So we have a customtype, a hasmany and a references. Let’s see how these are used in the report.

The report

There are several ways to include a report in an application. The default when adding  a new report (Add –> New Item –> Reporting –> report) is embedded. Which is clear and easy in deployment. When you need more flexibility you can switch to rdlc file or (back to) server side report.

The report needs a datasource. It will accept any datasource including an object datasource.

Add a new object datasource to the project and pick the Invoice domain object.

Just pick your POCO domain class. There is no need to set up any methods to get data, it is enough to have a model of the data.

This object data source is the datasource for your report. Having done so the domain invoice is available in the report designer. (To be precise the datasource has to be attached to a List, see the reportviewer docs for more information on that.)

A report is built from VB-like expressions. A simple one looks  like this:

Which displays the plain content of a property. But expressions can be more complicated. The report designer has a good expression builder which even includes intellisense. The next expression uses both the meaningful name and the value of my tax enumeration property in a string

The result is a string like “BTW Hoog 19%”.

Things get really interesting when displaying the contact information. In the domain model this expressed as

In the report the expression is quite similar

The subreport

The lines in the invoice are a one to many relation. The invoice datasource has a lines property but this property is not usable in the report. To display the lines I need a subreport. The subreport is based on invoice lines. After adding the InvoiceLine class as a datasource the subreport is drawn. It is a simple table displaying the lines.

A subreport is linked to its parent report using parameters. In my case an invoice and its lines are linked through a parameter named FactuurNummer.

All of this works just as in “traditional” reports. In the main invoice report this parameter is assigned the value of the invoice number

Check the gotreportviewer site for further information.

Binding the data to the report

Now the reports are ready. The next step is to get some real life data into them.

It takes code to bind real data to a report and display the result in the viewer. Also the subreport has to be fed with data. To wrap things up I have built a helper class which takes a reportviewer component and a list of invoices and wires them up.

The code requires some explanation. The constructor should be clear, it receives viewer and data.

The ShowReport method does the real work. The first line loads the report in the viewer. The second line loads the report data. This line is tricky. The name of the ReportDataSource has to match exactly the name of the datasource class used as model. The Invoice class name is Gekko.Administratie.DomainModel.Invoice which gets formatted as Gekko_Administratie_DomainModel_Invoice. This required magic string is not clear from any documentation, it took me quite some time to find out how essential it was.

All these datasource properties in the report definition are confusing. The model of the data is saved in the report definition. Changing the datasource  to another “dataset” will also lead to errors at run time. The report will complain about the former datasources. To get rid of the error messages you have to check the report defintion by hand. The report definition file (.rdlc) is plain xml. The last part contains dataset definitions. What helps is just deleting the unused nodes by hand.

Upon rendering a subreport an event is fired. In the third line of ShowReport a handler is attached. This SubReportProcessing method feeds the subreport with data. From the parameters the number of the invoice is read. Which is used to find the corresponding invoice. The datasource for the subreport is set just like the datasource of the main report. Again spelling the name of the reportdatasource right is essential. The data is the Lines property. Which is an IList of InvoiceLines.

Putting it all together

With all pieces in place it’s time to get some results. I have a windows form with a reportviewer on it.

This reads the invoices from the repository and sends them with the reportviewer to the helper class.

There is one more thing to watch. The repository is an nHibernate repository. By default nHibernate lazy loads related objects, like the Contact and the Lines of the Invoice. The contact and the invoicelines will not be read from the database until they are actually used. This default setting has some serious drawbacks.

In the first place the reportviewer‘s datareader does not operate very well (at all ?) with nHibernate’s proxies. In the report all the contact fields will contain error messages. The subreport on the  lines will work. But still the lines are not read from the database until rendering the sub-report. So for this lazy loading to work the report has to be run against a dataset with an open dataconnection. That’s bad, I try do do as little as possible, and most likely not presenting a report, with an open connection. A far better scenario would be to fetch all data in one go, close the connection to the database and render the report after that. This can be done by eagerly loading the Contact and Lines properties and disposing of the session before returning the data.

In the nHibernate API the fetch mode can be set on a property level. The code in the repository takes care of fetching all data in one big snapshot.

The repository reads all data from the database and closes the connection. After that the collected data are input to the reportviewer. Which now works completely as hoped for.

I have my report complete with printing and export functionality and am still working with my beloved domain objects.

This way reporting can almost be pure fun.

Several months ago a few people started hosting a Virtual ALT.NET meeting right here on the information super highway. I've been to a few and they're always packed with information and good conversation. It's a great way to learn new things or show off cool stuff you've been working on. If you haven't been to one, I'd recommend you check one out. There are two great looking meetings hosted by Ryan Svihla coming up soon:

Development with Castle Project with Ryan Svihla July 1 and July 8
Ryan will be doing a two part series on the Castle Project.  Mark your calendar for some Castle Project fun.
Ryan Svihla has been working as a C# developer Farm Bureau Bank in San Antonio since September 2007. Before that he worked as  a Consultant in Lincoln, NE for 3 years, where he had working experience with Php, some Perl, Python and of course C#.  Attemping Agile since early 2008 as an eager student with a focus on making programming more useful and relevant for the end user.

IoC and Dip through Castle Windsor
Ever wonder what acronyms like IoC and Dip mean? If you know what they mean do you wonder why anyone would use them in code?  This talk aims to deal primarily with those two questions through the use of Castle Windsor IoC container.  Intermediate level C# material with a couple of more advanced demos at the end for fun and pleasure.
Central Daylight Time
Start Time: Web, July 1, 2009 8:00 PM UTC/GMT -5 hours
End Time: Web, July 1, 2009 10:00 PM UTC/GMT -5 hours
Attendee URL: http://snipr.com/virtualaltnet (Live Meeting)

Web Development with Castle Monorail, Active Record and Brail view engine
Have a look at the first popular MVC .Net based web framework. Also will be covering persistance with ActiveRecord, and view templates using Brail.  Bonus, will demo a plugin framework for building CMS like applications.
Central Daylight Time
Start Time: Web, July 8, 2009 8:00 PM UTC/GMT -5 hours
End Time: Web, July 8, 2009 10:00 PM UTC/GMT -5 hours
Attendee URL: http://snipr.com/virtualaltnet (Live Meeting)

Hope to see you all there!

When we started our current project we did not use context/specification style testing, instead we used testcase-per-class with a four-phase test model (also known as arrange-act-assert). Although we followed story-test driven development (STDD) we were not explicitly Behavior-Driven Development BDD when we set out. Over time I began to see the path between STDD and BDD we shifted toward a BDD approach.

That is really background for what comes next. As part of this shift we started writing more context/specification style tests. We did not re-write our existing testcase-per-class fixtures (then and still now I would see the possibility of using multiple test organisational patterns within a project; some of our tests still seem better with testcase-per-class). Recently we have had new developers join the team. One interesting, if anecdotal, observation has been that those developers find it easier to use tests as a source of documentation for what the software is doing when they are in the context/specification style. This is a useful observation, because the rest of the team is able to supply much of the context for testcase-per-class fixtures by virtue of having worked on the codebase and so do not see the lack so easily.

To build a maintainable system you have to build one where the behavior can be understood. Tests offer this promise, but often we have found that tests failed us, because the tests were no easier to comprehend than the code under test. Context/specification style tests seem to be better on this account. Of course there is no silver bullet in software. You still have to put the effort into making your context/specification tests readable, but the form seems helpful toward achieving that goal.

I'll try to post more on my experiences with BDD style approaches as they come out. I would also recommend that anyone interested in the topic look at the RSpec book from the Pragmatic Programmers which offers a great overview of the topic.

Normal 0 21 false false false FR X-NONE X-NONE

During my interview on Code Metrics by Scott Hanselman’s on Software Metrics, Scott had a particularly relevant remark.

Basically, while I was explaining that long and complex methods are killing quality and should be split into smaller methods, Scott asked me:

looking at this big too complicated method and I break it up into smaller methods, the complexity of the business problem is still there, looking at my application I can say, this is no longer complex from the method perspective, but the software itself, the way it is coupled with other bits of code, may indicate other problem…

Software complexity is a subjective measure relative to the human cognition capacity. Something is complex when it requires effort to be understood by a human. The fact is that software complexity is a 2 dimensional measure. To understand a piece of code one must understand both:

• what this piece of code is supposed to do at run-time, the behavior of the code, this is the business problem complexity 
• how the actual implementation does achieve the business problem,  what was the developer mental state while she wrote the code, this is the implementation complexity.

Business problem complexity lies into the specification of the program and reducing it means working on the behavior of the code itself. On the other hand, we are talking of fabricated complexity when it comes to the complexity of the implementation: it is fabricated in the sense that it can be reduced without altering the behavior of the code. As an illustration here is a super/giant/complex method found inside the .Net Framework implementation System.Windows.Forms.DataGridView.GetClipboardContent():

GetClipboardContent() is made of around 300 Lines of Code and has a ILComplexity equals to 192. GetClipboardContent() does not have drastic performance requirement. As a consequence I don’t see any justification for not refactoring this massive method into smaller ones and maybe even a small classes hierarchy that could help in implementing the enormous switch/cases. Doing so would certainly discard a lot of fabricated complexity.

Fighting Fabricated Complexity with Simple Code Metrics

The simplest way to limit fabricated complexity is to abide by simple code metrics thresholds. This is why one of the default CQL rule proposed by NDepend is the following one:

// <Name>Quick summary of methods to refactor</Name>

WARN IF Count > 0 IN SELECT TOP 10 METHODS WHERE

     (  NbLinesOfCode > 30 OR             

        // http://www.ndepend.com/Metrics.aspx#NbLinesOfCode

        NbILInstructions > 200 OR         

        // http://www.ndepend.com/Metrics.aspx#NbILInstructions

        CyclomaticComplexity > 20 OR      

        // http://www.ndepend.com/Metrics.aspx#CC

        ILCyclomaticComplexity > 50 OR    

        // http://www.ndepend.com/Metrics.aspx#ILCC

        ILNestingDepth > 4 OR             

        // http://www.ndepend.com/Metrics.aspx#ILNestingDepth

        NbParameters > 5 OR               

        // http://www.ndepend.com/Metrics.aspx#NbParameters

        NbVariables > 8 OR                

        // http://www.ndepend.com/Metrics.aspx#NbVariables

        NbOverloads > 6 )                 

        // http://www.ndepend.com/Metrics.aspx#NbOverloads

Fighting Fabricated Complexity with Abstractions

Another popular way to limit fabricated complexity is to hide implementations behind some interfaces. Even though interfaces are not contracts, an interface itself often conveys enough information to make its purpose understandable. For example the IDisposable pattern is a complex topic but still, the IDisposable interface present only one method and we at least understand that it is an indication that some unmanaged resources must be released somehow. When a piece of code relies on an interface, at code review time, the interface relieves the developer from the burden of mastering every details of the implementation(s) hided by the interface. This is in essence the Liskov Subsitution Principle.

NDepend provides 2 ways to asses if and where your program should use more abstractions. First NDepend proposes the Robert C Martin metrics about Abstractness vs Instability. The idea is that the more a code element of a program is popular, the more it should be abstract. Or in other words, avoid depending too much directly on implementations, depend on abstractions instead. By popular code element I mean an assembly (but the idea works also for namespaces and types) that is massively used by other assemblies of the program. Abstractness of a group of types (like an assembly) is the ratio NbAbstractTypes / NbTotalTypes. There are debates about how these metrics should be computed and we plan in the future to make Robert C Martin's metrics more flexible but still, you get the point, it is not a good idea to have concrete types very popular in your code base. This provokes some Zones of Pains in your program, where changing the implementations can potentially affect a large portion of the program. And implementations are known to evolve more often than abstractions.

The second way to assess if more abstractions is needed is to rely on the Level metric. I won’t detail here this metric and its usage because I have already done it in the post Layering, the Level metric and the Discourse of Method. The idea here is that using more interfaces decrease the overall Level value of code elements (classes/namespaces). Thus, if many classes and namespaces have a high Level value (> 12), it means that you have a long stack of concrete layers sitting above each others. Introducing abstractions is then a good idea to split such long stack and benefit from the interface simplification describe above.

Fighting Fabricated Complexity with Immutability

A common source of fabricated complexity is mutable states. The human brain is not properly wired to anticipate what is really happening at run-time in a program. While reviewing a class, it is hard to imagine how many instances will simultaneously exists at runtime and how the states of each these instances will evolve over time. This is actually THE major source of problems when dealing with a multi-threaded program. If a class is immutable (meaning if the states of all its instances objects don’t change at runtime once the constructor is called) its runtime behavior immediately becomes much easier to grasp and as a bonus, one doesn’t need to synchronize access to immutable objects. For more information, I wrote about the benefits of immutable types and how NDepend can help in statically verifying immutability.

Fighting Fabricated Complexity with minimal Coupling

When trying to re-engineer/understand/refactor a large piece of code (I mean something made of dozens of classes like a big namespaces or an assembly), the difficulty is directly proportional to the coupling between considered code elements. Both these following graphs are made of dependencies between 23 classes, one with 53 edges and the other one with 175 edges: which one would you prefer to deal with?

While clear componentization is certainly the best way to fight against entangled/spaghetti code, keep in mind that using abstractions is also a good way to limit the over coupling-overhead. Indeed, if an interface has N implementations, then relying only on the interface is virtually like depending on the N underlying classes, except that from the static dependency point of view, you actually rely on only one type.

Could Fabricated Complexity be Measured?

There are plenty of other sources of fabricated complexity and I estimate that the 4 quoted ones are certainly the big 4 culprits. One could arguably appends also the ratio of code coverage + the number of automatic tests in the list, since a clean tests suite certainly forces the code to be better designed, simpler and more maintainable.

Interestingly enough, all these potential sources of problem can be controlled through NDepend. Could we find a formula that might process all these data to finally spit a number (a score) to measure the Fabricated Complexity? This approach is implemented in tools like Struture101 in the Java world that comes with a dedicated XS metric to measure fabricated complexity. Another example is the maintainability index range in VisualStudio that spits a number in the range 0-100. This number is linearly computed from potential source of fabricated complexity such as NbLinesOfCode, CyclomaticComplexity… and being lower than 20 is bad.

Such metrics are certainly interesting but I have a few caveats. The inventors of these formulas claim that in a certain range of value the code is crappy. The fact is that the value spitted doesn’t have any associated dimension. If a method has 120 lines of code and a Cyclomatic Complexity of 50, does it helps to add 120+50 and says that 170 something (something here is undetermined) is a bad thing? From the 2 values (120 Loc and 50 CC), isn’t it already obvious that the method should be splitted? And what about a method with 169 lines of code and no cyclomatic complexity (such as the massive Windows Form InitializeComponent() method)? I don’t think that such a method is as bad as the previous one and still they both measure 170 something.

Let's take for example the more and more popular CRAP metric that helps to detect crap code. The idea is that crap code is complex methods and poorly covered by tests.The proposed formula is

CRAP(m) = comp(m)^2 * (1 – cov(m)/100)^3 + comp(m) where comp(m) is the cyclomatic complexity of method m, and cov(m) is the test code coverage provided by automated tests.

By mixing it up, it is hard to understand the meaning of an estimation of the fabricated complexity value and worst, it makes harder to predict which refactoring will affect positively the value of the metric. This is the reason why so far in NDepend, we favored a more linearly independent/vectorial way to assess the exact causes of problems. To detect complex methods poorly covered with NDepend one just needs the following CQL rule and adjust thresholds at whim: Normal 0 21 false false false FR X-NONE X-NONE MicrosoftInternetExplorer4

WARN IF Count > 0 IN SELECT METHODS WHERE CyclomaticComplexity > 10 AND PercentageCoverage  < 90 ORDER BY NbLinesOfCode DESC

The CQL result panel provides all needed information to work with the crap methods detected and lets jump directly into VisualStudio to view and edit culprit methods:

Alternatively, the CQL flexibility lets correlate these metrics with other kind of relevant information, like for example detecting crap methods added recently to the code base (or also crap methods where ChangesObjectState, ChangesTypeState, CodeWasChanged, is declared inside or outside certains assemblies/nalmespaces/classes, not IsGeneratedByCompiler, is used by a certain code element, is using certain code elements, BecameObsolete...etc):

Normal 0 21 false false false FR X-NONE X-NONE MicrosoftInternetExplorer4

WARN IF Count > 0 IN SELECT METHODS WHERE WasAdded AND CyclomaticComplexity > 10 AND PercentageCoverage  < 90 ORDER BY NbLinesOfCode DESC

Also one can do experiments and compose metrics at whim:

Normal 0 21 false false false FR X-NONE X-NONE

This situation might evolve in the future. But so far I would prefer to avoid giving the NDepend user the feeling that fighting fabricated complexity is only a matter of reducing the value of a naked index (i.e an index without any dimension). Things must remain close to the code itself: if I have a method with 15 parameters it seems much more meaningful than having a class with a maintainability index equals to 18 or a CRAP value of 33.21.

Are you using these formulas? What’s your opinion on this?

Jeffery Olson (a fellow Eleutian guy) has put in some good effort to create a very readable, very clean MSpec DSL for Boo. You should give it a gander.

Scott Hanselman posted a podcast he & I recorded at NDC last week.  We nominally talked about applying conventions and opinionated software to .Net, then we went off on every possible tangent that presented itself.

Recently, there has been an effort launched called the “Anti-If Campaign” in which they deride the use of if statements and instead, focus on Object Oriented Principles in order to create more flexible designs.  Now certainly, I have a sympathetic ear to this cause as I’ve seen code that literally walks off the side of the screen due to nesting of if statements.  Pattern matching to me, especially at the top level of the function is actually quite beautiful in a way, such as the implementations in Haskell:

-- Haskell
lucas :: Int -> Integer
lucas 0 = 2
lucas 1 = 1
lucas n = lucas (n - 2) + lucas (n - 1)

And in Erlang, this also holds true:

% Erlang
-module(lucascalc).
-export([lucas/1]).

lucas(0) -> 2;
lucas(1) -> 1;
lucas(N) -> lucas (N - 2) + lucas (N - 1).

Simple, easy to understand and best of all, no if statements.  But, instead of focus on this debate, I’d like to propose another which strikes closer to this functional programmers heart, the “Anti-For Campaign”.  This is simply to say that we should create and use composable functions instead of explicit for loops.  This is actually an old post I had written months ago and until now had been unfinished, but now with some inspiration, it’ll finally be done.

What and Why?

Before you throw all sorts of questions asking what and why, let me instead ask a question.  When you’re writing a loop, ask yourself the question, “What am I accomplishing in this loop?”  Chances are, it might be one of the following:

• Query (Map, Filter, etc)
• Aggregation (Sum, Count, etc)
• Perform some side effect

If you’re doing more than one of those in a single loop, then well, you’re probably doing too much.  In fact, Martin Fowler’s Refactoring site has a refactoring called “Split Loop” which would solve that issue.  It is better for future refactorings and readability if we keep those loops pointed to do one thing, and one thing only.  Better yet, we could rid ourselves of that loop altogether, and that’s what we’ll focus on here.

Looking at first two bullet points, you’ll notice most of LINQ is indeed built around those two to be able to query data as well as aggregate.  The final bullet point, we perform some sort of side effect, perhaps writing to a file, printing to the console, or even perhaps sending messages. 

So, what’s my problem with them?  My problem is that it focuses more on the How instead of the What.  Let’s look at a quick example down below of what I mean.  First, we’ll attempt to find all prime numbers under 100 using C# as an example language.  First in the How:

static bool IsPrime(int i) {
    var lim = Math.Sqrt(i);
    
    Func<int, bool> check = null;
    check = j =>
        j > lim || (i % j != 0 && check(j + 1);
        
    return check(2);
}

static void Main(string[] args) {
    var numbers = Enumerable.Range(1, 100);
    var output = new List<int>();

    foreach(var number in numbers)
        if(IsPrime(number)) output.Add(number);
}

The problem is that if we then want to compose another operation, well, it’s really hard to do inside of these for loops.  We’re too focused on the how at this point.  Instead, getting to know generics and lazy evaluation, in .NET 2.0 and beyond, we were able to write generic functions to take advantage of some functional constructs.  This will give us a more declarative style that we can now focus on the what.

static IEnumerable<T> Filter<T>(
    this IEnumerable<T> items,
    Func<T, bool> predicate) {
    
    foreach(var item in items)
        if(predicate(item))
            yield return item;
}

static void Main(string[] args) {
    var numbers = Enumerable.Range(1, 100);
    
    var primes = items.Filter(x => IsPrime(x));
}

Of course we realize that LINQ already has such constructs built in, so we could rewrite the entire code above in just one statement.

var primes = Enumerable.Range(1, 100)
    .Where(x => IsPrime(x));

And what’s better is that it is composable that we could do other operations such as aggregations (sum, count, etc) without much additional code:

var primesCount = Enumerable.Range(1, 100)
    .Where(x => IsPrime(x))
    .Count();

And there you have it.  We not only have a query, but also an aggregation.  Try doing that with those non-composable loops!  Justin Etheredge has a nice writeup as well recently on the subject.

Coping Strategies

Functional languages tend to deemphasize the use of such constructs.  In fact, Haskell has neither a for loop nor a while loop, and languages such as F# and OCaml have limited support for such constructs in terms of no break and continue.  We tend to look at those two in particular with suspicion due to the fact that it cannot return a value and instead it mutates state in some fashion.  With that in mind, how do we cope with the fact that those aren’t available to us?  Above I showed a basic concept of a filter instead of an explicit loop, but what about some other considerations?

Some things we might want to consider with some links to some of my previous posts on the subject:

• Explicit Recursion
• Transforming each item (Map/Select & SelectMany in LINQ)
• Selecting items (Filter/Where in LINQ)
• Aggregating Data (Folds/Aggregate in LINQ)

In fact, many times that people could consider using explicit recursion could instead use a fold to aggregate the data which then cuts out the issue of tail call optimization.  By truly understanding the goals of LINQ as well as the concepts of functional programming, we can realize that most of the looping that we do can indeed be replaced by the above, outside of side effects of course.

Now What About Those Side Effects?

Is there a place where we draw the line and say that explicit loops are ok?  Eric Lippert was recently asked about the reasoning of the lack of the ForEach extension method on IEnumerable<T>.  His response was that he was philosophically opposed to such an endeavor as the whole approach is to cause a side effect.  As IEnumerable<T> collections are immutable, he doesn’t believe it makes as much sense because you wouldn’t be side effecting the collection itself.  Not only that, but introducing closures can complicate object lifetimes and all sorts of potential reference issues.

What about me?  I understand his concerns, and in C#, I can certainly see where he is coming from.  However, in F# we have such constructs readily available to us in the iter and iteri functions as shown below in F# Interactive:

> let flip f y x = f x y
- [1..10]
-   |> List.map((*) 2)
-   |> List.filter(flip (%) 3 >> (=) 0)
-   |> List.iter(printfn "%d");;
6
12
18
> [1..10]
-   |> List.map((*) 2)
-   |> List.filter(flip (%) 3 >> (=) 0)
-   |> List.iteri(printfn "%d\t%d");;
0       6
1       12
2       18

Outside of logging, writing to the console and such are rare in functional programming, so once again, I can certainly understand the concern.

Conclusion

I hope by going through some basic scenarios that we will indeed question our code the next time we see that we are writing that explicit loop with a for and a while.  With a tool chest filled with such functions as transforming every item, to filtering content, to aggregating data and so on, we can realize that we can create composable solutions instead of creating mutable collections or mutable variables and aggregating to them which are not as much.  So, come and join me in the “Anti-For Campaign”.

This post is over a week in the making. I attended AltNetBeers #9 last week in London which is making a quick rise in my list of top developer-related events, big or small. And that’s not just because I got one of the best compliments ever from someone who claims his non-IT friend reads a single IT-related blog (take that, Hanselman!). Anyway, it’s nice to know I’m reaching my target audience.

The event was not altogether removed from what Sebastien outlined at the one I attended last year which shows a degree of vision to which I only aspire. We spent a couple of hours talking on a topic of choice (as voted by attendees), then socialized until the place closed down at the unreasonable hour of 11pm. I’m told much merriment ensued afterward but, after posing for a picture, I made my way back to the hotel.

The topic at hand, and I’m paraphrasing: How do we become better developers today, and how do we create them tomorrow? Nice and subjectively vague, just how I like my open spaces. The runner up topic, brownfield applications, seemed suspiciously planted so I threw my weight behind the one we eventually chose.

Kinda concerned at how often this image is relevant to my posts

Conversation ran the usual gamut from apprenticeship programs (and thanks to Neil for the extensive historical thesis into the concept) to showing passion to defining professions to just going out there and doing it. My position through most of it was to focus more on the non-technical skillz, to the point where I think I pooh-poohed actual book learning more than I should have. If you decided, based on my advice, to drop out of university, I’d advise you to politely grovel to your dean to get back in. It’s still important. But take a psyche course or two when you go back.

Alas, we didn’t offer as much practical advice to the first part of the question as I would have liked so I’m expanding on the closing comment I made. It was essentially: Follow your instincts.

Now, there’s a danger with throwing out a broad statement like this because I made it in a very specific context. Namely, in a crowd of people who had taken time out of their lives to come to an IT-related event (albiet, one with more social activities than your average code camp; it *is* held in a pub after all). These are people who, at some level, have made a conscious decision to improve themselves on their own time. They could have had much better things to do but their instincts told them that this was an event that was more important than attending the new West End show, Deliverance: The Musical. (Side note: I know I just made that up now but MAN, the Duellin’ Banjoes scene would rock live; I suspect they’d lose a lot of people at intermission though.)

A lot of people have questions like, well, how do I make myself a better developer. The fact that they are asking those questions is the first step. The next step is to trust that they can muddle their way through their own answer. Because as we proved last Wednesday night, the same answer may not work for everybody. Me? I’m not much of a book learner. Nor, ironically, do I get much out of scanning blog posts (other than creating my own internal Google index so I know where to look later when the topic becomes relevant to me).

It’s dead simple to make yourself a better developer nowadays. Resources are plentiful and mostly free. And the ones you pay for, like conferences and training courses, are easy enough to rank with a little research (e.g. JP’s Nothin’ But .NET, Oredev, NDC). All it takes is a focused effort to make yourself better and to put some thought into how you go about it. You alone know which bloggers out there speak the same language you do. And you alone know which technologies will help you in your daily life and which ones interest you. Yes, SharePoint developers are in high demand but is it a product you want to learn and spend eight hours a day working with? (That’s not meant to be facetious actually, stop laughing.)

There will be much second-guessing and wondering which way you should go. And that’s fine. I won’t pretend I’ve made the right decisions all the time but like the software we’re supposed to be writing, most decisions are reversible. Humming or hawing about a contract in Dubai? Go for it. If you don’t like it, go home when it’s done. Worried about being away from your family that long? Well, you have some soul-searching to do but once you’ve made the decision, move on to the next one.

Was it the right decision? In my experience, unless you have direct and obvious evidence to the contrary, the answer is always yes.

Like I said, the topic is generic enough that pretty much everyone’s opinion will be valid, likely because it has worked for them personally. The underlying message I want to get across isn’t so much how to improve yourself (because you already know how to do it), but rather, if you’re in the London area, AltNetBeers is not to be missed. Many thanks to Neil, Toby, Scott, Andrew, Chris (both of them), Paul, Christian, and Lorenzo for the conversations and, of course, to the incredibly humble and reticent Sebastien Lambla for organizing and chairing the event, and also to Neil Robbins for the geek lunch two days later.

Kyle the Well-Shod

Hi there.

I got this question from a fellow named Martin today:

Real simple ... most of the discussion on DDD surrounds the design in terms of the ubiquitous language.

What if you are developing a product that is designed to be a tool for a certain industry, say marketing, but the processes and terminology all differ according to the marketing company that is in discussion.

To be specific, one company may use the term Client and the other Customer?

How do you accommodate the difference in terms?  Do you code to a common denominator or do you just pick one and go?

Great question; in product development we're often trying to make a general solution that solves the common problems of customers that have different languages. There's no way around that.

First, find those elements that are universal. Question if there's really a need for all of this customization or if your clients are OK with calling a client a customer. Having a ubiquitous language that can be used end-to-end, from developer to end user, is extremely valuable and kind of the original point.

For variable terms you should, in my estimation and experience, form an internal ubiquitous language that's adhered to across product management and software development. In cases like this, you'll can potentially also apply DDD guidance to a generic subdomain that handles terminology. This domain usually goes by the name "localization," only slightly extended to allow clients to change terminology. This is exactly how we handle the differences in terminology (silly) between agile methods like Scrum and XP in our product at VersionOne. We simply have a set of components that handle the change between backlog item and story, iteration and sprint, etc.

When I was reading through Real World Haskell, I was struck several times by the mention of providing safe function alternatives.  The idea is to provide a function that in all cases returns a value as well as the one which is meant to accept valid input and throw exceptions should that contract be violated.  There is a real performance consideration to be taken into account as a function which repeatedly throws exceptions as logic will tend to overwhelm a system and slow it down significantly.  Imagine if you will an application which reads a large directory to check each file for an X509 certificate, whether it has one or not, and it throws an exception if one is not present.  The problem of course is there is no way to determine whether a file was signed at the time using the .NET class without resorting to P/Invoke (my favorite).

Recently on Twitter, there was talk finally of the inclusion of Guid.TryParse in .NET 4.0, yet somehow, Enum.TryParse has not been addressed yet (gee, we’re consistent here).  The idea is to provide a way of determining whether the string is a Guid format without throwing an exception, and instead returning a flag indicating success and an out parameter with the value which is set to the value if there is success, else the default value.  Before we dig any further, let’s look at some terminology.

Partial Versus Total Functions

To get some terminology straight, let’s talk about partial functions versus total functions.  Partial functions are those functions which only return values for a defined subset of valid inputs, as throwing an exception is not considered a return value.  On the other hand, functions that return valid results over the entire input are considered to be total functions.  Let’s look at a quick example of what that means in F# interactive, the first being a partial function and the latter being a total function.

> open System.Linq;;
> ([] : int list).First();;
System.InvalidOperationException: Sequence contains no elements
   at System.Linq.Enumerable.First[TSource](IEnumerable`1 source)
   at <StartupCode$FSI_0003>.$FSI_0003.main@()
stopped due to error
> ([] : int list).FirstOrDefault();;
val it : int = 0

We see by the above that the First() function is a partial function because it only has defined behavior for non-empty collections, whereas FirstOrDefault() has defined behavior for empty lists and returns the default value for the given generic type.  When we’re writing our code, we better know which one we’re dealing with as default values can cause unintended consequences.  Without proper tests around these calls, this can be hard to manage. 

Mitigating these partial functions with calls to such things as the Count() extension method is not necessarily the answer either.  When dealing with non-lazy lists such as an array or a List<T>, then such things as the number of items in our collection is well known and part of the state.  In the case of the lazy sequence, this is not the case, so any call to Count() would cause an evaluation of all items in the list, which could lead to unintended consequences.  So, if we’re chaining together 4 partial functions together, we could be doing something wrong.  All in all, total functions should be preferred usage, but of course within reason.

Can We Do Better Than Try?

Going back to one of the original points, can we do better than the standard Try (Parse/GetValue/etc) pattern that is pervasive throughout .NET code?  After all, it tries to do two things at once, and the success flag can be missed if one so chose.   F# decided to take a slightly different angle to this problem by returning the success flag and the value as a pair tuple.  Such an example would be like this memoize function below:

open System.Collections.Generic

let memoize f = 
  let t = new Dictionary<_,_>()
  fun n ->
    match t.TryGetValue(n) with
    | (true , value) -> value
    | (false, _    ) -> 
        let res = f n
        t.Add(n, res)
        res

What we’re doing above is calling the TryGetValue method which takes a key and returns a success flag as well as the out parameter for the value (set if there is success).  F# automatically creates a pair tuple for us which we can then pattern match against to take appropriate action.  Unfortunately, for this go around at least, C# does not treat tuples as first class data types, so a solution like this exclusive to F# at this time.  Overall, a much cleaner design, but maybe there’s another option?

Maybe there’s an Option…

Instead of dealing with the clumsiness of the above code, I would much rather solve this in a clean way using the F# Option type.  This allows us to definitively specify whether we have a value or not.  Oh great you say, another version of null.  Not quite, as sometimes null can be a value…  More to the point, we can have a universal solution to having a value without having to resort to the Nullable<T> or reference type null-ness nonsense.  Simply put, the Option type is no more than the following:

type Option<'a> =
  | Some of 'a
  | None

You’ll notice two constructors for the Option type, the Some which takes a value, and the None which takes no parameters.  Now, using this to extend the Dictionary class much like the F# Map class, we can add a function which returns an Option type whether we have a value or not.

type System.Collections.Generic.Dictionary<'k,'v> with
  member this.Lookup(k:'k) =
    match this.TryGetValue(k) with
    | (true , value) -> Some value
    | (false, _    ) -> None

Then we can rewrite our memoize function to utilize this in a cleaner fashion using the Option type for pattern matching purposes to show our true intentions of whether we have a value or not.

let memoize f = 
  let t = new Dictionary<_,_>()
  fun n ->
    match t.Lookup(n) with
    | Some v -> v
    | None   -> 
        let res = f n
        t.Add(n, res)
        res

Much cleaner and shows better intent on our part.

Conclusion

How far do we take this?  After all, making full functions everywhere sometimes isn’t feasible as we want to constrain our system.  Adding both partial and full implementations might just clutter up our entire API.  With this comes balance, but there are certainly places where this comes into play such as parsing values, querying values from collections and so on.  Could Code Contracts in .NET 4.0 help us here in terms of statically verifying that we’re not failing on a chained partial function?  We’re not quite finished here with this discussion as I’ve yet to cover the many values of null.