Rod Paddock

Can a Fish Use a Bicycle?

Rod Paddock [MVP] — Sat, 28 Nov 2009 14:15:00 GMT

Can a Fish Use a Bicycle?

This post will appear in the Jan/Dec issue of CODE Magazine….

A few months ago, my friend and CODE Magazine contributor Ted Neward posted a statement on Twitter repeating something said by Billy Hollis: “Agile is treating the symptoms, not the disease”. This flame bait drew a number of responses that Ted later replied to on his blog (http://blogs.tedneward.com/)

From Ted’s blog we learned the origins of Billy’s statement. Billy was lamenting the complexities of the “stack” of tools we use when developing applications in .NET today. The number of technologies we use is staggering: SQL, MVC, EF, LINQ, WPF, WCF, MEF... and so on. I agree-whole heartedly that the stack of tools is complex and very difficult for developers to keep a handle on.

The blog post described how Microsoft and other companies moved from selling to “long tail” i.e., small development shops, into the “enterprise” space. This shift from small to large has left small shops in a difficult situation. Ted mentions in his blog post that not long ago, it was possible to build software with one or two developers. He asks “What chance do these developers have today?” Then he throws out the non sequitur (and I paraphrase at a distance here) “Agile development does not solve this problem.” And you know what? He is right. Agile does not solve this problem because agile has nothing to do with it. This statement in its radical context shift made me think of this statement:

“A woman needs a man like a fish needs a bicycle.”

This is a statement made by Irina Dunn and popularized by Gloria Steinem during the women’s movement; basically, it means that women are not dependant on men. Just like women and men, agile development and toolsets are unrelated processes.

I am going to illustrate this point with the basic values of the Agile Manifesto (http://agilemanifesto.org/).

· Individuals and interactions over processes and tools

· Working software over comprehensive documentation

· Customer collaboration over contract negotiation

· Responding to change over following a plan

Where do you see any mention of specific tools? In fact, the first line states individuals and interactions OVER processes and tools.

Agile is all about customer collaboration and building software not toolsets or specific techniques. If you drill down into the principles, you will see things like this:

· Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

· Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

· The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

· At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

These are just four of twelve principles. You will not see any details about specific tools or techniques. It is all about philosophy and how development teams interact with customers and each other and how they strive to build quality software. I will continue with some of my own questions to further illustrate my point:

· Can I do Agile development using Access?

· Can I do Agile development using FileMaker Pro?

· Can I do Agile development using SharePoint?

· Can I do Agile development using Visual FoxPro?

· Can I do Agile development using an assembly language?

The answer to all of these questions is YES. You can do agile development in any tool you choose because it does not pre-suppose any specific tool or technique. Another very valid point that Billy makes is that there are a lot of agile zealots that say if you are not doing/using “insert nouveau idea here” you are not doing “it” right. Simply put, those zealots need to learn and implement a little pragmatism in their discussions of agile. Are some tools or techniques better suited to the agile style of development? Yes, there are better tools, but your definition of better may be radically different than mine.

In summary, agile does nothing to address the complexity of today’s modern toolset. We will always be faced with an ever-growing list of tools and techniques we use. Agile is a process, plain and simple. In my next editorial, I will illustrate some ideas you can use to make your life simpler in tackling the “stack.”

Comments Suspended

Rod Paddock [MVP] — Wed, 15 Jul 2009 16:04:00 GMT

Because of spam bot whackness I have turned off comments.

Move The Chair

Rod Paddock [MVP] — Fri, 03 Jul 2009 14:17:00 GMT

Last Sunday I took my family to Red Robin (a family style restaurant) after church. I requested a table with a high chair for my 3.5 yr old son. Upon bringing this high chair the hostess simply moved an adult chair to the side and put the high chair in its place. (See Image)

This is where the fun began…

For the next hour or so I watched countless customers, servers and managers limbo their way between these two chairs. A lot of them did it repeatedly, some even carrying trays full of drinks and/or food. Finally I provided evidence to to my family that I am 100% full on geek. I got excited about the insight that this little experience game me (to the point of taking pictures)

I was simply amazed that people would allow this “friction to continue” It was a simple problem in this case. Move the chair!

I find it interesting that it’s built into our DNA incur friction in our lives without really addressing it. In real life its called denial. I know the chair was not that big a deal but it did open my eyes to our basic nature. Our basic nature follows through to software development as well.

The example of the misplaced chair provides some interesting parallels when it comes to technical debt (which is a cause of software development friction) We all have technical debt in our applications and it’s in our nature to shimmy, shuffle, limbo, mambo, and shake around these issues whenever possible. It takes a constant vigilance to address technical debt. We must be active in our quest to remove technical debt. We must take the time to do two things:

Recognize the debt. In our case: The chair.
Remove the debt: In our case: Move the chair.

That’s it for this blog post. Just a simple observation brought to you by the staff and management of Red Robin :)

Silverlight Frankenstein

Rod Paddock [MVP] — Tue, 19 May 2009 15:40:00 GMT

For the last 6 months I have been working on a massive call center application that uses JavaScript heavily to provide a rich client experience. While working on this application a feature request was made that threw me for a loop. The requirement was that should the users browser die or lock up their session state would need to be recoverable, meaning when the browser was restarted we would need to recover state. With a little head scratching and soul searching I came up with this idea:

“Why not user Silverlight, Jquery, JSON and .NET Isolated Storage to persist client information on the browser. It’s Alive!!!!”

I then went to work to create this monster.

Constructing a Monster

1. Create a new Silverlight Application with the test site included. Initially this application will have a visual representation. You will be changing this application to a “headless” version later.

2. Add a new class to your application. Lets call that class Frank.

3. Add a reference (and appropriate using statement) to System.Windows.Browser to your Silverlight Application. System.Windows.Browser is the assembly used to integrate Silverlight with its host application.

4. Now add a function called wakeUp to your Frank class. Decorate wakeUp with the [ScriptableMember] attribute.

[ScriptableMember]
public string wakeUp(string input)
{
    return input + System.Guid.NewGuid().ToString();
}

5. The next step is to expose your Frank class to your browser by registering it it with the Silverlight plugin. Registering classes is done in the Page.xaml.cs file.

6. Open the Page.xaml.cs file and add a reference (and appropriate using statement) to the System.Windows.Browser assembly.

7. Modify the constructor of the Page class adding the following code:

public Page()
{
    InitializeComponent();
    HtmlPage.RegisterScriptableObject("frank",new Frank());
}

8. Now your Silverlight Frankenstein is exposed to your browser’s scripting code. Now lets go to the browser and wake up the monster.

Waking Up the Monster

Our preferred method of working with the DOM and JavaScript is to use jQuery. jQuery found at www.jquery.com is a very useful library that takes the pain away from working with client side applications.

Along with jQuery we use the JSON library to serialize and de-serialize JavaScript objects. the JSON library can be found at http://www.JSON.org/json2.js

Now lets go to work and call our monster from our JavaScript application.

Important Step:

Before you begin this exercise modify the <object> tag in the HTML page created when you started the silverlight application. You need to add an ID attribute to that tag. Your Silverlight tag should look like:

<object id="frank" data="data:application/x-silverlight-2," type="application/x-silverlight-2" width="100%" height="100%">
 <param name="source" value="ClientBin/SilverlightFrankenstein.xap"/>
 <param name="onerror" value="onSilverlightError" />
 <param name="background" value="blue" />
 <param name="minRuntimeVersion" value="2.0.31005.0" />
 <param name="autoUpgrade" value="true" />
 <a href="http://go.microsoft.com/fwlink/?LinkID=124807" style="text-decoration: none;">
    <img src="http://go.microsoft.com/fwlink/?LinkId=108181" alt="Get Microsoft Silverlight" style="border-style: none"/>
  </a>
 </object>

Now you can proceed to add the code to talk to your Silverlight application.

1. Add the latest and greatest version of jQuery to your web application.

2. Add the latest and greatest version of JSON to your web application.

3. Add a new Jscript file to your application. call it Frank.main.js. This file will contain your custom JavaScript code.

4. In the .html page generated for your Silverlight application add script references to jQuery, JSON and your Frank.main.js files.

5. Add a script tag to your html page to launch your jQuery application. Your code should look like the following snippet:

<script src="Scripts/jquery-1.2.6.js" type="text/javascript"></script>
<script src="Scripts/json2.js" type="text/javascript"></script>
<script src="Scripts/Frank.Main.js" type="text/javascript"></script>
<script type="text/javascript">
   $(document).ready(main);
</script>

5. Add a function called main to the Frank.main.js file.

6. Now you can add code to call your Frank class. The first step is to create a function that will return a reference to your referenced frank class. The following JavaScript function does just that:

function getFrank() {
  var returnValue = document.getElementById("frank").content.frank;
  return returnValue;
}

7. You need to carefully examine that function. It does the following:

a) Grabs a handle to the Silverlight control via document.getElementByID(“frank”)

b) Bores into the Silverlight control’s content property and grabs a reference to

the frank variable you referenced in the first exercise.

8. Now you can call your wakeUp() method as follows:

alert(getFrank().wakeUp("hello"));

Now For Something Useful

Now that you have a basic understanding of how to call C# code from your web pages you can open up your horizons and do something useful.

Using the Visual Studio Debugger

One of the more interesting things I came up with was integrating the Visual Studio debugger into my JavaScript applications. Rather than debugging variables via the good old alert() function why not spit information to the debugger?

The following code blocks show how you can spit data from your JavaScript application into the Visual Studio debugger.

[ScriptableMember]
public void writeline(string output)
{
 System.Diagnostics.Debug.WriteLine(output);
}

function debuggerTest() {
    for (i=0; i<100; i++) {
        writeline(i);
    }
}

Calculating Ages

Lets face it JavaScript date support,especially when it comes to date calculations sucks.Date support in the .NET framework is much better. Especially when you use the “wrapped up” code found in the Microsoft.VisualBasic namespace.

The following code calculates a person’s age (in years) using the DateDiff() function found in the Microsoft.VisualBasic assembly. To use this code add a reference to the Microsoft.VisualBasic assembly to your Silverlight Application.

The following code snippets show how to calculate an age from a given date:

[ScriptableMember]
public long getAge(string inputDate)
{
    long returnValue = -1;
    try
    {
        var dateValue = System.Convert.ToDateTime(inputDate);
        returnValue = Microsoft.VisualBasic.DateAndTime.DateDiff(Microsoft.VisualBasic.DateInterval.Year,
                 dateValue,
                 System.DateTime.Now,
                 Microsoft.VisualBasic.FirstDayOfWeek.System,
                 Microsoft.VisualBasic.FirstWeekOfYear.System);
    }
    catch (Exception)
    {
        returnValue = -2;
    }
    return returnValue;
}

alert(getFrank().getAge("01/01/1981"));c

Final Frankenstein

The original purpose of this exercise is to be able to save and restore form state from a client browser session. This is where a good plan comes together. Saving state involves a number of moving parts.

0. Create a simple HTML form with some fields.

1. Create an array of the objects representing the data from a set of web input fields.

2. Serialize the array into a string using JSON (JavaScript Object Notation)

3. Pass that string to Silverlight and save it to disk using Silverlight’s Isolated storage mechanism.

4. Retrieve the JSON data from Isolated Storage and de-serialize it back to an array.

5. Re-populate the controls on the form from the array.

Simple HTML

This code is about the simplest HTML form you could create. Two text fields and two buttons. The two buttons will be used to save and restore state (we save and restore state via a timer).

<label>Last Name</label><input type="text" id="txtLastname"/><br />
<label>First Name</label><input type="text" id="txtFirstName"/><br />
    
<input id="cmdSaveState" type="button" value="Save State" /> 
<input id="cmdRestoreState" type="button" value="Restore State" />

The following code wires up the buttons to functions to save and restore state:

$("#cmdRestoreState").click(restoreState);
$("#cmdSaveState").click(saveState);

Gathering and Restoring Data

The next set of code creates an array of objects representing page content. It also serializes the content to a JSON string using the JSON library added to your project earlier.

function getContentAsJSON() {
    var inputControls = $(":input").not(":button").not(".ignorestate");
    var controlData = Array();
    for (i = 0; i < inputControls.length; i++) {
        controlData.push({ ID: inputControls.id, Value: inputControls.value, Type: inputControls.type });
    }
    var jsondata = JSON.stringify(controlData);
    return jsondata;
}

Note: this code gathers up data for all controls except for buttons and ones that have the .ignorestate style attached to them.

The next set of code is the converse of the prior. It takes a JSON string and de-serializes it back to an array and repopulates control values based on its content.

function restoreControlState(controlDataString) {
    if (controlDataString.length > 0) {
        var controlData = JSON.parse(controlDataString);
        for (i = 0; i < controlData.length; i++) {
            $("#" + controlData.ID).val(controlData.Value);
        }
    }
}

Saving and Restoring State

Now you need a mechanism for storing and retrieving the data to client workstation. To do this, you will use the Isolated Storage mechanisms contained in the Silverlight framework. Isolated storage is a data storage mechanism available in both standard .NET and Silverlight applications. Isolated storage is a set of APIs used to read and write files to a virtual file system managed by the .NET framework. Silverlight applications have a default 1MB (yes 1MB) of storage available to them for reading and writing files. To use the following code examples import System.IO and System.IO.IsolatedStorage to your applications:

The first is used to preserve state. It does the following:

1. Opens the “virtual file system” presented by IsolatedStorage

2. Checks to see if the “sessioncontents.txt” file exists . If so it deletes it.

3. Creates a file stream to write content to.

4. Uses a standard StreamWriter to push content to the file.

5. Flushes the buffer and closes the file handle

[ScriptableMember]
public void saveState(string stateString)
{
    //yes this is a MAGIC string for a demo
    var contentFileName = "sessionContents.txt";
    var userStore = IsolatedStorageFile.GetUserStoreForApplication();

    //whack the file if it exists
    if (userStore.FileExists(contentFileName))
    {
        userStore.DeleteFile(contentFileName);
    }
    var file = userStore.CreateFile(contentFileName);
    var writer = new StreamWriter(file);
    writer.WriteLine(stateString);
    writer.Flush();
    writer.Close();
    file.Close();
}

The next step is to return the data from IsolatedStorage. It does the converse:

1. Opens the “virtual file system” presented by IsolatedStorage

2. Checks to see if the “sessioncontents.txt” file exists . If so it opens it.

3. Creates a file stream to read content from.

4. Uses a standard StreamReader to read content from the file.

5. Closes the file handle

6 Returns the string to the caller.

[ScriptableMember]
public string getState()
{
    var returnValue = "";
    //yes this is a MAGIC string for a demo
    var contentFileName = "sessionContents.txt";
    var userStore = IsolatedStorageFile.GetUserStoreForApplication();

    //whack the file if it exists
    if (userStore.FileExists(contentFileName))
    {
        var file = userStore.OpenFile(contentFileName,FileMode.Open,FileAccess.Read );
        var reader = new StreamReader(file);
        returnValue = reader.ReadToEnd();
        reader.Close();
    }

    return returnValue;
}

Now all you need is the small glue code to put it all together:

function saveState() {
    getFrank().saveState(getContentAsJSON());
}

function restoreState() {
    restoreControlState(getFrank().getState());
}

That’s all you need to save and restore state from a HTML form

Removing the Head

Finally you can go “remove the head” from your Silverlight application. this is rather simple. To remove the head from your Silverlight application you simply go to the Page.xaml file and change the size attributes to zero. Now you have a “headless” frankenstein for your fun and enjoyment.

<UserControl x:Class="SilverlightFrankenstein.Page"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation" 
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" 
    Width="0" Height="0">
    <Grid x:Name="LayoutRoot" Background="White">

    </Grid>
</UserControl>

I hope you have enjoyed this post. You can find this code at www.dashpoint.com/downloads/silverlightfrankenstein.zip

10,000 Hours to Graceland

Rod Paddock [MVP] — Sun, 26 Apr 2009 20:29:00 GMT

Lets take a look at some old code:

I know what you may be thinking. Why the heck are we looking at 20 yr old procedural code. Well...what you just witnessed was the first paid code written by a greenhorn application developer. For the curious that code was written in dBaseIII+ hosted on a PC Limited (now Dell) server running Novell 2.0a.

When you take a look at that code and the application in its entirety you will find an application riddled with poor coding constructs, amateurish coding principles (use of publics for example) and an application that misses fundamental features available to it from its core language (xBase). Basically what you will find code written by a wet behind the ears application developer, fresh out of college, and unleashed on a property management company to write an application to be known as : "The Hot Tub Management System".

When looking at that code you will see the type of code written by a junior application developer. That junior developer started his career with with 1.5 years of college education and a burning desire to write code for a living.

The learning didn't stop there. The programmer that wrote that code went on to gain more education and more importantly more application development experience. That programmer moved from the property management company to a large hotel/resort where he was applied his craft for 18 months as an application developer/network admin. Eventually that programmer left small town Bend, Oregon for the bright lights and big city life in Seattle, Washington.

It's in Seattle where that developer went from being the lone wolf developer in a small company to to being a member of a team of developers in a somewhat larger company. When he was hired the developer told he would be joining a group of the best FoxPro developers assembled in the Seattle area and would need to rise to the challenge. That is exactly what he did.

It was at that company that the developer really came into his own. The developers on that team were a competitive and very talented group of developers. The primary concern of that group of developers was to create the correct solutions for the company and do it right the 1st time. By the time the developer left that company he was writing some of the best code of his life, creating high performance and feature rich applications and finally re-caught the writing bug and was drafting articles for the magazine Data Based Advisor. At that point the developer had been writing code (including college) for nearly 5 years. Which leads me back to the title of my post: 10,000 hours to Graceland.

The normal employee works approximately 2,000 hours a year (40 hours * 50 weeks). When you factor in 5 years of work that comes out to 10,000 hours. This is the minimum amount of time it takes to master a craft.

This epiphany didn't come straight from my gray matter it came from a book I picked up earlier this month called Outliers, by Malcolm Gladwell. In general I am not a big fan of overhyped books like Outliers. I couldn't have been more wrong about this book. I found that this book was much like Freakanomics which challenged basic assumptions we have as a society.

Malcolm starts this book with an analysis of a championship team of young Canadian hockey players. People were attempting to figure out why some hockey players seem to excel while others to not. When analyzed a startling conclusion was made: Kids with birth dates falling in a specific range seemed to play better than the others. As Mars Blackmon would say "Gotta be the stars!" People born under a specific sign are better hockey players right ? Nope... Basically it boiled down to this. The kids falling in a certain birthday range were allowed to start playing hockey earlier than the other kids. Basically they got a head start by 6months to a year. Now these kids did have talent...but talent nurtured earlier has a better chance of success right?

This was making sense. It got clearer as the book progressed. Gladwell wrote about two of the most regarded software developers of our day. The two Bills. Bill Gates and Bill Joy. One thing that Bill Gates and Bill Joy share was the fact that they had access to computers at a very early age and spent countless waking hours working on these machines. By the time real opportunities presented themselves in there 20's both of these me had the requisite 10,000 hours under there belts and were able to apply their craft to great advantage. Once again it was not just the amount of time they spent, they did have talent by the truck load, but that talent was nurtured from an early age to yield some some real talent.

With all the talk of Software Craftsmanship these days we must never forget that it takes a lot of time to master new skills. I moved from Visual FoxPro to Visual Studio .NET in 2001 it took me a few years to really master a new way of development. It didn't take me quite 10,000 hours, as I had a lot of experience in object oriented code and frameworks, but it did take a certain amount of time in the trenhes to master this new set of concepts.

What is true in Software Craftsmanship is true life's other endeavors as well. There is an old saying in poker:

"It takes 15 minutes to learn the game and a lifetime to master"

I completely relate to this statement. Around 10 years ago I sat down for my first real poker game in a casino. I was scared out of my wits but I held my own. I lost of course but I had caught the poker bug. I spent literally hundreds of hours grinding learning how to play this game effectively. Do I consider myself a master? Heck now I am no Daniel Negraneu. But I can definitely hold my own with the best of em in the casinos in Vegas. I feel this way now when it comes to Agile and XP development processes.

In 2000 I found a book called Extreme Programming Explained. This book seemed to be what I had been looking for when it came to developing software. At its core it talked about working directly with customers to develop exactly what they wanted correctly the 1st time. Kinda like that team I was on early in my career. Over the ensuing years I spent time attempting to implement these with customers and in my own practices. It seemed to me like I had started over once again as a web behind the ears programmer. I didn't have a lot of luck getting clients to adopt these principles but I did keep growing my own personal knowledge. Flash forward to October 2007 in Austin Texas. I attended my first ALT NET Conference it was here that I spent time learning more about implementing agile practices in my company and with my clients. I was going to master this if it killed me. Over the last year and a half I have been putting in my time towards my 10,000 hours to Graceland. I have been fortunate that I have the opportunity to be mentored by, dialog with and become friends with a number of agile developers I respect. It's this dialog that has been most important to my mastery of this stuff. Like poker I am not a master but I am beginning to "hold my own" with these folks. I am glad they have spent the time sharing their 10,000 hours to Graceland with me.

One other thing that did for me was to give me another level of perspective when dealing with people new to agile development practices. I am an agile teacher and need to take into account that it takes a number of hours practicing thus stuff before you can see the real benefit. A lot of developers will take a surface glance at a subject like Test Driven Design and rule it out as a a waste of time. We all have heard the question "Why do I need to waste my time with this?". We need to address these questions with patience and understanding. We need to relate back to ourselves as greenhorn wet behind the ears developers. We need to remember that it took us 10,000 hours to get here and it will take people new to agile 10,000 hours to find their Graceland.

jQuery 101 UG Samples and Slides

Rod Paddock [MVP] — Tue, 07 Apr 2009 13:48:28 GMT

Here is the slide deck and sample code from my jQuery 101 user group talks this week.Thanks you to everyone that attended my user group chats this week.

jQuery 101 Slide Deck

jQuery 101 Sample Code

Come Learn jQuery

Rod Paddock [MVP] — Mon, 30 Mar 2009 13:19:00 GMT

After a brief stop in Houston for the Alt Net Houston Open Space and another brief stop in Seattle for Emerald City ComicCon I will begin my trek north for another Canadian user group tour.

Topic: Using jQuery with ASP.NET

Web 2.0 is here to stay. jQuery is a JavaScript library that abstracts away all of the gory details of working with JavaScript for web applications. This session will demonstrate how to add jQuery to your ASP.NET applications today. This session will focus on proper uses of jQuery including how to organize your JavaScript code, how to use selectors in jQuery and how to manipulate your web content dynamically. Along with uses of the standard jQuery library, time will also be spent exploring some of the most useful jQuery plugs-ins.

I will be at the following user groups

April 6th .Net User Group of British Columbia (Vancouver BC)

April 7th Edmonton .NET User Group

April 8th Regina .NET User Group

April 9th Winnipeg .NET User Group

I am looking forward to seeing old friends and making new ones on this tour. Hope to see you there!

Underscoring A Point

Rod Paddock [MVP] — Sat, 28 Mar 2009 12:43:06 GMT

This last week I took a great class on TDD and BDD given by Scott Bellware. During the session he showed us a cool tool called AutoHotKey. AutoHotKey is a tool that allows you to control your keyboard behaviors. For instance you can use AutoHotKey to substitute the underscore character for a space when typing in a test name

I had a discussion with my co-worker Greg Lawrence about using underscores vs. CamelCase for our test naming conventions. I have always liked underscores in test names. Test names are like sentences and should be read as such. To drive this point home (and to demonstrate it for an underscore bigot :) I present the following:

Original Test

We hold these truths to be self-evident, that all men are created equal,
that they are endowed by their Creator with certain unalienable Rights, that among these are Life, Liberty and the pursuit of Happiness.

CamelCase

WeHoldTheseTruthsToBeSelfEvidentThatAllMenAreCreatedEqualThatTheyAreEndowed…WithCertainUnalienableRightsThatAmongTheseAreLifeLibertyAndThePursuitOfHappiness.

Underscored

We_hold_these_truths_to_be_self_evident_that_all_men_are_created_equal_that_they_are_endowed_…with_certain_unalienable_rights_that_are_among_these_are_life_liberty_and_the_pursuit_of_happiness

BDD CamelCase Notation

WhenMenAreEndowedWithUnalienableRights

ShouldIncludeLife
ShouldIncludeLiberty
ShouldIncludeHappiness

BDD Notation (Underscored)

When_Men_Are_Endowed_With_Unalienable_Rights

Should_Include_Life
Should_Include_Liberty
Should_Include_Happiness

As is self evident (pun intended) the underscored version is much easier to understand and read. When you generate documentation from your BDD specs which one make more sense? Which will is easier to use by your customers? Which ones can you substitute underscores for spaces when printing documentation? Its pretty obvious which one to choose.

Just a small thought on this march morning.

What's in a Job Title

Rod Paddock [MVP] — Sun, 04 Jan 2009 21:17:58 GMT

One of the most frequently asked for attributes on a credit or job application is Job Title. As an independent business owner I have answered this question in multiple ways:

Programmer
Software Developer
Software Engineer
Architect
CTO
Coder
President
Jedi Master
Etc...

The scary thing about this is there is no real way for a potential employer, banker or client to certify what I say is legit. This is a fundamental issue with our industry. There is no way to verify whether a person is a quality software engineer or simply a charlatan. And for the record there are more of the latter than the former.

For the most part programmers like to think of software development as engineering. We like to compare ourselves to engineers with a capital E, construction managers, manufacturing process managers, etc. But do we really practice engineering ? What is the bar for calling yourself a software engineer?

I know for a fact that if I need an expert licensed plumber, electrician or ironworker I can take myself down to the local union hall and find someone that has gone from apprentice to journeyman over the course of many years. And thanks to my friend Eric Anderson I learned that there is such a thing as a Professional Engineer. Here's the list of requirements for Professional Engineers as taken from the Wikipedia

Graduate with a degree from an accredited four-year university program in engineering.
Complete a standard Fundamentals of Engineering (FE) written examination, which tests applicants on breadth of understanding of basic engineering principles, and optionally some elements of an engineering specialty. Completion of the first two steps typically qualifies for certification in the U.S. as an Engineer-In-Training (EIT), sometimes also called an Engineer Intern (EI).^[5]
Accumulate a certain amount of engineering experience under the supervision of a P.E. In most states the requirement is four years, but in others the requirement is lower.
Complete a written Principles and Practice in Engineering ('PE') examination, testing the applicant's knowledge and skills in a chosen engineering discipline (mechanical, electrical, civil, for example), as well as engineering ethics.

For software engineering there is no similar accreditation process. Any Mort i.e (NOOB) with $500.00 and the ability to hit the keyboard with his/her forehead can become a software engineer overnight. "Yesterday I couldn't spell it. Now I are one". This has got to stop.

A few weeks ago I came across a commentary made by my friend Billy Hollis in Visual Studio Magazine Commentary: Is Agile Rock or Disco. In this commentary Billy mentions the following:

The Patterns and Practices group at Microsoft is heavily into agile development. Don Smith is in that group, and, at a conference last May, he said, "It's hard to do agile without really good developers." I've heard variations of the same thing said from others.

But not all developers are that good. Therefore, if Don is right, then agile isn't for everyone.

The second statement is the core problem we face. It raises the white flag of surrender to all the half ass coders out there. Why shouldn't we all strive to be good developers? Why is average acceptable? What does the author propose we do with the average programmer?

Because programmers are less than good (LTG) does he propose we have no process whatsoever? It's because of these LTG developers we need agile practices more not less. I want to point out something carefully at this point. It's agile practices... not agile! There is no push button template you can use and whoosh you are an agile coder or development shop. Nope agile is a set of principles and practices.

Adopting agile principles is not quick or easy. It is a lifetime commitment that takes lots of study, practice, mentoring, and continuous education and improvement. There are lots of new terms, concepts and practices to examine, consider and implement. Every company and situation is different and there are a number of useful practices to adopt.

Agile Adoption

In the last year we have moves very heavily from our old RAD way of doing development to a more agile one. Our goal in this process was to change how we design and develop software in a quality manner. These are the tools and practices we have adopted.

Shared State Source Control

The first practice we adopted was changing from a pessimistic style of source code control into a shared state source code control system. It made no sense to limit a source code file to a single developer (the default behavior of Visual Sourcesafe). We moved to Subversion. Subversion is an industry standard source code control system that's default behavior is multiple developers working on the same files and merging conflicts.

Automated Builds

Another principle of agile we have adopted is that of automated builds. Another terms for this is Continuous Integration. Whenever source code is committed to our repository an automated build fires off, checks out all the code, compiles it and runs a suite of automated tests. Our goal is to find whether or not our code is in a Red (bad) state or Green (good) state.

Test Driven Development and Design

Of all practices in agile I find this one to provide the most value. Any software you build should be accompanied by a suite of tests to verify its long term quality and integrity. The ultimate goal of TDD is a good design. While test driven development does not insure a good design your chances are better that your designs are more sound if they can be tested in an automated way.

I had a conversation with Scott Bellware and Chad Myers last night. We were discussing testing nomenclatures and Chad Developed this list of test types:

Proofs - Code performs as expected.
Examples of Client Access (API testing) - API Proving with tests.
Specification (BDD) - Documenting intent.
Contract Enforcement/Change Confidence - Regression testing

As you can see these nomenclatures provide you with a confidence that your code will work as expected, has its intent documented and allows for long term changeability inherent in all software development.

Pair Programming and Mentoring

Union plumbers and electricians go through extensive apprenticeships before they are accepted as journeymen. Software engineers should adopt similar practices. Pair programming facilitates knowledge transfer from master to apprentice, mentor to student and in lots of cases from student to mentor. In the last year I have spent a lot of time discussing these practices with a number of different people I consider my mentors. My mentor list includes Scott Bellware, Jeremy Miller, Chad Myers, Jeffery Palermo and numerous others. These practices are not always obvious or simple so it is a good idea to have a network of mentors to rely on.

Adoption of Dependency Patterns

One aspect of building testable systems is reduction of development friction. One way to introduce friction is to tightly couple processes together by allowing them to instantiate there own resources internally. It may seem strange but information hiding is not necessarily a good practice in the agile space. The concept behind dependency injection is this: if a component or method has a dependency on another class, then that dependency should be passed (injected) into the consumer. This give you untold flexibility when it comes to configuring and reconfiguring your systems.

Daily Stand Ups

In agile development a common set of practices revolve around daily developer meetings also known as stand ups. Every day developers stand in a circle, stands at there desks or in the hallway and talk about two items: what the did yesterday and what they plan on doing today. This has the benefit of keeping all developers in the loop and facilitating communication.

Small Iterations

We all know that estimating timelines on most software projects is a fools errand. Software estimates follow share a common trait with metrological predictions the shorter the span or time the more accurate the estimate is likely to be. Agile practices acknowledge this as a core principle. Your applications should be developed in short sprints. At the end of each sprint there should be "show and tell" for the client that demonstrates the deliverables for that sprint. After the sprint and accompanying demo the next sprint should be planned. And its rinse lather repeat from there.

Conclusion

While not definitive this list is a good start to moving you or your development team closer to being professional software engineers. When you are evaluating a client, consultancy or even a potential employer consider the practices discussed in the list.

Do you think your a great coder or do you think your development shop is top notch? I challenge you to prove it. Can you develop quality software that stands the test of time ? Do you have software you can modify with a high level of confidence that your modifications don't introduce software regressions? Can you repeat and improve on your successes? Do you have the processes in place that make this a possibility? When you can answer in the affirmative to some or all of these questions you are closer to being a true software engineering shop.

Twitter Conversations : First Refactoring

Rod Paddock [MVP] — Sat, 20 Dec 2008 04:29:00 GMT

Thanks for the feedback and encouragement so far. This project has been fun and I hope you keep enjoying it so far.

Refactoring Required

I knew going in that the first cut of this application needed some serious refactoring. The project had dependencies on internet connections , numerous classes contained redundant code and the user interface was lacking. In this first set of refactorings I hope to demonstrate how you can design your applications to be more testable and how to make the code cleaner at the same time. I will also attempt to make the UI better with my feeble UI skills.

First Refactoring: Extracting Interfaces

The first cut of the Twitter Conversations application took a dependency on being connected to the Internet for testing. In order to remove this dependency I decided to extract the an interface from the Twitter.Communications.TwitterRequest class. This process is simple when using a tool like ReSharper. I simply opened the class source code file, Right Clicked, Selected Refactor->Extract Interface from the popup menu. The following screen shows the Extract Interface dialog from ReSharper.

This process yielded the following interface

   1: Public interface ITwitterRequest   2:     Function GetTwitterRequest(ByVal URL As String) As String   3: end interface

After refactoring the interface from the Twitter.Communication.TwitterRequest class I created a new Twitter.Interfaces project. This project would all interfaces for this project including another new interface ITwitterCredentials. I created the ITwitterCredentials interface in order to provide a slightly cleaner design for creating instances of the TwitterRequest class.

   1: Public Interface ITwitterCredential   2:     Property UserName() As String   3:     Property Password() As String   4: End Interface

Once I created the ITwitterCredentials interface I implemented it on a new class Twitter.Communication.TwitterCredentials. I also needed to refactor the guts of the TwitterRequest class. The new code for this class now looks like:

   1: Public Class TwitterRequest   2:     Implements Twitter.Interfaces.ITwitterRequest   3:     Private _Credentials As Twitter.Interfaces.ITwitterCredential   4:     5:     Sub New(ByVal Credentials As Twitter.Interfaces.ITwitterCredential)   6:         Me._Credentials = Credentials   7:     End Sub   8:     9:     Public Function GetTwitterRequest(ByVal URL As String) As String Implements Twitter.Interfaces.ITwitterRequest.GetTwitterRequest  10:         Dim Credentials As New NetworkCredential(Me._Credentials.UserName, Me._Credentials.Password)  11:    12:         Dim Request As HttpWebRequest = HttpWebRequest.Create(URL)  13:         Request.Method = "POST"  14:         Request.Credentials = Credentials  15:    16:         Dim Response As WebResponse = Request.GetResponse  17:         Dim Reader As New StreamReader(Response.GetResponseStream)  18:         Dim Results As String = Reader.ReadToEnd  19:         Return Results  20:    21:     End Function  22: End Class

After extracting these interfaces I changed the constructors for Twitter.API.User and Twitter.Message.API to receive an ITwitterRequest rather than a concrete class.The following code demonstrates the constructor for the Twitter.API.User class.

   1: Public Class User   2:     3:     Dim Communicator As Twitter.Interfaces.ITwitterRequest = Nothing   4:     Dim JsonSerializer As New System.Web.Script.Serialization.JavaScriptSerializer   5:     6:     Sub New(ByVal TwitterCommunicator As Twitter.Interfaces.ITwitterRequest)   7:         Me.Communicator = TwitterCommunicator   8:     End Sub

Of course at this point all of our tests were broken but I waited to do a little more refactoring before fixing the tests.

Second Refactoring: Creating the Disconnected Twitter Service

The next step in the process was to create a disconnected version of Twitter. To accomplish this I took the following steps:

Extracted JSON strings from the various calls to Twitter and put them into our testing project as embedded resource files.
Added code to the testing project that could be used to extract the embedded text files
Implemented the ITwitterRequest interface on a new class called TwitterRequestMock.
Implemented the function GetTwitterRequest to return the appropriate text resource based on the URL.

The first item on the list was to extract the JSON data from actual calls to Twitter. To satisfy the unit tests I created three file extracts:

singleuser.txt - Contains the information for my user account
friends.txt – A list of people I follow
messages.txt – A list of twitter updates

All three of these files were marked as Embedded Resource files in the Visual Studio project.

The next step was to add code that would allow these files to be extracted from the testing assembly. the following code extracts resources from the currently running assembly (i.e. our unit tests)

   1: Imports System.IO   2: Imports System.Reflection   3:     4: Public Class ObjectMother   5:     Public Shared Function GetEmbeddedResource(ByVal ResourceName As String) As String   6:         Dim ResStream As Stream = System.Reflection.Assembly.GetExecutingAssembly.GetManifestResourceStream(ResourceName)   7:         Dim oStreamReader As New StreamReader(ResStream)   8:         Dim RetVal As String = oStreamReader.ReadToEnd   9:         Return RetVal  10:     End Function  11: End Class

Finally a new class TwitterRequestMock was created. The purpose of this class was to return the proper embedded resource files based on the URL passed to a request. The following code demonstrates the implementation of this:

   1: Public Class TwitterRequestMock   2:     Implements Twitter.Interfaces.ITwitterRequest   3:     4:     Private _Credentials As Twitter.Interfaces.ITwitterCredential   5:     6:     Sub New(ByVal Credentials As Twitter.Interfaces.ITwitterCredential)   7:         Me._Credentials = Credentials   8:     End Sub   9:    10:     Public Function GetTwitterRequest(ByVal URL As String) As String Implements Interfaces.ITwitterRequest.GetTwitterRequest  11:         Dim ReturnValue As String = ""  12:    13:         '-- urls we use so far  14:         'Const UserMessageURL As String = "http://twitter.com/statuses/user_timeline/<<USERID>>.json"  15:         'Const MyMessagesURL As String = "http://twitter.com/statuses/friends_timeline.json?count=200"  16:         'Const FriendsURL As String = "http://twitter.com/statuses/friends.json"  17:         'Const GetUserURL As String = "http://twitter.com/users/show/<<USERID>>.json"  18:         'Const CredentialURL As String = "http://twitter.com/account/verify_credentials.json"  19:    20:         If URL.Contains("verify_credentials") OrElse URL.Contains("users/show/") Then  21:             ReturnValue = ObjectMother.GetEmbeddedResource("Twitter.UnitTests.singleuser.txt")  22:         ElseIf URL.Contains("friends_timeline") OrElse URL.Contains("user_timeline") Then  23:             ReturnValue = ObjectMother.GetEmbeddedResource("Twitter.UnitTests.messages.txt")  24:         ElseIf URL.Contains("friends.json") Then  25:             ReturnValue = ObjectMother.GetEmbeddedResource("Twitter.UnitTests.friends.txt")  26:         End If  27:    28:         Return ReturnValue  29:     End Function  30: End Class

I discussed this method with Jeremy Miller and his comment was if possible the text for the JSON strings should be embedded into the unit test if at all possible. I didn’t follow this recommendation because the JSON returns strings are long and very ugly. What are your thoughts on this ?

Third Refactoring: Fixing the Unit Tests

Once this class was implemented I went in and fixed up the unit tests. This was actually pretty simple. Basically I refactored the constructor and <Setup()> of the UnitTests class.

   1: <TestFixture()> _   2: Public Class UnitTests   3:     4:     Dim UserAPI As Twitter.API.User = Nothing   5:     Dim MessageAPI As Twitter.API.Message = Nothing   6:     Dim Communicator As Twitter.Interfaces.ITwitterRequest = Nothing   7:     8:     <SetUp()> _   9:     Sub Setup()  10:         Me.Communicator = New TwitterRequestMock(New Twitter.Communication.TwitterCredentials)  11:    12:         Me.UserAPI = New Twitter.API.User(Me.Communicator)  13:         Me.MessageAPI = New Twitter.API.Message(Me.Communicator)  14:     End Sub

Now the tests for this application can be run in a disconnected mode.

Fourth Refactoring: Removing Redundant Code

The next step in this refactoring process was to clean up redundant code from the Twitter.API.User and Twitter.API.Message classes. In the first version each method was responsible for parsing data and creating collections of Messages and Users. The code started like the following snippet:

   1: Function GetUserMessages(ByVal UserID As String) As Twitter.Domain.Message()   2:     Dim ReturnMessages As New List(Of Twitter.Domain.Message)   3:     4:     Dim UserObjects As Object = JsonSerializer.DeserializeObject(Me.Communicator.GetTwitterRequest(UserMessageURL.Replace("<<USERID>>", UserID)))   5:     6:     For Each UserObject As Object In CType(UserObjects, Array)   7:         ReturnMessages.Add(New Twitter.Domain.Message With { _   8:                            .MessageID = UserObject("id"), _   9:                            .MessageDate = UserObject("created_at"), _  10:                            .MessageContent = UserObject("text"), _  11:                            .UserID = UserObject("user")("id"), _  12:                            .UserName = UserObject("user")("name"), _  13:                            .ImageURL = UserObject("user")("profile_image_url")})  14:     Next  15:     Return ReturnMessages.ToArray  16:    17: End Function

The redundant code is the code contained in the For loop. I decided to extract this code into its own method.

   1: Sub PopulateMessages(ByVal ListToPopulate As List(Of Twitter.Domain.Message), ByVal Contents As Object)   2:         For Each UserObject As Object In CType(Contents, Array)   3:             ListToPopulate.Add(New Twitter.Domain.Message With { _   4:                                .MessageID = UserObject("id"), _   5:                                .MessageDate = UserObject("created_at"), _   6:                                .MessageContent = UserObject("text"), _   7:                                .UserID = UserObject("user")("id"), _   8:                                .UserName = UserObject("user")("name"), _   9:                                .ImageURL = UserObject("user")("profile_image_url")})  10:    11:         Next  12:     End Sub

Now the code for parsing off message looks like this:

   1: Function GetUserMessages(ByVal UserID As String) As Twitter.Domain.Message()   2:     Dim ReturnMessages As New List(Of Twitter.Domain.Message)   3:     Dim UserObjects As Object = JsonSerializer.DeserializeObject(Me.Communicator.GetTwitterRequest(UserMessageURL.Replace("<<USERID>>", UserID)))   4:     Me.PopulateMessages(ReturnMessages, UserObjects)   5:     Return ReturnMessages.ToArray   6: End Function

The Twitter.API.User Class is had a slightly different refactoring. Because this API returns a lot of single user information it was necessary to create a function to return a single user object. When a method returned multiople user records a looping structure similar to the one for messages was used.

   1: Public Function GetUsers() As Twitter.Domain.User()   2:     Dim Users As New List(Of Twitter.Domain.User)   3:     Dim UserObjects As Object = JsonSerializer.DeserializeObject(Me.Communicator.GetTwitterRequest(FriendsURL))   4:     Me.PopulateUsers(Users, UserObjects)   5:     Return Users.ToArray   6: End Function   7:     8: Public Function GetUser(ByVal id As String) As Twitter.Domain.User   9:     Dim Users As New List(Of Twitter.Domain.User)  10:     Dim UserObject As Object = JsonSerializer.DeserializeObject(Me.Communicator.GetTwitterRequest(GetUserURL.Replace("<<USERID>>", id.ToString)))  11:     Return Me.GetUserFromObject(UserObject)  12: End Function  13: Public Function GetMyUser() As Twitter.Domain.User  14:     Dim UserObject As Object = JsonSerializer.DeserializeObject(Me.Communicator.GetTwitterRequest(CredentialURL))  15:     Return Me.GetUserFromObject(UserObject)  16: End Function  17: Function GetUserFromObject(ByVal UserObject As Object) As Twitter.Domain.User  18:     Return New Twitter.Domain.User With { _  19:       .UserID = UserObject("id"), _  20:       .UserName = UserObject("name"), _  21:       .ScreenName = UserObject("screen_name"), _  22:       .ImageURL = UserObject("profile_image_url"), _  23:       .Followers = UserObject("followers_count")}  24: End Function  25: Sub PopulateUsers(ByVal ListToPopulate As List(Of Twitter.Domain.User), ByVal Contents As Object)  26:     For Each UserObject As Object In CType(Contents, Array)  27:         ListToPopulate.Add(Me.GetUserFromObject(UserObject))  28:     Next  29: End Sub

The nice thing about these refactoring is that when a new property is added to a domain object you only need to change the code in one place.

Fourth Refactoring: The WPF Interface

The final refactoring I added was to implement a slightly improved interface. This basically consisted of changing the code that calls the Twitter API’s we write and adding three user interface elements. The elements added to the screen consisted of two fields for capturing username and password respectively and a new display element for the showing the Twitter users image. The VB code and XAML code now looks like this:

   1: Private Sub cmdGetThread_Click(ByVal sender As Object, ByVal e As System.Windows.RoutedEventArgs) Handles cmdGetThread.Click   2:     Dim Communicator As Twitter.Interfaces.ITwitterRequest = New Twitter.Communication.TwitterRequest(New Twitter.Communication.TwitterCredentials With {.UserName = Me.txtUserName.Text, .Password = Me.txtPassword.Password})   3:     Me.lstResults.ItemsSource = Me.LoadConversation(New Twitter.API.Message(Communicator), New Twitter.API.User(Communicator), Me.txtCriteria.Text)   4: End Sub   5:     6:     7: Function LoadConversation(ByVal MessageAPI As Twitter.API.Message, ByVal UserAPI As Twitter.API.User, ByVal TextCriteria As String) As Twitter.Domain.Message()   8:     9:     '-- create list of users from comma (,) delimited list of names in text box  10:     '-- TODO we should scrub this  11:     Dim UserQuery As New List(Of Twitter.Domain.User)  12:     For Each UserName As String In TextCriteria.Split(",")  13:         UserQuery.Add(UserAPI.GetUser(UserName))  14:     Next  15:    16:     '-- get messages for these users   17:     Dim messages As Twitter.Domain.Message() = MessageAPI.GetMultipleUserMessages(UserQuery.ToArray())  18:    19:     '-- filter messages based on who is in the contents  20:     Dim FilteredMessages = _  21:        From Message _  22:          In messages _  23:      Where MatchesCriteria(Message.MessageContent, Me.txtCriteria.Text.Split(",")) _  24:      Order By Message.MessageDate Descending  25:    26:     Return FilteredMessages.ToArray  27:    28:    29: End Function

   1: <Grid x:Name="LayoutRoot">   2:         <StackPanel >   3:             <StackPanel Width="Auto"  Orientation="Horizontal">   4:                 <TextBlock Text="User Name" Height="22" Width="80"/>   5:                 <TextBox Text="" TextWrapping="Wrap" x:Name="txtUserName" Width="151" Height="27"/>   6:                 <TextBlock Text="Password" Height="22" Width="67.957"/>   7:                 <PasswordBox  x:Name="txtPassword" Width="143" Height="27"/>   8:             </StackPanel>   9:                           10:             <StackPanel Width="Auto"  Orientation="Horizontal">  11:             <TextBlock Text="Screen Names" Height="22" Width="80"/>  12:             <TextBox Text="rodpaddock,bellware" TextWrapping="Wrap" x:Name="txtCriteria" Width="500" Height="22"/>  13:             <Button Content="Get Thread" x:Name="cmdGetThread"/>  14:             </StackPanel>  15:             <StackPanel>  16:                 <ScrollViewer Width="Auto" Height="600">  17:                 <ListBox Width="Auto" Height="600" IsSynchronizedWithCurrentItem="True" x:Name="lstResults"    >  18:                     <ListBox.ItemTemplate>  19:                         <DataTemplate>  20:                             <StackPanel Orientation="Horizontal" >  21:                                 <Image Height="100" Width="100" Source="{Binding Path=ImageURL}"/>  22:                                     <StackPanel>  23:                                         <TextBlock Text="{Binding Path=MessageContent}"/>  24:                                         <TextBlock Text="{Binding Path=MessageDate}"/>  25:                                         <TextBlock Text="{Binding Path=UserName}"/>  26:                                         <TextBlock Text="----------"/>  27:                                     </StackPanel>  28:    29:                             </StackPanel>  30:                               31:                         </DataTemplate>  32:                     </ListBox.ItemTemplate>  33:               34:                 </ListBox>  35:             </ScrollViewer>  36:             </StackPanel>  37:         </StackPanel>  38:     </Grid>

One item to notice is that all of the code for calling the Twitter API is no longer embedded in the Click() event of the Search button. It has been extracted into its own method and no longer relies on user interface elements. This refactoring was done with integration testing in mind. In a later installment we will be looking at testing the user interface along with just the API calls.

Now the user interface looks like:

Google Code

The source code for this project can now be found on Google Code The URL for this project is:

UPDATE

http://code.google.com/p/twitterplayground/

~~http://twitterplayground.googlecode.com/svn/trunk/twitterplayground-read-only~~

More Coming

I hope you have enjoyed these posts so far. Lets take a look at the list from the last post and strike out some tasks that were accomplished in this post:

1. Introduce a Dependency Injection container. For this application I want to use StructureMap.
2. ~~Introduce more mocking to remove dependencies on Twitter (maybe we can write a Twitter mock layer)~~
3. ~~Further refine the design of the libraries. There’s still redundant code left we can remove.~~
4. ~~Improve the UI~~ and create some alternate UI’s with MVC or maybe Silverlight
5. ~~Open up the code to contributions from other developers~~.

We accomplished quite a bit in this post. Is all of the code done? Not by any stretch. I plan on revisiting all of the items on the list in every refactoring (rinse-lather-repeat)… I will also add new items to the list, like:

Adding Proxy code to the communications class.
Adding error handling to the communications class.
Creating some mechanism for preserving credentials and other settings

RE #5: I did put it in Google code and would be happy to have other contributors. I would especially appreciate help with the UI.

NOTE: One of the cool things about this process so far is the confidence I have in changes I make to the code. Having a repeatable set of tests is invaluable.

Thanks
Rodman

Twitter Conversations an Agile Case Study

Rod Paddock [MVP] — Fri, 12 Dec 2008 04:47:00 GMT

Twitter Conversations

From time to time Twitter Users engage in conversations. It is a royal pain to follow these conversations using the native Twitter programs including the web version of Twitter. As an exercise I began creating an application that would allow me to follow Twitter conversations. This is the story of creating that application.

User Story
As a Twitter user I want to be able to follow conversations between other Twitter users.

This post will accomplish two things.
1. Fulfill the requirements of the user story.
2. Create a small application that can be used by the ALT.NET/Agile/Lean community to learn numerous agile software development concepts including: test driven development, dependency injection, how good design affects testing, etc.

Note: The second item in the list came from conversations at KaizenConf (www.kaizenconf.com kaizenconf.pbwiki.com). These conversations centered on the need to create a learning application. I am submitting this application as a case study so that we all may learn from it.
To accomplish the story we need the following parts:

1. A mechanism for communicating with Twitter.
2. Query Twitter UserID’s based on list of screen names.
3. Query messages for each user in list of screen names.
4. Filter messages where the message content contains any of the names in the list of screen names.
The first step is to define a domain model. The following domain model represents the different elements that will help in fulfilling the requirements of the user story.

• User
    o UserID(int)
    o ScreenName(string)
    o UserName (string)
    o Followers (int)
• Message
    o MessageID(int)
    o MessageDate(string)
    o MessageContent(string)
    o UserID (int)
    o UserName(string)

Unit Tests (Part 1)
After creating the domain model began creating unit tests for my API. I decided to divide my API into two subject areas: Users and Messages. The User API is used to query Twitter User information. The Message API is used to query Twitter Status(s) aka messages. The following code is the first set of unit tests.

<TestFixture()> _
Public Class UnitTests

<Test()> _
Sub Should_Return_My_Following()
Dim UserAPI As New Twitter.API.User
Assert.That(UserAPI.GetUsers().count > 0)
End Sub

<Test()> _
Sub Should_Return_Single_User()
    Dim UserAPI As New Twitter.API.User
    Assert.That(UserAPI.GetUser(UserAPI.GetMyUser().UserID.ToString).ScreenName _
       = "rodpaddock")
    End Sub

<Test()> _
Sub Should_Return_Single_User_ByName()
Dim UserAPI As New Twitter.API.User
Assert.That(UserAPI.GetUser("rodpaddock").ScreenName = "rodpaddock")
End Sub

<Test()> _
Sub Should_Return_My_User()
      Dim UserAPI As New Twitter.API.User
      Dim User As Twitter.Domain.User = UserAPI.GetMyUser
      Assert.That(UserAPI.GetMyUser().ScreenName = "rodpaddock")
End Sub

<Test()> _
Sub Should_Return_My_Messages()
Dim MessageAPI As New Twitter.API.Message
Assert.That(MessageAPI.GetMyMessages().Count > 0)
End Sub

<Test()> _
Sub Should_Return_Messages_By_User()
Dim MessageAPI As New Twitter.API.Message
Assert.That(MessageAPI.GetUserMessages("rodpaddock").Count > 0)
End Sub

<Test()> _
Sub Should_Return_Messages_From_MultipleUsers()
      Dim MessageAPI As New Twitter.API.Message
      Dim UserAPI As New Twitter.API.User
      Dim UserQuery As New List(Of Twitter.Domain.User)
      UserQuery.Add(UserAPI.GetUser("rodpaddock"))
      UserQuery.Add(UserAPI.GetUser("chriswilliams"))
      Assert.That(MessageAPI.GetMultipleUserMessages(UserQuery.ToArray).Count > 0)
End Sub
End Class

This set of unit tests does a good job of covering our API’s and insuring they all work as planned. But the tests do have a number of design flaws. For now we’ll leave them alone and look at some of the code they are testing.

Twitter Basics

Now that you have looked at the domain model and the unit tests, take a look at the process of fulfilling these unit tests. For this exercise consider the following unit test.

<Test()> _
Sub Should_Return_Single_User_ByName()
Dim UserAPI As New Twitter.API.User
Assert.That(UserAPI.GetUser("rodpaddock").ScreenName = "rodpaddock")
End Sub

To fulfill the requirements of this test the following things must occur:
1. Application must connect to Twitter
2. Application must retrieve user data based on the passed in username
3. The data returned must be transformed into a usable User domain object.

The following code fulfills the requirements of this test:

Const GetUserURL As String = "http://twitter.com/users/show/<<USERID>>.json"
Public Function GetUser(ByVal id As String) As Twitter.Domain.User

    Dim Credentials As New NetworkCredential("<<YourUserName>>", "<<YourPassword>>")
    Dim Request As HttpWebRequest = _
        HttpWebRequest.Create(GetUserURL.Replace("<<USERID>>", id.ToString))
    Request.Method = "GET"
    Request.Credentials = Credentials

    Dim Response As WebResponse = Request.GetResponse
    Dim Reader As New StreamReader(Response.GetResponseStream)
    Dim Results As String = Reader.ReadToEnd

Dim JsonSerializer As New System.Web.Script.Serialization.JavaScriptSerializer
Dim UserObject As Object = JsonSerializer.DeserializeObject(Results)

    Return New Twitter.Domain.User With { _
       .UserID = UserObject("id"), _
       .UserName = UserObject("name"), _
       .ScreenName = UserObject("screen_name"), _
       .Followers = UserObject("followers_count")}

End Function

This code performs the following tasks:

1. Created a NetworkCredentials object with your Twitter user name and password. All twitter requests use basic authentication.
Dim Credentials As New NetworkCredential("<<YourUserName>>", "<<YourPassword>>")

2. Created an HttpWebRequest based on Twitter’s REST API. This call will return data in JSON format as specified via the .json extension on the URL.
     Dim Request As HttpWebRequest = _
      HttpWebRequest.Create(GetUserURL.Replace("<<USERID>>", id.ToString))
      Request.Method = "GET"
      Request.Credentials = Credentials

3. Read data from the web request stream using a Stream Reader.
       Const GetUserURL As String = "http://twitter.com/users/show/<<USERID>>.json"
       Dim Response As WebResponse = Request.GetResponse
       Dim Reader As New StreamReader(Response.GetResponseStream)
       Dim Results As String = Reader.ReadToEnd

4. Deserialize the JSON data into an array of Name/Value pair data using the .Net JSON serializer.
Dim JsonSerializer As New System.Web.Script.Serialization.JavaScriptSerializer
Dim UserObject As Object = JsonSerializer.DeserializeObject(Results)

5. Turn the returned user information into a User domain object
       Return New Twitter.Domain.User With { _
         .UserID = UserObject("id"), _
         .UserName = UserObject("name"), _
         .ScreenName = UserObject("screen_name"), _
         .Followers = UserObject("followers_count")}

That’s pretty much how all communication works with Twitter. If you closely examine the code you will find a number of design flaws. Basically this code was developed as a spike: “Let’s see how we can pull data from Twitter”. The basic pattern: authenticate, request and parse was cut and pasted into each API call. I did this knowing that that the code (and tests) would be refactored into a proper design.

Refactoring the Code

After completing the first run through the code (getting it working) it was time to refactor. I gave myself some of goals:
1. Reduce the amount of redundant code.
2. Create a better designed set of test code (remove redundancy)
3. Prepare the code for dependency injection (current tests require internet connections)

The “authenticate and request” process received the first refactoring. This process can be pseudo coded as follows:

Given a valid set of user credentials and a REST URL return a string result.

From this pseudo code we created a Twitter communication class.

Imports System.Web
Imports System.Net
Imports System.IO

Public Class TwitterRequest
    Private UserName As String = ""
    Private Password As String = ""
    Sub New(ByVal UserName As String, ByVal Password As String)
        Me.UserName = UserName
        Me.Password = Password
    End Sub

Function GetTwitterRequest(ByVal URL As String) As String
Dim Credentials As New NetworkCredential(Me.UserName, Me.Password)

        Dim Request As HttpWebRequest = HttpWebRequest.Create(URL)
        Request.Method = "POST"
        Request.Credentials = Credentials

        Dim Response As WebResponse = Request.GetResponse
        Dim Reader As New StreamReader(Response.GetResponseStream)
        Dim Results As String = Reader.ReadToEnd
        Return Results

End Function
End Class

Now we have a wrapped class that needs two items in its constructor (UserName , Password) and has a single method GetTwitterRequest(URL). The GetTwitterRequest(URL)’s job is to return a JSON string that will be used by the subsequent API call.

All of the code necessary for communicating with Twitter is encapsulated into this class.

Next step was to refactor the User and Message API’s. The first step involved creating a constructor that accepted a TwitterRequest object. The constructor for the Message API changed to:

Dim Communicator As Twitter.Communication.TwitterRequest = Nothing
Dim JsonSerializer As New System.Web.Script.Serialization.JavaScriptSerializer

Sub New(ByVal TwitterCommunicator As Twitter.Communication.TwitterRequest)
Me.Communicator = TwitterCommunicator
End Sub

These two refactorings reduced the code radically. The following code shows the new GetUser() method.

Public Function GetUser(ByVal id As String) As Twitter.Domain.User
Dim UserObject As Object = JsonSerializer.DeserializeObject(_
Me.Communicator.GetTwitterRequest(GetUserURL.Replace("<<USERID>>", id.ToString)))

   Return New Twitter.Domain.User With { _
      .UserID = UserObject("id"), _
      .UserName = UserObject("name"), _
      .ScreenName = UserObject("screen_name"), _
      .Followers = UserObject("followers_count")}

End Function

The code went from 10 lines of code to 2. The same results were seen across the entire API.
Refactoring Tests
Once the API’s were refactored the tests were then refactored. The following code shows the new set of tests.

Imports NUnit.Core
Imports NUnit.Framework

<TestFixture()> _
Public Class UnitTests
    Dim UserAPI As Twitter.API.User = Nothing
    Dim MessageAPI As Twitter.API.Message = Nothing
    Dim Communicator As New Twitter.Communication.TwitterRequest("", "”)

    <SetUp()> _
    Sub Setup()
        Me.UserAPI = New Twitter.API.User(Me.Communicator)
        Me.MessageAPI = New Twitter.API.Message(Me.Communicator)
    End Sub

    <Test()> _
    Sub Should_Return_My_Following()
        Assert.That(UserAPI.GetUsers().Count > 0)
    End Sub

    <Test()> _
    Sub Should_Return_Single_User()
     Assert.That(UserAPI.GetUser(UserAPI.GetMyUser().UserID.ToString).ScreenName _
        = "rodpaddock")
    End Sub

    <Test()> _
    Sub Should_Return_Single_User_ByName()
        Assert.That(UserAPI.GetUser("rodpaddock").ScreenName = "rodpaddock")
    End Sub

    <Test()> _
    Sub Should_Return_My_User()
        Assert.That(UserAPI.GetMyUser().ScreenName = "rodpaddock")
    End Sub

    <Test()> _
    Sub Should_Return_My_Messages()
        Assert.That(MessageAPI.GetMyMessages().Count > 0)
    End Sub

    <Test()> _
    Sub Should_Return_Messages_By_User()
        Assert.That(MessageAPI.GetUserMessages("rodpaddock").Count > 0)
    End Sub

    <Test()> _
    Sub Should_Return_Messages_From_MultipleUsers()
        Dim UserQuery As New List(Of Twitter.Domain.User)
        UserQuery.Add(UserAPI.GetUser("rodpaddock"))
        UserQuery.Add(UserAPI.GetUser("chriswilliams"))
        Assert.That(MessageAPI.GetMultipleUserMessages(UserQuery.ToArray).Count > 0)
    End Sub
End Class

As you can see the testing class now has member variables for each API class and the <Setup()> section instantiates the APIs with the injected communication class.

User Interface

Finally a small WPF application was built to track conversations between users. The following code creates instances of the User and Message APIs, creates an array of user objects from a string (split by commas), retrieves messages for the specified users and finally queries them using LINQ to ferret out a conversation. This code is as follows:
Partial Public Class Main

    Dim UserAPI As Twitter.API.User = Nothing
    Dim MessageAPI As Twitter.API.Message = Nothing
    Dim Communicator As New Twitter.Communication.TwitterRequest("", "")

Public Sub New()
MyBase.New()

Me.InitializeComponent()

        ' Insert code required on object creation below this point.
        ' Add any initialization after the InitializeComponent() call.
        Me.UserAPI = New Twitter.API.User(Me.Communicator)
        Me.MessageAPI = New Twitter.API.Message(Me.Communicator)
    End Sub

Private Sub cmdGetThread_Click(ByVal sender As Object, _
ByVal e As System.Windows.RoutedEventArgs) Handles cmdGetThread.Click

        '-- create list of users from comma (,) delimited list of names in text box
        '-- TODO we should scrub this
        Dim UserQuery As New List(Of Twitter.Domain.User)
        For Each UserName As String In Me.txtCriteria.Text.Split(",")
            UserQuery.Add(UserAPI.GetUser(UserName))
        Next

        '-- get messages for these users
        Dim messages As Twitter.Domain.Message() = _
           MessageAPI.GetMultipleUserMessages(UserQuery.ToArray())

        '-- filter messages based on who is in the contents
        Dim FilteredMessages = _
           From Message _
             In messages _
         Where MatchesCriteria(Message.MessageContent, Me.txtCriteria.Text.Split(","))
        Order By Message.MessageDate Descending

Me.lstResults.ItemsSource = FilteredMessages.ToArray
End Sub

    Function MatchesCriteria(ByVal Content As String, _
         ByVal SearchCriteria As String()) As Boolean
        Dim llRetVal As Boolean = False
        For Each SearchString In SearchCriteria
            If Content.ToLower.Contains(SearchString) Then
                llRetVal = True
                Exit For
            End If
        Next
        Return llRetVal
    End Function

Lastly the information is displayed using the following XAML code:

The following screen shot demonstrates a conversation that occurred today between three people I follow on Twitter:

Next Steps
There is more work to be done here. In the future posts I hope to do the following:

1. Introduce a Dependency Injection container. For this application I want to use StructureMap.
2. Introduce more mocking to remove dependencies on Twitter (maybe we can write a Twitter mock layer)
3. Further refine the design of the libraries. There’s still redundant code left we can remove.
4. Improve the UI and create some alternate UI’s with MVC or maybe Silverlight
5. Open up the code to contributions from other developers.

Summary

What I hoped to accomplish in this post was a brief introduction to building a useful tool using the agile principles of Test Driven Development and Dependency Injection. Another goal of this post is to open a conversation on these agile principles. A lot of these practices may seem simple to a lot of the folks I blog with here but a lot of folks out there they seem foreign. I appreciate the comments.

The full code for this post can be found at www.dashpoint.com/downloads/TwitterPlayGround.Zip

Note: There are two sections where you will need to use your own username and password.

Thanks
Rodman

Roundtables vs. Lunch Counters

Rod Paddock [MVP] — Tue, 02 Dec 2008 08:28:00 GMT

I have a strange pet peeve. I hate rectangular tables. It’s a strange pet peeve but its mine and I stick by it. I hate rectangular because they don’t scale. They don’t scale conversations for groups of people greater than say 6 people. I became aware of just how much I hate rectangular tables last week. I was at QCON in San Francisco and went to dinner with some friends at a great seafood restaurant. There were 4 of us and because there were no tables available but we had to sit at the bar. The bar had 4 empty seats side by side so we took them. Dinner was great and some good conversations took place. The problem is that there were basically two conversations of two people. It would have been my preference to have one conversation with all four people involved. I cemented my hatred further two days later at another dinner with another rectangular table. We had a larger group (12+ people) this time. The communication situation was marginally better for the “middle” people. A larger conversation “cluster” formed in the center with smaller satellites of communication for the “edge” people.

Flash back two weeks earlier to Kaizen Conf. Kaizen Conf is a conference centered on concepts of continuous improvement, agile and lean software development practices. Most of the sessions I attended were large groups (some 50+) sitting in large circles having healthy and involved discussions on some healthy topics. The benefit of the structure is that everyone can see and hear everyone else. Communication is more fluid and minimizes the “clustering” effect. Another benefit of the roundtable structure is its channel balancing effect. No channel is more important or larger than another. There is no “head” of the circle.

I find a lot of similarity in software development practices today. Classic waterfall development is what I call lunch counter development. Communications happen in differing stages along the lunch counter with the bulk of quality communication happening in the middle. The edge people are generally left out of the main communication and never have their concerns every really addressed. Just think of your customer sitting at one end of the table trying to communicate there needs to the other end of the table.

Now take a look at agile/lean and RAD (iterative) software development practice. These practices are what I am seeing as round table development. It’s all about value systems. Agile/Lean/RAD developments value communication between all stakeholders at all stages of the development process. Every stakeholder has seat at the table. The cornerstone of these practices is the active communication between all constituents at all times. Stand up meetings, iteration planning meetings, retrospectives. All of these (and many more) practices relate to facilitating communications between project stakeholders.

The more I develop software using round table (agile/lean) methodologies and practices the more I find the lunch counter development (waterfall) detestable. Software development practices that emphasize and promote communicating down lunch counter are destined to disappoint and generally leave bad tastes in stakeholder’s mouths. This is not limited to just users being disappointed. All stakeholders are disappointed. How many developers do you know that don’t really care about their end users or the quality of their software? I take it personally when my software doesn’t work, doesn’t deliver its expected value or frustrates my end users who I truly care about. I am betting you feel the same way. Let’s not forget that it’s not just you the developer or the end user that feel the effects of software development process. Along with end users and developers; project managers, managers, executives and customers all feel the true quality of the software process.

Stories differ but legend says King Arthur created his round table to emphasize equal status among all that had a seat. The most important aspect of any software development process is the communication of requirements. Does your company communicate down a lunch counter or do they sit at the round table?

It's been too long between blog posts so here ya go. This post will end up as my editorial in CoDe Magazine next month. I have more posts in the works related to my recent spelunking into all things agile. Stay tuned.

crossposted from blog.dashpoint.com

Unlearn

Rod Paddock [MVP] — Fri, 09 May 2008 13:37:00 GMT

“You must unlearn what you have learned.”

Yoda to Luke Skywalker Empire Strikes Back 1981

I have become convinced that we are entering a new dawn of software development and we must take time to unlearn that which we know. Over the last 2 months I have been researching building business applications using Silverlight 2.0. In case you don’t know Silverlight 2.0 is a tool used to build Rich Internet Applications (RIA’s) Silverlight 1.1 (released in 2007) gave us a small glimpse of the future of RIA development. This glimpse proved that Silverlight 1.1 was wholly lacking for business developers (it didn’t even have a button). Silverlight Version 2.0 (released at Mix 08 in Las Vegas, NV) is a different story. Silverlight 2.0 has all of the features a business developer might need. The control set includes: textboxes, buttons, watermark textboxes and, drum roll please, a DataGrid component.

During my research I began experimenting with the DataGrid. I started (in XAML) adding columns to the DataGrid’s columns collection, setting data binding attributes, column widths and header properties. I then turned to the code portion of my research and DataGrid’s ItemSource property to a list of objects. I ran the application and lo-and-behold it worked (not the first time of course but eventually). After getting my simple grid demo working I wanted to implement a simple, but common use case. I wanted to allow the user to sort the data by clicking on the column. Here’s how it went:

1. Look for DataGrid properties called something like “AllowSort”
2. Look for header properties that would turn this feature on.
3. Look into the documentation for the DataGrid (samples, online, etc)
4. Download and open the source code for the DataGrid (yup you read that right its available to developers)

I struck out on all accounts. It looked like I would be waiting for a new rev for this feature. I did take one more step though. I turned to my trusty search engine Google and typed in “silverilght 2.0 datagrid sort”. And look what I found: Matt Berseth (http://mattberseth.com/blog/) had already figured this out in a very creative and enlightening way. Matt’s unique solution was to put a HyperlinkButton control into the DataGrid’s Header Content property. The Header object’s Content Property is defined as Object. The normal use case for this property is to specify a string as your header. But this is Silverlight and when a property says it’s an object it’s an object. This means you can replace the header content with a Hyperlink rather than a textbox. This is where my epiphany occurred:

My typical path took me to the control itself. I searched for properties, events and even considered for a moment hacking up a hit test set of code to do this. It hadn’t occurred to me to composite some controls together to do what I wanted. This is where I told myself I must unlearn what I know. I must think of new and interesting ways to combine these controls to do new and interesting things.

This epiphany raised its head again over the last 2 weeks. Over the last 6 months we have started the trek down the Agile Development path. This path has included installing and researching agile development applications including: Nant, Nunit and Subversion. We are using Nant and Nunit to automate our testing, and in the last two weeks we moved a large quantity of our source code into a Subversion repository. It’s this last application that has provided me with another epiphany. Source code control in Subversion is “just different” than in Visual SourceSafe (our previous source code provider). Visual SourceSafe is a pessimistic version control system (you lock files when you check them out) Subversion is just the opposite. All files are worked on in shared mode. Meaning two developers can work on the same file at the same time. It’s up to the developer to merge changes into the development branch of code when conflicts occur. Along with a different philosophy Subversion uses different terms: In Subversion Checkout means get a fresh set of code from the repository, Import means send source code to the repository. Commit means send your changes to the repository. Visual SourceSafe had different meanings for all of these terms.

When I started using Subversion I found it difficult until I told myself what Yoda said: “You must unlearn what you have learned.” I closed the book on Visual SourceSafe and opened a new one on Subversion. I consulted the documentation to see what the Subversion philosophy is and what the terms in Subversion really mean. When I “forgot” what I knew about source code control from the VSS POV things began to become clear and the process of migrating much simpler.

In today’s development world new technologies require new thinking. You need to unlearn what you already know. To find comfort in the unlearning process, take shelter in the words of Jedi Master Obi-Wan Kenobi to Luke Skywalker: “Luke… Let go. Trust the force”.

crossposted from blog.dashpoint.com

DevTeach Toronto 2008 = Agile Immersion

Rod Paddock [MVP] — Tue, 29 Apr 2008 17:30:00 GMT

If you want a good general immersion in Agile practices look no further than DevTeach (www.devteach.com)

Last DevTeach (in Vancouver) I sat through a number of Agile sessions with a fellow development partner. This parter had virtually no experience with Agile practices. After DevTeach we began discussions on implementing Agile practices in his 15+ developer shop. It was good for him to hear from other developers about their experiences with Agile.

So if you want to get your management, developers or development buddies on board with Agile then send them Toronto in May for DevTeach.

PS: You can also stay for a post-con session on Building Business Apps with Silverlight 2.0 given by me and my business partner Jim Duffy

PSS: Small conferences like DevTeach are MUCH better than large ones. Why: Because the speaker to attendee ratio is much lower and we can all interact much more…. I personally prefer small conferences over large ones.

crossposted from blog.dashpoint.com

OT: SilverLight 2.0 = iPhone SDK

Rod Paddock [MVP] — Thu, 06 Mar 2008 17:08:00 GMT

Maybe it's just a coincidence but I am thinking its not.

I am at MIX 08 in Las Vegas and am eagerly awating today's keynote with Steve Ballmer and Guy Kawasaki.

Funny thing is the whole timing of it all....

1. Apple's iPhone SDK Meeting is at 1:00pm Pacific

2. MIX 08 Keynote is at 1:00pm Pacific.

3. Guy Kawasaki is the former evangelist for Apple. Heck he started the whole software/hardware platform concept.

4. Silverlight is a full functioned player on the Apple.

5. Steve Jobs hates Flash http://www.informationweek.com/blog/main/archives/2008/03/jobs_flash_not.html

We got to see that Silverlight is a player in the Nokia space. Will we be developing iPhone and iTouch apps with Silverlight? As the Apple ad's say: Imagine....

UPDATE: Oh Well! It was a fun thought for a few hours in any case.

crossposted from blog.dashpoint.com