At the last Agile Austin lunch we were talking a little bit about the infrastructure improvements we’d all like to be making, if only there was time.  We talked about improving our continuous integration builds, retrofitting legacy code with more automated tests, and getting rid of VSS.   One of my friends commented that there were so many things he wanted to improve, but there wasn’t enough time.  I feel the same. 

We want to improve our code and architecture for better reliability and improved efficiency.  It’s not that simple though, because we really need to be creating new business features.  The marketplace is competitive and we’ve got to keep ahead or at least abreast of our competition.  Many of our clients are very large corporations that inevitably have unique requirements, and they need them now!  All the same though, my team is going to have to spend some time dealing with configuration management and access to a shared database or we’re going to create a support monster.  It doesn’t create any apparent business value, but we’re jeopardizing our ability to continue making changes to the code if we don’t get it under control.  The point being, we’ve got some technical debt that needs to be paid down because it decreases our efficiency while increasing the risk of any modification to the code – and the code is always changing.

So we’ve got a common dilemma.  How do we make improvements in the existing code to make us more efficient and reliable while delivering new stuff?  Some times you really have to stop and clean things up first.  How exactly do you communicate this to senior management?  How in the world do you convince management that it’s a necessary task?  How do you quantify the risk of not doing some sort of infrastructure improvement so management can make an informed decision versus new functionality? 

In a prior life as an enterprise architect I failed to convince management to create a proposed naming repository for vendors across our manufacturing, inventory, and supply chain systems.  Without that structural improvement the security model continued to be haphazard and wrongly allowed access to very sensitive data to the wrong folks.  The problem was pretty obvious to me and my peers, but we didn't manage to communicate the impact of their decision to forego the new naming repository.

Life’s a lot easier when there is a good trust relationship between the development team and the program management.  If that trust isn’t there you might be forced to make the situation worse by ignoring problems or do the right things under the cover of darkness (leading to people burnout).  It seems to pretty natural for project/program managers to mistrust software developers, maybe because they’ve known developers to waste time on unnecessary “goldplating.”  My experience has been that most project managers really don’t have a very good understanding of the technical situation, and I’ve found this to be every bit as true for project managers that are ex-developers.  We’ve got to be feeding them the right information in a way that they can comprehend and they’ve got to be able to have some sort of faith in what we’re saying.  You really need to guard your own credibility by being honest and at least somewhat accurate with management.  Spending months on a failed attempt to rearchitect some subsystem is going to result in a shorter leash.  You might also inherit the mistrust of your management for your predecessors.  That's a bit unfair, but it's happened so many times to me that I almost expect it.

Sometimes it’s obvious that you’ve got to clean something up or that doing a refactoring first will make the new functionality easier.  If the situation is really clear, I say you just do it.  You really shouldn’t need to ask permission from a project manager in these cases.  It’s simply a matter of you coming up with the best mechanical path to your end goal.  Anything that is big enough to affect velocity probably needs to be justified to a project manager and tracked as a user story.  That doesn’t have to stop the idea in its tracks, but it makes it harder to get it done.

Other times you just have to bide your time and look for opportunities to make the refactoring work as you’re building new functionality in the same area.  You take an attitude that anytime you make changes in an area of code that you try to leave it in a better state when you’re finished.  Kind of a more active version of the Hippocratic Oath for code.  You make incremental changes to break coupling, increase cohesion, speed up the build time, and increase the unit test coverage whenever the opportunity is presented.  We spend a lot of time being frustrated because we have many more ideas of how to improve things than we really have time to accomplish.  I say that it’s still important to think and talk about the improvements that you’d like to do.  You’ll never make any kind of improvement if you aren’t creating a vision of some kind, and you never know when an opportunity will present itself.

Of course the real answer is not to allow technical debt to build up in the first place.  You simply cannot cut corners in design, coding, or build management (damnit!) and sustain a rapid pace – not even in the short term.  Slacking off on good development practices will make you slower over time.  Another way to say this in MBA language for the suits is “Opportunity Cost.”  When you write badly factored code or skimp on test or build automation you’re making future endeavors more difficult and costly.  Agile development includes the practice of Refactoring.  I’ve often seen an apt analogy comparing refactoring work to washing the dishes in your kitchen (I think it originates with Ron Jeffries, but don’t quote me on that).  It gets a lot more difficult to cook in the kitchen if you didn’t wash the dishes.  Washing the dishes is a lot easier if you do it right after using them. 

It does help if you can just be omniscient and never make any mistakes ;)  Thank you for listening, I feel better now.