A Coalition of the Willing

Jeremy,
I share your frustration.

I work in a management consultancy firm where the prevailing wisdom is to address software delivery risk with “more tools, more processes, and more upfront planning”

in this case I feel I have to continue to argue for change, because if I don’t our clients will suffer, and the firm I work for will be put at risk as we try to address these massive multimillion dollar projects with waterfall approaches and BDUF.

I wouldn’t give up with trying to evangelize your software development values. These ideas are gaining wider and wider traction in even the conservative circles, and many of the “traditionalist” while not becoming full converts, aren’t we starting to understand that there’s something real in all of this.

It’s true it’s no use preaching to those that refuse to listen, but you’re making a big impact on all kinds of people who are “sitting on the fence” and I think this is the year that many of these things didn’t really sentence and traction.

Regarding why Jeff

The approach you propose is good and and is successfully employed by Buddhism for thousands of years now.

Everybody makes progress at their own pace. Things that are good may eventually not work out. Certain thought structures and approaches in Buddhism are just there to bring you to a higher level of understanding. From there on, they become useless!

Example Testing. I am quite happy with unit testing, even though I often end up testing interactions and behavior and my test meth names tend to be overly large In contrast, I find the pure given/when/then stuff I can find in the Internet quite verbose and still leaves me wondering what can be driven by it. You have obviously reached a different level of understanding.

What is important is that at every level when a developer becomes aware that there can be much more method in his working than just trial and error, there must be teachers and literature ready to pick him up at that level. It is quite difficult to jump levels. Pupils can often walk the levels faster, provided they have a good teacher, but jumping them will lead to frustration.

It’s indeed shocking how many people still refer to that stone-aged post about procs today, in never ever ending bickering about something so irrelevant.

So in general, I understand from your post that you’re saying online debates are pretty useless, and I agree with that. The only thing that you’ll get from an online debate is more frustration. I made the mistake again to go into such a debate very recently (about soft-deletes) and I concluded it’s been enough.

What I think is the core of this whole problem, and I think you ran into the same conclusion, is that it’s useless to think that you can convince somebody else that what YOU’re doing is great and what somebody else is doing is bad. It’s never that black-white, though often these discussions/debates are taking place in such a setting.

What might be better perhaps is simply stating what you think about the technology you’ve chosen to concur the problems faced every day and why you’ve chosen that technology, leaving the choice if it’s what the reader should choose or not to the reader. If the reader isn’t able to make that choice, so be it.

In that light, I found it a bit sad that you ended your great post with a weak ending where you indeed implicitly state that what ALT.NET provides is good, rest is bad, which IMHO is precisely what you argue against in the first half of the post: the time of stating what’s good/bad is over.

I agree with that guts of what you’ve said about moving on rather than hoping to drag every developer kicking and screaming into the future.

However, on the topic of “wasting energy” trying to engage with developers who are happy with their I’ve-been-doing-it-like-this-forever status quo, I’d make the point that it’s a daily reality for most of us that won’t go away quickly. I’d respectfully suggest that as a well-known, well-credentialled, “celebrity” developer (and I use the c word in a respectful sense and not as some kind of insult), you’re in a better position that most to insist upon working in a particular way. Those of us who don’t have that leverage are left to chip away at the old mentalities as best we can.

It’s just as with small kids. You can plea: “Hurry up, please” you can shout “Hurry up, dammit!”

Or you can walk ahead and be sure that the kids will hurry up as soon as you are getting out of sight.

Of course from time to time you might loose a kid which isn’t acceptable for actual kids, but might be ok with some kinds of developers

I would completely agree with the thrust of this so long as in the rush to avoid trying to drag *everyone* along with us we don’t lose sight of the very real responsibiliy to continue outreach and info-sharing efforts that get our ideas in front of the willing-but-unware developers out there who are actually sympathetic to our value system but simply unaware there is a better way out there.

Thanks for the compliment Jeremy … it was precisely the realisation you have had that made me start that series … trying to convince most people, or even guide them towards better ways is frustrating at best, and futile at worst.

I am settling for your approach now, I’ll put out the water, and if the horses are thirsty they can drink

Well said Jeremy and I can wholeheartedly get behind this idea.

As someone who buys into the alt.net philosophy, but doesn’t always have the ability to practice it as faithfuly as I’d like, I’m very glad to hear this and hope more of “alt.net” takes on this tone. I’ve always felt the us vs. them arguments were counter-productive to teaching new techniques. The personal attacks that these arguments tend to devolve into, inevitably alienate developers who can’t always progress as fast as some would like.

I think things like the EF are a good example of Frank’s comment:

“Everybody makes progress at their own pace. Things that are good may eventually not work out. Certain thought structures and approaches in Buddhism are just there to bring you to a higher level of understanding. From there on, they become useless!”

The EF isn’t the right tool for you becuase you have moved beyond its current abilities. However, it is the right tool for many because it will get them thinking in ways they haven’t in the past and start them on the path. A vote of no confidence just adds another barrier and likely hinders the progress you are trying to achieve.

Simon, even though Jeremy is a “celebrity” developer, you really need to consider who his readers are. The people who come here are one of two people, a) people who already agree with most of what he is saying, or b) people who are looking for answers on how to do their job “better”. The c) group, meaning those who are stuck doing things the “old” way, they aren’t coming to this site.

So really, he is preaching to the converted or to the pepole who are willing to listen. The developers/managers who insist on building 2000 sprocs for every application aren’t here and never (or at least rarely) will be.

Overall it was a great article and recently I had come to the same conclusions. It’s really not worth the energy arguing with someone who will never see your side, you just do what you can with those who will listen and always remain respectful of those who won’t.

I can see this working very well for the masses out there, but what do you do with the developers that are your direct colleges? I want to increase the quality of my colleges and not all of them are as open to this as I wish, but mostly they just don’t know any better. It would be great if you could share your ideas about that.

-Mark

Good post Jeremy. Someone has needed to say something like this for a while.

There’s a reason these debates get termed “religious”. They’re not very different from their spiritual cousins. You’re rarely going to convince a person of one religion to convert to another just based on your own testimony.

People convert from one idea to another when they seek answers and find them, and their own brain/logic/reasoning skills tell them what to do.

Our best chance of having these ideas spread, as you and Frans have pointed out, is to simply provide solutions, talk about those solutions in the context of problems and pain we face, and why the solutions helps us. The reader will find your words if they’re looking for a solution or a way to overcome their own pain. But if they aren’t looking, then you can’t convert them. You can’t evangelize them into crossing over.

I agree with abandoning the argument. And, I’m optimistic about the direction of ALT.NET. My agreement is based on a mix of experience, intuition, and some (but insufficient) evidence.

Flexibility and adaptation are good strategies in the general sense, but they aren’t evidence.

What we need are a series of measures and justifications for those measures to provide the evidence for claims.

“I’m implicitly stating that the ALT.NET way is the way…”

That statement out of context in isolation would alarm me.

How do we test that we haven’t, ourselves, become religious?

I find it somewhat disappointing that you feel the need to even make a post of this type. I really respect the stuff you’ve done vis-a-vis StructureMap and the thrust of Fluent NH, and I’m learning a lot from it – both code-wise and in the general philosophy as well.. However, I still use stored procs and when the situation more or less requires it, I don’t have a big problem staying with this “old style” paradigm.. Sometimes you just have to move ahead and not try to change the world, no?

I generally take the libertarian approach to this topic. If my neighbor insists on cutting his firewood with a coping saw, when I demonstrate everyday that a chain saw is 25x more productive, then that is ultimately his business and loss.

However, the problem arises when we have to work under coping saw wielding managers that are resistant to improving the state of the art. You may have to argue with them, but I think the answer is to find the space and freedom to lead. If that mean finding another organization, or starting your own business, then so be it.

Just lead, and leave the arguing for others.

@Peter: Using sprocs is OK. They have a place and purpose. Using sprocs to wrap simple CRUD Select/Update/Insert/Delete statements is an abuse of sprocs and adds lots of complexity and overhead to a process that’s more easily solved elsewhere.

@Peter,

I would still use a sproc for edge cases, but I think those edge cases are getting fewer and fewer as the years go by. I also don’t write batch processing systems or reporting systems anymore, and I think that’s the most likely spot to need/want a stored procedure.

@Garry,

“How do we test that we haven’t, ourselves, become religious?”

By the simple fact that we constantly change our mind. You did read that section right? I think that the fact that our tools and approach is always in a constant state of churn is a sign that we’re not really religious. No, we’re not going back to RAD programming the Microsft way, but we’re definitely still moving.

@ Jeremy & Gary

“How do we test that we haven’t, ourselves, become religious?”

As an atheist, people always like to remind me that my lack of belief is just another form of belief. And they’re right, of course.

So I’m going to disagree with Jeremy on this and say we (.Net folks) ARE religious. We have a “belief” or set of beliefs. Our belief happens to be “adapt, change, constant state of churn, find a better way, etc.” but it’s a belief system nonetheless. There are principles that guide the way we develop software.

And there’s NOTHING WRONG WITH THAT

Besides – how can you write opinionated software without having an opinion?

Stored procs are used as a mean of separation of concerns between the data and the code.

I understand it doesn’t suit RAD principles but in a large enterprise, the data is too valuable to be allowed to be directly touched by code. They have SQL dinosaurs that review every change of SQL you do to ensure data integrity. I don’t like, you don’t like it, but the business really doesn’t like it when someone screws up their data. Ultimately we do write software for the business.

Whether it’s a stored proc or an interface, what is the difference? Why would I be viewed as ‘not cool enough to use alt.net principles’ ?

@Philosophil,

That might be reality, but I don’t accept the gist of your argument. It’s easier to keep data from getting screwed up if you can easily test the system, and sproc’s will always be harder to test than business logic in the middle tier.

Would you rather depend on human eyeballs alone, or human eyeballs plus automated tests? Plus you should keep in mind that the tooling for middle tier languages are vastly better than that for T-SQL or PL/SQL.

It’s not “I’m just being cool ’cause I don’t use sprocs” Give me a little more credit than that. I really don’t believe that stored procedures as a gateway to the raw data is a viable or valuable strategy unless you’re stuck with a shared database between apps — and I think the shared database is a massive anti-pattern in enterprise architecture.

@Jeremy

I did read the section about changing our minds.

I also believe I understand the implication that you’re making which isn’t explicit, that there is some way to distinguish between healthy churn and unhealthy churn.

The conversation I’d like to participate on is finding and measuring that distinction.

No metric is perfect, and some people will game the measurement system (adding lots of trivial test to improve coverage, for instance). But, those of us working for real improvement could use a concrete measurement beyond just our current projects.

As business owners, and software product owners, we might use it like depreciation. If we can show that a product loses value over time because it gets out of sync with current practice, we can make the case that investments in current practice are capitalizable even if they don’t add or improve features.

Amen. I tired of all of the time wasted arguing with people who aren’t making an attempt to understand things. Like you said, there are plenty of people out there who do get it and we can just hire those people, and all the “cowboys” can continue to do their own thing if they want.

Although the next time someone comes out and says something crazy like the Stack Overflow podcast did, everyone will jump up and defend our principles.

*standing applause*

And this is in part what I meant (and said) to my friend Scott Bellware a few days ago (I think he fights WAY too much to try and get his points across, and IMHO it actually hurt what he says; he is crazy, but in actuality if all that vapor is removed, we’ll find more value in what he says …

To listen and accept different perspectives is hard for anybody, sure we can argue and be simply flabbergasted, but at the end of the day, only through personal experience and hard work I believe true awareness can be achieved.

I’m a MAJOR beginner in these alt.net techniques, I’m starting, reading and experimenting with a bunch of new concepts, and believe right now I will get a lot of value from this; however, even tho I’m willing, I can’t do as I’m told just because… I must ALWAYS put myself in a position in which I can reach my own conclusions, I’ve found it is the only way to learn…

So, Scott, like jeremy says at the beginning… by all means disagree with me, but stop ranting and being so mad at me for trying to do BDD with nant; wrong? maybe yes… but at least I will be able to take your words as advice and not simply disregard as $!$#@$!%@!@$@$!.

Sorry to use your post as a proxy Jeremy, but I thought it was a very suitable… and thanks

Geo
from the caribbean

Well, Geo, you wouldn’t be the first impudent child in the alt.net community who immediately sought to change forms and practices that he had only a trivial understanding of. And this is why I caution against this kind of self-indulgent vanity when you have barely even begun to take your first steps. It will force you to take the long way to knowledge rather than the direct route, and if you continue to choose to indulge yourself, you might join others who have gone before you and not get there at all.

I would be cautious about using Jeremy’s style as a model for my work. We work to very different aims that inevitably beget different tactics. You’re failing to take your analysis here beyond the trivial, but I recognize from our brief time together that this is more of a persistent pattern than a momentary lapse, and so I make allowances for it.

In the future, it would likely be more appropriate to take this kind of personal interchange to another medium, otherwise I might presume that your approach to debate is predicated on having safe and familiar surroundings and utterly lacking in courage, which I’ll likely just ignore in the future for its utter lack of honor and its conspicuous absence of particular glandular support.

It’s also rather discourteous to Jeremy and his readers to invite tangential conversations into meaningful contexts that they are working to build through dialog and exchange.

Jeremy,

The multi-faceted activism built into the entire effort goes above and beyond the very valuable goal of making EF better. I appreciate that you see no purpose in protest, but that doesn’t mean that there wasn’t more in-play than the effort to improve the product, and that many of the objectives didn’t result in categorical successes.

The EF effort achieved two of the primary objectives I had in mind when setting the campaign in motion: it raised awareness, and it brought Microsoft to the table. The hard part is sustaining, and let letting up, knowing that Microsoft has a long tradition of reverting back to less effective means when it feels that the heat is off. Well, that’s what activists are for, and not all signatories understand the imperatives or can sustain the tactics of activism.

Not every problem can be solved in entirety with well-wishes and a thoughtful article.

AndrewLV,

Considering that “never” is an infinite measure of time and that you can’t really know if you’ll “never” change someone’s mind, what is the criteria for desisting from the effort to sustain the debate?

@Jeremy

I agree with you, but the business doesn’t and changing their minds will not happen. They saw too many examples of people screwing up the data to believe me. I can’t really blame them, they have teams of 100s of people working on their system and if one of them screws up a commit once every 6 months it’s enough for them to justify a lock down of SQL. They still believe in waterfall and control of the schedule. Changing their mind will not work.

I use the term believe since it’s not based on any facts. It’s based on the analogy of a building a house. Since they understand the analogy, they believe in the process.

What I’d like is a way to work with SQL dinosaurs and not against them. I’m an optimist…

When I read this article I can’t help but notice that I was the one that Twitter’d Jeremy the comment that deploying a sproc is “easier” than redeploying code. I also commented that I was after ammunition for why sprocs are bad.

@Jeremy I really appreciate the things you guys (alt.net) do. I am still pretty “green” when it come the developing (I just graduated in June) and CodeBetter has been one of my go to places for design guidance.

I have recently stepped into more of a tech lead/architect role on my team. I want to steer the team more in the direction of OR/M and TDD. The move has been met with a considerable amount of resistance because they just do not want to let go of the sproc.

I am met with arguments such as: 1.) the sproc is an abstraction layer (contract) between the underling complexity of the database and the application 2.) the “behavior” of your application can be easily changed without redeploying 3.) the sproc is more secure and is recompiled (better performance) 4.) Mapping “hot to the tables” is simply unacceptable

I know you all feel that the sproc argument is “beating a dead horse” or a dead issue. I just want you all to know (alt.net) that i appreciate each and every discuss regarding the pros/cons of sprocs. Just remember that there are guys like me out there that need all the ammo they can get.

Keep up the good work!

Scott,

“Considering that “never” is an infinite measure of time and that you can’t really know if you’ll “never” change someone’s mind, what is the criteria for desisting from the effort to sustain the debate?”

I probably shouldn’t have used the word “never” when talking about people resisting change, instead I should have used something like “unlikely”.

In the past 15 years of doing what I do I’ve found is that job security seems to trump all else for most people. If change will risk job security, there will always be a group of people who will fight to maintain the status quo. Those are the people who I believe will “never” change.

As to you question, my answer would be that you need to take things on a case-by-case basis. No two people or organizations are identical. I know that’s a BS answer but I’m not sure what else to say. For example, if the person resistant to change is a CIO who is a “my way or the highway” type of person who constantly brags about past successes while not even remotely delivering at their current position (I’ve worked for two of these in my days), it’s not even worth the discussion, either suck it up or move on (in both cases I chose move on).

I should clarify that I think the percentage of people who refuse to change or accept new ideas is very low.

Ryan,

I’ve never really understood the “Proc’s are easier/safer to deploy” reasoning, because it’s simply not true. While working at a company who used Oracle there were two incidents where the DBAs brought the entire Database to a screeching halt while deploying modified Stored Procedures due to the horrible structure of their Packages.

There are some cases where Stored Procedures make sense (ETLs come to mind), but deployment definitely shouldn’t be a criteria. Let’s face it, a bad deployment on the DB can cause a lot more problems than a bad deployment of a single application.

Ryan,

re: Abstractions
A sproc is *an* abstraction layer. The issue is whether it’s the best choice for an abstraction layer, all things considered. Sometimes, the answer might be yes, but those cases are almost infinitely more rare than vendor-driven technology users understand.

The issue isn’t whether you have *an* abstraction layer, it whether you have *the right* abstraction layer. Without the right stuff in place, the potential for productivity takes a sharp dive. That doesn’t make good business sense.

re: Behavior
If you can change the behavior of your application’s core logic without re-deploying, then you’re likely not doing the due diligence to prove that those changes haven’t introduced malfunctions into the business system and damaged the business’s ability to achieve it’s goals.

Quick, easy, and assured deployment is a given: press a button, run a script, run some smoke tests. If you don’t have the resources to do this kinds of assured automation, then you might need to look to reclaiming some of the productivity lost to using the wrong abstractions.

re: Performance
All parametrized SQL, regardless of the abstraction from where it originates, is compiled by data servers like SQL Server. There is very little business-significant performance gains to be had here.

re: Security
The same security mechanisms that are used in, for example, SQL Server, to secure access to the execution of stored pros can also be used to secure access to the execution of commands against tables.

There’s little difference between the work needed to be done to accomplish either, and little difference in the outcomes.

the “behavior” of your application can be easily changed without redeploying 3.) the sproc is more secure and is recompiled (better performance) 4.) Mapping “hot to the tables” is simply unacceptable

I think we can all agree that sometimes its best to use a sproc, sometimes applications just ARE data-centric and sometimes (dare I say it) System.Data.DataTable is the right tool for the job.

A codebetter article describing best practices for these things? THAT I would love to see. And I really mean that!

“Bitching at Microsoft about EF was basically useless. ”
Don’t be too hard on yourself here, that whole episode gave a lot of us the leverage we’d been lacking to avoid going down that road.