Isn't the issue really about the choice between a RDBMS vs NoSQL'esque database systems? If you're set on using an RDBMS then using an ORM is a no brainer is it not?

This post was mentioned on Twitter by FL_Interactive: Using an ORM is like kissing your sister: I am pretty sure I have heard that phrase somewhere else before, no idea... http://bit.ly/9h6JVO

... other considerations:

- Time spent arguing for NoSQL solution with DBAs, management and assorted other eyebrow-raisers vs. time spent due to ORM related overhead.

- Time spent getting everybody up to speed with new persistence technology most won't be familiar with.

The Forces Of Habit take courage and good (non-technical) reasons to resist.  

I, as many of us developers, work for a development company and make business applications for a living.

In this corporate world, we have been able to push a number of innovations like new GUI's tecnologies, web architectures and you name it.

But clients have understood well enough that their biggest asset are not architectures or guis or even business rules, but data.

In this context, huge inversions have been made to backup, monitor, test and secure data storage solutions. These solutions, of course, are relational databases.

Relational databases is what we have, and what our clients are use to work with, and they work remarkably well for what they do, and there's  a huge infrastructure around them that is highly specific.

My point is that the data storage, for real applications, is not a decision to make on the development team anymore, it is imposed by the client or the IT dept. And that's not going to change until object/document/whatever databases prove in some way that they are a hugely better solution than relational databases, provide at least the same guaranties, and are backed up by corporations as Oracle or Microsoft.

Data is a serious thing, and no one is willing to play with it just because there's a new hot way around.

And by the way, we have been doing this relational thingy a "couple of years" now, we should be good at it by now, don't we?.

What choice do you have when you're stuck in a Microsoft shop, where you have to use the only database program they offer?

ORM's come to the rescue when you're constrained to a particular technology stack.

What is it about relational databases that might necessitate their use for OLTP?

I think I totally disagree with the sentiment here.  I'm on a large project, we have serialized ruby objects stored as blobs in a mysql table, it's effectively a k-v store.  It works great, it's pretty fast, and...

...do you know how many MySQL DBA's there are?  How many organizations are willing to put an SLA around MySQL, understand how to backup and restore it and monitor it, etc?  

How many orgs are offering SLA's around Redis?

The NoSql "movement" is in serious danger of jumping the shark (if it hasn't already)...mysql is a daemon, data is stored in some files on disk and you have all sorts of options around the form the data takes within it.  It's extremely fast at certain operations.  NoSql is not a revolution, and each of these k-v stores have their own set of problems.

I don't understand the point... If we could start from a blank page maybe we could make all the right architectural decisions, but more often we have to start from a legacy existing system and this is a huge obligation.

I don't know any customer who let me start from another type of "repository" which wasn't his existing RDBMS.

Too much money investment on RDMS for our customers not to build applcations on them.

From design perspective - if you properly abstract the Data Storage from the business model, it shouldn't mater 'where' it's stored.

Use the data-store that allows you to focus on the business problem domain. Why not use a K-V or object Db to accelerate development.

At some point, if you choosing to (or forced to) persist to a RDBMS, then you can quantify the amount of work to do the CRUD ...

... it's similar to having (or planning to have) object cache.

It is probably appropriate for I.T to dictate this just as they dictate the OS the servers run on.  The real trouble is the amount of risk involved in having a one application data silo as opposed to something that is more readily  structured, queryable and understood.  I would like nothing more then to drop this relational model burden.  But to do so will add an extra burden to sales, and to IT.  And if we can't sale our wares, well we just wasted our time.  

One of the reasons I came to the company I'm at now is that Tool-driven design was eschewed.

An RDBMS makes a lot of sense for a lot of applications, especially if you want the application data to be available for re-use to any number of off-the-shelf applications (in our case, reporting and workflow systems).

In enterprises striving to produce an SOA platform for their business, exposing the data through a SQL is antithetical to their end goal; a suite of composable web services that present the data AND guarantees business integrity of that data in a way that no SQL constraints or triggers can achieve.

The argument that developers know SQL holds little water.  Developers learn new skills all the time.  A well thought out data abstraction that fits the data usage will still involve less development effort than shoehorning SQL via an ORM into your object models.  Maturity of tools has more credibility, but even when screws were first invented, pounding them in with a hammer was the wrong way to use them.

@Greg

Don't get me wrong here. I know object databases have proven themselves on fields where there is always a chance to start with the best tools every time, like science or high data volume/processing   systems, which are backed up with formal CS research.

With all mi heart, I would *love* to forget about persistence and just push objects/documents to a store, but I do realize that there are other concerns than my comfort as developer.

When we worked with files, and the database came, it did solve a lot of real problems for the business, like concurrency, corruption, security management, performance, reliability and consistency, among others.

Object/document databases may be better at some or all this concerns, but moving takes a cost that is lower than the befit obtained on doing so.

Only problem I have with K-V stores and object DBs are that they don't have any referential integrity (unless this has changed recently).  Also, they make it harder to audit data history.

Not to speak for Greg, but I assume its because most people don't even contemplate using anything but a RDBMS when starting something new.

People will argue to death over what testing method to use (TDD, BDD or "Test?  We don't have time for tests!"), what ORM to use (if your DBAs don't have a heart attack first), whether to use an IoC container or not, and even what data to store.  But the single most important decision, the backbone of your application is generally not given any consideration at all.

You're a MS shop?  Use SQL Server.   You're not a MS Shop? Use Oracle.  Done.

Obviously there are other considerations like you are adding a new app to a suite that all share a common DB or that your decision has been made for you since you are being paid to do the work.

But, how many people have actually proposed using a NoSQL Solution to the powers that be?  It's easy to say "they'd never let me do it", but how many people have tried?

@Greg

>My point exactly why is the IT department (a team of >system administrators) making the largest >architectural decision of an app

Because this IT/SA team is responsible for buying, maintaining, and manage the infrastructure and services. It's their responsibility to decide what enters and what stays out of the servers. Normally if development needs or recommends something it gets in, but in this case, since is data, they strongly want it to stay the way it is.

Another voice to the NoSQL push. Love it! I like your approach here - asking thought-provoking questions rather than what I've done, which is stick my digital foot in my mouth with lame attempts at humor.

Would love to see how CQRS meshes with OODBs...

Great post Greg!!!

Couple of thoughts!

1) I have never used Object Database but I am extremely interested in it. I love DDD but at work we don't use it. I think Object Database will make sense when you are storing objects. But I have seen that in many organizations they are still not using DDD and leveraging DataSets and DataTables.

2) Another problems is introducing new kind of storage systems in big organizations. Currently, it is extremely hard to introduce NHibernate, SubVersion, Git etc. The database is a HUGE architectural decision and the IT people or Managers will never ever approve this new design.

This will be hard transition and I think it will take couple of years to introduce Object Databases into main stream.

this article makes no sense.  If you are trying to poke systems using relational data in the eye, talk about the negatives of using relational data.  What have you said that has anything to do with the decision to use an ORM or not?  Get off the soapbox.

You're all wrong. Stored procedures is the only way to go. ;)

I like how people are turning up their noses at relational systems as if they haven't used them for YEARS now, even though they just recently got turned onto some OODB product.

Relational databases don't suck as much as you say, and object databases are not a cure-all so people need to stop acting so indignant about them.

"It pains me to no end that developers more often than not make the single largest architectural choice on their system without applying any rigor to the decision."

Where do you work that developers make architectural choices? You sound really disconnected from the coders in the trenches here. In my organization, we have "architects" that decide the naming conventions for stored procedures! Choices like which persistence platform to use are the most sought-after technology decisions in any organization. Sales staff from big vendors spend big money wooing management-level staff to their platform. Hundreds of thousands of dollars are spent in licensing, implementation and training as a result of deciding on a given persistence platform.

Some of us get pushback just trying to implement ORM, much less some kind of futuristic object data store!

In the past few weeks, Ive seen a handful of .NET bloggers all announcing their buy-in to "no-sql" technology, each post preaching a "free your mind from RDB!" theme, topped off with a catch-phrase headline.

In the majority of my experience, I work applications where data is crucial, and applications/reporting are relatively disposable in terms of importance.

I studied BigTable and CouchDB on an introductory level, but never actually used it, because it feels like the application model and persistence model structures are required to match apples to apples for the whole thing to work.

I'm interested to learning  more about is the replication seam between the "no-sql" persistence, and larger, more static storage tiers.

I agree with the fact that the RDBMS should not be a constant starting point in a project. Every element in the design should be considered carefully based on the user requirements and architecture principles for the system in question.

At the moment I am designing a system that will probably never have a database. IMHO It should be a conscious decision to use an RDBMS, as with any design decision, where you weigh the risks, constraints and consequences. Ad-hoc Query/Reporting requirements on tuples can be a hint for needing an RDBMS :) And even then the RDBMS doesn't have to be central in your design.

@Roberto,

That is why Greg talks about Greenfield, where you have a blank sheet.

@Davidlaym,

This is not about Relational vs Non-Relational databases. I have often seen this kind of justification in many areas, from having IT imposing security restrictions that influence the architecture to not being able to get the exact requirements because of politics. Maybe it's us failing at our job by taking the easy route out.

Greg, I find the point you're making about capturing the intent of the changed data very interesting. Do you have any resources where I can find more information on that (ie how to implement such a mechanism, things to keep in mind, etc.)?

Also, do you have any experience with Object Databases? If so, which are good/bad for you? I'd like to look into that, because I am developing an application now. I just started so I could still change the data storage if it suits me better.

What I have always been told however, is that RDBMS's are faster and more scalable. Or is that not true anymore?

I was thinking about this recently too with my involvement in an OSS project, in wondering if we really need the RDBMS and the complexity and layers of persistence code...  Last time I implemented save/load for a small professional tool, it took half an hour to develop and smoke test the feature because .NET makes XML serialization of a properly designed object model a breeze.  I've been wanting to check out a couple object database options, but I don't know where to start.  Could you recommend a couple good, free object database options to consider, maybe with some opinions on the key strengths of each?

@Hadi Hariri

I think it is. sort of.

The posts expresses that ORMs make developers feel pain and that the relief comes from object/document dbs.

In this case, the pain a developer feels (which should be decreasing given the tools we have on our disposal by now and in the future) is not enough reason for looking for alternatives (for the mayority, there's always people like us who will discuss the new or alternative ways).

A defacto standard is stablished, much like ms office or even windows, people just don't think on alternatives.

I can't agree more on how unknown this alternative solutions are for most developers because of the defacto standard stablished on databases.

Now, I must recognize that Greg's intention is also to express that this solutions should get more coverage for all to know them and apply them when they make sense and stop the idiocy to throw relational dbs at every problem. Granted and understood.

My point is that the way this is presented, is wrong, because relational databases in MOST cases make sense. Relational databases are presented like evil or torture, and I simply can't agree on that.

I'm pretty sure line comes from "Ties are like kissing your sister" from Bad News Bears.  

@Greg, you're absolutely right.  We should really be keeping the context of our needs in mind when making this important architectural decision.  For my next personal project I will strongly most likely end up using an object db.  At work I know that it is a battle that I just would not be capable of winning.

Eventually object dbs will go mainstream, but until that happens most of us will not get an opportunity to work with them in our professional settings.

I agree with the person who said you sound disconnected. Not just from developers in the trenches, but also from team leads like myself, who have a hand in making choices regarding technology.

Sure, you make some points regarding something that 'may' become a standard for us in the future. But for present day, I don't quite get where you are coming from. It's not just a matter of "hey, RDMNS are sooooo last year, lets do OODB!!!'. Do I say to the fortune 500 companies we deal with, "OK, I know you have millions of dollars invested in your data, and go to great ends to protect it, but we are going to implement a new, unproved, technology as a data store for the new app we are building for you. Sound good? Great!!!"

Again, you are way out of touch.

"... a couple good, free object database options to consider, ... "

http://couchdb.apache.org/

http://www.mongodb.org/

Check out the client/sites that are using them today, in live, production environments.

I'm for relational-data  ... I've been a DBA, and I'm sure it will be around for a long time ... I've written 100,000 store procedures, used LinqToSQL, NHibernate .... all good ....

....but on my next project, I'm positioning it this way:

"I can use a object db, and cut development time down, reduce costs, get our prototype out sooner. If we want(need) an relational database backend, no problem - my app design will allow us to swap-in a different data layer .... later".

.. .and we can keep the object DB in place as a 'caching' layer :)

You're talking about the relational model, but the title is "ORM", which is not the problem comes from.

If you're not use relational model, you need an "OXM" just like ORM to bridge the gap between object model and k/v model.

I'm in a incubation project using Tokyo Tyrant and MongoDB without RDBMS, it's quite a pain until we build our own mapping mechanism.

.NET Pulse 2/19/2010

Greg concerning your first comment "...as opposed to storing a series of events in an event log that they rebuild current state from? eg: they capture the intent of changed data not just the change itself, it is well recognized that the storage of such information is highly valuable."

Do you have some reading suggestions to learn more about this approach?

I didn’t necessarily mean to piggyback off Greg’s two posts on ORMs but c’mon, what’s a hillbilly to

i haven't listened to it but there is a .NET Rocks episode on MongoDB

www.dotnetrocks.com/default.aspx

@people who think Greg is "out of touch".

Greg's out of touch because he doesn't know your particular situation?  That's a bit silly, isn't it?  At every development shop, regardless of size, someone is responsible for choosing what architectures/systems/IoC/ORMs/OSS projects/Coding Standards/Kitchen Sink that the developers use to write new systems.  If you don't don't have influence on those decisions, you have a few options:

1) Find a way to get that influence

2) Find another job where you have that influence

or

3) Just grin and take it.

There really are no other options.  If you can't be the somone (or on the team of "someones") making important decisions, and don't want to be, then there's really nothing to complain about.

When I started where I am now, I came in with a bunch of wacky ideas like Unit Testing, following the SOLID Principles and adopting OSS projects to speed up/improve our development.  I got some huge resistance, but I've gone 3 for 3 in about 18 months.  I had to drag some of the old guard kicking and screaming, but if I'd listened on day one when I was specifically told by a "Senior" dev: "Unit Testing, yeah, we don't want to do that here" I might as well have gotten my ball and went home.

A smart developer who can present ideas well (especially if there is a business case) can do wonders.  For example, just find out how many LoC you have in hand rolled DAL/Stored Procedure code and start there.  

Greg,

"@Steve  I was arguing against the need of a relational model ... you just said you werent using one ... good job.

Would you like another strawman to go hit with your bat?"

From your post:

" The choice to use an RDBMS drastically changes the options of where your architecture can go."

And I'll just say that your tone here and in your follow-up accusing your readers of not understanding you is irritating, to put it mildly.  I understood you quite well.

Seems pretty simple to me... Greg is saying that when you're green-fielding an app (white piece of paper), don't just say "we need to store data so we'll use an RDBMS".

you may very likely still end up there (our model requires a relational backend, our reporting tool works best with a sql db, too much work to fight our standards, too much fear around support, easier to claim sox compliance, whatever)... but, at least you'll know why.

Data Storages