Don’t Let the Database Dictate Your Object Model

Mike,

Jeremy is specifically referring to “business logic intensive” systems. For most simple applications, a 1:1 ratio between tables and classes works fine. In fact, i would go further to say that it works fine for many moderately complex applications. Where it doesn’t work well is:

- Legacy databases that are not well-normalized and/or have weird schemas. I have a real life example for a health related application. There are 4 tables that all hold diabetes information for patients. The only difference is there is different data in each table based upon a flag. Its a bad design and all should be in one table with an extra column to flag which type of diabetes patient it is. I don’t really want 4 DiabetesInfo classes. Of course in this case, most O/R Mappers won’t do you much good either, except maybe as a wrapper for data access.
- Applications that have a lot of business complexity. The example i like to use is a Address class. You may have address related fields in your database in an Orders table, an Address table (with a CustomerID relating back to Customers) or an Employees table all in the same database. However, you don’t want address logic in 3 places if you’re doing more than simple CRUD stuff. What if there is business logic to compare the distance between two addresses based on City, State and Zip. This may need to be rolled into its own class.

There are a number of ways to get around this:
-Duplicate (copy & paste) the business logic from one mapped class to another. (BLEH!)
-Don’t use the O/R Mapper for those specific tables and instead roll your own business objects and data access code.
-Create “Helper” classes (AddressHelper, DiabetesInfoHelper) in a very procedural fashion that have static methods which grab all the related data from the OR Mapped classes and performs the business logic and return the results.
-Create another domain object that the O/R Mapper classes use or refer to that handles the business logic. (NHibernate does something similar to this out of the box with a feature called components.)
-Create another domain object that wraps the O/R Mapped classes where necessary.

I’m sure there are more ways, but what i’ve noticed is that almost all the solutions cause your class/object model to be different than the actual tables, IF you don’t want to copy and paste.

I don’t want to copy and paste.

This post is right in line with a decision I must make. On my current project, the data model started bottom up (the DB schema first — the domain expert is a DB guy). The persistence layer is built with the DAO/entities pattern, with JPA and hibenate. We are building the domain model on top, following DDD patterns.

We defined entities with JPA annotations that map to the DB schema. This database will contain a lot of data: it stores electrical meter data from million homes with usage snapshots at every 5 minutes. Therefore, the DB schema is optimized for that, which results in annotated entities that are not always best suited for the domain model where we need to implement a lot of complicated behaviors and data analysis code.

I am tempted to define ddd domain entities that are different from the persistence JPA entities. This has many implementation consequences: we need to convert between JPA entities and DDD entities back and forth, we can’t use the JPQL from the client side, we can’t take advatage of many of the ORM facilities.

As a bonus, some of these DDD modules may be used by the legacy application (the current project is a rewrite of this legacy app)

For these reason, this post comforts me in the idea of having a clear separation between my domain and the persistence layer.

Still not sure thouth… not an easy choice. This has a clear impact on the long run and will be difficult to refactor.

< >
What is Object Model anyway?

OODB – It is so interesting: The order I experienced is Poet (an OODB) by C++ and then Java, then, Poet’s OR mapping, then, other OR mapping tools. The history is that in modern C++ age, we have OODB hype; then, during Java age, we moved to more realistic OR; I really hope .Net can revive OODB, so that we can be proud of something new.

So after catching up on all of this (took me way too long to read) I don’t think my mind was changed. There are a million ways to skin a cat. Regardless how you do it, don’t let your database design dictate your object model design and don’t let your object model design dictate your database design. However, quite naturally the two will look a lot alike more often than different.

Noone really understands half of what Survic is saying. His English is so bad and his thoughts so incomplete that you all should just ignore him instead of responding.

the souls of behaviors are data

Survic,

Rather than cluttering up Jeremy’s blog, i’ve posted a more detailed response to you here:

http://dotnettricks.com/blogs/craigbowesblog/archive/2007/03/03/255.aspx

Thanks,
Craig

Fregas: I read your posts again.

You said: “For .NET 2.0, i can’t remember a time where i really need my own, custom, strongly typed collection. It doesn’t mean it will never happen, but so far, I just use generics.”

– we agree here.

You also said: “”Ah ha!” I thought. SectionList is probably nothing more than a strongly typed collection, with some custom business rules added. And this led me to think of another obvious one: shopping carts! “

– And this led me to think of yet another even more obvious one ……..

Do you see the problem? DDD is crap. You have to agree with me, because you said, you did not use “strongly typed collection” in .net20. If DDD is right, you should have used that evil thing all over the places!

I bet a coke that after one more year, if somehow you are required to use .net 1.1 (why, do not ask me, in M$ world, time can go back easily ;-), you will use casting, instead of the STC. Casting is ugly, but it does not mess up your thinking.

You may call me a snob, however, because of my continuous VB background (from VB3, then, a lot of VB6, some VB.Net), and because I always have a soft spot toward it because of its lightweight “get things done” attitude (RAD attitude, or, agile attitude), to be fair to myself ;-), it would be a stretch to say that.

Having said that, when I use .Net, even in VB.net (I try very hard avoiding it), I follow exactly what I do in Java. That is the source that I do not like strongly typed collection so much, because you do not have that kind of thing in java (just like you do not have dataset in java!), and you are used to it. The style is actually just like generics, paying some price on performance and type safety.

Then, some people pushed using strongly typed collection systematically, and braging it as “best practice”, that basically made me think and pushed me to the edges, that was when the “do not like it” became “dislike it”.

And I justified my “dislikeness” by creating a theory, saying that modern OO does not use the concept of “custom strongly typed collections”. If you need some logic, create a non-collection class that contains a general collection (list or dictionary). — I am using a cynical tone to keep myself honest here, because perehaps it is all biassed; however, I really believe it is actually true: in modern C++ and Java (I am not sure smalltalk, but I suspect it is the case also), the concept of “custom strongly typed collections” is simply dropped.

So, if you use strongly typed collections, you are messing up people’s brain, do not do that! It is not about how easy it is, it is about thinking! A typed colleciton simply throw people off.

By the way, I observed that most people here treat Martin Fowler and DDD as unambiguous “source of truth”, so to speak. I do not share that at all. I love Martin’s “Analysis Patterns” (still, I always believe it should be read together with David C. Hay’s book). However, I put his other stuff and DDD in the category of “antique OO”. Surprise? Not at all. By “antique OO”, again call me a snob, I basically mean that a person have no serious experience in modern C++ AND (not or!) Java, or, did not learn very hard from modern C++ AND (not or) Java.

After being in Java and reading Java book, I simply cannot see any value in Martin’s recent work, and the DDD. There are just too much renaming and confusion. Trying to be “neutral” between M$ and Java, or, trying to “invent” something by renaming things, it is simply not right; also, in enterprise computing, M$ has nothing to offer (except RAD, as I said, we should be brave to take it into enterprise computing), why even bother, call “dataset” as … — I resent it the renaming scam to such a degree, I do not even remember it – it is totally ridiculous. I say it again, ridiculous – dataset does not become better or justified by being given a “pattern” name, give me break!!! To me, the whole thing is simply a strategy to get M$’s contract, that is all about it.

My source of modern OO? Experiences in EJB, and books on J2EE. One of the best among them is “ejb design patterns”.

With this disclosure, you know what I am going to say about SectionList

Another example; however, before I say it, I want to point out that I recommend loudly CSLA to everybody, anybody. However, I use it by tearing it apart, use its good parts, and show its terrible parts as anti-patterns.

Now back to strongly typed collections. If you like them, take a look of CSLA, then, you will know the real nature of strongly typed collection: pure evil.

Survic,

I don’t see how strongly typed collections offend you so much. In most cases they were a means to get around the lack of generic support in .NET 1.1. However, in a couple of weird cases, it may STILL make sense to make a custom, strongly typed collection. Perhaps you need an OrderLine collection that will not accept more than 100 lines. You could create a custom collection (strongly typed) that throws an exception or triggers a validation rule or something when the count goes over 100. I realize this is a weird, rare case, but it could happen. There are other ways to accomplish this, but making your own collection (that inherits from CollectionBase so its very little custom code) just isn’t very hard. I don’t see how that constitutes “PURE EVIL.”

Additionally, I wrote that blog article when i was still doing .NET 1.1. For .NET 2.0, i can’t remember a time where i really need my own, custom, strongly typed collection. It doesn’t mean it will never happen, but so far, I just use generics. But if i had to do a .NET 1.1 project, i might use codesmith or something to generate my strongly typed collections for me.

-Craig/Fregas

By “Strongly typed collections”, I mean pre-generic ones. Strangely, they are carried over in .Net 20, even when we have generics (i.e., you use both generics and custom collections). People can argue whatever, I will always see it as a clear sign that ‘this guy is still in VB6’!

The test stone is that, when you were in .Net 1.1, did you use “strongly typed collections” or not. If you did not, then, we know you are a hardcore of modern OO; if yes, you are somehow still in VB6. Consider being warned

As for old databases, I certainly agree that in that situation, OR mapping sometimes is more trouble than solution. Actually that is a rule number one of OR: the technology you choose must be compatible with manual DAO, and you should feel free to it anytime! A lot of times, the sql queries are used freely. Who need another semi-completed query language!

Then, we have to concede to ADM-only. You can do whatever in DAO, as long as you have ADM. Really, OR or not, it does not matter, the real key is ADM. It is not strange: ADM is the core of SOA, anyway.

Strangely, I find myself totally agree the “parallel evolution” concept — as long as they are not “separated”. The implementations are certainly in parallel, however, conceptually they are one.

Because of ADM, down with the “behavior” thing. I keep fighting this thing for so long, and just recently I have realized its nature: antique OO! In modern OO, “behavior” must immediately be split into two: validation or “real” behavior (service-like behavior). Validation is with the ADM, and it does not hurt anything. The “real” behavior is not with the ADM, and rightly so – the antique OO is very misleading here.

Vega: I like you say “sprocs/triggers do not count”, for me, the reason is that they are just evil. Once you got the idea of OR and application server (it started from “entity bean”, which itself is evil, ironic!), you really begin to treat DB just as DB.

Survic, you have some good points, and I agree completely with getting away from what I call impeded OO models (mostly due to deficient OO capabilities in certain languages, i.e. VB pre .NET for example). However, with regards to some of your other arguments, the problem is that while what you suggest works well for new projects, there are a lot of databases out there that were not designed to be used with ORM technologies, and it’s difficult (sometimes impossible) to use certain ORM products on these. These companies have been running for years on that data and aren’t willing to so easily migrate. As a developer, there’s nothing you can do about that situation except deal with it.

Secondly, what’s wrong with type-safe collections? After all, generic collections produce type-safe collections (either at compile time with Java or run-time with C#/VB.NET). A List is a Customer-type-safe list. You cannot add an OrderLineItem to it.

I read Fregas’s blog, I noticed:

>>>>I prefer strongly typed collections (or possibly generics in .NET 2.0) so my Content object would have a HistoryItems property which is Read Only and of type ContentHistoryCollection.

————–
My goodness! It is another example of antique OO directly from the time-froze machine VB6. “Strongly typed collections” are pure evil, period.

Of course, use generics – but it is pure, bare bone generics, no strongly typed collections at all.

Further, if you use .Net 1.1, then, use casting. Type safe and performance? Give me break, in Java, people have been doing that for many years, for serious enterprise applications – again, modern OO, not antique pseudo OO from VB6.

I know the following can be perceived as offensive, especially for long time experts in a different tradition. However it is the price to be on a platform that is undertaking a huge quantum jump.

1. OR mapping is the way to go. Java did it, M$ now finally makes up its mind to follow suit. So, do not even try to be different, just follow it.
2. This means, the so called “anemic domain model” is actually the way to go. J2EE entity bean started it, OR mapping makes it ubiquitous in Java world (come on, in this regard, there is no difference between entity bean and OR mapping). Again, do not even try other way around, just follow it.
3. This also means stored procedures are gone forever, of course, you can use them in some weird cases, but as a “layer”, it is gone forever. Again, do not even try to complain, just follow it.
4. Because of all the above, the only reason to emphasize the “first-ness” of “entity classes” over database schema is political – to shut up people who cannot understand “entity classes”, and, it is a good political technique. However, truth being told, it is only a political tool, nothing else.
5. As for OO, it is odd that so few people realize that the core of OO is indeed ER. I am glad that Frans knows it. I guess the reason for that is that to understand it, you really need first understand that inheritance and design patterns are easy and not that important. Once you really get that (it takes years, of course – sorry for being offensive, but truth must be told ;-), then, you will really appreciate that the real core of OO is indeed ER, or, normalization. To speed up the process, read Martin Fowler’s analysis patterns, together with David C. Hay ‘s Data Model Patterns: Conventions of Thought.

DevPrime,

One of the things i read in a book called “Object Thinking” is that objects/classes are defined by BEHAVIOR not DATA (nor where or how that data is stored.) This principle has served me well over and over.

In Frans’ world, everything is modeled on the data, the relational data structure, and your classes HAVE TO follow suit, which really sacrifices a lot in your object model. I believe that Frans is probably correct that the original meaning of the word “Entity” was from the RDBMS world, but language inevitable evolves (for good or ill) and that doesn’t mean classes should be the same as database entities/tables, just because some people seem to think or talk as if they are always the same. In many cases they can be, (in simple applications especially) but complicated business logic often dictates that they should differ if you want to avoid code duplication etc, as you and Jeremy have already pointed out. A good ORM product or a good ORM strategy if your hand-coding it all yourself, should allow for these differences.

Jeremy,

I think you’re wasting your time with Frans. His “O/R Mapper” and his whole thought process is that the database is the center of the universe, and that the business classes are merely a by-product of that. He is too arrogant and defensive to consider any other thought process as being worthy.

I totally agree with you that a good O/R Mapper should let you define business objects separately from the database tables. On simple stuff it doesn’t matter, but like you said, when you get to a system that has lots of business logic, or in your case a (legacy?) database that has over 500 tables, you’ll end up with lots of duplication.

Think how many tables have address information duplicated in them. If you simply gen the business classes from the tables, you’ll end up with many classes with duplicate address information (and likely duplicate business logic.) At the same time, if you start with the domain layer and make “Address” a separate class, then you may not want to generate a separate address database table. There may be some relationships where the information should be in its own table and others where it should be part of a parent table such as customer, because its a one-to-one relationship. Instead, having a single address CLASS separate from various TABLES that store address information, is the way to go in complex applications.

You article is similar to one i wrote a while back:
http://dotnettricks.com/blogs/craigbowesblog/archive/2005/09/05/33.aspx

Great article Jeremy. Keep up the good work.

I don’t think this dilemma is different from any of the other design dilemmas – the answer is still it depends.

I would tend to agree that in most system that aren’t just data entry or reporting you shouldn’t let the database dictate the object model – but many if not most systems are a mix of both data centric tasks and logic centric tasks – so you either mix approaches or you use one and suffer where it isn’t appropriate.

Regarding Ted Neward’s post- I don’t agree with him. It was true once but O/R Mapping is quite mature now. Yes, it isn’t a panacea but it works well where it fits.
You can also see a paper I published awhile ago on O/R mapping (http://www.rgoarchitects.com/Files/ormappin.pdf)

vikas (http://vikasnetdev.blogspot.com/) mentioned this discussion to me. Obviously, I am on Frans’s side Take a look of this site: http://www.geocities.com/tablizer/whypr.htm I do not agree with all of it, but it really an antidote of DDD or OO.

However, do not get me wrong. My background is NOT VB6; and I am constantly trying to change VB6 culture.

As a result, as you may guess, I have another side of the story. Recently, I feel that it is very important that we have to insist the domain entity model “first”.

Note that as Fran pointed out, even you start from database schema, you can still do that, because here “first” is logical, not chronological.

My experience is that a lot of VB6ers simply cannot use a “conceptual model”. It does not matter where that model comes from. My observation is that even they know the database very well, very strangely, they simply cannot leverage that when they do coding. You may think, my goodness, how difficult can that be! However, that is the reality. Poor guys. The paradigm shift is so difficult!

My guess is that the logic importance of the “model” makes some of us believe it must be chronologically first. That is not true. However, I now do believe that because the “model” is so important, so, “politically”, it is worth it to say that it must be done first.

Frans,

“If you think starting with classes is any different than starting with an NIAM/ORM model which is transfered into E/R for tables you’re clearly mistaken: they both describe the same concepts: entities and relations between them, which attributes are in which entity, which entity inherits from which entity, what are the constraints etc. etc.”

It’s not completely about data, relationships, and constraints. I’m also worried about cohesive classes, encapsulation, and reducing the duplication of business logic in the domain model. Yes, Frans, I get it, they represent the same logical data. Duh. But the domain model is responsible for the behavior and business logic of the application, and the exact structure of the database may not be the best way to build the domain model to carry out that business logic. What I should probably say more clearly is that the domain model is shaped by a little bit different set of forces and deserves its own design, which might easily need to diverge from the database model.

I am generally against creating a database model first, or at least making the database the paramount piece, for a logic intensive application because I think it always seems to lead to an anemic domain model, procedural code, duplication, and tight coupling. If all of the early focus is on the database model the domain model tends not to get enough attention.

When it comes time to choose a persistence tool, I want the flexibility to allow the domain model to diverge from the strict table model and an easy ability to evolve both models. Personally, I dislike any tool or framework that forces me to codegen my domain model classes, but I’m willing to write that off as a preference.

I wouldn’t necessarily advocate trying to dump the database out of the object model ala NEO either. I’ll compromise with you and say there’s no reason you couldn’t happily shape the domain model and the database model at the same time, but I’ll never agree with database first on anything but reporting applications.

Jeremy

If you think starting with classes is any different than starting with an NIAM/ORM model which is transfered into E/R for tables you’re clearly mistaken: they both describe the same concepts: entities and relations between them, which attributes are in which entity, which entity inherits from which entity, what are the constraints etc. etc.

So if you reverse engineer a relational model to a model which is at the same level as a NIAM/ORM model (and ORM means ‘Object Role Modelling’ http://www.orm.net) you simply can use that model to create entity classes which comes down to th same things as you would have when you would created the classes by hand first.

Better yet, if you think you can simply create 500+ entity classes without thinking them through, you’re mistaken as well. No big relational model is created without a proper abstract model, and it’s perfectly doable to get back to that model with a relational schema at hand.

Furthermore, the concept of the entity doesn’t change if you talk about classes or tables, they’re both containers which are physical representations of the entity definitions.

I wrote an essay about this once, it’s in Jimmy Nilsson’s ADDD&P patterns book as well:
http://weblogs.asp.net/fbouma/archive/2006/08/23/Essay_3A00_-The-Database-Model-is-the-Domain-Model.aspx

Oh that Ted Neward article… What a joke. The thing is that the problem is very simple and it doesn’t matter at which side you’re starting, the END RESULT is the same. At least with the proper tools when you’re starting from the db side and with the proper level of knowledge when you’re starting at the classes side.

So the situation actually is that you start with a set of abstract definitions of entities, which we could define as an entity model, at the level of a NIAM / ORM diagram for example. With that model you can for example create entity classes (if you’re into S&M, you can write them by hand for example for all 500 tables in your system) and you can also create the tables which will contain the entity instances. (You all do know the difference between an entity definition, entity class, entity class instance, and entity instance?)

So really, I disagree with what you’re trying to say, Jeremy, because your article breaths ‘Don’t start with the database model!!’, while that’s completely not what most o/r mappers do. They reverse engineer the relational model to a higher level, which is the same level on which you’re thinking when you’re creating the entity classes.

Because, and this is perhaps the hardest part, what’s in the DB and what’s in memory in your entity class instances, that’s THE SAME THING.

The problem is vaguely this –
when designing a system, you have a conceptual model, a logical model, and a physical model (which in a minute, I’ll make multiply, so find a seat :-)). The conceptual model is a thinned version of the how the business sees the processes and entities that will eventually make up the system. It’s not rare to see conceptual models only showing bare minimum details to get the major ideas across. The logical model more acurately shows what needs to be done to create real entities in full detail, capable of enacting a real-life system. The physical model is a real-life implementation of the logical model. However, between the conceptual model and the logical model, you have a number of things that go on, for example, database object normalization. So “entities” don’t really map one-to-one between those two models. Now, so far I’ve only talked about data models, so what happens with domain model classes?
Strictly speaking, domain model classes don’t necessarily care about things like normal forms, at least in the traditional relational data sort of way. In fact, they try to be the conceptual model with full details (after all, they are modeling the business, not the database). So there is in fact a big disconnect between domain entity classes and domain entity database objects, bigger than just the relational-to-object rift. It’s a huge mistake to assume that anyone *should* map objects to tables or views one-to-one for many reasons. Having said that, current O/R tools fall into camps that are mostly described as:
1) don’t support any model other than one-to-one
2) support other models, but only with some “backhand” magic and elbow grease (sometimes more, sometimes less)

To answer another comment, while I can’t comment (yet) on much of the new progress with LINQ entities, I can say that I’m not 100% satisfied that it will be everything we want (and for more reasons than just the O/R mapping aspect), though the functionality has some great ideas and features in it.

On a more positive note, I am currently working on such a system that allows very rich domain model classes and has an extremely high level of flexibility in terms of persistance (in fact, it will even take entities from service output rather than databases if need be). On the down side, it will be a while before I finish all the tools to go with it that will eventually make this framework as simple to use as current O/R mappers Basically, the technology is more than possible, but current offerings just seem to be coming up a bit short, mostly because of the original intent and design of the tools.

Just to clarify here, are you saying that using something like the Entity Data Modeller in Visual Studio Orcas is a bad thing because it does act as glue between data model and object model? Or is it a good thing because it allows the two to be distinct and then translates between the two?