This post comes from a discussion I had with Clemens Vasters at NDC but is a much bigger discussion than the context we had it in which was Azure specific. An exaple of where this also comes into play is with the REST crowd who also want all idempotent operations.
Sure distributed transactions are evil … until well they aren’t. There is a trade off between capital investment and *insert whatever reason you don’t like distributed transactions*. First let’s look at the alternative.
To briefly cover idempotency from Gregor
“In the world of distributed systems, idempotency translates to the fact that an operation can be invoked repeatedly without changing the result.”
This is a huge benefit because in failure conditions etc, I can just send you the same message multiple times and it won’t affect you in a bad way. As an example I peek a queue and process the message writing something. Then suddenly my power goes out before I have actually removed the item off the queue. When I start up, I can just re-process the message (it won’t cause destructive change).
Many operations are idempotent to start with. Consider a read or the setting of a property. Other options are not idempotent and need code to be written to make them idempotent as an example charging someone’s credit card what happens if you get that message twice, ouch. The trick here is the operations that are not naturally idempotent, they require us to write code to make them idempotent. That code has a cost associated with it in terms of creation, maintenance, testing, and conceptually.
Clemens gave an example of a system he worked on to illustrate his point, a credit card processing system.
A credit card processing system has a very limited number of possible transaction types. For sake of discussion let’s say that there are 15 and they are called very often, each has a large amount of logic associated with each transaction type (a similar example might be a trading system, not a lot of command types but a lot of logic associated with each). In these cases the idempotency is not a big deal because the amount of code being added is very small for it as a percentage of the total code.
This is very different than a stereotypical business application where there will orders of magnitude more transaction types and they will be often called rarely with small amounts of logic behind them … for sake of discussion let’s say closer to 1500 types with the detection code being vastly larger as a percentage (I have seen business systems where there was more code to ensure the idempotency than to perform the actual operation).
We can make everything idempotent but we will have a very different cost associated with it in the stereotypical business system than with the first system. Out of our 1500 operations how many of them are not naturally idempotent? How much code will we have to write in order to make them idempotent (quite likely ALOT!). Looked at as a percentage of the overall code it will be much higher than in the first example as there is much less code per transaction type on average.
Now what are we gaining by this overhead in order to have all idempotent operations? Scalability and Availability are two things that come immediately to mind. It is very hard to scale distributed transactions to an extremely high level (think Azure scale), many other trade offs start to come in at this level of scaling.
Are you scaling to Azure level? What is the ROI of that code you wrote to make things idempotent for the business system? This is not to say to use distributed transactions haphazardly (you always want to limit their usage as much as possible) but to point out that there are certainly situations where they may be favorable, there is a tradeoff involved. Again this is not to pick on Azure alone, REST also does this.
It is important to consider this tradeoff when you decide to say put an application onto Azure. How well do their non-functional requirements as a platform meet yours as an application? What types of decisions like this have they forced upon you and what will be the long term cost of those decisions.