I spend a lot of time and energy teaching systems designers how to deliver code that would be considered well architected. For the past three years, that has been predicated on the tenets of Service Oriented Architecture. What I continue to see 9 out of 10 times is deliverables that miss the mark. This list of seven sins is my therapy for the frustrations of designing to the SOA tenets (in other words I apologize ahead of time for the ranting :). Please leave comments, flames, counterpoints ... I'd love to hear your thoughts on all of these.
Without further adieu ... "The Seven Deadly Sins Of SOA"
1) The only way to ensure high availability or high performance in a SOA system is to replicate data between systems.
Commentary: A past colleague and myself wrote an entire article on this anti-pattern for SOA in The Architcture Journal. The idea that you need to physically move data between systems in order for them to work is about as logical as saying your entire database needs to be in memory on your app server in order for that to work. If you have that type of availability requirement then sure these types of solutions must be considered. More often, there's very small edge cases that could create a temporary delay in a business process. Usually, the issue isn't the availability of the service, instead its the manner in which it is consumed (i.e. batch or synchronous transactions instead of async transactions).
Fault tolerance is a design goal that all consuming applications must deliver based on their requirements, if the requirement is that it can absolutely never fail then you have to eliminate all external dependencies. That might mean caching the output of the service but you have to make sure you use it only as a cache. You can't start using it as a permanent data store and joining to the cache with other queries. Once you've done that you've built a very tightly coupled dependency.
The other myth I hear all the time in this space is that if I focus on "data residency" with my database design then I'll be able to avoid some of this replication. That's interesting but dangerous becuase I think it starts to move you down the path of putting together an enterprise database (or at a minimum one database per line of business). It also gives you an easy way to cheat and grab data that logically violates your defined boundaries. Again, the concern there is the coupling of business components down at the data tier. This is why I favor middle out domain driven design as opposed to a data tier up approach (more on this later).
Now as far as performance goes ... well you're going to see me talk a lot more about that one in other sins so I won't harp on it here. For more information on this one read "Data Replication as an Enterprise SOA Antipattern" at http://msdn2.microsoft.com/en-US/library/bb245678.aspx.
2) In order for us to achieve optimal performance we must pre-process all business transactions and persist the results using a batch process.
Commentary: I deal with this design strategy almost every day and it is the most damaging design approach I know of to the overall health of the enterprise architecture. This really boils down to a batch processing mind set versus an event driven architecture mind set. The latter being the one that provides you with better adaptability, supportability, and extensibility. The most obvious pitfall of pre-generating results in a batch process is that those results are immediately stale. If any of the data that feeds into these processes changes then the results we calculated in the batch job are now bad. There's also the horrible job of babysitting these super long running processes to make sure they do at least 90% of their work 100% of the time. Very often this requires people to be "watching the jobs" or building in things like retry logic or checkpoint/restart. In my opinion this is probably the most expensive and inflexible way to build an enterprise system. This ususally will result in overproducing results (batch processes don't know what the users are going to ask for) and adding horrible capacity and logging demands (when I talk about logging I mean things like archive logging in the database ... imagine a system that has to pre-create millions of results that would normally happen throughout the day when a service is called ... that's one big freaking transaction log file).
If you drive all of the processing back to the core events/transactions that begin all of this activity and if/when they become long running you favor asynchronous managed transactions over store and wait batch processes you'll find that you're in a much better position to adapt to your customers needs. Just about everything that happens in an enterprise can be traced back to some type of an event that some user is initiating ... the only things that don't have event triggers are those things that are time phased (for example triggering archiving of data after it's 14 days old). I can't even count how many times I've heard that we can just "add another step to the nightly batch" as an answer to a business problem that should have been as simple as adding a few lines of code into the managed business transaction/process/service. I often call myself the "batch killer" because I find if you force designers to think about how they would solve their problem without the batch crutch they use a lot more ingenuity and deliver a much more useful piece of software. This sin is a big one and I think it's the number one obstacle for any company trying to do enterprise SOA. Batch has to become a dirty word or you'll never see half the benefits you should be seeing with SOA.
3) In order for us to be service oriented we simply need to have a layered architecture that forces web services between our user interface and busines layer.
Commentary: I've seen first hand what can happen when you follow this approach. This strategy can at first seem like a good way to get the enterprise to buy into web services and SOA as a standard distributed design goal. The problem is, if you don't have good design practices that are based on appropriate component boundary definition then you're likely going to be putting web services in front of components that lack cohesiveness. This will result in the phenomenon known as "private services". I hear that from designers at least once a week. "Why do I need to define a web service if my application is the only one that will consume it". The answer to that is easy ... you are not building web services for your application you are building business components that are scoped to the enterprise and you are exposing business services using a neutral technology like HTTP/Text so it could be consumed by others. Ok ... maybe it's not that easy of an answer :)
Seriously though, the goal with SOA is to take the meaningful domain components and expose their logic via an interface that can be interoperable (this is why I love the flexibility in WCF .. I don't have to start with HTTP/Text ... I can move there when I need to). There's nothing magic about ASMX or WCF in terms of designing components so they can be useful to your enterprise. That burden falls back on the strategy architects and solution architects. You have to know that when you deliver a system you are delivering pieces of business functionality. You might be aggregating three or four interesting business components. If you don't define those components as seperate assemblies and encapsulate all of the aspects of those business components then I'm sure your services will suck. A service that simply exposes the logic to save some cobbled together composite business entity that a user dreamed up is probably not going to be useful to expose. However, two different services that expose data from very cohesive business components that are simply aggregated for viewing purposes will be. If you find yourself saying your web services are only useful to my application ... your design is off track. Stop .... back up ... and look at your component design boundaries. That's the problem.
4) The only services I need to create are CRUD services.
Commentary: This happens all the time and I believe it's because there is a serious lack of focus on the business processes/transactions that drive all of the work in the enterprise. I believe this comes down to what drives your software design. There's a lot of strategies in this realm, things like "Test Driven" or "Domain Driven" or "Architcture Driven". I think the important one is actually "Business Process Driven". A good friend of mine wrote a nice article on this called "Purporting the Potence of Process" (http://www.code-magazine.com/article.aspx?quickid=0703021&page=1). He and I are very like minded when it comes to enterprise systems development so I won't go into too much of rant here ... he's already done that for me ... thanks Ambrose :)
I will say this though, if you aren't making sure that business processes are helping to drive out your system then you probably are going to end up with relatively innocuous services like the CRUD services mentioned in this sin. The real business processes have very interesting steps and very interesting state that they will represent during their life cycle. If your design is lacking any sort of state diagrams or business process diagram then you're probably designing services that are based on an extremely limited view of what that business component is supposed to do.
5) I am in a .NET to .NET scenario here so why should I care about interop.
Commentary: This type of thought process just lacks vision. When a developer tells me they don't think any Java client will ever call their service I tend to say this "Your crystal ball is no clearer than my crystal ball". That being said, we don't want to add a ton of complexity just to win an interop award and put it on the wall. That's why it's so great that the WS standards bodies have helped us to understand what we can and can not do. This doesn't add any serious overhead to your design if you keep it in mind up front. Well, I'll take that back, with the onset of WCF and .NET 2.0 I would say that it doesn't add a lot of overhead to your design. In the days of .NET 1.1 when you tried to avoid things like datasets or collection types you found yourself having to do weird little hacks like manually genterating the proxy classes or manipulating the generated code to re-reference your data transfer entities. It really sucked.
That's why I love what is now available in WCF. This makes your interop decision an easy one, make sure you understand what should be available based on the WS-BasicProfile and if you want to define other access points that can do the fancy .NET to .NET stuff then go ahead and do it with another endpoint. This type of flexibility would have saved me hours of design arm wrestling with system designers who were 100% sure their services would never get used by a different platform. Oh by the way ... it's not just about can a Java client call your service either ... see my last post on BizTalk server or consider some of the other portal environments out there or third party systems that might be able to consume services in the future. You simply don't know what the future holds ... that's why architecture always tries to make strides toward the most broadly reusable design specifications (assuming of course you can still satisfy the requirements of the system).
6) I know that my logic in this component will never be reused by anyone other than my application ever until the end of time.
Commentary: This is a little different than the one you see above which is about reuse across platforms. This is where a designer tells me unequivically that the logic we are designing for this system is only useful to this one single deliverable. It's logic that no one anywhere will ever want to invoke ever. This is a very "one-off" thought process and also lacks vision for just about everything in the distibuted development world. I think just about any business service that gets defined around a cohesive business component has a potential to be used in a different way in the future. Let's use an example, lets say I am building a business service that is defined to be used as part of a batch process (yes ... that's right ... I see a lot of this). Well what if in the future we wanted to start calling that same business service realtime. It wouldn't be hard to imagine another entry point that wasn't responsible for processing 1 million business transactions in a single bound but instead it could handle batches of 10 or 20 that were being called from a user interface or from an asynchronous workflow. When you consider the increased capabilities of application servers to run parallel processes with multi core processes you have to expect that building scalable systems in the future will be different than today.
This brings me to another interesting point about this one-off approach to delivering distirbuted systems. It seems to me the same designers that feel like there is no way this process step could be used anywhere other than a batch process are also the ones that are not really strong at defining and delivering to an object based or domain driven design paradigm. When you think about the aspects of your system in terms of domain concepts you can't help but think that everything your defining has value beyond the scope of your deliverable. Isn't that what we as designers would want to strive for? It is amazing to me how many designers want to put up walls around their components because they want to deliver just "to the specs" of their problem. I guess part of me understands it. We don't usally have a parade for people when something they delivered is easier to reuse or update 2 years later. We do on the other hand reward those that meet the timelines of the project and deliver what was asked for in that time window (no matter how short sighted that may be).
It really is a shame though, I always find myself wanting to desperately build stuff that is interesting and useful to as broad a set of consumers as possible. It just feels more rewarding/fun to know that what I designed is now useful to 10 systems instead of 1. Maybe that's because my role as an architect affords me the ability to have broader scope. Maybe that's because the people I interact with are all similar in their approach at delivering systems. Not sure what it is but I just think it's really boring to deliver stuff that only satisfies one function and then privatize it so it has no flexibility going foward. That is procedural one-off thinking and it produces code that will have a finite lifetime because someday ... I promise you ... there's going to be some other use for that function you didn't think of. This is actually more of an issue of enterprise development agility than anything else. I actually think that topic deserves a blog entry of its own so I'll stop there.
7) Performance is not "a" requirement it's "the" requirement.
Commentary: Ok this one has been crawling all over me for the past 3 months or so. There always seems to be this extremely passionate set of people that make every decision based on uber-performance goals. Often times these are very smart people. They can explain every millisecond of overhead possible in every component of the system. There is usually a super heavy focus on how to optimize everything at the database and then make the rest of the architcture fit into whatever design goals exist for fetching and persisting data. It all makes all the sense in the world ... if performance is "the" requirement.
The problem with this is that it becomes almost a battle cry for every developer and designer in your enterprise. What was initially a design direction that should be followed when there's an extreme case of a process that has to read 15 million rows of database in 15 minutes (guess what that is ... can someone say batch process) becomes a strategy for everything that exists. Developers are always avoiding things like round trips to the database (even though they have very little overhead) or serialization to XML. That's where web services and even SOA start to get a dirty name in the "Performance Oriented" design world. Like anything with architecture/systems design performance has to be considered and has to be evaluated against the requirements and the long term goals of the architceture.
Very often (like 90% of the time) the performance overhead associated with calling a web service is insignificant. The alternative is always to take the logic and either rewrite it (often using cut and paste techniques) or just redploy the components locally (contributing to versioning, maintainbability, and code fallout problems). All for the sake of avoiding milliseconds of xml serialization. I've already said it earlier in this post but I'll say it again. I can't wait to have these performance discussions once WCF is mature and available to me as a tool in my enterprise customers. I would love for a designer to argue that the overhead associated with a named pipe transport or a TCP/Binary call will kill their performance. I guess this is another reason I have nighmares about batch processes. Usually these arguments are only interesting when we are talking about millions of transactions in a compressed time window. If you can remain creative and use some ingenuity to avoid the batch process all together then you'll never be dealing with a "we can't use a centralized component because it adds .0000001 of overhead".
No ... I'm not done .... I actually could rant about this for another couple of paragraphs but I'm going to stop ... for now :)
In closing, I hope you found some of these items interesting and maybe even had your eyes opened so some issues in your enterprise that lead to poorly designed SOA components. The good news is, I think this is all getting a lot better as the frameworks and tools for delivering solutions to the SOA tenets improve. WCF and .NET 2.0 took a lot of the problems and made them simple configuration switches. I also see more and more designers and developers understanding and appreciating the long term goals of SOA. If you want to hear more about this topic you should pick up the TechEd 2007 DVDs and listen to my talk titled "Best Practices for Designing Reusable SOA Services". In there I go through a bunch of very concrete strategies that'll help you deliver services that are flexible enough to avoid these sins. Lastly, please sign up on SOAPitStop.com and give me your feedback on these. I really want to hear what you think!!!!!