JBOWS

Joe McKendrick has an interesting post on opportunity areas for SOA. One in particular I would like to expand on. Here is his quote:

An effective, functioning service-oriented architecture requires governance, and the ability to share services across multiple business units and enterprises. It’s easy to build Web services. You could build 10 of them in an afternoon. But, then you end up with a JBOWS architecture (Just a Bunch of Web Services), which will grow into a different sort of SOA — a Spaghetti-Oriented Architecture. SOA requires orchestration between the services and governance. This is an opportunity for those proposing or offering governance tools and methodologies as a way to clean up a growing tangle of point-to-point services.

I have a very difficult time when talking with developers about this topic. Most developers I speak with are very creative. They like to write code and there is a strong since of ownership. There is really no faulting them for this. These are traits normally seen as good in an individual developer. However when taken in the Enterprise context, they can be a harmful factor. These code warriors are very distrustful of third party solutions that have to be intermixed with their own solutions. Things come up like: Unnecessary hops, transaction overhead, more failure points, span of control, I can do it better myself, etc.

These are all valid concerns. However as we move into the SOA world, the role of the intelligent intermediary is crucial. Having each developer write their own policy management implementation, their own content based router, their own security handler, their own orchestration engine seems to make little sense. What about the governance aspect around service registration and usage? How do you know who is using what and in what context? Where is the latest version of the interface spec?

My position is without the intelligent intermediary (Both tools and people) in the SOA infrastructure, you will end up with the same brittle architecture that is represented so well in the traditional spaghetti diagrams. However the impact to the enterprise will be far greater than before.

Here is a simple example of the problem. Let's say I create a nice interface into my trouble ticketing system. The trouble ticket vendor is following the Web Services trend so they expose their API via Web Services. I pass along the interface definition (The WSDL) to the team that commissioned my work. All is well and good.

The team being the sharing type (stop laughing) decides to share the WSDL with anyone who asks. And they tell two friends and they tell two friends and so on and so on. I now have a lot of very different uses of the original interface of which I know about only one.

So now let's say the original team now comes back and wants a modification or it's time to upgrade the trouble ticketing system or there is a planned outage/maintenance. Who do I notify? How critical are the other applications that are using the interface? What context are they using it in? How do we regression test all that stuff we don't know about? This isn't some code someone has checked out and incorporated into their own. This is a live running interface. The impact is immediate.

The intermediary is critical in helping establish the governance for SOA. It plays many roles and may consist of multiple products. It may also be played in part by people. Web Services Management, ESB's, Registry/Repositories, Policy Enforcement, Governance, XML Appliances are all things that make up what I consider the intelligent intermediary. Without governance (both runtime and design time), a Service Oriented Architecture is doomed to repeat the past.

This probably seems obvious to most but in practice it is more difficult than one would think.

XPath Query

One minor annoyance while working with Celtix (not really a Celtix issue) was a content based routing test. One thing you find out when working with open-source projects is that they can be made up of many open-source projects. I had the following issue with an XPath query.

This query:

//urn:CustomerAccount/urn1:accountdescription = 'yoda'

Is really not the same as this query:

//*:CustomerAccount/*:accountdescription = 'yoda'

In Celtix ESB land, this last query (which is a valid XPath query) throws an error. It turns out the XML parser they are using does not support wildcards like this. Instead it will match any prefix as long as the prefix is presence. In other words, urn1:CustomerAccount and urn:CustomerAccount will both match the query. That's cool once you know it, but it is annoying when you don't. Dom4j is the culprit. No relation to Celtix but used in Mule which is used in the Celtix ESB.

You just have to do a little family tree tracing when using open-source solutions.

JAX-WS, WSDL and Tools

I've mentioned that I'm looking at Celtix's Open-Source ESB. One thing you notice right off the bat is the incorporation of the new JAX-WS standard. Annotations in particular make it much easier to expose an existing java service as a web service or create a new one as a web service.

Here is an example -

Interface Definition

package pgn.objects.wsdl.full.devservices.customerrequest.ver1;

import javax.jws.WebParam.Mode;
import javax.jws.WebParam;
import javax.jws.WebService;
import javax.jws.soap.SOAPBinding.Style;
import javax.jws.soap.SOAPBinding;
import javax.jws.WebMethod;
import javax.jws.WebResult;

/**
* This class was generated by the CXF 2.0-M1-IONA
* Wed Dec 13 15:45:39 EST 2006
* Generated source version: 2.0-M1-IONA
*
*/

@WebService(wsdlLocation = "file:/C:/IONA/samples/service_creation/customerinfo/wsdl/
customerrequestver1.wsdl", targetNamespace = "urn:pgn:objects:wsdl:full:devservices:customerrequest:ver1", name = "customerrequestPT")
@SOAPBinding(parameterStyle = SOAPBinding.ParameterStyle.BARE, use = SOAPBinding.Use.LITERAL, style = SOAPBinding.Style.DOCUMENT)

public interface CustomerrequestPT {

@SOAPBinding(parameterStyle = SOAPBinding.ParameterStyle.BARE)
@WebResult(targetNamespace = "urn:pgn:objects:xmlschema:messagetypes:devservices:
customerrequest:ver1", partName = "CreateCustomerResponse", name = "CreateCustomerResponse")
@WebMethod(operationName = "getCustomerRequest")
public pgn.objects.xmlschema.messagetypes.devservices.
customerrequest.ver1.
CustomerResponseType getCustomerRequest(
@WebParam(targetNamespace = "urn:pgn:objects:xmlschema:messagetypes:devservices:
customerrequest:ver1", partName = "CreateCustomerRequest", name = "CreateCustomerRequest")
pgn.objects.xmlschema.messagetypes.devservices.customerrequest.ver1.
CustomerRequestType createCustomerRequest
);
}

The annotations defined the communication style and structure that match what is in the WSDL.


An excerpt of the implementation of the Interface

package pgn.objects.wsdl.full.devservices.customerrequest.ver1;

import pgn.objects.wsdl.full.devservices.customerrequest.ver1.
CustomerrequestPT;
import pgn.objects.xmlschema.messagetypes.devservices.
customerinfo.ver1.*;
import pgn.objects.xmlschema.messagetypes.devservices.
customerrequest.ver1.*;
import java.util.*;
import java.io.*;


@javax.jws.WebService(serviceName ="CustomerRequest", portName = "customerrequestPort", endpointInterface ="pgn.objects.wsdl.full.devservices.customerrequest.ver1.CustomerrequestPT", targetNamespace ="urn:pgn:objects:wsdl:full:devservices:customerrequest:ver1")

@javax.xml.ws.BindingType(value = "http://cxf.apache.org/bindings/xformat")


public class Customer extends Thread implements
CustomerrequestPT {
CustomerRequestType requesttype;
CustomerAccountType accountinfo;
String programname;

Notice the two annotations. That's really it for exposing this as a Web Service. Now for the important part. The interface was auto generated by some of the bundled tools in the Celtix distribution. It is important to note that the implementation interface was generated from the WSDL (which was written first) and not vice versa.

Joe Mckendrick points out that tools can make it easy to generate your interface definition (WSDL) from an existing implementation. I think the generally accepted best practice is to develop the WSDL and XML Schemas first. Hopefully following that approach will lead to a less brittle interface that has the right granularity. It should also help reduce the potential number of interoperability issues that can arise from auto-generation of Web Services from existing code.

As far as Celtix goes, if you are looking for an Open-Source ESB to mess around with, I have found Celtix's to be straight forward to work with so far. I haven't really done anything over than some content based routing tests and fail-over routing.

Blog-tagged

Fuzzy put the tag on me so here I go with five things most folks don't know about me:

-I was a Captain in the US Army National Guard for 8 years. I commanded a M1 Tank company as my last assignment. I even jumped out of working airplanes. Now I write software, sure they go together. :)

-I was a history major in college (East Carolina University - Go Pirates). I took a Fortran class with punch cards and hated it. Now I write software.

-I lost the conference championship singles tennis match my senior year of high school after going undefeated during the season. Still #$% me off when I think about. But I choked in that match.

-I was a horrible student but I really enjoyed High School and College.

-My favorite thing to do is watch the new Dr. Who with my eight year daughter. Actually any with the family is on the fav list. We have a great time together.

Okay for the victims, I'm tagging Mark Carlson, Nathan Lee, Todd Biske, Scott Mark, Sam Lowe.

Open SOA Collaboration

Sure it sounds good but unless Microsoft is playing it won't matter. It's a sad fact but Microsoft is a pretty dominate player in corporate US. This will go the way any other thing goes that doesn't include them.

Celtix ESB

Iona released their latest version of Celtix Enterprise ESB as an open-source offering. I've been toying around with this product a bit. I was impressed at the initial install and configuration. It worked right out of the box which can be a bit unusual for open-source products.

The thing I liked the most is the lightweight nature of the product and the ease of configuration. In particular I liked the routing functions, content-based, fail-over etc. They have several good samples included with the product to get you started. The documentation is not bad for an open-source product although the configuration files that drive most of the functionality could use a bit more documentation.

I'm not sure I would deploy this as a main ESB solution but it might be a good addition to an existing infrastructure. Particularly where you might want to deploy some lightweight distributed components. Overall a very good effort. I'll post more on it as I dive deeper into it.

Web Services ''Reuse'' is Number One Driver For SOA @ SOA WEB SERVICES JOURNAL

Here is another study on reuse and SOA. The main point of the article is that reuse is the number one benefit in adopting a SOA strategy. Of course the research is sponsored by a vendor whose product happens to be...yes that's right a registry and repository. The study goes on to say that most don't have a registry or repository. Surprise! No not really.

Do you think the lack of a registry is holding back reuse? Not me. I think complex business logic is difficult (if not impossible) to reuse. There are probably some enterprise wide services, but not a lot, that have the potential for reuse. Most services I suspect will not be in the enterprise category but more specific to lines of business.

Of course the other factor is the type of business you are in and the types of IT systems you use. SOA is not a cookie cutter approach but unique to each business. Ther e is no one size fits all in my opinion.

Open Source Java Claims Its First Casualty: Graham Hamilton, Sun Fellow and VP, Quits @ JAVA DEVELOPER'S JOURNAL

Is this a warning sign? Brenda Michelson asked this question a few weeks ago. Compatibility is a key factor with Java. However I have found that once you move beyond simple POJO type implementations and move into the world of app servers, compatibility is generally lost.

Have you ever asked a vendor to support their application on more than one app server platform? The word port usually comes up.

Web Services Adoption

I was reading a post from Mike Herrick on SBA and EDA. Mike made a statement about the death of Web Services. Mike was talking about the WS-* set of standards in particular and not Web Services on a whole. It did get me thinking about the adoption rate of Web Services in general.

It's clear that the software industry thinks a lot of the future of Web Services. The plethora of acquisitions this past year is a testament to that. webMethods buys Infravio, HP buys Mercury (Systinet), Progress gets Sonic and Actional, IBM buys everybody else. Most of these products are heavy on governance. That would seem to imply a demand in the market for this type of technology.

I have been recently working on putting in a basic UDDI server. I was toying around with Microsoft's UDDI server which is included in their Windows Server 2003. What struck me was the lack of chatter on UDDI? If the adoption rate of Web Services is high then I would think UDDI would be a topic of conversation. It is not.

So to me the question becomes: Have most enterprises bought into the concept of Web Services? If so is the adoption rate just slow? Or are the vendors way the mark with their governance products? What do you think? How are Web Services really doing in the enterprise?

A Quantum Leap in Data Encryption

I'm not a security guy and I am certainly not a physics guy. But this is pretty cool on the cool meter. I have always wondered what folks did with quantum physics degrees.

TIBCO freeing up Business Studio - Column 2 - ebizQ

Alright I see a trend. :) Tibco releasing their BPM modeling tool for free is probably going to shake things up a little bit. They are changing the rules with the pure play EAI vendors. I wonder how webMethods will react to this news?

soapUI 1.6 final, web service integration and testing tool, released

I've used this tool a lot. It is a good one to have in your tool box. Very easy to use, not a lot of fuss.

Eclipse STP Project

There is an interesting story that is taking shape here. This is a recent article discussing the upcoming release of the STP platform for Eclipse. If you remember, I posted a thought (Culture Clash) about the webMethods purchase of Infravio. The quick summary of that was a discussion about Infravio's contribution to Eclipse and Apache versus the more closed culture at webMethods.

The STP tools project got a major infusion from Infravio (according to Infravio, the founding member). I also notice that the STP tool set is going to use the Celtix Open Source ESB. This is a product seeded from Iona.

Anybody else think this is weird? All of these folks are competitors or were at one point. How will all of this play out? Don't know but it will be interesting to watch. I'm especially interested to see how webMethods will absorb Infravio and how its own tool set will evolve. Will they abandon their own proprietary tool set in favor of the open Eclipse based tools? Or will they consume the functionality of Infravio into their own tools?

Web Services and SOA, Better that EAI?

I found this article over at the SOA Web Services Journal. While it is a good article from an attempt to explain WS-Security, WS-Addressing and BPEL from a technical perspective, it also generated some "say what?" questions for me.

This quote at the introduction:

People sometimes ask what a service-oriented architecture enables today that could not have been done with the older, proprietary integration stacks of the past 5 to 15 years, such as those from Tibco, IBM, or Vitria. One such ability is the greater degree of interoperability between heterogeneous technology stacks that is made possible by the standards SOA is built on, such as Web services and BPEL. Although interoperability is only one facet of the SOA value proposition, it is one that has become increasingly more important, due in large part to the evolving IT environment, merger and acquisition activity, and increased partner connectivity.

It becomes more understandable once you read who the authors are. They of course work for Oracle and a Microsoft Partner who are really promoting a product suite. The problem with this quote of course is it is simply not true or at the very least misleading. I don't think most folks bought Tibco, Vitria, IBM, webMethods, SeeBeyond etc to integrate technology stacks like Java and .Net. They bought these EAI platforms to integrate multiple proprietary software packages. Lets say SAP, Oracle, PeopleSoft, Remedy, Mainframes, Java, .Net, EDI (Trading Partners)and the list goes on and on.

Those integration needs are still there. Some vendors like SAP, Oracle and Peoplesoft are evolving their offerings to include Web Services. But for the other million package applications out there, integration is an issue. Web Services doesn't solve the problem of legacy integration. Even for companies moving towards a SOA style of architecture, legacy integration is still an issue. The "private side" of a service still has to communicate/work with legacy packages in a lot of cases. I do think the concept of creating service wrappers around these legacy integrations is a good idea. However you should not think by using Web Services that you somehow don't have to deal with this issue. By the way all of the platforms referenced in the quote offer the same functionality that the vendors in the article are touting.

The other issue I had with the article is the concept of asynchronous processing. But I'm going to leave that for a future discussion.

Red Hat woes

You have to wonder if Microsoft is serious about this or are they just trying to smack Red Hat around a little bit.

Novell Shares Spike 20% As Wall St Journal Reports Microsoft To Start Reselling SUSE LINUX
— In a move that caused Red Hat shares to tumble 3.2% and Novell's to surge by 20% in the same period of trading today, the Wall Street Journal is reporting that Microsoft is going to start providing support for SUSE LINUX and that it is working too on Microsoft-Linux interoperability.

I guess it could be a response to Oracle's support of Red Hat. It is interesting to see both the upside and downside of Open Source at the same time. For customers, the upside is no lock-in to a particular vendor (Assuming you are not using any proprietary extensions). For vendors, the downside is you really only have your service/support offering to support you.

NetBeans 5.5 has been released

Here you go, the official release of NetBeans 5.5.

ESB market report

A decent report on the state of the ESB market and the players in it. This is a good follow-up to some on-going discussions on the ESB versus XML appliances and content switches/smart routers. Scott Mark posted some thoughts on it which was followed up with a posting from Todd Biske. I think you will see from the report that Anne Thomas Manes over at Burton is an advocate of the XML appliance. My company is a client of the Burton group. Anne came and spoke with us about this and other SOA related topics earlier this year. The XML appliance was discussed a lot.

I'm still undecided on the appliance concept. You can see my thoughts over on Scott Mark's blog. Still it is becoming interesting and probably worth a look. It's now on my radar which is the first step to more serious research.

Culture Clash?

webMethods recently acquired Web Services Management Leader Infravio. For those of you who don't know, Infravio has contributed a lot to the open-source movement. You can read about it here. If you are familiar with Apache Synapse or the Eclipse SOA Tools Project (STP) then you can thank Infravio.

I think this brings up several interesting questions. Will webMethods snuff out Infravio's support of Apache Synapse and Eclipse STP? Will Infravio bring a more open mindset to webMethods? How will Jack Bauer escape from the Chinese government? Stay tuned it should be very interesting.

Netbeans 5.5 Beta 2

I think Netbeans is a pretty good Java IDE from a first glance. But it is really a whole lot more than just Java. It is packed , without being bloated can you say WebSphere, with features. One of the cool features is the XML schema designer.

Check this out -


Notice the Selected Design Pattern: Venetian Blind. This is pretty cool. It will apply the design pattern to whatever XML schema you currently have it in. It supports Salami Slice, Venetian Blind, Garden of Eden (New to me) and Russian Doll.

Of course the tool has about a million other features as well including a pretty nice UML tool. I don't want anyone to think I've done any kind of serious evaluation on this but at first glance it is promising. And of course it is free from Sun like most of their software these days.

A better POD

So the next time you are thinking about getting one of those PODS that seem to sit in everyone's yard forever, try one of these out. Way cooler.

Enterprise Data Integration @ SOA WEB SERVICES JOURNAL

Articles (pretend articles) like this really get me going. Erin happens to work for Altova which makes XMLSpy, a very useful tool by the way. I think at least a pretense of being objective would have been nice. Otherwise a big disclaimer at the beginning of the article as opposed to the end. Not to pick on Erin because a lot of folks do it but these types of articles which are really just sales pitches should be appropriately titled. I don't think Industry Commentary cuts it.

xkcd - A webcomic of romance, sarcasm, math, and language - By Randall Munroe

A little Unix humor for today...

Enter The JBoss Matrix

I don't know about you but this clustering solution from JBoss eh I mean Redhat looks promising. The in memory persistence replication in particular will be fun to try out. It actually kind of makes sense when you consider the average life expectancy of a single transaction is not very long. Message clustering has always been a challenge with most of the products out there. I like this more innovative approach. We will have to see how well it holds up.

"When someone mentions SOA reuse, I cringe"

Here is another article on the reuse war that is just getting started. My perspective from my little corner of the world is that it is very difficult to achieve business reusable services. Just as the article points out, getting the top down cross organizational business perspective is a very difficult task in the real world. Don't believe me, go try it.

Another thing the article said was around vendors forcing IT into SOA with their products. I'm not buying that. Shipping me a product that has its API's exposed as Web Services is not SOA which is what most vendors are doing. I do think vendors are putting a lot of pressure on IT to buy into the SOA concept buy trying to upsale their products. But still not SOA.

Gartner: SOA will drive next round of acquisitions by Oracle, SAP | Service-Oriented Architecture | ZDNet.com

Okay, this was a useful forecast by Gartner after the fact. Can we all say duh? As most have been purchased already, the question remains who is left?

webMethods Blogs

webMethods has started blogging. Check it out, they have some pretty smart people over there doing some cool stuff.

soapUI - soapui - tool integrations - ws-i tools

A cool little tool for WSDL compliance testing and test case generation I just started using. It even does load tests which is pretty slick. It's basically a graphical front end (Java based) to the variety of command line tools that are out there. Easy to use and setup.

AT&T buys poster child of ASP boom years | Software as services | ZDNet.com

It's good to see my old company from the .com era mentioned again, Interpath not USinternetworking. We actually built some pretty neat stuff at Interpath in the early days contrary to Phil's comments about services versus hosted applications.

Ah, those were the days. Unlimited budgets, brand new software models, lots of shiny new hardware. And our data center was the bomb baby. We worked like crazy but it was fun.

IBM prepares open-source systems management initiative for Eclipse - Yahoo! News

More on the SOA front. This one on Systems Management with relation to SOA and Open-Source. IBM is leading this effort.

The SOA "Trough of Disillusionment"

This is more discussion on the real world trys at SOA. Those that come from the EAI world know how this works. Many of the EAI projects failed for a lot of the same reasons.

I do think it is important to define what the "business" is. IT is itself a business and SOA has value there. Yes even in a corporate IT shop. There are numerous "business systems" within a corporate IT shop that can benefit from this type of architecture. A lot of these systems are not customer business systems but yet still have an impact on the overall cost and agility of the overall business.

It is an underestimation of the value of SOA to not include the "business of IT".

Pouring cold water on SOA 'reuse' mantra | Service-Oriented Architecture | ZDNet.com

An interesting article on reuse in the SOA world. I would add to the article by saying if you are successful with reuse of services another host of problems can crop up. Once a live running interface is in use by lots of folk, maintenance, upgrades, patching, outages, enhancements etc can become a real issue.

I'm not saying you shouldn't do it but you better have a good plan on how you are going to handle it.

Web Services Software Factory

Microsoft has just released a Web Services Software Factory. From what I have been able to digest so far it's not to bad especially from a getting started standpoint. I'm actually working on something similar for our webMethods Fabric layer. It follows a similar architecture but with a lot more messaging injected into it. It's a bit of a hybrid approach to EDA and SOA styles of architecture with messaging and web services as a foundation.

Obviously the Microsoft provided architecture could apply and be implemented with any other platforms that support web services. Microsoft does provide actual examples of the implementations with the download they provide. That means you are going to have to have VS 2005 in order to run the stuff. Yes I can hear the collective sigh.

Size Matters

At least according to astronomers ..... Pluto a planet no more.

Slashdot | DoD Study Urges OSS Adoption

A very good study/report on OSS that was presented to the DoD. Be sure to follow the links to get to the pdf. It's 79 pages long but a very good read. Vendor lock-in is a key theme.

webMethods acquires Cerebra

Here is the announcement Metadata is taking a front seat at webMethods.

Money Side of SOA

Todd Biske has started an interesting discussion on SOA Consulting. This was prompted in part by Brenda Michelson's announcement of a new business venture. Good luck to Brenda by the way.

Frustration

Integration architecture can be very frustrating some days. I'm having one of those weeks. Fortunately I'm off on vacation next week so I can clear what's left of the mind.

As most large organizations experience, we stay in an almost constant cycle of maintenance, upgrades and other vendor driven changes. Very little time is left for true innovation and adding real business value. Unfortunately explaining the way out to clients whose attitude consist of "git r done" is a very frustrating task. Anybody counting how many times I've used the word frustrating.

The scenario works basically like this. Hey client, if you spend a little more time up front, you can help isolate your application and others from changes down the road. Client says, hey integration architect dude I just need to "git r done". Fast forward a year later. Hey client, we have to upgrade that application because the vendor is dropping support for that version. Client says, but I wrote a hundred interfaces directly into that app, it's going to cost me a fortune to upgrade and it's going to be so disruptive. Client then says, oh by the way I've got another app to put by next week and I just need to "git r done".

The only sound at this point is the integration architect banging his/her head against the brick wall. I'm running out of Advil.

Home - Open SOA Collaboration

Not everyone is at the party though.

Turning Service-Oriented Events into Business Insight @ SOA WEB SERVICES JOURNAL

Interesting article. webMethods has a product that does some of this. A few of their customers have used to some success. I wonder how much traction this will gain and will it become a true differentiator in the business space? Seems a lot of enterprises are still trying to grasp the model part and haven't made it to the measure part.

SOA Governance

The HP deal that got Mercury which included Systinet is getting a lot attention. It seems that SOA was a primary driving force behind the acquisition. When you actually need a registry and run time governance for SOA is still up for debate. What do you think? Registry now or registry later.

Vendor Driven IT Strategy

One primary advantage behind using Service Oriented Architecture and Event Driven Architecture is the flexibility these concepts/practices provide an IT organization to drive its own strategy and activities. One might think that this ability exist regardless of the software vendors and architectural styles in use by an organization. The reality is when you hook up with a software vendor you inherit their vision or lack there of the future. This vision will ultimately translate into time and effort for an IT organization.

The typical IT organization goes into a planning/budget cycle for each fiscal year. Generally speaking it has to address these areas: Things it has to do (Maintenance of existing applications, security, support of existing stuff etc), Things the business clients want to do (new projects and enhancements), Innovation efforts (Looking at new stuff, R&D), Pain Points (Things that causing the organization problems). Ideally you would like your organization to spend most of its time and money addressing what the business clients want/need and innovation. I think in reality a lot of organizations spend most of their time with maintenance and pain points. Why does this happen?

Most IT organizations are heavily invested into software from multiple vendors. IBM, Microsoft and Oracle are some of the big ones but there are a lot of smaller ones that have deep penetration into IT organizations. These companies have a goal and that goal is to sell more software and services to the IT organization (Yes I know they want to "Partner" with you). Makes sense right? The problem is that this goal is usually in direct contradiction to the IT organizations goals. What's the typical IT organizations single goal? Produce more business value at a lower cost. Anybody got a goal out there to product less business value at a higher cost? I think probably not but that's what happens in a lot of cases after significant investment in these vendors.

So what cost so much time and money? How about patching existing software, upgrading existing software, rewriting code because of end of life issues with vendor provided software, vendors that head off in a new direction, vendors that are purchased or merged with another vendor, vendors that disappear from financial issues. All of these have a significant impact on an IT organizations budget and staff. The second part of this equation is how all of the above software was implemented and wired together.

So the IT organization invests heavily with the vendors and then does what most organizations do. They implement it in silos and then come back and wire it together in the most tightly coupled brittle manor possible. That completes the one two punch. So now not only is the organization in a constant state of change due to the vendor, that change has far reaching impact to applications that have no direct relation to the problem vendor.

So how do we fix all of these issues? How do we move the IT organization to a state where it defines its own strategy and is not at the mercy of the vendors? The real answer is that it's not possible to completely fix this. The goal should be to reduce the disruption as much as possible.

SOA and EDA have a lot in common. One key point they both address (if done correctly) is the loose coupling of applications. This is a big step in reducing the impact of software changes within an environment. Vendor A decides that version 90 of its widget repair suite is going to get a new data model. You have 20 other applications that directly invoke Vendor A's API. That upgrade just got a lot harder. It happens almost every day in a lot of IT shops. Vendor A's strategy has now impacted your IT organization on a much greater scale than just the original implementation group. If the organization had taken a different approach with let's say messaging (asynch when possible), intermediaries, and common XML data representations then the impact has just been reduced significantly.

Layering standards based implementations on top of SOA/EDA architectures can help reduce the impact even more. It can also give the organization the flexibility to make changes at the vendor level. Let's say Vendor A decides to drop support for a component your organization is heavily reliant on. You still must provide the implementation that component provided but now with a different supplied component from that vendor or one from a completely different vendor. With a standards based interface, the impact to the downstream applications will be kept to a minimum level. Standards based interactions can also be used within an application suite. Most application suites are made up of multiple components. These components interact usually with the vendor provided proprietary implementation. There can be significant advantage to abstracting these interactions.

Ultimately moving to SOA and EDA styles of architecture can be difficult and even expensive. It's not the same way of doing business. I do believe that allowing your vendors to drive your organization is ultimately more costly than shaking up the IT organization to a new way of designing and delivering applications. Just remember in the end vendors will make business decisions that are best for them, not you.

Combining SOA and Event-Driven Architecture Using an Enterprise Service Bus @ SOA WEB SERVICES JOURNAL

Nothing really new here that Brenda Michelson and others haven't already covered. But the article is still a good read and another confirmation that SOA and EDA can coexist and should.

Microsoft, Xen to run Windows, Linux side-by-side - Yahoo! News

XenSource goes over to the dark side. Forever will it dominate their destiny. On a serious note maybe it will be good, XenSource could use some help on the install department.

SOA Technology Adoption Reaches 90% in 2006, New AberdeenGroup Report Finds @ SOA WEB SERVICES JOURNAL

I don't know if anyone has seen this report but it is a load of you know what. 90% adoption rate, more like 10% would be my guess. A lot of companies are studying SOA and it's impact. But 90% implementing it? I don't think so.

Microsoft Virtual PC 2004 System Requirements

You got to love Microsoft. They are giving away their virtual pc software for free. Check out the listed OS support, notice the OS/2 Warp. Yeah that's right, all you warped freaks out there can now run OS/2 in a free Windows vm software system. I hear the rush of the crowd.

In a small footnote at the bottom of the page, they did mention that it will support most x86 operating systems. So if you wanted to do something useful like run Solaris or a Linux variety you could.

Sun's open-source odyssey | Tech News on ZDNet

Sun maybe late to the game but they have some good software. It will take time to tell if the strategy will work.

Curious about WS-Addressing

I'm plowing through the mountain of standards relating to web services. One that is interesting to me and that I'm kind of on the fence on is WS-Addressing. I am curious to what others think about this. I can see some of benefits especially the address (endpoint) and reply-to parts. But I can also see increasing the complexity for the developers by requiring them to know a lot more about the implementation than they had with just a simple URL.

Without WS-Addressing, the developers just implemented their web services consumer with the provided WSDL which basically was just manipulating objects in their native format (Pretty easy stuff assuming you are using Doc/Literal). With WS-Addressing they would be required to populate the header of the SOAP message. This is not contained within the WSDL. It does requires more in-depth understanding of the SOAP message structure. Of course I guess that they will have to learn sooner or later based upon the other WS-I standards coming out. Either that or run away screaming, kind of like J2EE.

I'm wondering from a practical (not purist) standpoint if implementation based upon some other method is not better? What does everyone think? WS-Addressing the way to go? Or is it too much added on to the developer? By the way I use an intermediary or middleware layer as the entry point, I don't allow point-to-point connections or interfaces.

Eclipse Callisto

Is now available for your coding pleasure.

Apache Synapse

I'm messing around with Apache Synapse. It's an open source mediation framework from Apache. It seems pretty straight forward to use. I was looking for a light weight mediation server to show capabilities and uses of mediation servers or intermediaries. I'm just starting out with it. I'm also looking at ServiceMix which is much more robust but a bit more complicated. It does mediation but a whole lot more like messaging for persistence which is very important in my mind. It's more of a full featured ESB type platform although it lacks the tooling associated with more expensive commercial products. LogicBlaze is based on ServiceMix which I hope to get too as well sometime in the future. Time is always an issue. My ultimate goal is to compare development efforts with more expensive platforms to that of open source solutions which are cheaper or free but have primitive tooling.

The Power of the Marginal

I've always liked Paul's essays. This one in particular is very good and pushes a lot of buttons. His quote "If most of your ideas aren't stupid, you're probably being too conservative. You're not bracketing the problem.", reminds me of something I tell myself and my daughter often. If you are not making mistakes then you are not in the game. It's easy to be perfect and safe standing on the sideline.

Still waiting for patterns

And now there are anti-patterns (saw this over at Sandy Kemsley's ebiz blog) for SOA. What happen to the patterns? I guess the anti of the anti-pattern is the good pattern. :)

Oracle Unveils Event-Driven Middleware Suite: Next-Generation of SOA @ SOA WEB SERVICES JOURNAL

I don't really care about this product from Oracle. They will probably just abandon it in about a year after they buy someone that does the same thing. But the interesting thing is the "SOA 2.0" mentioned in the article. It will be interesting to see if they can get this new buzzword version to gain traction. How much power does a big vendor like Oracle have? We will have to wait and see. There was a grassroots movement to try and stop this in the form of an online petition. I really think something like SOA .98 would have been more appropriate. ;)

SOA

This is coming to you from the beautiful Kiawah Island, SC. I'm still trying to figure out how to win the lottery so I can sit and stare at the waves all day. Okay back to SOA. Brenda Michelson has a conversation started on preparing the organization for SOA. There have been some discussion points brought up. Check it out and chime in.

Enter The JBoss Matrix

Ah the pace of the software industry. Hope you were not getting to attached to JBossESB. If you were I'm sure that there is grief counseling available from Redhat.

Ozzie takes top Microsoft software job - Yahoo! News

I'll be perfectly honest. I don't care for Microsoft products all that much. But the two men that are discussed in this article are pretty amazing individuals. I use to go to Lotus Notes conferences back in the early nineties and listen to Ozzie speak. It was well worth the trip. Ozzie is a very talented individual and the kind of person Microsoft needs. It is funny though because he use to bash the hell out of Microsoft during those talks.

Bill Gates who was blasted and vilified (remember the pie in the face) as a businessman is doing some pretty amazing things for the world's population. I doubt the vast majority of the folks who did the blasting have ever or will ever contribute to the greater good of the world as Bill Gates is doing. Some may argue that Bill obtained his money through ruthless business tactics but I think few can argue with what he is doing with the money.

How to rebuild an airplane.....

In-flight. That's the trouble I sometimes see with IT architecture. Rarely are we given the luxury of rolling out a new model. Instead we have to rebuild or replace components of complex architecture while it's still in use. I like the analogy of replacing the aircraft engine while the plane is in-flight. A pretty impossible task I would say.

That's why building the architecture on a solid foundation is so important. Of course that is easier said than done in the real world. The real world is full of demanding clients, tight deadlines, tight budgets and applications/infrastructure that have to be up almost all the time. Of course the other big problem is that we are not perfect. Sometimes the right architecture at first is not always the right architecture later on. Ever remodel a house? Sometimes your needs change beyond what the house is capable of providing. Sometimes it's as simple as adding a room while other times it calls for more drastic measures like tearing out foundations, walls and other large things (not to mention the portable toilet that sits out front of your house for months).

The big promise of a SOA style of architecture is the potential to handle change better. If done semi-correctly, large destructive remodeling should not be as frequent or necessary. How to do it semi-correctly? That's the million dollar question. Let's explore that more in future conversations.

ACM Queue: Interview with Amazon's Werner Vogels on Services, SOA, and Business Innovation - Business-Driven Architect - ebizQ

Another good article from Brenda Michelson and a good interview from ACM Queue with Amazon's CTO. It's about Amazon and the possible nirvana of computing.

Wary of Analysts

Most folks who work in an IT shop for a non-technology oriented company know all about the impact IT industry analyst can have. Upper management much to the angst of the architects and senior tech leads, depend on these analysts for advice or opinions. Whole projects can be derailed from a simple comment from an "expert". Entire IT industry movements can be started from these opinions from some of the more powerful analyst.

Technology vendors are not immune to the affect either. Most vendors of size have to spend time and money coaching the analyst on their products, market presence and viability. Do you think the amount of effort the vendor spends with the analyst is linked to the opinion of the analyst? hmm...I often wonder.

My own personal experience is that a lot of analysts don't have the background to render the opinions they are giving. Most have not done any significant work in real IT shops or it is so dated as to be not relevant ( I really don't care that you use to drop your Fortran punch cards). There are exceptions though and it's important I think to understand those. There are some good analysts out there.

When an analyst comes to visit or you are tied into a conference call with one, make sure you get them to explain their own experience with the technology they are rendering an opinion on. In other words if they have never actually implemented an "ESB" (and who has?) at a real company or if they have never actually integrated anything other than maybe some pasta with the sauce then chances are their opinions are entirely academic.

There is nothing wrong with getting those types of opinions as long as everyone understands and they are presented as academic. One final thought, make sure you understand the analyst's opinion on a given subject before they get in front of your management. Otherwise you may find yourself in a less than fun situation.

Distributed in a box (DIB)

Okay so I just made up a new acronym. I am speaking EAI terms today but it also applies to SOA (At least it should or could maybe I don't know). EAI architecture can come in many flavors. Some examples would include fully distributed, hub and spoke, bus, fully centralized or some combination of them all. This mainly has to do with where you place your software components including messaging components, enrichment services, business logic etc. A lot of influence will come from the vendor and how they have architected their platform. Another influence will be the skill set and maturity of the IT organization. Fully distributed is more difficult to manage while fully centralized is easier to manage. Or is it?

One lesson that I have learned the hard way is that a fully centralized architecture can quickly become very difficult to maintain. This is especially true if you experience a lot of early on success and start growing rapidly. What kind of problems you ask? Maintenance including patching, upgrades, code deployments and application co-dependencies where adequate separation of services has not been done (of course distributed can have this problem too).

The concept I'm getting to here is loose coupling. It really becomes very important with large environments with lots of services. This term has been around for a while in the EAI space (and now in the SOA world). It's not only how you distribute your components but how those components relate to and or depend on each other. The more dependency, the more brittle the architecture becomes. Of course in reality this is not always as easy to achieve as folks might think.

There are some contributing factors that can stall a distributed architecture. 1-The cost of the software that you have to deploy. Large environments with lots of endpoints can quickly add up if you are being charge by the processor. 2-The management instrumentation of the software. Was it designed with distributed administration in mind? Do you have to log in to a hundred different endpoints to manage the infrastructure? 3-Ownership of target systems. Yes believe it or not sometimes getting software installed on systems is difficult when you don't own them. Or the software is not considered a core part of the system. 4-And of course your developers ability to architect their solutions in a loosely coupled way. Once again the maturity of your organization can play a big role in that.

There is no clear cut answer to all of this. These decisions will always have to vary by company. I think the more distributed you can become, the more reliable and less brittle you infrastructure will be.

So what is DIB other than something I just made up? It's the concept of distributing your architecture within a centralized server cluster. You get some of the benefits of a centralized architecture while also getting some of the benefits of a distributed architecture. It is by no means a perfect solution but it does help overcome some (but not all) of the inherent problems with a centralized architecture and a fully distributed architecture. And we all know of course that architecture is all about trade offs.

New Links

I've added some new links to the site. Check them out under Enterprise/Integration Architecture on the right. These folks have real experience in this space and have a lot of valuable insight.

How to open source Java? That is THE question. | Between the Lines | ZDNet.com

David Berlind has an excellent post on the Open Sourcing of Java. He touches on a lot of points with Open Source in general as well as the particulars of Java. The particular area that hit home with me is compatibility. He used Linux as an example. The main point there was that there really is not guaranteed compatibility between distributions. And that migrating off one distribution to another has a lot of potential issues which most customers would shy away from. So much for avoiding lock in.

I'm a fan of Open Source software. However achieving vendor independence is not a trivial task. The act of just selecting Open Source software does not provide that independence.

Multitasking

I finally got Solaris 10 (Intel version) installed in a Parallels VM. I had a bit of a mouse problem that was holding me up. Just for the pure geekdom I had Solaris 10, Windows XP (Base System) and Fedora Core 5 (complete with running webMethods Integration Server Instance) running at the same time. It taxed my laptop pretty hard but it was pretty cool. It struck me as funny that I was doing this as I was writing up our company's Integration\SOA\Web Services strategy. On the surface two very unrelated tasks to say the least.

I have found in my role that effective Integration Architecture requires a wide range of skills. It goes well beyond pure development, touching things like Networking, OS's, Databases, MOM systems, Servers, Systems Management and more. It may seem obvious or it may not. But I find a lot of the things I read failing to take these things into account. What do you think? What skills have you found make a good integration architect?

It's up, it's down, it's up

From time to time you may notice that this site becomes unavailable for brief periods of time(once every two or three months). Yes I know you are crushed. If you have read the site note, you will know that this server is hosted in a closet in my house. It is connected to the Internet via a wireless Ethernet bridge. The bridge is pretty reliable but it does have its moments from time to time. Today was one of them.

It's Easy!

Underestimate the complexity of implementing Web Services, SOA, or EAI and you will likely pay a steep price. Vendors and some industry analysts like to tout this as the easy part. Just use standards and this stuff is pretty easy. All you need is the 850 standards and their toolset.

I'm sure the getTemperature("Insert your zip code here")they tried once was pretty easy. In reality Integration work is pretty complex. There are a lot of contributing factors to the complexity that no one really addresses when hyping it up. Some of these factors include: Complex Business Rules, Data Enrichment, Infrastructure dependencies, Interoperability, Standards Interpretation, Distributed or Silo oriented Development teams.

Integration Architects have their hands full trying to make all of this stuff work. When you consider that some of the biggest reasons for adopting EAI, Web Services, and SOA is reuse and agility, these are in fact some of the hardest benefits to obtain. My advice from hard earned experience, do your homework before you go down these paths. Nobody wants to explain to their business units why its not possible to reuse any of the services you just deployed for several quadrillion dollars. Not to mention they aren't reliable and they don't interoperate with any of your other technologies or platforms.

The bright side is that is possible to achieve these benefits. But did I mention the homework part? Oh and when the vendor says it's easy, Run Away!

Business-Driven Architect - ebizQ

Brenda Michelson has a new blog. How she has time to write in it, I have no idea but it should be good. I'm personally on my second major production issue for the week, I believe I'm starting to hallucinate. Does everyone see guys in orange bunny suits standing in the corner?

Web Service Skills

Skill sets needed to consume web services is a frequent topic of conversation. I often hear from vendors and analyst that the consumer of a web services doesn't need to know anything about WSDL, SOAP or XML. I think many of them say that because a lot of the tools out there help abstract away that complexity from the consumer. For example in .Net when you consume a web service, the XML is marshaled and unmarshaled for you. The user ends up with a series of classes to work with instead of XML.

This is great assuming everything works correctly. The reality though is things don't always work correctly. More often than not, a developer with have to trace the messages being sent and returned from the web service. This means using something tcptrace to inspect the messages. Once the messages are being capture, comparing them to the WSDL is usually required. Understanding XML Schema, SOAP message structure and WSDL structure is required in order to do this. Things like namespace issues, type incompatibilities or improperly formatted SOAP messages are the usual suspects. But you have to know what you are looking at.

So does the developer need to be a XML expert? No, but a base level of skill is required. At a minimum, the developer should be able to read a WSDL and understand how to construct a proper SOAP message against it. The developer should also be able to tell when the SOAP message is constructed incorrectly. Without this minimum set of skills, developers will struggle to consume web services.

Iona launches open source ESB - Yahoo! News

Iona is launching a new open source ESB. It begs the question, do all software vendors need to open source at least a partial part of their offering? Does it really help drive market adoption faster by having a version developers can get their hands on?

Parallels Virtualization Software

I just started using Parallels Workstation 2.1. Do yourself a favor if you need this capability, try out Parallels product. They have a free trial. The product could not be any easier to use. I have Fedora Core 5 and Windows XP running side by side as we speak. I'm adding Solaris 10 this weekend. They support almost every OS in the known universe including OS/2 (remember that one). Okay they don't support AIX or HPUX but who really cares. ;) They don't have all the features that the higher priced VMWare product has but it is still pretty well featured.

Great Support

Don't know if any of you have or are using Eclipse for anything. I'm using it to do some interoperability testing with their WTP module. I've talked about their WSDL support in a earlier post, its pretty good for free. I wanted to mention their support/development community. It's staffed by a lot of folks at IBM. There are others but a whole lot come from Big Blue. I recently found what I thought was bug. I got a response back the same day I entered it. And it was resolved by the next day. Turns out it was not a bug but a desired ableit somewhat quirky behavior.

My point about all this is of course the Open Source support fear that a lot of companies still have. I find that in general Open Source software support is better than paid support. I think there are a couple of reasons for this. 1 - Pride of ownership in a much smaller company or community. Most of the time you are dealing with the people who wrote the software or are very close to it. 2 - You are dealing most of the time with the people who wrote the software or are very close to it. You are not going through some 1st level support help desk that has questionable training let alone any in depth knowledge of the product. The answers are quick and usually very accurate. Of course a lot of Open Source companies offer the support contract if you are so inclined.

Reliable Integration

I'm working on a "side project" to create a Web Services adapter. The basic guiding principals of this adapter are following my previous post on "Real Time Integration". The ultimate goal is to produce an easy to use but extremely reliable adapter that can be distributed and connected to existing systems. This is an exercise where I'm using Open Source products to construct. I'm then going to compare the outcome (time/effort, quality, ease of use, supportability) with some packages platforms that do the same thing.

The adapter is designed to interface with third party applications who are still using proprietary API's. Here is a high level and still in-flight design of the adapter:



Here are some more details of where I'm at.

Products Used: ActiveMQ 3.2.2, Apache 5.5, Axis 2 (rel .94)

I've got just basic functionality working right now. That is a doc/literal wrapped web service publishing directly to a JMS queue. It is very raw and has none of the "reliability" stuff I've been talking about built in yet. That's Phase 2 which is next.

Since this is a "side project" which means on my own time, it will be some time before it is complete. I'll try and post more details as I go along. Things like the WSDL design, interop issues, schemas, any issues/gotchas with the open source products etc.

McNealy steps down at Sun | Tech News on ZDNet

Big news from Sun. Scott McNealy is stepping down.

Fedora Core 5

I've been working with Fedora 5 specifically on a USB installation. The goal there is to boot from a USB drive instead of hard drive to keep my work image clean (laptop). It's working pretty well so far. Here is a link to a good set of instructions if you are interested in working with Fedora but don't want it on your primary hard drive. Look for the post by masterzzz for the detailed instructions. The only additional thing I would add is remove your hard drive (assuming laptop here) prior to install. Otherwise the installation is going to want to put the boot loader Grub on your primary hard drive. Oh and make sure you have USB 2.0 support with your device, otherwise slow......

My ultimate goal here is to do some work with Xen virtualization and some distributed Web Service's adapter services I'm toying with. More on that a little later.

Slashdot | IBM to Oracle - You Can't Buy Open Source

Their buying, no their not buying, their buying, no their not buying. Oracle makes a good database.

Report: Oracle Studies Novell Acquisition - Yahoo! News

More Open Source impact.

How is SOA like EAI?

Eric has a good post on some common themes between SOA and EAI. I couldn't agree more and in fact have said similar things. A lot of the problems that are present (notice I didn't say were present, EAI is still alive and well thank you very much)in EAI will be present in an SOA style of architecture.

No EAI is not a toolset as one of the commenter's tried to say, it is an architectural style complete with best practices, patterns etc. Things like abstraction, loose coupling, reusable services are all part of a solid EAI strategy.

There are some differences but not as many as some would imply. EAI shops will have a leg up on SOA. Contrary to what a lot of vendors are saying about SOA, defining a well defined, abstract, loosely coupled, reusable service is not nearly as easy as they make it out. We have already learned that from our EAI experience.

The difference I do enjoy the most is the "SOA is all about the business" and EAI was an IT thing. You have to smile about that. It's always about the business folks, that's why we are here. Any of you out there implement EAI systems and architectures and not work with the business?

So one day we will have dozens of highly decoupled, composable, abstracted, dynamically discovered, reusable services that our business users will simply string to together with their favorite BPM tool with little effort and no real involvement of IT. This is tied directly to world peace and the end of poverty and hunger.

Okay I'm off my "SOAP" box now. I like SOA and where it is trying to go from an architectural style. But eyes open when starting. Vendor fed delusions of grandeur can be costly to the aforementioned business users.

Red Hat to Buy JBoss for $350 Million - Yahoo! News

In case you missed this.....A big buy for Red Hat.

A bit of irony

I spend a lot of my time architecting solutions that are designed to be robust and highly available. Redundant this and redundant that. Clusters, documentation, procedures, change management. The list goes on and on. These solutions are all designed so that the customer does not lose time because of hardware or software outages.

I spend so much time doing this that I sometimes forget about my on system, my laptop. Well crash, boom on Friday. The old hard drive had enough. While I recovered all of the data, I spent most of the day reinstalling software (still have more to go). And while I could have made an image from time to time of the hard drive, I never seemed to find time to do that. I was lucky in that the data was still reachable since my backup was not as current as it should have been. A nice refresher lesson.

Review: ESB Suites | Make Way for the Enterprise Service Bus | Mar 10, 2006 | Network Computing

ESB test results from the folks over at Network Computing. Unfortunately webMethods did not participate. None the less, the testing procedure and results were interesting. I found the fact the Oracle was in the top three to be a surprise.

New Virtualization Software Lets Mac OS X and Windows Share Desktop - Yahoo! News

If you don't think virtualization is hot, you are not paying attention. Can you see a day when we are all running three or four OS's on one machine. Not the average consumer mind you, I am talking developers, architects etc. Of course you will need a pretty beefy workstation. But it beats having three or four different systems to do interoperability testing.

Microsoft starts supporting, er, Linux | The Register

Let the war begin. Poor VMWare, I would hate to be the target of a Microsoft price war. It does show how important virtualization is becoming. It also shows a partnership between Microsoft and an Open-Source software provider which is interesting as well.

Busy Busy

I haven't had a chance to post much lately. I'm in the middle of a major webMethods upgrade. I have been playing around with the new Fedora 5 on the side. I'm really interested in the embedded Xen support. I'll put more out here about that and some distributed services concepts I'm toying with a little later as time becomes less of a constraint.

SOA Web Services Journal: Save Our Architecture @ SOA WEB SERVICES JOURNAL

There are over 492 new acronyms in this article. On the somewhat serious side it is a decent read.

Slashdot | Fedora Core 5 Available

This is the distribution that is suppose to have Xen embedded into the OS as opposed to just available as a package. It holds some promise if you believe what Redhat is saying about it. 100ms switch times is pretty impressive if it is true. For those of you who are experienced in clustering, you know the complexity that can be involved. Bringing up the whole OS container somewhere in 100ms is fast and less prone to error than traditional active/passive clustering.

Application Availability

Software design and testing is directly related to the availability of an application. We all know that already. No amount of money spent on hardware can overcome a flaw in the application. We call these undocumented features sometimes. They can be very challenging to troubleshoot in a production system.

With Integration, these challenges become more intense. Multiple systems wired together and yes even loosely coupled systems can impact each other despite what all of the analyst and software vendors tell you. The question is how to overcome this complexity and turn out reliable systems that have little to no impact on each other. Well I would love to say I have the answer but the truth is there is no one answer.

Good sunny day and rainy day testing will take you far as well as good design and good documentation. Well trained operations and application support staffs will also contribute. But in the end the combination of variables in a large integrated environment both known and unknown are just to great to overcome. There are going to be issues.

So what to do? A few of pointers from hard learned experience:

1-Educate your management and your clients; setting expectations for realistic availability is key. ie deploying your app that you designed while coding it to a single server is not going to give you 5 nines.

2-Educate your staff. It's great that you know the app but you are not available 24x7 unless of course food, sleep, vacation are not something your require.

3-Practice recovering your app. Don't know how? When it is hosed in production is not the time to learn.

4-Know what else your app depends on. This is really key in integration work. Change management is a big deal.

5-Did I mention educate your management and your clients?

Just a few thoughts for the day as the fires have finally died out.:)

SOA Pipeline | Business Management << How Software Consolidation Will End Up Squeezing You >> February 26, 2006

More and more folks are coming to this conclusion. The hype is high but the reality is low for SOA. If you have seen the mixed adoption of OOP and the partial success of EAI and you were wondering if SOA somehow made all of those issues go away, in short the answer is no. SOA is a good thing but it is not mature, not yet.

Folks that found success with OOP and EAI will probably transition well into an SOA style of architecture. Everyone else will struggle. Why? Because integration is hard (esp. real time). Developing interfaces that are abstract and reusable is hard. Convincing clients and management that spending extra time on common information models is hard. Understanding the 8 million Web Services standards well let's not even go there.

98% of Java developers just don't need it

Well what do you think? I've often read that 90% of Java development is servlets. I have definitely seen folks buy expensive app servers to do very basic stuff. But I've been kind of trolling the Axis support list and I can say there is more going on with Java than just servlets and basic development.

Java EE 5 Tools Preview

To give Sun credit, they have put out a pretty good preview release of the new Java EE 5 along with a new tools preview. A couple of things I noticed were BPEL support and UML support. It's also pretty fast. It's got XML schema support like Eclipse along with a WSDL editor. I think its XML support is a little less robust (from a graphical standpoint) than the Eclipse version. But I did find at a casual quick glance that the UML support and the BPEL support were fairly easy to work with. I was even able to import the BPEL generated by the Sun tool into webMethods. It's worth a look.

Here is a screenshot of the BPEL composer:

Figure 1



Here is a screenshot of the UML editor:

Figure 2

Eclipse 3.1 with WTP 1.0

Still working on some interoperability testing with Eclipse and the new release of the WTP toolset. I found something interesting with working with the document/literal wrapped style of WSDL. Normally in you WSDL design with a document/literal wrapped style, you would have your operation name the same as your input message. And this in fact works for a large portion of the toolsets including, Axis 1.3, webMethods, .Net, and Suns Studio Creator. This does not work however with Eclipse. In Eclipse if you try and generate a proxy or a web service based off that type of WSDL, you will not get the messages deserialized into java beans. In other words instead of a method like createTicket(ServiceRequest request), you will get createTicket(arg1, arg2, arg3, arg4, arg5).

The work around I have found for this is to change the operation name into something other than the message input name. See the screenshot below. Notice that the operation name is different from the input message. With the WSDL setup like this the beans are generated correctly.



Not really the ideal thing to have to do. I'm attempting to work with the Eclipse folks to see if this is a bug or intended behavior.

More on WSDL Design

This is kind of a strange thing to happen but if you run into it this might help. I was importing xml schema into some wsdl using http as the location and got a java.net.URLConnection exception when using Axis 1.3 from the command line and with Eclipse. Here is the import statement:

schemalocation="http://sn000046:8081/schemas/servicerequest.xsd"

Very common import statement, so I couldn't figure out why the error was being thrown. Turns out I was hosting the .xsd schema on an Apache Tomcat server that I haven't defined the content-type for xsd or wsdl as xml. So the response header coming back from the server had no content-type specified. Easy to fix just modify the web.xml either in the individual project or in the global conf/web.xml file.

This is why interoperability testing is important. This same wsdl consumed by other platforms ie .Net and webMethods did not have a problem with the content-type being absence.

WSDL and Eclipse 3.1

I am doing some work on modular WSDL. I was looking for a good WSDL editor that was low cost as in free. Don't know how much hand coding you have tried with WSDL but it can become a bit tricky especially when trying to troubleshoot. It comes in handy to have a tool that shows you where some of your links may be off.

Altova is okay for syntax checking but to see graphical WSDL, you have to purchase a licensed copy. I had been playing around with Eclipse 3.1 and their web tools for a while doing some interoperability testing. I did not realize at the time that they had a built-in XML schema editor as well as a WSDL editor. Turns out it is not really bad for the price. They have just released version 1.0 of the web tools.

Here are a couple of screen shots to illustrate what I mean by graphical layout. In figure 1 you can see the correct linkage being made. This saves a lot of time.

Figure 1.


Figure 2 shows the import schema linked into the messages section. This is also nice in that it lets verify that your linkages are correct.

Figure 2.


Here is a site with some more good tips on using XML schemas and WSDL. I also found another site that where the authors are using modular WSDL in their design. It's a good reference if you are just getting started.

Real Time Integration aka EDA

Brenda Michelson has another excellent post on Event Driven Architecture. For those of you from the EAI world, we use to call this Real Time Integration. Jump over and read her article on Event Driven Architecture and then come back and read about some issues associated with it. I've posted below an excerpt from a white paper I wrote a while back on Real Time Integration challenges. Feel free to chime in with other challenges you have run into when doing Real Time Integration.

Excerpt

Issues associated with real time integration are numerous and can have a large impact on the success of a real time integration project. This white paper is designed to give the developer an overview of these issues before undertaking an integration project.

Background

What is meant by real time? For the purpose of this discussion real time is a business event that is to be processed as it happens or within a short time period there after by another application(s). This time lag could be few seconds or several minutes. This differs from a batch type of integration which is typically defined as a grouping of multiple of events or data points in a scheduled one time event.

This need for real time integration introduces multiple issues and challenges which must be overcome to produce a successful, reliable integration project. A thorough understanding of these design issues is necessary before beginning an integration project.

Integration Challenges

Real time integration brings forth a host of challenges driven by the need to consume business events as they happen. These challenges can be categorized into the following topics: Event Notification, Event Delivery, Event Persistence, Event Recovery, Event Consumption, and Event Monitoring.

Event Notification

In a typical integration scenario, there is at least one event producer and at least one event consumer (multiple consumers are discussed under Event Consumption). The first challenge is notification a business event in the producing application has occurred. How is this event generated, how is it captured by the integration layer, how is it persisted in case of connectivity failure between producing application and the integration layer, and how is it rolled back if needed?.

Most applications do not provide an internal mechanism for providing event notification and storing event information for use by integration platforms. Applications typically have application programming interfaces (API's) for interfacing however these API's usually do not address real time event notification (capture). Api's typically allow event generation (i.e. create an invoice), data interrogation i.e. return a specific invoice, and even import and export of data.

Third Party integration platforms such as webMethods provide "adapters" for common types of third party applications. I.e. Oracle, DB2, Peoplesoft, etc. These adapters are designed to be aware of events in the applications, persist the events at the source when possible, and aid in the deliver of the event and its content to the integration layer. So to summarize, the adapter is responsible for observing the event, capturing the event, persisting the event and aiding in the delivery to the integration layer. The adapter also may be responsible for assisting in "at least once"delivery to the integration layer and/or "once and only once" delivery to the integration layer.

Event Persistence

Persisting the event once it has happened, is the responsibility of the integration adapter. Event persistence is important to maintain the history of the event, the contents of event and the notification of the event occurring. To insure that the event data is recoverable in the event of a transient infrastructure type failure, the persistence should always occur on the source system. This also aids in de-coupling the source application from the rest of the integration layer infrastructure as well as any target receiving system. For example if the integration platform were to be down for maintenance, failure etc then the event would be collected on the source system and sent to the integration layer once connectivity was re-established automatically.

Event Delivery

Delivering the event data to the integration layer once the event has occurred is generally the responsibility of the adapter. There are multiple ways to do this and different adapter types use different methods.

Event Recovery

There are multiple recovery situations that can occur within a typical integration. Designing the integration to be recoverable is a key step in insuring the event does not get lost.

Event Recovery - Infrastructure Failure

Infrastructure failures can come in a variety of forms that the integrations have to take into account. Network failures, application failures, database failures, integration software failures, hardware failures. These types of failures can be accounted for during the design and event recovery can be automated within the integration.

Event Recovery - Logic Failure/Error

Logic failures can occur in the source application, integration layer and target application(s). These failures are typically not associated with automatic recovery. However notification of the event logic error is critical. Once notification of the error is received, the event can be repaired and resubmitted/reprocessed within the integration layer, if the error occurred within the integration layer.

Event Persistence and Event Recovery are tightly tied together during the design phase. If the event is not persisted correctly and at the right place(s) then recovery becomes difficult.

Event Consumption

Event consumption can be separated into two parts: Integration layer event consumption and target application event consumption.

Integration layer event consumption occurs after the hand off from the event source application. Typically in well designed integrations, the event is persisted before it is consumed. Consumption may consist of simply forwarding the event on to interested target applications. In most cases however, the event needs to be translated into the target application's desired format.

The transformation process is an important part of the integration layer. It is at this layer that a common data model can be inserted, also called a common document. This common document plays an important role in the de-coupling of source and target interfaces. By translating the source data into this common document, the target applications do not know about or care about the data structure of the event producing application.

As the common document is passed through the integration layer to the target applications, it will be translated again into the format supported by the individual target application. These different persistence points allow for further de-coupling of the source and target applications.

Target application event consumption occurs when the event has finished its route through the integration layer. The event data has been translated into the target applications data model. The interface into the target application is dependent on how the target application has exposed its methods.

Important design considerations: What happens if the update of the target application fails due to a data error? What happens if two separate applications are being updated and one succeeds and one fails? Is there an event sequencing requirement? What happens if the update of the target application fails due to infrastructure failures?

Event Monitoring

As events are happening in real time, event monitoring becomes critical. Event monitoring can be very complex depending on the overall design of the integration layer. Event monitoring can be divided into several layers: Source and Target Application monitors, Infrastructure monitors, Integration Layer Component monitors, and Transaction monitors.

Application Monitors

Application monitors are specifically designed to monitor the running application. I.e. an Oracle database monitor. These monitors typically have specialized knowledge for the application they are monitoring however they can be as simple as is the application up or down.

Infrastructure Monitors

Infrastructure monitors typically monitor the underlying server hardware, operating system and network.

Integration Component Monitors

Integration component monitors typically monitor the individual pieces of an integration. An example would be the custom queue monitor which alerts when queue depths for a particular integration have exceeded a certain limit.

Transaction Monitors

Transaction monitors are used to monitor an individual transaction from end to end. This is the most sophisticated type of monitor and yields the best results for monitor an integration. It is also the most difficult to implement.

Conclusion

Real time integration can provide great business value to the corporation. Events consumed as they happen can reduce cycle times, make the business more responsive to customers and increase the competitive advantage. Real time integration can also bring a host of issues as this paper has outlined. Understanding these issues prior to undertaking the project will significantly increase the integration's chance of success.

Jonathan Schwartz's Weblog

This is a good post from Jonathan on free software. It is good because of one phrase he uses in justifying giving away software, "it amplifies adoption". To grab the new customers, to get the software in the hands of the folks who will be working with it will be the key in the next few years for gaining market share and keeping existing market share. A developer, an architect, a programmer will always tend to recommend, lean toward a solution that they are familiar and comfortable with.

If you are a software vendor with closed source and your strategy for the next few years is to keep it that way, I think we can all see the ending. There is no longer a middle ground for software vendors. There is the upper tier ie IBM, Oracle, Microsoft and then there is everybody else. Everybody else is competing with open source solutions. Competing with solutions that are "good enough". Even the upper tier players are feeling this affect. But the lower and middle tier vendors tend to have a more fragile bottom line that is greatly affected by even a percentage point or two in market share.

Another thought on this. Companies that can successfully implement a changeover to a SOA style of architecture will make this even worse for the lower and middle tier vendors. A SOA based upon the WSF standards and good design principals will make it easier to swap out vendors.

iWay steps gently into ESB - Yahoo! News

Another ESB coming to town. The question is how long to it becomes open source.

5 careers: Big demand, big pay - Feb. 3, 2006

I thought this was interesting especially in contrast to my previous post about free software. Notice the demand for .Net developers. Of course it also mentions a demand for software quality management analysts. Are the two related? :)

Slashdot | VMware to Make Server Product Free (as in beer)

More software going to the free side. The pressure is increasing on vendors. We appear to be in the middle(maybe the late beginning, its hard to tell) of a transformation of the software industry. Business as usual which is high license fees and high maintenance contracts for questionable quality and even more questionable support is going by the wayside.

This is probably a good thing. Increased quality, better support and innovation could all be the result of this pressure. Only time will tell how it is going to play out. I say again, you should know where you vendors strategies are and how they are going react to this. If they don't have a plan, watch out.

Who's killing the software industry the fastest? | Service-Oriented Architecture | ZDNet.com

Joe's got another good article on SOA and Open Source. I do believe that the greatest threat to traditional software vendors is Open Source and not so much SOA. Although as I said before the companies that can harness both correctly will be out in front of their competition. The key is the harness correctly part. Good effective SOA is easier said than done.

E-Trade Goes into the Great Wide Open

I liked this article on E-Trade and its embracing of open source technologies as well as the development methodology behind it. It hits on SOA as well. Its been said before that the companies that can figure out how to make open source work for them and embrace SOA at the same time will be far ahead of the competition.

Might Oracle Buy JBoss? @ JAVA DEVELOPER'S JOURNAL

Will Oracle enter into the open source movement? Oracle partnered up with Sun recently. I wonder if Sun's move to open source the majority of their software had any influence on Oracle. We will see how things shake out this year.

» Sun Services CTO says utility computing acceptance is slow going | Tom Foremski: IMHO | ZDNet.com

How about Sun building your data center? You have to admit it would be pretty cool (and expensive). :)

IBM Proposes Open-Source AJAX Project to Eclipse

IBM continues its impressive support for open source software. The AJAX support is much needed in my opinion and IBM looks to have a number of key players involved.

Will SOA work?

Yes, no and maybe. How's that for a straight forward answer? I qualify the answer because your mileage may vary when trying to move forward with the concept of SOA. Why you ask? We all know there has been a tremendous amount of hype associated with SOA. Vendors trying to rebrand old products, new vendors try to market new products, and industry analyst touting it as the next best thing.

Remember the 90's when OOP (Object Oriented Programming) promised to make software reusable, cut down on development time, save the world etc. There was a lot of talk about abstraction, encapsulation, interfaces etc. What happened? Why aren't all software shops specifically IT shops using OOP design principals? When is the last time you saw a sequence diagram, a class diagram...? The answer as it turns out is that it is hard and it takes time.

So what happened with OOP? Well some (few) IT shops did embrace it and are still using it today. Some IT shops formed special units or groups to handle OOP. They were the elite developers for special projects. Other IT shops tried it and failed horribly.

It turns out the SOA and OOP have a lot in common. Interface design (think of contract, WSDL in Web Services terms), abstraction, encapsulation are all design principals used in both OOP and SOA (specifically service design). There are differences as well between OOP and SOA (loose coupling, coarse grain versus fine grain etc) but I would argue that the discipline required to achieve the touted benefits are the same. In fact I would put forth that SOA is much more difficult because of the scale and scope of the interfaces.

Software vendors tend to understand these concepts better because the majority of the big software vendors use OOP design principals on a regular basis. IT shops on the other hand are a mixed bag. So if you are an IT shop, I would ask this question. How did you react to the OOP song and dance when it arrived back in the 90's? If you don't have a good answer for that, SOA may not be your thing.

LCblog | What should an SOA repository look like? | Jan 18th 2006 10:32am

I'm excited about SOA repositories, no really I am, really. Phil states "What's less clear is what a successful SOA repository actually looks like." I would add its pretty unclear what a successful SOA looks like. In fact if you ask ten different people what is SOA, you going to get some interesting answers.

If you are new to SOA, and most of us are, I would suggest concentrating your efforts on writing good services first and understanding what a good service is. I do think repositories are very important and will become more important as services continue to grow within and outside of organizations. But I still think we have a lot to do on the services side first.

SYS-CON Announces Readers' Choice Awards for SOA, Web Services, Java, and XML Technologies @ SYS-CON AUSTRALIA

Congrats to webMethods. They came out very well in the reader's choice awards.

Is there an open source community? | Open Source | ZDNet.com

I found this article by Dana Blankenhorn over on zdnet. There is also another article by Joe Mckendrick along the same lines. Both articles talk about software as services. The summary is where business and users get software is evolving. One part of this is because of the emphasis on SOA and viewing the world of software and applications as a set of services. The second part of this is the Open source movement.

There are a number of folks now talking about harnessing these two things (SOA and Open Source) to make a business more competitive. Of course harness is the key. Done incorrectly you can increase your cost pretty dramatically. More on this later.

AJAX & Java: Feature Interview with "AJAX in Action" Coauthor Dave Crane @ JAVA DEVELOPER'S JOURNAL

Here is an interview with Dave Crane the author of "AJAX in Action". The interview is good. Dave gives more insight into the AJAX movement and what's going on with frameworks, IDE's and other toolsets. Dave has a way with communication and it shows in the interview as well as his book.

UDDI: policy enforcer or dead parrot? | Service-Oriented Architecture | ZDNet.com

I like the dead parrot reference. So we have already discussed that no one is doing dynamic runtime binding to services or discovery. Most organizations are not going to have multiple services doing the same thing. A clarification here, this is not to say organizations do not duplicate work unintentionally but rather I am talking about writing multiple services doing the same thing on purpose. So for organizations with a small number of services, UDDI probably isn't all that necessary. I'm not saying it would not be useful just not necessary.

Of course as services grow, the ability to manage them diminishes. Something like a UDDI based repository would come in very handy at that point. I'm still not convinced about the other stuff such as SLA management, failover, dynamic discovery etc. It sounds pretty cool but I'm not sure how many companies will really need that. It probably has a bigger role with external service providers.

AJAX moves toward open source | Open Source | ZDNet.com

I haven't looked at the products in depth yet but this is potential good news for AJAX. The current crop is not so good, anything that makes it more reasonable is welcome in my book.

IBM to freeze pension program - Yahoo! News

It always kind of surprises me that IBM seems to slip under the radar of the tech world. Everybody knows IBM and everybody knows they are huge. They contribute an enormous amount to the tech world but its strange how no one ever talks about them. Maybe because they are so big and have been around for so long that everyone kind of takes them for granted. I know the IBM employees don't. IBM's recent announcement to freeze pensions has got them up in arms.

It's really just a sign of the times. Companies (not unique to tech companies) continue to struggle with costs. Whether its pensions, health care, or salaries, the cost pressure in today's business world is immense. Companies are trying everything from offshoring, pension cuts, benefit cuts and layoffs.

SOA, EAI and Open Source can all play a part in helping companies reduce costs. Of course they can also increase costs if done incorrectly. As IT folks, we can make a big difference to the bottom line. The way we purchase/license software, the way we construct applications, the way we manage infrastructure and change can have huge impacts. SOA and Open Source are not just the cool new things, they can how a profound impact. EAI is included in this, it's just not new.

I think for those of us who work in large IT departments struggle with the reality of making these concepts come to life. There are a lot of industry analyst and others who talk about these concepts but few who talk about the practical application of these technologies and architectural styles in the real world. I am hoping to bring more of my experiences in one large IT shop over the course of 2006. No "expert" opinions, just one architect's daily struggle with trying to make this stuff happen.

BEA and JBoss Blogs Stir SCA Open Source Debate @ JAVA DEVELOPER'S JOURNAL

Competing standards, now how did that happen? At any rate the standards war is likely to be with us a while longer if not always. For an internal IT shop, and this is just my opinion, but I wouldn't worry to much about the standards at this point. There is a lot to be gained from an SOA style approach to application development and wiring. Messaging, abstraction, loose coupling etc have lots of benefits to an organization even in a proprietary format.

If you do that, your services will become more agile, more reusable, more reliable and you will be shielded at least somewhat from vendor change and churn. Standards will come and go. And even with standards there will still be interoperability issues. It's really strange how vendors interpret stuff. If you haven't run into interoperability issues with web services, it's probably time to move beyond "hello world" and the single platform.