Create collaborative and dynamic method content using Web 2.0

developerWorks, SOA and Web services zone has just published the first in a series of papers that looks at addressing some of the ideas around context to content to enable asset consumability that I have been talking about on this blog, This paper titled: Create collaborative and dynamic method content using Web 2.0, talks about how to leverage Web 2.0 technologies to extend software development process content, which is typically published static as HTML. This article describes how you can develop the ability to collaboratively edit method content and have access to the latest dynamic content within a method context.

For this paper, and indeed many other papers, we have been fortune enough to work with developerWorks own, Patrick Flanders, and Ashleigh Brothers who do such a professional job in terms of reviewing, editing and laying out the content.

A common message

Last month I did some service modeling work for a well known insurance company in the US. After five years of helping customers with SOA this particular company was very mature in terms of their cultural alignment between their business and IT stakeholders and was also adopting some of the best practices and patterns around SOA enablement.

A little background, this company is typical of so many other Fortune 500 companies in that the business struggles with it silo-ed nature, that has been built up over decades. Two simple silos within this organization are the personal and commercial insurances lines, both offering insurance products to different customer bases.

SOA is all about aligning the business with the IT and for this to happen they have to all speak a common language. To help understand this better lets take a simple analogy of the worlds favorite organization the United Nations. Now I am not privy to the internal working of the UN but I do know that it employs a lot of interpreters to interpret from one language to another. Take the example of an interpreter working for the French ambassador. This interpreter will need to be able to interpret from French to Chinese, from French to English, etc or to whatever the native language of the person the ambassador is talking with. This is clearly a very stressful job for the interpreter. Now lets pretend that I have been put in charge of the UN and have decreed that from now on the common international language will be Irish (this would be a complete disaster by the way, as Irish is notoriously a very difficult language with needlessly complicated grammar and syntax, I know this because I spent 14 painful years in school learning it). Now all the interpreters need to know is how to convert from their host language to Irish and back again.

Scaling this up to the enterprise, let us examine three best practices around a enterprise common message model:

1. The first best practice is simply to have one. The business and the IT need to agree upon an enterprise message format, or lingua franca across the organization. Thus what was traditionally a silo-ed organization in terms of personal and commercial insurance (each has a difference via of what a customer/party is) could now start to share service based on a common understanding about what what was a claim, a policy or indeed a customer/party. This really is not news to anyone starting to mature in the SOA space, however, the next best practice is a bit trickier.
2. The second best practice is not to build this enterprise common message model yourself. This can not be overstated. Building a enterprise strength common message model in a big job. To get a sense as to how big, consider IBM Insurance Application Architecture or IAA models. Now I am not here to plug these models, but they have been built up over ten years worth of experience, of practitioners, in the insurance industry. As a result these models are highly normalized and contain some of the best patterns and lessons learned and are what I would consider a mature enterprise model. There are many other enterprise strength message models out there such as ACORD, SID (telco), STAR (automotive) . One also needs to differentiate between industry standard models and priority models and here again I want to outline a best practices in terms of their usage. Industry standard model such as ACORD have been debated and agreed upon by concerned parties within an industry vertical, these models are there typically to level the playing field and provide a good, typically flat, model structure to enable B2B exchanges between companies in that particular business vertical. Priority models on the other hand such as IAA and IBM Information FrameWork Service Models (IFW), are highly normalized models, and are there to provide a company with core differentiation within the marketplace. The best practices then become to use a priority model internally within the organization, to define the internal services and as a lingua franca for the messaging infrastructure and then to use one or more industry standard models to define services that are exposed external to the organization and typically involved in B2B exchanges and delegate to the message infrastructure to convert between them.
3. Finally the enterprise message model must be managed and governed. New versions of these message model are being released all the time as new markets and opportunities arise and your company needs a coherent and consistent way of managing this change. Let us go back to our UN example and imaging that the Irish language was continuously changing (it is not, the Irish language in a considered a dead language), think of the nightmare for the interpreters having to continuously learn a new version of the common language.

For those you foolish enough to want to learn Irish, to champion the cause of having the UN adopt it as a common international language, if refer you to Des Bishop web site

Slán go fóill...

SOA information as a service pattern specifications

Guenter Sauter et al have three excellent SOA Information as a service pattern specification papers on data federation, data consolidation, and data cleansing. You can find these three papers on developerWorks:

• Data federation pattern: The data federation pattern virtualizes data from multiple disparate information sources. The pattern creates an integrated view into distributed information without creating data redundancy while federating both structured and unstructured information. This article describes the federation of structured information (data) with a focus on the SOA context. This pattern specification helps data and application architects make informed decisions on data architecture and document decision guidelines.
  


• Data consolidation pattern: The data consolidation pattern specification helps data and application architects make informed architectural decisions and improve decision guidelines. In this article you will see how you can apply the pattern in the SOA context. The primary business driver for the data consolidation pattern, also referred to as the data population pattern, is to gather and reconcile data from multiple data sources before this information is needed. To do so, it extracts data from one or more sources, transforms that data into the desired target format, and loads it into some persistent data target. The prepopulation of a persistent target is a key differentiator between this pattern and the data federation pattern.
  


• Data cleansing pattern: The data cleansing pattern specifies rules for data standardization, matching, and survivorship that enforce consistency and quality on the data it is applied against. The data cleansing pattern validates the data (whether persistent or transient) against the cleansing rules and then applies changes to enforce consistency and quality. In this article, you learn to apply the data cleansing pattern within a Service-Oriented Architecture (SOA) context. This pattern specification helps you, as a data or application architect, to make informed architectural decisions and to improve decision guidelines.
  

Services-based enterprise integration patterns made easy

A new article from Dr. Waseem Roshen, in the series titled: Services-based enterprise integration patterns made easy, appeared today on dW SOA and Web services zone. Here is the abstract:

This series of articles explains services-based enterprise integration patterns in an easy-to-understand, step-by-step way. In this installment, Part 1 of the series, you learn about the two earliest integration patterns—data sharing only and remote procedure call (RPC)—which help introduce the concepts of service provider and service consumer, platform independence, and connectivity. Exploring RPC helps you get familiar with the basic steps necessary for two applications to share functionality. This article also includes a general description of the concepts of loose coupling, code reuse, and layering and componentization. Part 2 of the series will continue the discussion of the early patterns, while Parts 3 and 4 cover the Service-Oriented Architecture (SOA)-based integration patterns, including examples.

The asset graveyard

One of my favorite movies of all time is Sergio Leone spaghetti western classic: The Good, The Bad and the Ugly. Part of the appeal of this movie is the simplicity of the plot; at its heart it is, a treasure hunt, with the three main characters, the good, the bad and the ugly hunting for confederate gold. Between them they know the location of the treasure, buried in a grave in a graveyard, yet only their combined knowledge will lead them to the prize. Towards the end of the film the Ugly a character called Tuco, played by Eli Wallach, comes upon the graveyard, called Sad Hill, and here in one of cinema truly great moment, backed by a unforgettable score, he begins his frantic and haphazard search for the grave of Arch Stanton.

What is impressive about this scene is the sheer scale and size of the graveyard itself, a conservative estimate would put it at over 2000 graves, arranged in a circular fashion. Tuco starts from the center of Sad Hill, where the graves are oldest, and works his way outward initially in concentric circles but soon his search becoming more haphazard as he despairs at the magnitude of the task. Finally, after a dizzy and Groundhog Day like scene of multiple graves passing by he come upon the grave of Arch Stanton, only to find it devoid of any gold, indeed all that remains is a rotting corpse.

Today, this story is echoed in ever IT projects I have ever been a part of, one of my favorites is from within my own group in IBM. It should be noted here that our team is geographically dispersed so although we talk on the phone regularly,we do not get the chance to chew the fat at lunch or around a water cooler. A good friend and colleague of mine on our team, lets call him Bob, was leading the engineering of one the first SOA service registry for an automotive customer in California. One requirement the customer had for the service registry that it performs within certain limits. Unknown to Bob, our group has tackled a very similar problem a couple of years earlier on another engagement. We realized the importance of this work and created a number of assets around it. These assets feel under the broad classification of the requester side caching pattern of which we created pattern specifications, pattern implementation, pattern documentation (including, dW articles, flash movies and a detailed case study). A full listing of these and other pattern asset related articles can be found on my home page.

Bob unaware of the previous asset work we had done in this space and having I am sure ran in circles around his own repository graveyard a number of times, he did what any other good architect would do, he started from scratch and reengineered the exact same asset as we had two years earlier.

Just like Tuco desperately searching Sad Hill for the confederate gold, Bob too faced the same dilemma, in fact I have witnessed that same dilemma on every IT engagement and software project I are involved in. I dare say that this dilemma is not exclusive to IT but that you will find the exact same problem is almost every human endeavor whether it be an engineer building a rocket to put a man on mars to a lawyer drafting claims for a patent application, How do we provide the right assets,the content, to help solve the problem at hand, the context. In other words, how do we automate a context to content mapping to suggest the best assets to our architects, engineers, lawyers to help them?

IBM Redbook on Strategic Reuse with Asset-Based Development

A good friend of mine Lee Ackerman let me know last week that a IBM have just publicly released a draft of a new IBM Redbook, titled: Strategic Reuse with Asset-Based Development.

The Redbook is structured as follows:

Part 1. Introduction

• Chapter 1. Introduction to Asset-Based Development


• Chapter 2. ZYX Electronics case study


• Chapter 3. Tools in support of Asset-Based Development


• Chapter 4. Introduction to the IBM Rational Unified Process


• Chapter 5. Asset-Based Development and Asset Governance Rational Method Composer plug-ins


• Chapter 6. Value, impact and return on investment

Part 2. Reusable Assets

• Chapter 7. Adopting Asset-Based Development


• Chapter 8. Configure your asset repository


• Chapter 9. Produce reusable assets


• Chapter 10. Consume reusable assets


• Chapter 11. Manage reusable assets

Part 3. Service Assets in an SOA

• Chapter 12. Introduction to service assets


• Chapter 13. Producing service assets


• Chapter 14. Consuming service assets


• Chapter 15. Managing service assets

Part 4. Patterns

• Chapter 16. Introduction to patterns


• Chapter 17. Produce: model-to-text pattern implementations


• Chapter 18. Produce: model to model


• Chapter 19. Produce: packaging for deployment


• Chapter 20. Consuming pattern implementations


• Chapter 21. Managing pattern implementation assets

Part 5. Appendixes

• Appendix A. Rational Asset Manager: installation and administration


• Appendix B. Integrating Asset-Based Development into a process


• Appendix C. Additional material

The value and use of Rational Data Architect in SOA

The IBM Information management yesterday announced that the paper "The value and use of Rational Data Architect in SOA" was published today as part of the series "The Information perspective of SOA design"

Here is the abstract from that paper:

Discover how you can use the IBM® Rational® Data Architect, IBM Industry Models and the unified metadata management of IBM Information Server to align process, service, and data models. Use these tools to accelerate your SOA project. The fifth part of "The information perspective of SOA design" series describes the key features of the products that support the data modeling pattern in SOA.

Pattern (reusable) assets classification

Reusable software assets represent significant IP in terms of lessons learned from previous engagements and can to address both functional and non-functional requirement (e.g. a performance non-functional requirement could be addressed using the requester side caching pattern asset). However, because assets are not currently cataloged against meaningful criteria, such as functional and or non-functional requirement, they are often not leveraged or reused effectively by architects to help make more consistent architectural decisions.

The importance of Architecture in an SOA can not be overstated. Yet when it comes to making architectural decisions in our software labs or on engagements there is often no consistency, tractability or accountability in terms of the decisions made. These architectural decisions are driven by non-functional requirements (e.g. performance, transactionality, tractability, scalability, etc).

Assets such as software patterns assets are often represented by many assets such as a pattern specification and/or pattern implementations and these assets are often hard to locate because of this lack of consistent and meaningful cataloging. Also the knowledge and use of a asset, such as an industry model asset is often local to a particular practitioner or architect. This leads us to a situation both in software development labs and more importantly on field engagement where is no consistency way of consuming assets and leveraging lessons learned to achieving architectural consistency traceability and accountability.

Below is a meta model that shows how reusable assets can be used to create a solution and goes into more detail and classification around a particular type of reusable asset: the software pattern reusable asset.

The many buckets problem

As I mentioned in my previous post one of the problems that we run into with mapping context to content is the multi bucket problem. Consider the following. At home I organize all my clothes into buckets.

Each bucket has a different item of clothing in it e.g. one bucket contains all my shoes; another bucket contains all my pants; yet another bucket has all my jackets.

How let's imagine for a moment that these are very sophisticated buckets. These buckets have the ability to classify all of the items contained within as well as matching items together. For example my shoe bucket has the ability to classify all of my shoes by color and size as well as matching all of my left shoes with their corresponding right shoes. To top it off these buckets have a shinny panel on the front that gives me detailed information about any item in the bucket. So with my bucket of shoes I can see from the front panel all the information that i need about a certain shoe, such as when I bought it, how often I have worn it etc.

This is all fine and dandy, but I run into a problem when for example I am invited out to a work related cocktail party. Since this is my first cocktail party I am unsure about what to wear and would like to be able to say to all my clothes buckets, please give me an ensemble to make me look good for this cocktail party. My clothes buckets I am sure will look back blackly at me, with the shinny digital displays, beeping nervously as I start to frantically root through one bucket after another to try and put together some vestige of an outfit that will make me presentable at the cocktail party.

This simple story serves to illustrate the problems inherent in any repository based information management system. However, there are a couple of lessons to be learned from this simple analogy.

1. No one bucket can be responsible for management relationships between the items in its bucket and the items in another bucket as the would quickly be unmanageable for all but the smallest number of bucket items
2. I could put all my clothes into one bucket and call it a clothes bucket, however by treating all of clothes as just generic clothes items I lose a lot of flexibility I has in the multi bucket scenario.

An SOA pattern for performance

In my days as a consultant in London, I used to work for a French consultant company based in the center of London, just off Oxford Street. One of my assignments was to train recent computer science college graduates on how to become consultants. This resulted in an intensive one week boot camp that focused on UML, OOAD, and patterns in a model driven development environment. One of the pattern that I liked was the decorator or wrapper pattern.


As a pattern it could not be simpler and yet it has a subtlety that made it a great pattern to teach and to learn. It clearly shows the cherished object orientated principles of object encapsulation and polymorphism working in harmony, yet at the same time the pattern could be applied to any interface to enhance, extend or decorate its existing implementation. Now five years later, on an engagement in Texas the buss words were field based development of SOA patterns. SOA provides a whole new set of challenges in terms of performance and business flexibility and as such a new set of patterns and best practices to deal with these. One of these challenges is that the bottom up enablement of existing legacy applications as services often poses problems as the non functional requirements for these services in an SOA environment change considerably from the requirements when the application was originally created. In the case of the Texas engagement we had a well defined service and a legacy implementation that did not perform. Use the decorator/wrapper pattern I hear your shout and you would be right. This was the start of what was to become know as the requester side caching pattern. The requester side caching pattern can decorate an interface or service on the requester side with caching capabilities to significantly improve performance.


The requester side caching pattern was applied to the client of the underperforming service, and the non function requirements were satisfied. In my last blog entry I talked about a pattern specification and a pattern implementation. Here is the pattern specification for the requester side caching pattern. An implementation of this pattern using the Rational Software Architect pattern engine will be made available as a Reusable Asset Specification (RAS) asset on the developerWorks RAS repository and a supporting developerWorks article showing how to use this pattern will be also be published within the next couple of weeks. This developerWorks article will be part of the series on Building SOA Applications with reusable assets The requester side caching pattern is testament to the vision of the original Gang of Four Design Patterns. These patterns come up repeatedly in different contexts to help us understand and formulate new best practices for an ever changing IT landscape.