Making it easier for Organizations to Publish Services

Publishing a service is far more complex an exercise then simply registering a WSDL in a UDDI registry. The technical problems that have to be addressed by a service provider are non-trivial. Equally important are the political, financial and governance issues that surround providing a service to a community like the DoD. The Federal Government and the DoD and its NCES program, in its implementation of a shared-services infrastructure needs to make it easy for a service to be published on the Global Information Grid (GIG).

The technical tasks involved are only one aspect of service production. These generally include: Registering a service description in the service registry; making sure that the application infrastructure such as the application server hardware, network, etc. are scalable and up to the task of handing the kinds of loads that are expected by a DoD wide service; keeping track of service quality metrics and detecting and reporting violations in Service Level Agreements (SLAs). However, these are only a small part of the kinds of operations that should be required from a shared-services infrastructure.

Setting up the mechanisms for a competitive services marketplace is a key element to allowing service providers a chance to get their service used by potential service consumers. The following is a representative (and certainly not exhaustive) list of possible service infrastructure capabilities. Some of these topics will be covered in more detail later posts on this blog.

Provide the means for suppliers of services to advertise their service. Simply relying on the pull model of existing service registries is not sufficient for service providers. There needs to be a well-known and standard push channel for service providers to advertise their service(s).

Provide the inherent capability for hosted services to automatically scale to meet demand. This should be based on how well Service Level Agreements are being kept. This is capability will assure service providers that their service will be able to continue to satisfy the contractual obligation for service while not burdening the service supplier PEO with this non-trivial requirement. It should be noted here that this type of capability will most likely be a critical success factor in the overall success of a Shared Service Infrastructure program. Having this capability inherent will greatly increase the trust factor of both consumers and producers of brokered services.

Provide real-time and historical Quality of Service reports regarding brokered services. This, in conjunction with associated service SLAs will give service providers the ability to compete on a quantitative basis where consumers are either making a choice between two or more possible services to include or in the situation where the service they are using is failing and they are shopping for another to use.

Another easily overlooked characteristic of the shared-services infrastructure is one that helps protect producers of services from becoming victims of abusive consumers. Service providers live and die by how well they satisfy their SLAs and Quality of Service objectives. For example, assume there is a consuming system that somehow utilizes a service in a manner not consistent with how it should be consumed (such as an inadvertent over-zealous invocation pattern … sort of a programming bug that causes a denial-of-service attack on a shared service). The infrastructure needs to provide a mechanism that helps protect the shared service and its all-important service metrics. In short, service providers need to trust that the shared services infrastructures (such as NCES) will provide a fair place to conduct business and provide technical and procedural mechanisms (a.k.a. governance) to help mitigate the risk of publishing a service.

The lynchpin in the entire infrastructure effort is to provide a set of services that allow consumers and producers of services to work though a single trusted environment. The infrastructure must provide a consistent language for governance; for example, service level agreements, violation notifications as well as processes and policies for non-compliant providers and consumers to regain compliance. It must include a path for consumers to switch services if they feel the need. It must provide providers of services a set of mechanisms by which incentives for the service provider can be realized. This speaks directly to your question: “How can we maximize the flexibility and reusability of core services?” The core services will only be used if and when NCES can provide a consistent and trusted shared services environment.

Service Infrastructure Value: Making SOA Easier

Service-Oriented Architecture seems like a simple idea. Produce a bunch of services, publish them through some type of service infrastructure for governance, and then consuming these services will realize the obvious benefits of SOA. It seems pretty straightforward. There are lots of articles out there that tell you how to build shared services utilizing various protocols such as WebServices, REST, etc. There are many technologies you can use to consume this newly created plethora of shared services. The important part is that the service protocols are interoperable. As for the service infrastructure, there seems to be no end to the catalog of enterprise-class software products you can buy that will provide you all the services governance you could possibly need. At least this is the promise. The reality is that there are various road bumps along the road to SOA nirvana. Certainly there are some technological gaps in the whole SOA story; but those gaps will be filled and the technology will work. What problems I’m talking about are a bit squishier. There are business, organizational, political and financial issues that need to be addressed.

Here’s the idea: shared services are published via a service broker via one or more interoperable interfaces that will be connected to and invoked by some consumer. There are some practical issues that surround this. The practical issues that I would like to focus on in the next several posts include what I call the “friction” involved in consuming or producing services through a shared services infrastructure. And the ideas here are more targeted toward those that are contemplating standing up a central service infrastructure or brokering services within your organization. Remember that simply because you have a service infrastructure, it doesn’t mean that service consumers and producers will come flocking. There is a level of responsibility with the service infrastructure provider that can significantly reduce the “friction” involved in getting producers and consumers to come together. As it turns out, it’s not quite as natural as you might think.

The Federal SOA Watershed.

The Federal Government finds itself at a watershed. The old practices of procurement to satisfy government requirements are being squeezed by a strong need for agility, visibility of information and our ever decreasing ability to pay to “reinvent the wheel.” There is an old saying that goes like this: “It’s not that you have what you want, but do you want what you have.” When it comes to looking at how to create new systems and processes to handle new requirements, government agencies won’t be asking how to build or buy something new, but asking what exists now that can be reused and shared among programs to get the best value for government dollars spent. And the time for this is now, not 5 or 10 years from now; so we’d better learn to want what we have.

With world events being what they are, there is an enormous pressure on federal budgets and organizations to not only do more with less, but share what they have so that their systems, organization and people have the greatest value. Service-Oriented Architectures (SOA) and Service Infrastructure software enable the Federal Government to take the critical steps toward sharing information and processes among other agencies. This article provides an overview of what a federal agency needs to think about both organizationally, politically and technologically with regards to SOA technologies and practices.

There has been much hype about SOA in the last few years; however, the thing to understand about SOA technology is that it represents a significant leap in the maturity of distributed systems. Sharing resources is the foundation of a sound SOA strategy. These shared resources should be available to consumers with as little effort as possible. Organizations should expose their shared system via completely standards driven interfaces and does not require specialized software or hardware to be purchased by those wishing to use that shared service. The most important software category of SOA software with respect to strategic SOA initiatives is SOA Infrastructure software such as Enterprise Service Bus (ESB), Service Enablement Platforms and Data Service Platforms. The reason infrastructure software is so important to consider is because it provides the technology platform that enables proactive sharing of services while mitigating many of the risks and issues that have killed other distributed systems or data/service sharing efforts.

Strategic SOA presents some significant challenges to the organization of any federal agency. Most significantly, shared services represent a completely new way of doing business. Instead of a “need to know doctrine” where agencies do not share information unless there is a specific need from some specific other agency. The doctrine now is better characterized as a “need to share” doctrine. Agencies need to proactively begin sharing data that is or maybe useful to other agencies. The organizational preparation that an agency needs to accomplish this reversal of doctrine should not be underestimated. Part of that preparation begins with selecting the correct SOA Infrastructure software but even more vital to this transition is partnering with a System Integrator that offers extensive consulting practices that that covers many organizational areas such as cost, budgeting, governance and business strategy.

The politics of SOA will most likely be quite simple. Government agencies shall share data proactively; actively encourage other organizations to utilize their shared services and utilize the shared services of others to the greatest extent possible, or face budget cuts and poor performance reviews. The technology is there and is ready to be utilized today to get maximum value from the federal governments existing assets. The “need to share” doctrine can be realized and the momentum toward a shared service infrastructure across the government is growing and doesn’t look like it’s going to stop any time soon.

Why write about SOA for the Federal Government?

I suppose that it might be might have been easier simple to write about SOA in the abstract without framing it in some specific context. As I work in and around the US Federal Government I have some knowledge and insight as to how the government (Civilian, Military and Intelligence community) does business. Armed with that knowledge and my position as Federal SOA Architect for BEA, it's clear to me that the benefit of SOA in this space is simply enormous.

Of significant note is the sheer size and mass of the US Federal Government. In the world of Enterprise Architecture and distribruted engineering, I would be hard pressed to find an organization larger, more complicated in scope or gravity nor one that can compare politically. The military alone is a vast organization that will benefit is ways I think even it does not quite understand. The Civilian side of the fed with its effort to accomplish its "line-of-business" consolidation dwarfs even the largest of private companies with the scope and complexities involved. The Intelligence Community has an obvious need to share information that can only be realized through SOA principles and a shared service infrastructure.

In all, I think it's clear that the Federal Government crystallizes and clarifies the needs, challenges and solutions for which SOA was invented. This blog was meant as a place to describe the thoughts, challenges and great ideas that I've run across during my time practicing SOA in the Federal space.