CHAPTER
The Interface and Change
13
All things change. This simple statement describes the most complex concept for any SOA—change. A simple service to read the Item database and return details to the consumer for a particular item will no doubt change over time. The change might be adding a new element to the item database and returning it in the reply message. Or it might be a complete restructuring to the data and as a result the message. Each of these scenarios and any number of other variations represent the types of change that a service will undergo over time. Once a Web service (or any type of service) has been designed, developed, tested, published, and deployed, it is then also available to consumers. Consumers will search for services (i.e., “discover” or “find”), and when found, the consumer will then “bind” to the service interface. The process of binding associates the service consumer to the service interface and messages and is analogous to signing a contract. By virtue of publishing the service interface and making it available, the service commits to supporting the defined service interface, service operations, message formats, and supported messaging protocols. Similarly, when service consumers “bind” to a service interface, they commit to sending request messages in the stated format to the exposed endpoints on the ESB and using the supported protocols. Between the service, the consumers, and the service interface, a contract for collaboration has been defined. This concept of a contract that is largely based upon the service interface is fundamental to SOA. It removes ambiguity and potential anomaly from consumer and service interactions. It also constrains or limits the scope and context of messages that are exchanged. However, when changes occur to the service that would in some manner be exposed to consumers, this contract can be broken. The terms and conditions implied in this interface contract are important because the message structures and message content are used to either build or map the consumer’s object model. The structure, location, naming, and the metadata characteristics of the message are reflected by the internal processing of the service consumer. If something in a request message has changed, and the consumer is unaware of that change, the service will reject the request and return a fault message. Similarly, if something has changed in the reply message, and the consumer is unaware of it, the message will not be deemed as consumable, or an error will be raised. To keep the SOA up and running, to avoid unnecessary and ongoing code changes, where possible, service consumers must be insulated from change.
© 2010 Elsevier Inc. All rights reserved.
259
260 Chapter 13 The interface and change
The significance of a change is what determines whether the consumer can be protected from it or not. There are numerous types of change that will occur naturally and over time to a service. A simple change, such as adding a new element to the response message, can have significant impact. However, if the new message element were optional, the potential for impact to consumers might be somewhat reduced. Examples of change with less significant potential for impact to consumers include the following: Add a new service operation. Add a new, optional element to either the request or reply message. n Change an element of either the request or the reply message, from mandatory to optional. n Remove simpleType constraints from any element (remove type restrictions). n n
The preceding examples are still of potential impact and should not be ignored or taken lightly during service interface design. Rather, the service interface designer should anticipate that every service will undergo some type of change and include patterns and techniques to help mitigate the potential impact on consumers. Other types of change are of greater significance, and, unfortunately, service consumers will most likely incur some type of impact if they do not rebind to the new or revised service interface. These types of changes will require structure, content, type, or behavioral modifications, and potentially a refactoring of the consumer’s underlying object model. Following are a few examples of significant change: Add a new mandatory element to either the request or reply message. Modify an element’s type or type constraints (change to a simpleType or its facets). n Modify the placement of an element (move an element from one location in the message to another). n Change an optional message element to mandatory. n Delete an element from either the request or the reply message. n n
For any of the previous “significant” service interface changes, the consumer will at minimum be required to rebind to the new or revised service interface, and most likely will also have to modify its internal object model and behavior. Anticipating change is something that can help to reduce potential impact on and optimize the design and development investment for already deployed and bound service consumers. Change management for Web services has been struggled with for many years, and it is an area that introduces more complexity than should be expected. This is not to say that Web services cannot change over time or that there isn’t a solution. Rather, the challenge is in mitigating the impact from change to service consumers. Some practitioners will rely entirely upon the SOA and Web service design tooling to manage change. The approach often taken here is to just design, build, and deploy a new version of the service whenever a change occurs. Existing service consumers remain bound to their current version of the service, and new service consumers will have their own versions of the service. While this approach initially seems to make sense, it is fraught with complexity and potential cost. Each new service deployment will be a “one-off” from the original with significant potential to introduce disparity and anomaly. Further, each deployment of a service into the SOA infrastructure includes cost. Each of these services requires additional management, monitoring, maintenance, and governance.
13.1 Schema extension 261
13.1 Schema extension An experienced service designer will adopt the position that a service interface will always require future modification. Determining exactly when those changes might occur and the degree of significance is not easy, but the designer can make an optimistic assumption that one or more changes will be of minimal significance, such as adding a new, optional element to the reply message. For these less impactful types of change, the service designer can prebuild extension areas into the service interface and resulting message structures. This technique is not quite the same as that of more traditional and procedural languages such as COBOL (Common Business Oriented Language), where “FILLER” elements are defined to preallocate a fixed length of space in a structure. With XML and as supported by XML Schemas, the service designer can incorporate an optional extension element that is also a complexType structure. Additional and nested structures and elements can be defined subordinate to the extension element at a later time (see Figure 13.1). This extension element approach still has some limitations. Depending upon the binding process, an optional element can still be prescriptively defined to the consumer’s object model. On receipt of a reply message, the consumer would correctly process when the optional extension element is present in the message or not. However, if the element contains additional and subordinate structures and elements, processing becomes more complex. To avoid anomaly, and when present in a message, the consumer’s object model would have to ignore any nested content of an optional extension element. This approach can be somewhat limiting, but it does help to mitigate the potential for impact to the consumer. The value of this technique is exploited when the consumer’s object model has been refactored to ignore the optional element and its subordinate content. As new, optional elements are added to the message, the consumer can ignore those changes. The use of XML Schemas to define and constrain a Web service interface provides additional advantages. Not only can the service interface be designed to include a predefined and optional extension
01 Item-Details. 05 Item-ID 05 Desc 05 Version 05 Length 05 Color 05 Diameter 05 Weight 05 Extension-Area
PIC X(4). PIC X(20). PIC 9(3). PIC X(10). PIC X(12). PIC X(12). PIC X(10). PIC X(30).
05 Filler Redefines Extension-Area. 10 Version. 15 MajorVerNo PIC 9(2). 15 MinorVerNo PIC 9(2). 10 Base-Packaging-Unit PIC X(6). 10 Bulk-Packaging-Unit PIC X(6).
Figure 13.1 COBOL “Filler”
262 Chapter 13 The interface and change
element but the underlying and subordinate structure and element content of the extension can also be defined to evolve somewhat dynamically. This is accomplished by defining a schema “wildcard” to the optional extension element.1 A “wildcard” allows additional and variable structural content to be incorporated at the point of implementation in the service interface. The schema declaration is known as an “any” wildcard declaration. The schema wildcard “any” declaration allows another separate schema to be referenced and defined as additional content of the message. The “any” declaration allows the service designer to reference a namespace (in this sense, a namespace representing another separate schema) that matches the targetNamespace of the current schema, an explicitly declared namespace (specified by namespace URI as the name), a local namespace, or another namespace that does not match the targetNamespace of the current schema (no-namespace schemas can also be referenced with this option). The wildcard pattern is also somewhat of an indirect namespace association pattern, where the primary XML Schema references a separate wildcard schema (see Figure 13.2). In addition to identifying the namespace of the separate wildcard schema, the XML Schema “any” declaration provides an attribute-defined parameter to determine whether the constraints of the wildcard schema will be strictly applied or not. The “processContents” attribute of the “any” declaration instructs the validating XML parser to strictly apply the referenced schema constraints (with a processContents value of “strict”), to loosely apply the constraints (with a processContents value of “lax”), or to ignore the wildcard schema (with a processContents value of “skip”). XML Schema
Subset fragment
In this example, the “namespace” attribute value matches the targetNamespace URI value of the parent schema.
In this example, the“processContents” attribute value is set to “strict”, which will instruct the validating XML parser to strictly test against and apply the XML Schema constraints of the “wildcard” schema.
Figure 13.2 Example XML Schema Wildcard Syntax
In this example, the wildcard content is set to optional (“minOccurs” attribute value of “0”), and an unlimited number of occurrences are allowed (“maxOccurs” attribute value of “unbounded”).
13.1 Schema extension 263
The determination of which additional wildcard schema will be applied to the service interface and resulting XML message is also determined by contract. That is, the service interface specifies which namespace-qualified schemas are allowed to define the subordinate structures and content of the wildcard extension. Additional parameters determine how strictly these additional schema structures will be enforced when the message is parser validated. The consumer also plays a role in the definition of the contract. The consumer of the service will specify which namespace and namespace-qualified XML Schema (of those that are allowed by the service interface) will be used. When the service is initially deployed, the optional extension element of the service interface message does not include any additional content. At a future point, the service is enhanced, and the optional extension point might then include a new element. The originally bound consumers of the service can ignore the optional extension element and are unaware of the underlying content. New consumers that are “aware” of a separate wildcard schema will make an association between the service interface, the message, and the wildcard schema using the schemaLocation attribute of the XML message. The schema location attribute will contain a pair of values to represent the namespace of the wildcard schema and its file name (a directory path is optional but recommended). Prior to implementation, these new consumers will bind to both the original service interface and the identified wildcard schema. The previously bound consumers will not include the additional wildcard namespace reference in their schemaLocation attribute values, and as a result, they will be “unaware” and unaffected by the additional element from the new schema (see Figure 13.3). In this case, the new element will not be exposed in the message. This allows previously bound service consumers and new service consumers to invoke the same service and process messages with and without the additional wildcard content as appropriate to their understanding of the contract. As described, the wildcard schema pattern to extend a service interface is quite powerful. It can also be complex. The service designer must ensure that the interface content model is still deterministic. That is, the interface should not include schema declarations that are in conflict, or where the validating XML parser cannot determine which element is next in the corresponding XML message. To avoid these potential problems and to optimize the wildcard pattern, the service designer should follow a set of simple best practices: Ensure that XML Schemas “any” wildcards are supported by the environment, parsers, and binding utilities. n Position the extension element at the end of the element set for the message. n Define the extension element to the primary interface schema as an optional complexType. n Specify the wildcard “any” declaration “namespace” attribute with an explicit namespace value. n Specify the wildcard “any” declaration “processContents” attribute with a value of “strict”. n Test the wildcard extension. n
Positioning an optional extension element at a specific location in the message plays a significant role (see Figure 13.4). When the extension element has been located at the end of the message, consumers of the message will have increased ability to ignore any additional content. During the XML Parser validation process, the consumer can elect to stop parsing with the last element of the XML message that maps to the lowest-level termination point of the object model. If the extension element is placed somewhere before the last element, some consumers may encounter difficulty in processing
XML Message
Subset fragment
…
The schemaLocation attribute values of this previously existing consumer’s message does not identify or reference the wildcard schema. As a result, the additional “ExtensionArea” element is not included. XML Message for a new Consumer
XML Message
Subset fragment
… …
The schemaLocation attribute values of this new consumer’s message identifies and references the wildcard schema. As a result, the additional “ExtensionArea” element is included.
Figure 13.3 Existing and New Service Consumers with a Wildcard
264 Chapter 13 The interface and change
XML Message for a previously existing Consumer
13.1 Schema extension 265
XML Message with an extension area
XML Message
Subset fragment
5678 Blue Stick Pen 2 8.0 in Blue 0.25 in 1.5 oz
The Extension Area of the message (in this example, an element named as “) is defined near the end of the message.
Figure 13.4 Position the Extension Element at the End of the XML Message Structure
the elements that trail after the extension. This is often the case with a structured and procedural language such as COBOL, where the elements of a file definition have been statically mapped one for one to the elements of a message. If an unexpected element occurs somewhere between the boundaries of the file definition element grouping, an error may result. The service designer should define the extension element as an optional complexType structure. The extension element will be defined by a globally declared and named complexType element with a minOccurs value of “0”. To allow for multiple occurrences of the wildcard schema reference, the extension element should also include a maxOccurs value of “unbounded”, which will allow for any number of occurrences (see Figure 13.5). As a recommendation, the service designer should define the wildcard schema with an explicit targetNamespace value. Avoid matching of namespaces across schemas where possible (this will prohibit reuse) and avoid using an open “##other” namespace value. The value of the referenced wildcard schema namespace will also be declared as its internal targetNamespace value. The referencing XML Schemas will identify the wildcard schema as an explicit namespace value of the “namespace” attribute for the “any” declaration. The XML message will also include the namespace value along with the name of the wildcard schema as a value of the schemaLocation attribute. While this is somewhat limiting, it also simplifies that pattern and adds a degree of referential consistency. Service consumers that include the additional extension content in their XML message processing will need to include a reference to the wildcard schema as a schemaLocation value that matches the targetNamespace of the wildcard extension schema (see Figure 13.6). This approach will also avoid having multiple unmanaged and potentially disparate wildcard schemas being defined with the same targetNamespace.
266 Chapter 13 The interface and change
Figure 13.5 Optional Extension Element
XML Message with an Extension Area
XML Message
Subset fragment
…
XML Schema with Reference to “any” Wildcard Schema
XML Schema
Subset fragment
… wildcard refernce is the same as that specified in the consumer’s message. …
Figure 13.6 Consistent Namespace Values Across Schemas and Messages
13.1 Schema extension 267
XML Schema Subset fragment ...
“v1.1”
The version of the example wildcard extension schema is exposed as part of the namespace URI value and, in this example, has been assigned to: • Default namespace (xmlns=“”) • Qualified namespace (xmlns:EXT=“”) • targetNamespace (targetNamespace=“”)
Figure 13.11 Exposing the Version in the TargetNamespace
a new and incremented version number in an existing schema will result in different binding models between previous and new service consumers, and it can result in anomalous references between subordinate “included” schemas (those with no namespace). If a reusable schema that is referenced by many service interfaces is versioned in this manner, it will cause a ripple of changes across all of those higher-level import namespace references. For this reason, I do not recommend using the namespace URI as the primary vehicle for exposing the entire service version (both major and minor version identification). Rather, I cautiously recommend including only the “major” version in the targetNamespace URI. This follows the previous approach to version identification and granularity. A major version should be used to identify significant change that requires service consumers to rebind. The minor version should be exposed in a different and separate manner (see Figure 13.11). As the definition of a service interface, XML Schemas also provide an additional annotation capability. Annotations are XML Schemas declarations that can be added to any XML Schema definition. Annotations have two child types, and each has a somewhat different intent. The documentation annotation element is largely that—documentation. It is primarily intended for human consumption. Alternatively, the “appinfo” child element is consumable by both humans and in some cases by validating XML Parsers. The content of an appinfo element can be identified and extracted during parser validation and exposed to the parsing application. This schema capability can be used to include the version number of the service. Using schema annotations to expose the service version can be effective (see Figure 13.12). However, not all validating parsers will expose the appinfo content. Also, the appinfo declarations of an annotation are largely unconstrained. That is, they must be well-formed, but they do not have explicit rules that govern the number of occurrences, the data types, and so on. The
272 Chapter 13 The interface and change
XML Schema Subset fragment ...
“v1.1”
The version of the example wildcard extension schema is exposed as elements defined within an XML Schema “annotation” and in this case, as “appinfo” elements.
Figure 13.12 Exposing the Version in a Schema Annotation XML Schema Subset fragment