- Email: [email protected]
Conformance testing of Office Document Architecture (ODA)
Conformance testing of Office Document Architecture (ODA) Using ODA as an example, Richard Carr* and Frank Dawson t discuss conformance testing for standard document interchange formats
Both the difficulties and solutions for testing the conformance of standard document interchange formats are considered here. The difficulty lies both in the general nature of standards, as well as in the restricted scope of the document interchange standard specification. The Office Document Architecture standard is u s e d as an example to show h o w the conformance testing of interchange format standards is being addressed. In addition, s o m e consideration is given to 'lessons learned" from the c o n formance testing of ODA. Some suggestions are offered for the developmental framework of future interchange format standards to facilitate conformance testing.
Keywords: standard document interchange formats, conformance testin& Office Document Architecture
Document Interchange Format (DIF) standards such as Office Document Architecture (ODA) 1 have been developed to facilitate interworking between dissimilar text processing, office and publishing systems. DIF standards are in the same class as other interchange format standards such as the Initial Graphics Exchange Specification (IGES)2 and the Computer Graphics Metafile (CGM) 3. These standards specify a generic data stream architecture for the interchange of a representation of some application object, e.g. part geometry or computer graphics pictures. In the case of DIF standards, the representation is of a *National Computing Centre, Oxford Road, Manchester M1 7ED, UK tlBM Corporation, 03433-50, 5 West Kirkwood Boulevard, Roanoke, TX 76299-0001, USA
compound document. As illustrated by the example in Figure 1, representations for compound documents are in terms of their structure and content. The structure might consist of various logical and layout structures, such as a body object consisting of paragraphs and figures to be laid out on pages with some automatic page numbering scheme. The content could consist of several different content types such as text and graphics objects, or even some automatically generated object. Figure 2 is intended to offer a simplified view of the process involved in interchanging DIF data streams from one system to another. The System A document editor accepts a different document internal format than System B. Interchange is accomplished by converting from the internal format of the particular system to that of the standard interchange format. While the example in the illustration depicts a floppy disc transfer, DIF standards, such as ODA, are designed to be media independent. ODA is applicable to magnetic media as well as other digital transfer methods such as standard electronic mail and file transfer protocols. In general, the scope of DIF standards does not focus on the method for generating or interpreting the data streams. Instead, the standards address the semantics of the structures representing the interchanged objects and the syntax for a transfer format for the structured representation of objects. As these standards are intended to facilitate inten~vorking between dissimilar systems, the conformance of implementations based on these standards is of concem both to the implementer and the client.
CONFORMANCE Standards conformance generally means that the tech-
0140-3664/89/020102-05 $03.00 © 1989 Butterworth & Co (Publishers) Ltd 102
computer communications
Structure
Content examples
examples
l
Header
Graphic Objects
~-- Graph,Obj c ects
Section title
I y,~ ~ ;- TIPIESTRAN~=OlCCI, i~ t~I
F gure made up of illustration caption ~ Paragraph ~
~ ~ I ~"'~ ~ " ~ ~
Formatted text content
.
I ~(~ ~ ~. @
#L~ ~L
--'--------- Geometric graphics content
I. . . . . . . . . . . . . . . . . . . . . . . . . . . . I
Unordered
list
-~ " " - ~
~!}~~i}}i !! ~2!!!~i !;i !!! "~}}!i~i~}i!!!i}! !*° I
Ordered list Automatic page numbering
~-
,. o. . . . , . . . . . . .
~o.~.o o° . . . .
................................ ~:.~,~;°;~':'4;~o~;,=:~'~'.:',~:~';'~:',.~,.~'~'~':~ ~ 1 ~ ~0
Revisable text content
--------- Generated content
Pers0nal PuDL=snIn(]Systems ;nrr0ouction
Footer
Compound document representation
Figure 1.
System A
System B
I ] ~
Document~ Internal FormatB
Document Internal Format A
Export Import
Export -"" Import
: Document Interchange Format
Figure 2.
DIF data stream interchange process
nical specifications defined in the individual standard are being met. In the case of DIF standards, conformance implies that the data stream adhering to the standard conforms to: (1) Some prescribed syntax that ensures individual data values are within range - - context free measurement. (2) Some abstract semantics that ensure individual data values are valid in combination - - context sensitive measurement. The first of these is fairly straightforward to quantify, and conformance to (1) can easily be verified. However,
vol 12 no 2 april 1989
measuring (2) is more complicated and for some abstract semantics may require a reference implementation to verify that reception is at least theoretically possible. Most standards are specified in a descriptive prose, but this format has the inherent weakness of interpretation. The value of conformance is depreciated if different, conforming implementations can be incompatible. Another characteristic of standards that makes conformance quantification difficult is that not all of the functionality of a particular standard may be mandatory. Most standards provide for optional features to increase the application of the standard to different uses. Certainly, implementers will not support all features of a standard. This raises the issue of which features of a standard must be supported in order to conform to the standard. DIF standards have a unique quality that makes verification difficult. Unlike protocol standards such as those defined for the Open System Interconnection (OSl) Reference Model, D IF standards and data stream standards are 'blind' interchange formats, i.e. no provision is provided for specifying the process to be used for the initiation or termination of an interchange session between an originator and a recipient of the data stream. Also, in many of these standards no provision is provided for negotiating the set of semantic features to be used in describing the document. Finally, no provision is provided for exception condition reporting, handling or recovery. The DIF standards merely define the data stream to be exchanged between an arbitrary originator and recipient. One exception to this is the ODA standard which, by use of Document Application Profiles (DAP), provides an indication of the features interchanged. Under these circumstances, conformance testing of DIF standards would seem to do little to assure interworking. However, experts working within the ODA community have provided a framework to complement the limited conformance specification of the standard,
103
that has permitted the development of at least one practical approach to testing ODA conformance.
ODA CONFORMANCE The ODA standard specifies conformance in terms of a conforming data stream. The data stream must either conform to the semantics and syntax of the standard as a whole, or the data stream must conform to a subset of ODA as specified in a DAP. The DAP is identified by the DAP Attribute in the document profile section of the data stream. (DAP identifiers are to be registered with the International Organization for Standardization (ISO) registration authority.) While the ODA standard provides rules for the definition of a DAP, it does not define individual DAPs: this is left to implementers of ODA. Three organizations have emerged as regional workshops for ODA expert groups developing DAPs. These include the National Institute for Standards and Technology (NIST) (formerly the National Bureau of Standards (NBS)) ODA Special Interest Group (SIG), the European Workshop on Open Systems ODA Expert Group, and the Asian-Oceania Workshop on Open Systems ODA SIG. ODA implementation agreements developed by these three groups are being internationally aligned by the Profile Alignment Group for ODA (PAGODA). It is expected that the internationally aligned DAPs will be submitted to ISO for processing as International Standardized Profiles (ISP). These DAPs provide a comon or core set of semantic and syntactic features to be supported by implementations interested in interworking with dissimilar systems. In addition, the Consultative Committee on International Telephone and Telegraph (CCITT) has an ongoing effort to develop recommendations regarding ODA DAPs. The implementation agreements being developed within these ODA expert groups are also providing solutions for other difficulties in testing DIF standards. Conformance specifications in these agreements have identified common classes of implementations 4. These include an originator, recipient and originator~recipient class of implementation. Originators generate ODA data streams; recipients interpret and translate ODA data streams into internal document formats; and originator/ recipient class implementations accomplish the function of both an originator and recipient. The agreements further specify minimal requirements for the generation and interpretation of ODA data streams. The precise format and specification for these constraints on the classes of implementations are currently being addressed by the ODA expert groups mentioned above. In the past, such specifications were referred to as the Implementation Conformance Statement (ICS). The ICS, along with the DAP, provides the specification of an implementation's conformance to the ODA standard. The weakness of the descriptive prose specification of ODA has been addressed by the development of a formal specification for ODA (FODA) s. The aim of FODA is to provide an unambiguous interpretation of ODA. It is a predicate calculus-based function-by-function formal description of ODA, which is intended to be used as a supplemental reference for implementing ODA, as a validation tool for verifying the conformance of systems, and as a reference point for examining future extension and revisions to ODA. The specification is being processed
104
by ISO as an addendum to the ODA standard. The predicate calculus format of FODA can be processed, in a straightfoward manner, by rule-based programming languages, eliminating most of the interpretation weaknesses of the descriptive prose format of the standard 6.
TODAC TEST ARCHITECTURE Since the process of generating and interpreting the ODA data stream is outside the scope of the ODA standard, ODA conformance testing is limited to an analysis of ODA data streams. While this makes the job of conformance testing easier for implementers, it does little to assure that implementations will be able to interwork with each other. Hence, most ODA test architectures have attempted to go beyond strict ODA conformance testing to provide additional tests that may aid in determining the likelihood of an implementation interworking with other, dissimilar ODA implementations. Such a verification method is being developed by an international project, Testing ODA Conformance (TODAC) 7. Staffed by the National Computing Centre (NCC) in the UK and the Department of Communications in Canada, TODAC intends to develop ODA conformance testing tools by the end of 1989. Within the TODAC test architecture, an implementation is referred to as an Implementation Under Test (IUT). An IUT is categorized as either an originator, a recipient or an originator/recipient class of implementation. The IUT is specified with an ICS and one or more DAPs: the DAPs are specified in an unambiguous, machine readable format. The TODAC format has been accepted as the basis for an addendum to ODA on a recommended proforma and notation for ODA DAPs. ODA expert groups participating in PAGODA have indicated their intent to use this recommendation when it becomes available. The TODAC test architecture consists of three components (see Figure 3): the Test Document Specification component generates a set of parameterized, generic test cases specific to an individual version of ODA. FODA is used as the formal specification for ODA. Specific test cases, in the form of test document specifications, are generated for an IUT by processing the requirements specified in the ICS and the DAP against the generic test cases. These test document specifications are input to the Generation Test and Reception Test components. The generation test component creates verification test reports for originator class implementations. For an originator, the test document specifications are used as activity scripts for generating IUT results in the form of ODA data streams. A reference document analyser processes these output data streams from the IUT and generates the test reports. The parameters of the test results are based on the criteria specified by the test document specifications, the ICS and the DAP. The Reception Test component creates verification test reports for recipient class implementations. For a recipient, only those systems that can image a document can currently be tested. Gateways that take in an ODA data stream and convert it into an output data stream in some non-ODA format are outside the scope of this test environment. Recipient class implementations that can only generate 'soft copy' images on terminals are also outside the scope of this test environment. In the Reception Test component the'test document
computer communications
Test DocumentSpecificationSub-System
I RepresentativeI~1 i structure I ~ . ~
Attribute ValuelList
~ Test Document Specification Generator
Test identification
I I
Implementation J Conformance Statement
= Test ~.~ Document = Specification
I
E
Generator
II
I ODIF Generated
By Test System
I II Reference Imaging System
I
II '°T
Receiver
II
,UT [
Generator
ODIF
I Imaged Document
Generated
By IUT
I
t I Reference
Document Analyser
I
I m age
Comparison& Analysis
I m ages
I
ReceptionTestingSub-System Figure 3.
) /
II
! !
Test Document Specification
!:
GenerationTestingSub-System
TODAC test architecture
specifications are input to a reference ODA data stream generator that creates ODA data streams intended to test specific features of the IUT specified by the ICS and DAP. The test ODA data streams are imaged by a reference ODA imaging system to produce reference images. The recipient portion of the IUT receives the test ODA data streams. The IUT must be able to image the test documents so that an image comparison and analysis can be completed between the reference and test images. When analysing the result of a reception test, an analysis would examine the test image, with reference to the test cases specified in the Test Document Specification, to understand the intended result and therefore conclude whether the test has been executed successfully or not. The reference image is used as a guide to assist in determining the success of the test, but is of secondary importance to the Test Document Specification. The result of this analYSiS is in the form of a test report. For originator/recipient class implementations, the originator and recipient portions of the IUT are tested separately in the same way as originator and recipient class implementations (see above). The TODAC test architecture goes beyond merely testing the conformance of an IUT to ODA. It is expected vol 12 no 2 april 1989
that these test tools will be able to provide information to implementors and clients on the potential for inte~vorking with other ODA implementations. LESSONS
LEARNED
Interchange format standards such as ODA provide a challenge to conformance testing. The difficulties in meetingthis challenge could be lessened by changingthe framework of the standards. In particular, the scope of the interchange format standard needs to be increased. Figure 4 shows the layered components involved in the interchange of a DIF between two systems. An application in system A produces the data stream to be exchanged with system B by invoking the data steam generator process through some Application Program Interface (API). System B receives and interprets the data stream exchanged from system A by invoking the data stream interpreter process through another API. At present, DIF standards such as ODA go no further in this model than specifying the semantics and syntax of the data stream. Increasing the scope of these standards to include both the abstract and real specification of the 105
Application ProcessB
Application ProcessA
API
API Generator Process
~0~
I nterpreter [ Process
Document Interchange Format
Figure 4. scope
Document interchange standards proposed
REFERENCES* 1 2 3 4 5 6
features of the processes used to generate and interpret the data stream, as well as including the programming language bindings to these specifications, would more readily permit the development of uniform and useful conformance test architectures. It should be noted that an alternative to increasing the scope of the standards would be to augment the existing scope with supplemental standards covering the necessary 'services' for generating and interpreting the DIF data streams (CCITT recommendations on 'service definitions' is an example of the latter approach). Unambiguous specification of the standards could be achieved through the use of formal descriptive techniques such as FODA to complement the descriptive prose specification. The formal descriptive technique would also greatly assist in the development of automated test tools.
SUMMARY The important issues involved in the conformance testing of standard document interchange formats such as ODA have been addressed here. Testing the ODA conformance of interchanged data streams can assess conformance to the ODA standard, but it may not provide a strong indication of whether an implementation will interwork with other ODA implementations without some value added testing, such as image comparison. The implementation agreements for ODA have augmented the conformance semantics of the ODA standard. Such specifications complement the conformance statements in the ODA standard with requirements for specific categories of ODA implementation. Also, a verification method being developed in the TODAC Project was described. The testing tools being developed within TODAC address more than the mere testing of ODA conformance -- it is believed that such tests greatly assist in providing information on the likelihood that an implementation will be able to interwork with dissimilar ODA implementations. The development of ODA tests would be be facilitated if the scope addressed by interchange format standards were extended to support a conceptual framework that addressed the need for conformance testing.
106
7
Information Processing -- Text and Office Systems -Office Document Architecture (ODA) and Interchange Format (ISO 8613) (November 1987) Initial Graphics Exchange Specification (IGES) ANSI Y14.26M, Version 3, ANSI, USA (1987) Information ProcessingSystems -- ComputerGraphics Metafile for the Storage and Transfer of Picture Description Information (ISO 8632) (February 1987) Stable Implementation Agreements of the NIST Workshop for Implementors of OSI National Institute for Standards and Technology, USA (May 1988) Formal Specification of ODA Document Structures Proposal for Draft Addendum to ISO 8613/2 (ISO/IEC JTC1/SC18/WG3 N996) (March 1988) Conformance Testing of ODA Document Structures Based on FODA (ISO/TC 97/SC 18/WG3 N881) Dr W Appelt, GMD, Bonn, FRG (August 1987) TODAC Methodology Requirements Documentation National Computing Centre, UK/ Department of Communications, Canada (March 1987)
*International Standards (IS) are available from the International Organisationfor StandardizationCentralSecretariat,1 rue de VarembE, CasePostale56, CH-1211Geneva20, Switzerland. Richard Carr MSc MBCS CEng has been involved with voice and data communications since 7971. He has worked in Australia on the introduction of PABX systems, and has lectured in mathematics and computing. Since 1984 he has worked for the UK National Computing Centre, where he currently leads the TODAC project He is co-author of the Formal Specification of ODA, editor of ISO notation for DAPs, and convenor of the ISO SWG on ODA conformance. He has designed two profiles for the UK GOSIPproject and is chairman of the D TI's ODA Implementor Group.
Frank R Dawson received his MSc in electrical engineering from the University of Missouri. He has over 10 years of experience in architecture, standards and the implementation of computer graphics and office systems. As a member of the Office Systems Architecture and Standards department of IBM's Application Systems Division, his responsibilities include the chairmanship of the NIST ODA SIG. He is head of IBM's delegation to PAGODA, and IBM representative to the COS Document Architecture Subcommittee.
computer communications
Both the difficulties and solutions for testing the conformance of standard document interchange formats are considered here. The difficulty lies both in the general nature of standards, as well as in the restricted scope of the document interchange standard specification. The Office Document Architecture standard is u s e d as an example to show h o w the conformance testing of interchange format standards is being addressed. In addition, s o m e consideration is given to 'lessons learned" from the c o n formance testing of ODA. Some suggestions are offered for the developmental framework of future interchange format standards to facilitate conformance testing.
Keywords: standard document interchange formats, conformance testin& Office Document Architecture
Document Interchange Format (DIF) standards such as Office Document Architecture (ODA) 1 have been developed to facilitate interworking between dissimilar text processing, office and publishing systems. DIF standards are in the same class as other interchange format standards such as the Initial Graphics Exchange Specification (IGES)2 and the Computer Graphics Metafile (CGM) 3. These standards specify a generic data stream architecture for the interchange of a representation of some application object, e.g. part geometry or computer graphics pictures. In the case of DIF standards, the representation is of a *National Computing Centre, Oxford Road, Manchester M1 7ED, UK tlBM Corporation, 03433-50, 5 West Kirkwood Boulevard, Roanoke, TX 76299-0001, USA
compound document. As illustrated by the example in Figure 1, representations for compound documents are in terms of their structure and content. The structure might consist of various logical and layout structures, such as a body object consisting of paragraphs and figures to be laid out on pages with some automatic page numbering scheme. The content could consist of several different content types such as text and graphics objects, or even some automatically generated object. Figure 2 is intended to offer a simplified view of the process involved in interchanging DIF data streams from one system to another. The System A document editor accepts a different document internal format than System B. Interchange is accomplished by converting from the internal format of the particular system to that of the standard interchange format. While the example in the illustration depicts a floppy disc transfer, DIF standards, such as ODA, are designed to be media independent. ODA is applicable to magnetic media as well as other digital transfer methods such as standard electronic mail and file transfer protocols. In general, the scope of DIF standards does not focus on the method for generating or interpreting the data streams. Instead, the standards address the semantics of the structures representing the interchanged objects and the syntax for a transfer format for the structured representation of objects. As these standards are intended to facilitate inten~vorking between dissimilar systems, the conformance of implementations based on these standards is of concem both to the implementer and the client.
CONFORMANCE Standards conformance generally means that the tech-
0140-3664/89/020102-05 $03.00 © 1989 Butterworth & Co (Publishers) Ltd 102
computer communications
Structure
Content examples
examples
l
Header
Graphic Objects
~-- Graph,Obj c ects
Section title
I y,~ ~ ;- TIPIESTRAN~=OlCCI, i~ t~I
F gure made up of illustration caption ~ Paragraph ~
~ ~ I ~"'~ ~ " ~ ~
Formatted text content
.
I ~(~ ~ ~. @
#L~ ~L
--'--------- Geometric graphics content
I. . . . . . . . . . . . . . . . . . . . . . . . . . . . I
Unordered
list
-~ " " - ~
~!}~~i}}i !! ~2!!!~i !;i !!! "~}}!i~i~}i!!!i}! !*° I
Ordered list Automatic page numbering
~-
,. o. . . . , . . . . . . .
~o.~.o o° . . . .
................................ ~:.~,~;°;~':'4;~o~;,=:~'~'.:',~:~';'~:',.~,.~'~'~':~ ~ 1 ~ ~0
Revisable text content
--------- Generated content
Pers0nal PuDL=snIn(]Systems ;nrr0ouction
Footer
Compound document representation
Figure 1.
System A
System B
I ] ~
Document~ Internal FormatB
Document Internal Format A
Export Import
Export -"" Import
: Document Interchange Format
Figure 2.
DIF data stream interchange process
nical specifications defined in the individual standard are being met. In the case of DIF standards, conformance implies that the data stream adhering to the standard conforms to: (1) Some prescribed syntax that ensures individual data values are within range - - context free measurement. (2) Some abstract semantics that ensure individual data values are valid in combination - - context sensitive measurement. The first of these is fairly straightforward to quantify, and conformance to (1) can easily be verified. However,
vol 12 no 2 april 1989
measuring (2) is more complicated and for some abstract semantics may require a reference implementation to verify that reception is at least theoretically possible. Most standards are specified in a descriptive prose, but this format has the inherent weakness of interpretation. The value of conformance is depreciated if different, conforming implementations can be incompatible. Another characteristic of standards that makes conformance quantification difficult is that not all of the functionality of a particular standard may be mandatory. Most standards provide for optional features to increase the application of the standard to different uses. Certainly, implementers will not support all features of a standard. This raises the issue of which features of a standard must be supported in order to conform to the standard. DIF standards have a unique quality that makes verification difficult. Unlike protocol standards such as those defined for the Open System Interconnection (OSl) Reference Model, D IF standards and data stream standards are 'blind' interchange formats, i.e. no provision is provided for specifying the process to be used for the initiation or termination of an interchange session between an originator and a recipient of the data stream. Also, in many of these standards no provision is provided for negotiating the set of semantic features to be used in describing the document. Finally, no provision is provided for exception condition reporting, handling or recovery. The DIF standards merely define the data stream to be exchanged between an arbitrary originator and recipient. One exception to this is the ODA standard which, by use of Document Application Profiles (DAP), provides an indication of the features interchanged. Under these circumstances, conformance testing of DIF standards would seem to do little to assure interworking. However, experts working within the ODA community have provided a framework to complement the limited conformance specification of the standard,
103
that has permitted the development of at least one practical approach to testing ODA conformance.
ODA CONFORMANCE The ODA standard specifies conformance in terms of a conforming data stream. The data stream must either conform to the semantics and syntax of the standard as a whole, or the data stream must conform to a subset of ODA as specified in a DAP. The DAP is identified by the DAP Attribute in the document profile section of the data stream. (DAP identifiers are to be registered with the International Organization for Standardization (ISO) registration authority.) While the ODA standard provides rules for the definition of a DAP, it does not define individual DAPs: this is left to implementers of ODA. Three organizations have emerged as regional workshops for ODA expert groups developing DAPs. These include the National Institute for Standards and Technology (NIST) (formerly the National Bureau of Standards (NBS)) ODA Special Interest Group (SIG), the European Workshop on Open Systems ODA Expert Group, and the Asian-Oceania Workshop on Open Systems ODA SIG. ODA implementation agreements developed by these three groups are being internationally aligned by the Profile Alignment Group for ODA (PAGODA). It is expected that the internationally aligned DAPs will be submitted to ISO for processing as International Standardized Profiles (ISP). These DAPs provide a comon or core set of semantic and syntactic features to be supported by implementations interested in interworking with dissimilar systems. In addition, the Consultative Committee on International Telephone and Telegraph (CCITT) has an ongoing effort to develop recommendations regarding ODA DAPs. The implementation agreements being developed within these ODA expert groups are also providing solutions for other difficulties in testing DIF standards. Conformance specifications in these agreements have identified common classes of implementations 4. These include an originator, recipient and originator~recipient class of implementation. Originators generate ODA data streams; recipients interpret and translate ODA data streams into internal document formats; and originator/ recipient class implementations accomplish the function of both an originator and recipient. The agreements further specify minimal requirements for the generation and interpretation of ODA data streams. The precise format and specification for these constraints on the classes of implementations are currently being addressed by the ODA expert groups mentioned above. In the past, such specifications were referred to as the Implementation Conformance Statement (ICS). The ICS, along with the DAP, provides the specification of an implementation's conformance to the ODA standard. The weakness of the descriptive prose specification of ODA has been addressed by the development of a formal specification for ODA (FODA) s. The aim of FODA is to provide an unambiguous interpretation of ODA. It is a predicate calculus-based function-by-function formal description of ODA, which is intended to be used as a supplemental reference for implementing ODA, as a validation tool for verifying the conformance of systems, and as a reference point for examining future extension and revisions to ODA. The specification is being processed
104
by ISO as an addendum to the ODA standard. The predicate calculus format of FODA can be processed, in a straightfoward manner, by rule-based programming languages, eliminating most of the interpretation weaknesses of the descriptive prose format of the standard 6.
TODAC TEST ARCHITECTURE Since the process of generating and interpreting the ODA data stream is outside the scope of the ODA standard, ODA conformance testing is limited to an analysis of ODA data streams. While this makes the job of conformance testing easier for implementers, it does little to assure that implementations will be able to interwork with each other. Hence, most ODA test architectures have attempted to go beyond strict ODA conformance testing to provide additional tests that may aid in determining the likelihood of an implementation interworking with other, dissimilar ODA implementations. Such a verification method is being developed by an international project, Testing ODA Conformance (TODAC) 7. Staffed by the National Computing Centre (NCC) in the UK and the Department of Communications in Canada, TODAC intends to develop ODA conformance testing tools by the end of 1989. Within the TODAC test architecture, an implementation is referred to as an Implementation Under Test (IUT). An IUT is categorized as either an originator, a recipient or an originator/recipient class of implementation. The IUT is specified with an ICS and one or more DAPs: the DAPs are specified in an unambiguous, machine readable format. The TODAC format has been accepted as the basis for an addendum to ODA on a recommended proforma and notation for ODA DAPs. ODA expert groups participating in PAGODA have indicated their intent to use this recommendation when it becomes available. The TODAC test architecture consists of three components (see Figure 3): the Test Document Specification component generates a set of parameterized, generic test cases specific to an individual version of ODA. FODA is used as the formal specification for ODA. Specific test cases, in the form of test document specifications, are generated for an IUT by processing the requirements specified in the ICS and the DAP against the generic test cases. These test document specifications are input to the Generation Test and Reception Test components. The generation test component creates verification test reports for originator class implementations. For an originator, the test document specifications are used as activity scripts for generating IUT results in the form of ODA data streams. A reference document analyser processes these output data streams from the IUT and generates the test reports. The parameters of the test results are based on the criteria specified by the test document specifications, the ICS and the DAP. The Reception Test component creates verification test reports for recipient class implementations. For a recipient, only those systems that can image a document can currently be tested. Gateways that take in an ODA data stream and convert it into an output data stream in some non-ODA format are outside the scope of this test environment. Recipient class implementations that can only generate 'soft copy' images on terminals are also outside the scope of this test environment. In the Reception Test component the'test document
computer communications
Test DocumentSpecificationSub-System
I RepresentativeI~1 i structure I ~ . ~
Attribute ValuelList
~ Test Document Specification Generator
Test identification
I I
Implementation J Conformance Statement
= Test ~.~ Document = Specification
I
E
Generator
II
I ODIF Generated
By Test System
I II Reference Imaging System
I
II '°T
Receiver
II
,UT [
Generator
ODIF
I Imaged Document
Generated
By IUT
I
t I Reference
Document Analyser
I
I m age
Comparison& Analysis
I m ages
I
ReceptionTestingSub-System Figure 3.
) /
II
! !
Test Document Specification
!:
GenerationTestingSub-System
TODAC test architecture
specifications are input to a reference ODA data stream generator that creates ODA data streams intended to test specific features of the IUT specified by the ICS and DAP. The test ODA data streams are imaged by a reference ODA imaging system to produce reference images. The recipient portion of the IUT receives the test ODA data streams. The IUT must be able to image the test documents so that an image comparison and analysis can be completed between the reference and test images. When analysing the result of a reception test, an analysis would examine the test image, with reference to the test cases specified in the Test Document Specification, to understand the intended result and therefore conclude whether the test has been executed successfully or not. The reference image is used as a guide to assist in determining the success of the test, but is of secondary importance to the Test Document Specification. The result of this analYSiS is in the form of a test report. For originator/recipient class implementations, the originator and recipient portions of the IUT are tested separately in the same way as originator and recipient class implementations (see above). The TODAC test architecture goes beyond merely testing the conformance of an IUT to ODA. It is expected vol 12 no 2 april 1989
that these test tools will be able to provide information to implementors and clients on the potential for inte~vorking with other ODA implementations. LESSONS
LEARNED
Interchange format standards such as ODA provide a challenge to conformance testing. The difficulties in meetingthis challenge could be lessened by changingthe framework of the standards. In particular, the scope of the interchange format standard needs to be increased. Figure 4 shows the layered components involved in the interchange of a DIF between two systems. An application in system A produces the data stream to be exchanged with system B by invoking the data steam generator process through some Application Program Interface (API). System B receives and interprets the data stream exchanged from system A by invoking the data stream interpreter process through another API. At present, DIF standards such as ODA go no further in this model than specifying the semantics and syntax of the data stream. Increasing the scope of these standards to include both the abstract and real specification of the 105
Application ProcessB
Application ProcessA
API
API Generator Process
~0~
I nterpreter [ Process
Document Interchange Format
Figure 4. scope
Document interchange standards proposed
REFERENCES* 1 2 3 4 5 6
features of the processes used to generate and interpret the data stream, as well as including the programming language bindings to these specifications, would more readily permit the development of uniform and useful conformance test architectures. It should be noted that an alternative to increasing the scope of the standards would be to augment the existing scope with supplemental standards covering the necessary 'services' for generating and interpreting the DIF data streams (CCITT recommendations on 'service definitions' is an example of the latter approach). Unambiguous specification of the standards could be achieved through the use of formal descriptive techniques such as FODA to complement the descriptive prose specification. The formal descriptive technique would also greatly assist in the development of automated test tools.
SUMMARY The important issues involved in the conformance testing of standard document interchange formats such as ODA have been addressed here. Testing the ODA conformance of interchanged data streams can assess conformance to the ODA standard, but it may not provide a strong indication of whether an implementation will interwork with other ODA implementations without some value added testing, such as image comparison. The implementation agreements for ODA have augmented the conformance semantics of the ODA standard. Such specifications complement the conformance statements in the ODA standard with requirements for specific categories of ODA implementation. Also, a verification method being developed in the TODAC Project was described. The testing tools being developed within TODAC address more than the mere testing of ODA conformance -- it is believed that such tests greatly assist in providing information on the likelihood that an implementation will be able to interwork with dissimilar ODA implementations. The development of ODA tests would be be facilitated if the scope addressed by interchange format standards were extended to support a conceptual framework that addressed the need for conformance testing.
106
7
Information Processing -- Text and Office Systems -Office Document Architecture (ODA) and Interchange Format (ISO 8613) (November 1987) Initial Graphics Exchange Specification (IGES) ANSI Y14.26M, Version 3, ANSI, USA (1987) Information ProcessingSystems -- ComputerGraphics Metafile for the Storage and Transfer of Picture Description Information (ISO 8632) (February 1987) Stable Implementation Agreements of the NIST Workshop for Implementors of OSI National Institute for Standards and Technology, USA (May 1988) Formal Specification of ODA Document Structures Proposal for Draft Addendum to ISO 8613/2 (ISO/IEC JTC1/SC18/WG3 N996) (March 1988) Conformance Testing of ODA Document Structures Based on FODA (ISO/TC 97/SC 18/WG3 N881) Dr W Appelt, GMD, Bonn, FRG (August 1987) TODAC Methodology Requirements Documentation National Computing Centre, UK/ Department of Communications, Canada (March 1987)
*International Standards (IS) are available from the International Organisationfor StandardizationCentralSecretariat,1 rue de VarembE, CasePostale56, CH-1211Geneva20, Switzerland. Richard Carr MSc MBCS CEng has been involved with voice and data communications since 7971. He has worked in Australia on the introduction of PABX systems, and has lectured in mathematics and computing. Since 1984 he has worked for the UK National Computing Centre, where he currently leads the TODAC project He is co-author of the Formal Specification of ODA, editor of ISO notation for DAPs, and convenor of the ISO SWG on ODA conformance. He has designed two profiles for the UK GOSIPproject and is chairman of the D TI's ODA Implementor Group.
Frank R Dawson received his MSc in electrical engineering from the University of Missouri. He has over 10 years of experience in architecture, standards and the implementation of computer graphics and office systems. As a member of the Office Systems Architecture and Standards department of IBM's Application Systems Division, his responsibilities include the chairmanship of the NIST ODA SIG. He is head of IBM's delegation to PAGODA, and IBM representative to the COS Document Architecture Subcommittee.
computer communications