- Email: [email protected]
Standards and protocols
Data standards in international trade and transport The UK Simplification of International Trade Procedures Board was set up in 1970 to 'guide, stimulate and assist the rationalization of international trade procedures and the documentation and information flows associated with t h e m . . . , particularly in view of the widening use of computers and telecommunications links'. Its membership is drawn from a wide range of interests in the UK and includes shippers, carriers, forwarders, ban kers, insurers and government representatives. Its working groups call on the specialist expertise of many commercial and official interests. Funds are provided by the UK Department of Trade through an annual grant, but the board is independent within its agreed terms of reference. To help those trade and transport organizations who, because of lack of standardization, are experiencing difficulty in the collection-and production of information needed to service the movement of goods, SlTPRO have produced 'Computers in international trade and transport - data standards'*.
SCOPE OF MANUAL A major section of the manual contains a data element glossary (or dictionary) of the most important data elements required for computer processing and production of international trade and transport documentation. The glossary lists standard formats for each data element and also recommends or lists relevant coding systems. Where possible, internationally agreed codes are referred to, and in other cases UK national codes. Each data element contained in the glossary is crossreferenced to a list of major trade documents. The data-element glossary covers the major commercial, governmental and transport requirements for importers, exporters, forwarding agents, carriers (road, rail, sea and air), port authorities * See also 'Future trends in computer and communications systems' in the Interface section of this issue of Computer Communications (Vol 1 No 5 1978).
272
problems of preparing, despatching and/or processing, quickly and accurately, the information required nationally and internationally for the efficient servicing of a trade transation.
and for HM Customs and Excise, and can be used for inhouse applications or The use of computers and telecommunifor local or international data in terchange. cations is largely confined at present to The manual also deals with standards 'in-house' systems, e.g. for the producfor the interchange of data over teletion of conventional documents or for communications links between computhe transmission of data in computerters, intelligent terminals and telex (in readable form between two offices of any combination) or by means of a the same company. Airlines, shipping physical medium such as magnetic tape. companies, importers, exporters, port The standards have been designed to be authorities, freight forwarders, HM independent of the type of interchange Customs and Excise, banking and insurmedium being used, the type of comance companies provide examples of the puter and the system application. The increasing use of computers for inhouse standards therefore act as a 'bridge' applications. between inhouse systems which are In future, more organizations will currently incompatible. A set of 'syntax need to interchange data with other rules' and guidelines are listed for messeparate organizations in computersage designers. The guidelines demonreadable form, either by using telecomstrate how logically related data elemunications links or a physical medium ments listed in the glossary can be comsuch as magnetic tape. Some data-interbined into basic segments (or data change systems are already functioning, groups) and how these segments can be for example a scheme operated by HM combined to form interchange messages. Customs and Excise. Full details of The manual is designed to help users import consigments are passed periodito avoid some of the technical difficulcally to the Customs by magnetic tape. ties of interchanging data between difOnly a simplified entry document is ferent makes of computer. A list is prorequired at the time of importation. vided of the technical standards relevant HM Customsand Excise are also carrying to data interchange produced by the out trials for the interchange of exportInternational Organization for Stanentry data. dardisation (ISO), the International Although the use of computers and Telegraph and Telephone Consultative data-interchange systems is increasing, Committee (CCITT), the European many organizations will continue to Computer Manufacturers Association use more conventional methods of pre(ECMA) and the British Standards paring and transmitting information. Institute (BSI). The production of documents by manual The following material is based on means or with the assistance of equipextracts from those parts of the manual ment such as automatic typewriters will that cover computer communications. also continue. Similarly, the transmission of data by telex will play a prominent part in the activities of many organizations for some time.
BACKGROUND There is an increasing use of computers by organizations in international trade and transport, because • equipment is becoming relatively cheaper, thus bringing computer applications within the reach of more organizations, • computers and telecommunications help solve the daily but important
SCOPE The manual contains standards for the data most commonly used in international trade and transport procedures and documentation. The standards also cover the exchange of data on conventional documents, in computer-readable form or via telex. The main principles
computer communications
and much of the detail of the data standards have been agreed within joint committees of the United Nations Economic Commission for Europe (ECE) and the ISO. The ECE are considering the preparation of a similar manual at the international level.
Data elements The manual contains standards for the length and other characteristics of the most commonly used items of information or, as they are referred to in the manual, 'data elements'. It recommends some appropriate coding systems. These data-element standards can be used in 'inhouse systems' e.g. for the production of conventional documentation, including aligned documents, by means of computer, by word-processing equipment, by automatic typewriter or by manual methods. They can also be used in 'data-interchange systems'.
Data interchange Other standards applicable to datainterchange systems are provided. A set of 'syntax rules' and guidelines for message designers are also provided. These show how to combine data elements into interchange messages. The rules and guidelines provide an alternative to conventional computer records which is independent of the type of interchange medium or the type of computer. The manual also helps users to avoid some of the technical difficulties of interchanging data between different makes of computer and includes a survey of the technical standards produced by the ISO, the CCITT, the ECMA and the BSl.
BENEFITS The data-element standards are, wherever possiblel independent of individual trade sectors or modes of transport, i.e. they are applicable to rail, road, air and sea.
vol ] no 5 october 1978
The standards are designed to provide compatibility between systems whether these are based on documents or data interchange. A trade or transport document produced by one organization will invariably be processed by another. The use of common standards will prevent queries, delays or errors caused by having to 'translate' information from one internal standard into another. The use of common data elements in inhouse systems will reduce costs when introducing data-interchange systems later, because there will already be compatibility between organizations wishing to interchange data. The decision to introduce data-interchange systems will depend on commercial justification rather than on the availability of standards. Once such decisions are taken, however, standards are essential to act as a bridge between the computer systems of the various participants. The syntax rules contained in the manual provide such a bridge. They are independent of the type of system, e.g. batch or online, they are independent of the medium used, e.g. magnetic tape, telex or telecommunications link; and they are also independent of the methods by which data can be formatted on different types of computer. These are important considerations for organizations that may join several interchange systems and for interchange system designers who do not know what equipment will be used by all the eventual participants.
ACCEPTABI L I T Y OF STANDARDS The standards and guidelines in the manual have been prepared in consultation with representatives from national and international organizations and companies. The research and development phase of the proiect lasted for 18 months. During that time, representatives of 88 organization worked with SITPRO staff in developing the contents of the manual. Over 1O0 other organizations were consu Ited once draft conclusions had been reached. During the development phase, many organizations expressed the view that HM Customs and Excise policy towards
the standards would play a significant part in determining how far they would be implemented by other participants. HM Customs and Excise were among the first to endorse the proposals in principle. The main international forum for the discussion of data standards affecting international trade and transport is the ECE, which includes East and West European countries, representatives from other countries such as Australia, Canada, Japan and the USA, and representatives from the ISO and other international organizations. The United Nations Conference on Trade and Development UNCTAD) Facilitation Programme (Falpro) were closely involved in the work. Many of the major dataelement standards included in the manual have already been endorsed by the ECE. Other international and national organizations consulted included the International Chamber of Shipping (ICS), the International Air Transport Association (I ATA), the General Council of British Shipping (GCBS), and the BSI. The standards will also be discussed with the Commission of the European Economic Community (EEC).
DATA-ELEMENT GLOSSARY General The data element glossary includes standards for • commercial data, which could appear on an export invoice etc., • transport data, associated with the movement of goods by air, road, rail and sea, including general cargo, rollon/roll-off and containerized shipments. These are data elements which could appear on a bill of lading, a waybill or a consignment note etc., • HM Customs and Excise data, which could appear on export entries or import entries etc. It is intended for use in the interchange of data between organizations • via computer-computer links (either by telecommunications or by a physical medium such as magnetic tape), • via telex, • via documents printed by a computer.
273
Formats A maximum length is set for each data element in the glossary,e.g. 17 characters for names of parts and 5 lines each of 35 characters for names and addresses. For each data element, the entry characteristics are numeric, alphabetic or alphanumeric.
Codes Some data elements may be represented in coded form, e.g. country names, as well as in clear language. Wherever possible, a single coding system has been selected, e.g. the ISO 2 alpha code for countries (ISO 3166). Where it is not practical to recommend a single code for the time being because of established detailed codes - for example those already developed for different modes of transport - details of these codes are shown. Code lists are included in the glossary wherever practicable, but, where a code changes very frequently or where the list is very long, only examples from the code are shown, although the source of the code is stated.
• the ISO country and currency codes and the container size and type code, the IATA standards for data elements specific to air transport, the standards for data elements specific to maritime transport as agreed by the ICS and the ECE. Future ECE standards will cover other modes of transport and commercial requirements in agreement with the relevant organizations, and, as they are agreed they will be produced as amendments to the manual as part of a general maintenance programme.
S Y N T A X RULES The syntax rules are used to assemble data elements within messages interchanged between organizations in such a way that the messages are independent of constraints imposed by individual types of equipment, media or systems. Methods of using the rules are described and the advantages of the rules over conventional computer records are discussed.
Scope and purpose
Character set For these standards, a restricted character set is recommended. The set is limited to those characters which can be printed on almost all computer printers and which can be communicated over telex links. The recommended character set is as follows: • •
A - Z the space character, 0-9.,()/-=:'?
The use of characters = + ? ' within data elements should be avoided because these have a special use in the syntax rules.
International compatibility Existing international data standards have been widely adopted as part of these standards, for example
274
As the rules are independent of machine, media and systems, they are suitable for the interchange of data between computer systems, by a telecommunications link or by a physical medium. They can also be used for the interchange of data via messageswitching systems or telex where data has to be prepared or interpreted manually, without computer processing. The rules are intended for interchange between organizations. Methods for the initial data capture and the final output of data are outside the scope of the syntax rules, except where interchange does not involve computer processing, i.e. in message-switching systems.
ioust o ver,oI
record layout
syntax layout
Transfer Medium
The syntax rules provide the means of structuring the required information for interchange. They can therefore be used to structure interchange messages specified at international, national and application level whatever standards for data elements and messages are used. The rules only apply to the data interchanged. They do not apply to the communication protocols and media labels within which the data is transmitted. Privacy of data is outside the scope of the rules. Where required, encryption or other techniques can be agreed between interchange partners. Facilities are provided, however, for the transmission of control totals to facilitate the safe transmission of data. The syntax rules are designed to be independent of the media used for interchange and of the structure of data within users' internal computer systems. They are intended as a bridge between systems, for use at the point of interchange. The sender would convert from his internal record layouts and the recipient would convert into his internal layouts (see Figure 1). The syntax rules do not dictate the extent to which data should be validated prior to transmission. The onus is on the sender of a message to provide accurate information which has been correctly assembled. The recipient of a message should carry out validation checks however, to help ensure that invalid data does not corrupt internal computer records.
TECHNICAL CONSIDERATIONS It is not part of SITPRO's task to develop technical standards such as those for the labelling of magnetic tapes, but a guide is provided to existing technical standards and to those under development. Included in the manual are recommendations on:
,[ Convert from ~] syntax layout
] In house ~ record layout Recipient
Sender Figure I. Use o f syntax rules as a bridge between systems
computer communications
•
which magnetic tape standards should be adopted, to ensure that tapes can be accepted by the largest possible number of computers, • comments on physical media such as flexible (or floppy) discs and magnetic-tape cassettes as interchange media, • suggested methods of circumventing some of the problems of directly connecting computers by telecommunications links.
List of standards An appendix contains a list of technical standards from the ISO, CCITT, BSI and ECMA relevant to interchange. The preface to the appendix indicates how the standards can be obtained, and there is also a summary of the work of each body and their interrelationships.
Recommendations to users Press manufacturers to conform to standards One general point on the observance of standards by computer manufacturers needs to be made; when standards are developed, they may be ignored by manufacturers or only partially observed. Users are strongly recommended by SITPRO to exert pressure on manufacturers to observe international standards and also to complain to manufacturers, user associations etc. if they deviate from a standard which they claim to observe. It is also important to find out exactly which standard a manufacturer conforms to. It is not good enough for a manufacturer to state that his cassettes conform to ECMA standards because there are two standards, one for physical characteristics and one for labelling - the product may only conform to one. References to individual standards are important. Recommended physical interchange media
½ inch magnetic tape The recommended physical interchange medium for use between computers of different types
vol 1 no 5 october 1978
is ½ in magnetic tape. This is available on all mainframe computers as well as many small business computers and some terminals. Manufacturers, on the whole, conform to the international standards for 1/2 in magnetic tape. Magnetic cassette Magnetic tape cassettes are available on many small business computers and terminals that do not support 1/2 in magnetic tape. To ensure compatibility with as many other computers as possible, magnetic-tape cassettes should conform to standards BS 5079 Part 1 for physical and magnetic properties, and ECMA 41 for labelling and file structure. However, cassettes are poor alternatives to ½ in magnetic tape because there is insufficient observance of the standards by the computer manufacturers, there is a difference between the ECMA and ANSI standards, the recommendations on character set in the standards are not rigorous enough, and, with one or two exceptions, magnetic-tape cassettes are not supported directly on mainframe computers. Flexible for floppy) discs Flexible discs are in some respects an ideal interchange medium, being cheap, easily transportable and becoming increasingly available on many small computers and terminals as well as on some mainframe computers. Currently, there is only a draft standard for physical characteristics and standards for labelling and structuring have reached only an early stage. Once standards are established, however, it is believed that flexible discs will certainly become a second choice to ½ in magnetic tape and will be much easier to interchange than magnetic-tape cassettes. The only relevant standard is the draft ISO/DP 5654, 'Data interchange on flexible disc cartridges'. There is also an ECMA 'White cover document', which is not a standard, covering flexible disc labelling. Magnetic tape cartridges These are used on only a small number of minicomputers, small business systems and terminal systems. None of the mainframe manufacturers use magnetic cartridges for any major item of equipment. As there has apparently been little collaboration between suppliers of cartridges and manufacturers of cartridge readers, it is likely that the level of comparability between different types of computer will be fairly low. The relevant stan-
dards are ECMA 46 and ISO/DIS 4057 (Draft) for physical and magnetic properties, and ISO/DIS 4341 (Draft) for labelling and file structure. Punched cards and paper tape These are becoming less common, especially on small business systems where input is normally through a VDU. For those computers that do have the necessary peripherals, however, punched cards and paper tape do present a way of avoiding many of the problems associated with interchanging other media.
Telecommunications links Apart from telex links, the interchange of data over telecommunications links causes more problems than physical media. Manufacturers have developed and are still developingtheir own telecommunications standards. The new international HDLC standards, however, are at least accepted as a basis upon which to work by most manufacturers. The development over the next few years of packet-switched public data networks should also force some standards on computer manufacturers, because the X.25 standard, which embraces elements of HDLC, is likely to be adopted by almost all PTTs. Even with X.25, many 'high-level' options, such as terminal-control commands, are left open to manufacturers, so that interchange between different makes of computer will not be assured unless the gaps in standards are quickly rectified by the standards bodies. Organizations such as airlines and banks have their own standards, e.g. the SITA and the SWIFT networks. Computerscan be connected directly into the telex network, but this is relatively rare. Paper tapes can be transferred between a computer and a telex terminal and transmitted over the telex network. Standards for the telex network are well established. However, the transmission speed of the network makes it unsuitable for the interchange of large volumes of data, while the possibility of data corruption makes it necessary for the interchange participants to include many fundamental error controls in their data which are built into magnetic media or more sophisticated telecommunication links. (This is one reason why the syntax rules allow users to include
275
hash totals when interchanging data over telex.) If computers or terminals are connected by other than the telex network, i.e. by private circuits (or leased lines) or the public switched network (PSN), data can be transmitted faster than via telex links and in higher volumes. In addition, the chances of data corruption with the majority of the techniques used is negligible because of inbuilt error controls which make it necessary for the user to include only simple audit checks on the data, The standards for the basic hardware connection into the telecommunications circuits are universally established. Apart from this, there is very little standardization between different computer manufacturers with one exception, the support of teletypewriter and 'teletypewriter compatible' terminals is almost universal. Unfortunately, these have a limited speed and unsophisticated error-checking procedu res. On the faster, more sophisticated links, the major computer manufacturers have developed their own protocols which are only loosely based on the 'basic-mode control procedures' defined by the BSI, ECMA and ISO. This makes communication between different makes of computer impossible unless programs are available that will emulate the other manufacturer's protocol. It should not be taken for granted that it is easy to connect together computers from the same manufacturer; this can be a particular problem when connecting two large mainframes together. The current telecommunications scene is not very conducive to interchange. Efforts are being made, however, to establish new standards with which all manufacturers can comply. At the basic level, the HDLC protocol has been adopted by the BSl, ECMA and ISO, and is intended to replace the 'basicmode-control procedures' that are so loosely adhered to by the major computer manufacturers. Most major manu-
276
facturers and many of the small manufacturers already support or intend to support HDLC. Unfortunately, international agreement has not been reached on some of the high-level procedures of HDLC, and already some major manufacturers have established their own while conforming to the internationally agreed low-level parts of the HDLC standard. One of the most significant developments will be the introduction of public networks by the PTTs nationally and internationally. These networks will make it easier to exchange low-tomedium volumes of computer data between terminals and/or computers. The PTl's are almost certain to select the X.25 standard, for packet switching, with which all manufacturers will need to conform to connect their terminals and computers to the network. X.25 includes elements of HDLC. Some computer manufacturers have their own network software, such as SNA and Decnet, but these manufacturers will almost certainly support X.25 interfaces to public data networks. At the highest level of technical protocols, sometimes called the 'user' level, there is no immediate prospect for standardization, although work is being undertaken, for example, by the BSI. The prospects for the user wishing to connect to other makes of computer can be summed up as follows: •
using the X.25 interface should become practicable. Throughout this period the telex network will continue to be supported and, within its limitations, presents no standardization difficulties.
Recommendations The manual recommends that preference should be given to terminals and computers that can emulate the communication protocols of different computer manufacturers. The terminals and computers should be capable of operating to the HDLC standards and suppliers should be committed to supporting the X.25 interface. Teletypewriter-compatible terminals offer a high degree of compatibility with different mainframe computers. If the number of participants in an interchange are limited but there are a variety of makes of cam puter involved, a useful technique is the offline transmission of 1/2 in magnetic tape using the same make of terminal. The problems of emulation are then overcome, as only the magnetic tapes have to conform to a common standard. There are several remote batch terminals that will carry out offline transmission of 1/2 in magnetic tape. Users are strongly advised to carry out trial data exchanges well in advance of implementation, even where manufacturers claim conformity to international standards.
in the short term, say one to three years, the user must be prepared to emulate the communication protocols of the major manufacturers unless the teletypewriter-compatible terminals are adequate for his needs. ('Data standards for trade documenta• in the medium term, say three to five tion and data interchange' (two years, the use of HDLC should imvolumes) is published by 51TPRO, and prove the prospect of intermanufacis available (price £30) from technical turer compatibility, providing there bookshops, UK National Computing is international agreement for some Centre regional centres and J M Dent, of the higher-level procedures. Durhams Lane, Letchworth, Hertz SG6 I LF, UK. For further details, • in the longer term, say five to ten years, when public data networks contact the EDP Director, 51TPRO, are established in the UK and other 11/12 Waterloo Place, London countries, the more general intercon- 5WIY 4AU, UK. Tel." 01-8393393, nection of computer and terminals Tx: 919130)
computer communications
SCOPE OF MANUAL A major section of the manual contains a data element glossary (or dictionary) of the most important data elements required for computer processing and production of international trade and transport documentation. The glossary lists standard formats for each data element and also recommends or lists relevant coding systems. Where possible, internationally agreed codes are referred to, and in other cases UK national codes. Each data element contained in the glossary is crossreferenced to a list of major trade documents. The data-element glossary covers the major commercial, governmental and transport requirements for importers, exporters, forwarding agents, carriers (road, rail, sea and air), port authorities * See also 'Future trends in computer and communications systems' in the Interface section of this issue of Computer Communications (Vol 1 No 5 1978).
272
problems of preparing, despatching and/or processing, quickly and accurately, the information required nationally and internationally for the efficient servicing of a trade transation.
and for HM Customs and Excise, and can be used for inhouse applications or The use of computers and telecommunifor local or international data in terchange. cations is largely confined at present to The manual also deals with standards 'in-house' systems, e.g. for the producfor the interchange of data over teletion of conventional documents or for communications links between computhe transmission of data in computerters, intelligent terminals and telex (in readable form between two offices of any combination) or by means of a the same company. Airlines, shipping physical medium such as magnetic tape. companies, importers, exporters, port The standards have been designed to be authorities, freight forwarders, HM independent of the type of interchange Customs and Excise, banking and insurmedium being used, the type of comance companies provide examples of the puter and the system application. The increasing use of computers for inhouse standards therefore act as a 'bridge' applications. between inhouse systems which are In future, more organizations will currently incompatible. A set of 'syntax need to interchange data with other rules' and guidelines are listed for messeparate organizations in computersage designers. The guidelines demonreadable form, either by using telecomstrate how logically related data elemunications links or a physical medium ments listed in the glossary can be comsuch as magnetic tape. Some data-interbined into basic segments (or data change systems are already functioning, groups) and how these segments can be for example a scheme operated by HM combined to form interchange messages. Customs and Excise. Full details of The manual is designed to help users import consigments are passed periodito avoid some of the technical difficulcally to the Customs by magnetic tape. ties of interchanging data between difOnly a simplified entry document is ferent makes of computer. A list is prorequired at the time of importation. vided of the technical standards relevant HM Customsand Excise are also carrying to data interchange produced by the out trials for the interchange of exportInternational Organization for Stanentry data. dardisation (ISO), the International Although the use of computers and Telegraph and Telephone Consultative data-interchange systems is increasing, Committee (CCITT), the European many organizations will continue to Computer Manufacturers Association use more conventional methods of pre(ECMA) and the British Standards paring and transmitting information. Institute (BSI). The production of documents by manual The following material is based on means or with the assistance of equipextracts from those parts of the manual ment such as automatic typewriters will that cover computer communications. also continue. Similarly, the transmission of data by telex will play a prominent part in the activities of many organizations for some time.
BACKGROUND There is an increasing use of computers by organizations in international trade and transport, because • equipment is becoming relatively cheaper, thus bringing computer applications within the reach of more organizations, • computers and telecommunications help solve the daily but important
SCOPE The manual contains standards for the data most commonly used in international trade and transport procedures and documentation. The standards also cover the exchange of data on conventional documents, in computer-readable form or via telex. The main principles
computer communications
and much of the detail of the data standards have been agreed within joint committees of the United Nations Economic Commission for Europe (ECE) and the ISO. The ECE are considering the preparation of a similar manual at the international level.
Data elements The manual contains standards for the length and other characteristics of the most commonly used items of information or, as they are referred to in the manual, 'data elements'. It recommends some appropriate coding systems. These data-element standards can be used in 'inhouse systems' e.g. for the production of conventional documentation, including aligned documents, by means of computer, by word-processing equipment, by automatic typewriter or by manual methods. They can also be used in 'data-interchange systems'.
Data interchange Other standards applicable to datainterchange systems are provided. A set of 'syntax rules' and guidelines for message designers are also provided. These show how to combine data elements into interchange messages. The rules and guidelines provide an alternative to conventional computer records which is independent of the type of interchange medium or the type of computer. The manual also helps users to avoid some of the technical difficulties of interchanging data between different makes of computer and includes a survey of the technical standards produced by the ISO, the CCITT, the ECMA and the BSl.
BENEFITS The data-element standards are, wherever possiblel independent of individual trade sectors or modes of transport, i.e. they are applicable to rail, road, air and sea.
vol ] no 5 october 1978
The standards are designed to provide compatibility between systems whether these are based on documents or data interchange. A trade or transport document produced by one organization will invariably be processed by another. The use of common standards will prevent queries, delays or errors caused by having to 'translate' information from one internal standard into another. The use of common data elements in inhouse systems will reduce costs when introducing data-interchange systems later, because there will already be compatibility between organizations wishing to interchange data. The decision to introduce data-interchange systems will depend on commercial justification rather than on the availability of standards. Once such decisions are taken, however, standards are essential to act as a bridge between the computer systems of the various participants. The syntax rules contained in the manual provide such a bridge. They are independent of the type of system, e.g. batch or online, they are independent of the medium used, e.g. magnetic tape, telex or telecommunications link; and they are also independent of the methods by which data can be formatted on different types of computer. These are important considerations for organizations that may join several interchange systems and for interchange system designers who do not know what equipment will be used by all the eventual participants.
ACCEPTABI L I T Y OF STANDARDS The standards and guidelines in the manual have been prepared in consultation with representatives from national and international organizations and companies. The research and development phase of the proiect lasted for 18 months. During that time, representatives of 88 organization worked with SITPRO staff in developing the contents of the manual. Over 1O0 other organizations were consu Ited once draft conclusions had been reached. During the development phase, many organizations expressed the view that HM Customs and Excise policy towards
the standards would play a significant part in determining how far they would be implemented by other participants. HM Customs and Excise were among the first to endorse the proposals in principle. The main international forum for the discussion of data standards affecting international trade and transport is the ECE, which includes East and West European countries, representatives from other countries such as Australia, Canada, Japan and the USA, and representatives from the ISO and other international organizations. The United Nations Conference on Trade and Development UNCTAD) Facilitation Programme (Falpro) were closely involved in the work. Many of the major dataelement standards included in the manual have already been endorsed by the ECE. Other international and national organizations consulted included the International Chamber of Shipping (ICS), the International Air Transport Association (I ATA), the General Council of British Shipping (GCBS), and the BSI. The standards will also be discussed with the Commission of the European Economic Community (EEC).
DATA-ELEMENT GLOSSARY General The data element glossary includes standards for • commercial data, which could appear on an export invoice etc., • transport data, associated with the movement of goods by air, road, rail and sea, including general cargo, rollon/roll-off and containerized shipments. These are data elements which could appear on a bill of lading, a waybill or a consignment note etc., • HM Customs and Excise data, which could appear on export entries or import entries etc. It is intended for use in the interchange of data between organizations • via computer-computer links (either by telecommunications or by a physical medium such as magnetic tape), • via telex, • via documents printed by a computer.
273
Formats A maximum length is set for each data element in the glossary,e.g. 17 characters for names of parts and 5 lines each of 35 characters for names and addresses. For each data element, the entry characteristics are numeric, alphabetic or alphanumeric.
Codes Some data elements may be represented in coded form, e.g. country names, as well as in clear language. Wherever possible, a single coding system has been selected, e.g. the ISO 2 alpha code for countries (ISO 3166). Where it is not practical to recommend a single code for the time being because of established detailed codes - for example those already developed for different modes of transport - details of these codes are shown. Code lists are included in the glossary wherever practicable, but, where a code changes very frequently or where the list is very long, only examples from the code are shown, although the source of the code is stated.
• the ISO country and currency codes and the container size and type code, the IATA standards for data elements specific to air transport, the standards for data elements specific to maritime transport as agreed by the ICS and the ECE. Future ECE standards will cover other modes of transport and commercial requirements in agreement with the relevant organizations, and, as they are agreed they will be produced as amendments to the manual as part of a general maintenance programme.
S Y N T A X RULES The syntax rules are used to assemble data elements within messages interchanged between organizations in such a way that the messages are independent of constraints imposed by individual types of equipment, media or systems. Methods of using the rules are described and the advantages of the rules over conventional computer records are discussed.
Scope and purpose
Character set For these standards, a restricted character set is recommended. The set is limited to those characters which can be printed on almost all computer printers and which can be communicated over telex links. The recommended character set is as follows: • •
A - Z the space character, 0-9.,()/-=:'?
The use of characters = + ? ' within data elements should be avoided because these have a special use in the syntax rules.
International compatibility Existing international data standards have been widely adopted as part of these standards, for example
274
As the rules are independent of machine, media and systems, they are suitable for the interchange of data between computer systems, by a telecommunications link or by a physical medium. They can also be used for the interchange of data via messageswitching systems or telex where data has to be prepared or interpreted manually, without computer processing. The rules are intended for interchange between organizations. Methods for the initial data capture and the final output of data are outside the scope of the syntax rules, except where interchange does not involve computer processing, i.e. in message-switching systems.
ioust o ver,oI
record layout
syntax layout
Transfer Medium
The syntax rules provide the means of structuring the required information for interchange. They can therefore be used to structure interchange messages specified at international, national and application level whatever standards for data elements and messages are used. The rules only apply to the data interchanged. They do not apply to the communication protocols and media labels within which the data is transmitted. Privacy of data is outside the scope of the rules. Where required, encryption or other techniques can be agreed between interchange partners. Facilities are provided, however, for the transmission of control totals to facilitate the safe transmission of data. The syntax rules are designed to be independent of the media used for interchange and of the structure of data within users' internal computer systems. They are intended as a bridge between systems, for use at the point of interchange. The sender would convert from his internal record layouts and the recipient would convert into his internal layouts (see Figure 1). The syntax rules do not dictate the extent to which data should be validated prior to transmission. The onus is on the sender of a message to provide accurate information which has been correctly assembled. The recipient of a message should carry out validation checks however, to help ensure that invalid data does not corrupt internal computer records.
TECHNICAL CONSIDERATIONS It is not part of SITPRO's task to develop technical standards such as those for the labelling of magnetic tapes, but a guide is provided to existing technical standards and to those under development. Included in the manual are recommendations on:
,[ Convert from ~] syntax layout
] In house ~ record layout Recipient
Sender Figure I. Use o f syntax rules as a bridge between systems
computer communications
•
which magnetic tape standards should be adopted, to ensure that tapes can be accepted by the largest possible number of computers, • comments on physical media such as flexible (or floppy) discs and magnetic-tape cassettes as interchange media, • suggested methods of circumventing some of the problems of directly connecting computers by telecommunications links.
List of standards An appendix contains a list of technical standards from the ISO, CCITT, BSI and ECMA relevant to interchange. The preface to the appendix indicates how the standards can be obtained, and there is also a summary of the work of each body and their interrelationships.
Recommendations to users Press manufacturers to conform to standards One general point on the observance of standards by computer manufacturers needs to be made; when standards are developed, they may be ignored by manufacturers or only partially observed. Users are strongly recommended by SITPRO to exert pressure on manufacturers to observe international standards and also to complain to manufacturers, user associations etc. if they deviate from a standard which they claim to observe. It is also important to find out exactly which standard a manufacturer conforms to. It is not good enough for a manufacturer to state that his cassettes conform to ECMA standards because there are two standards, one for physical characteristics and one for labelling - the product may only conform to one. References to individual standards are important. Recommended physical interchange media
½ inch magnetic tape The recommended physical interchange medium for use between computers of different types
vol 1 no 5 october 1978
is ½ in magnetic tape. This is available on all mainframe computers as well as many small business computers and some terminals. Manufacturers, on the whole, conform to the international standards for 1/2 in magnetic tape. Magnetic cassette Magnetic tape cassettes are available on many small business computers and terminals that do not support 1/2 in magnetic tape. To ensure compatibility with as many other computers as possible, magnetic-tape cassettes should conform to standards BS 5079 Part 1 for physical and magnetic properties, and ECMA 41 for labelling and file structure. However, cassettes are poor alternatives to ½ in magnetic tape because there is insufficient observance of the standards by the computer manufacturers, there is a difference between the ECMA and ANSI standards, the recommendations on character set in the standards are not rigorous enough, and, with one or two exceptions, magnetic-tape cassettes are not supported directly on mainframe computers. Flexible for floppy) discs Flexible discs are in some respects an ideal interchange medium, being cheap, easily transportable and becoming increasingly available on many small computers and terminals as well as on some mainframe computers. Currently, there is only a draft standard for physical characteristics and standards for labelling and structuring have reached only an early stage. Once standards are established, however, it is believed that flexible discs will certainly become a second choice to ½ in magnetic tape and will be much easier to interchange than magnetic-tape cassettes. The only relevant standard is the draft ISO/DP 5654, 'Data interchange on flexible disc cartridges'. There is also an ECMA 'White cover document', which is not a standard, covering flexible disc labelling. Magnetic tape cartridges These are used on only a small number of minicomputers, small business systems and terminal systems. None of the mainframe manufacturers use magnetic cartridges for any major item of equipment. As there has apparently been little collaboration between suppliers of cartridges and manufacturers of cartridge readers, it is likely that the level of comparability between different types of computer will be fairly low. The relevant stan-
dards are ECMA 46 and ISO/DIS 4057 (Draft) for physical and magnetic properties, and ISO/DIS 4341 (Draft) for labelling and file structure. Punched cards and paper tape These are becoming less common, especially on small business systems where input is normally through a VDU. For those computers that do have the necessary peripherals, however, punched cards and paper tape do present a way of avoiding many of the problems associated with interchanging other media.
Telecommunications links Apart from telex links, the interchange of data over telecommunications links causes more problems than physical media. Manufacturers have developed and are still developingtheir own telecommunications standards. The new international HDLC standards, however, are at least accepted as a basis upon which to work by most manufacturers. The development over the next few years of packet-switched public data networks should also force some standards on computer manufacturers, because the X.25 standard, which embraces elements of HDLC, is likely to be adopted by almost all PTTs. Even with X.25, many 'high-level' options, such as terminal-control commands, are left open to manufacturers, so that interchange between different makes of computer will not be assured unless the gaps in standards are quickly rectified by the standards bodies. Organizations such as airlines and banks have their own standards, e.g. the SITA and the SWIFT networks. Computerscan be connected directly into the telex network, but this is relatively rare. Paper tapes can be transferred between a computer and a telex terminal and transmitted over the telex network. Standards for the telex network are well established. However, the transmission speed of the network makes it unsuitable for the interchange of large volumes of data, while the possibility of data corruption makes it necessary for the interchange participants to include many fundamental error controls in their data which are built into magnetic media or more sophisticated telecommunication links. (This is one reason why the syntax rules allow users to include
275
hash totals when interchanging data over telex.) If computers or terminals are connected by other than the telex network, i.e. by private circuits (or leased lines) or the public switched network (PSN), data can be transmitted faster than via telex links and in higher volumes. In addition, the chances of data corruption with the majority of the techniques used is negligible because of inbuilt error controls which make it necessary for the user to include only simple audit checks on the data, The standards for the basic hardware connection into the telecommunications circuits are universally established. Apart from this, there is very little standardization between different computer manufacturers with one exception, the support of teletypewriter and 'teletypewriter compatible' terminals is almost universal. Unfortunately, these have a limited speed and unsophisticated error-checking procedu res. On the faster, more sophisticated links, the major computer manufacturers have developed their own protocols which are only loosely based on the 'basic-mode control procedures' defined by the BSI, ECMA and ISO. This makes communication between different makes of computer impossible unless programs are available that will emulate the other manufacturer's protocol. It should not be taken for granted that it is easy to connect together computers from the same manufacturer; this can be a particular problem when connecting two large mainframes together. The current telecommunications scene is not very conducive to interchange. Efforts are being made, however, to establish new standards with which all manufacturers can comply. At the basic level, the HDLC protocol has been adopted by the BSl, ECMA and ISO, and is intended to replace the 'basicmode-control procedures' that are so loosely adhered to by the major computer manufacturers. Most major manu-
276
facturers and many of the small manufacturers already support or intend to support HDLC. Unfortunately, international agreement has not been reached on some of the high-level procedures of HDLC, and already some major manufacturers have established their own while conforming to the internationally agreed low-level parts of the HDLC standard. One of the most significant developments will be the introduction of public networks by the PTTs nationally and internationally. These networks will make it easier to exchange low-tomedium volumes of computer data between terminals and/or computers. The PTl's are almost certain to select the X.25 standard, for packet switching, with which all manufacturers will need to conform to connect their terminals and computers to the network. X.25 includes elements of HDLC. Some computer manufacturers have their own network software, such as SNA and Decnet, but these manufacturers will almost certainly support X.25 interfaces to public data networks. At the highest level of technical protocols, sometimes called the 'user' level, there is no immediate prospect for standardization, although work is being undertaken, for example, by the BSI. The prospects for the user wishing to connect to other makes of computer can be summed up as follows: •
using the X.25 interface should become practicable. Throughout this period the telex network will continue to be supported and, within its limitations, presents no standardization difficulties.
Recommendations The manual recommends that preference should be given to terminals and computers that can emulate the communication protocols of different computer manufacturers. The terminals and computers should be capable of operating to the HDLC standards and suppliers should be committed to supporting the X.25 interface. Teletypewriter-compatible terminals offer a high degree of compatibility with different mainframe computers. If the number of participants in an interchange are limited but there are a variety of makes of cam puter involved, a useful technique is the offline transmission of 1/2 in magnetic tape using the same make of terminal. The problems of emulation are then overcome, as only the magnetic tapes have to conform to a common standard. There are several remote batch terminals that will carry out offline transmission of 1/2 in magnetic tape. Users are strongly advised to carry out trial data exchanges well in advance of implementation, even where manufacturers claim conformity to international standards.
in the short term, say one to three years, the user must be prepared to emulate the communication protocols of the major manufacturers unless the teletypewriter-compatible terminals are adequate for his needs. ('Data standards for trade documenta• in the medium term, say three to five tion and data interchange' (two years, the use of HDLC should imvolumes) is published by 51TPRO, and prove the prospect of intermanufacis available (price £30) from technical turer compatibility, providing there bookshops, UK National Computing is international agreement for some Centre regional centres and J M Dent, of the higher-level procedures. Durhams Lane, Letchworth, Hertz SG6 I LF, UK. For further details, • in the longer term, say five to ten years, when public data networks contact the EDP Director, 51TPRO, are established in the UK and other 11/12 Waterloo Place, London countries, the more general intercon- 5WIY 4AU, UK. Tel." 01-8393393, nection of computer and terminals Tx: 919130)
computer communications