The specification, engineering, and measurement of information systems quality

J. SYSTEMS SOFTWARE 1992; 17:205-217

The Specification, and Measurement Systems Quality

205

Engineering, of Information

G. P. A. J. Delen and D. B. B. Rijsen~rij Cap Gemini Pandata,

Utrecht, The Netherlands

The article opens decomposes the notion of quality for information systems to 40 quality attributes. This provides a frame of reference for structuring discussion between EDP professionals and clients on the subject of quality. The article describes a way of working, based on the frame of quality attributes, which enables EDP professionals in cooperation with the client to specify, realize, and measure the quality of information systems (here seen as the products of the automation process). This “quality loop” is subdivided into three steps-quality requirements, quality engineering, and characteristics-which will be positioned within a normal system development method. A practical application of this approach and the experiences obtained from it are described. 1. INTRODUCTION Process

Quality

Product

Quality

This process quality system is a first step, and may help software managers get a sound night’s rest. But clients, on the other hand, want more. The software development process is only of minor interest to them; they want guarantees for the quality of the end product. If we compare this to a more mature industry such as the automobile industry, we see that customers are interested only in the quality of the car (the end product) in terms of speed, safety, comfort, and durability. The optimization of the process has become a matter of concern for factory managers only; they are concerned with efficient organization of the assembly line, inspection of intermediate products, reduction of the failure rate, and so on.

and Standardization

The idea of quality awareness, which originated 40 years ago in industry through the work of Juran, Demming, Crosby and others, has reached software at last. In 1987 the international industry standard for quality systems, IS0 9001 [ 11, was defined by all the important industrial countries. It was adopted immediately as the quality standard for the European Economic Community and was applied to software engineering as well as other disciplines. As a result, clients are demanding that European software companies organize their production process according to the IS0 9001 standard. The problem of applying a general standard to the specifics of software production resulted in the “Guideline for the Application of IS0 9001 to the Development, Supply, and Maintenance of Software” (IS0 DIS 9000-3) [2], which was adopted in 1990. In many European countries certification institutes now grant quality certificates to companies that meet the IS0 standards. This explains the recent increase in publications on process quality (e.g., [3-51). 0 Elsevier Science ~biishing Co., inc. 655 Avenue of the Americas, New York. NY loOtO

Quality

Dimensions,

Aspects,

and Attributes

give more structure to the discussion on process and product quality, we have framed a stepwise decomposition of the entire quality notion in four dimensions, 21 aspects, and 40 attributes (Figures 1 and 2). This decomposition approach was inspired by Boehm, who presented a similar quality tree [6]. The four dimensions were introduced by Veldhuizen [7] and Verhaeven [S] . Dimension IV describes the impo~ance of information to a company. The role information plays for the company supersedes all the other quality requirements. However, it is difficult to formulate quality requirements based directly on information; therefore, quality requirements are formulated in terms of product dimensions II and III, which describe the information system that provides the information. That information system in turn is the product of a development process described by quality dimension I. In this article we will pass over the process quality, which has received so To

G. P. A. J. Delen and D. B. B. Rijsenbrij

f.

SYSTEMS SOFTWARE 1992; 17:205-217

206

If applied to automobiles, dimension II would indicate how soon a car begins to rust or how easy it is to replace a defective part; dimension III would describe its driving properties. Such a frame of reference, elaborated in aspects and attributes, may be used to specify quality requirements, and as such it is a necessary first step But more is needed for the deveiopment of information systems: one should know how to ~~#~~r~ the quality attributes, and how to realize an info~ation system that fits those quality requirements. In this article we will describe a way to realize product-quality and to make it measurable. In so doing, we will also determine the current limits of quality engineering and quality measurement.

OUALITY

4 Dlmenslons

21 Asrrects

40 Quality attributes

2. THE PRODUCT QUALITY LOOP: REQUIREMENTS, ENGINEERING, AND CHARACTERISTICS

Figure 1. Decompositionof the quality notion.

According to the definition of ISO 8402 [!3] (the “quality vocabulary” standard to the IS0 9000 series), a quality loop is a “conceptual model of interacting activities that influence the quality of a product in the various stages ranging from the identification of needs to the assessment of whether these needs have been satisfied. ” Following De Moe1 and Van Hulzen [lo], we divide the product quality loop into three steps:

much attention already, and focus on the two dimensions of product quality. The static dimension (dimension II) describes characteristics in the area of information system control, and the dynamic dimension (dimension III) describes characteristics for the users of the info~ation system. Appendix C defines all 27 quality attributes of the two product quality dimensions.

of

a Prof.

information

skills

b account

mgt.

c project

mgt.

Ccntribution

by

IA

developer:

R

maintenance

a external

8 control

b

internal

0 B

111

DYWiMXC

l.iteliabiiity

I 2. conttiuity

3.Lfficirnty

4Sffsctivtnra*

a speed - internal

a coverage of b"~.pWCe~*~S

functioning of

v

the

*yetem the

C

for

user

a correctnass

a uninterrupted

b completeneee c authorizednees

b robustneee

d timelineaa T

- total

e restorability ,d degradation poesibility ~7 diversion possibility

friendliness

availability - in - on

time location

c economy d match with

c usability

manual proc. e workability

support a end ~sQr

rnbnusL

IV

b

b "SE%-

INFOlWATION

l.COr?ZeCtneSe

Z.Completeneee

importance for company

3.IJp-to-dateneee

I.Accuraey

proc.

Figure 2. Overview OFquatity dimensions, aspects, and attributes.

d decision

SUppart

5.Vwifiability

Information

J. SYSTEMS SOFTWARE 1992; 17:205-217

Systems Quality

tion will not take place until the next step, when specifying the quality engineering actions. In that step one reviews which requirements are feasible, whereupon they are stated in such a way that they may be verified later on.

QUALITY REQUIREMENTS

CHARACTERISTICS

Quality

Engineering:

Implementation

I

I

Figure 3. Product quality loop.

quality requirements, teristics (Figure 3). Quality

quality

Requirements:

engineering,

Identify

and charac-

and Normalize

According to the IS0 definition, the quality loop starts with identification of the quality requirements. When formulating the quality requirements, one naturally uses the language of the user, i.e., one considers the information provision function from the outside as a black box. At first, the requirements can only be identified using the frame of reference of the quality attributes shown in Figure 2 (dimensions II and III). Normaliza-

Technical system atchitactu

of

Specification

and

of Actions

To be able to specify a set of engineering actions that can implement the list of quality requirements, the designer has to know the internal structure of the information provision function, and so switches now from the black box to the white box approach. Going back to the example of the car: the customer specifies how fast the car must be able to accelerate from 0 to 100 km/h (black box approach); the engineer then specifies the motor volume in cc’s, the ignition mechanism, and so on needed to meet that requirement (white box approach). For the purpose of describing the complete structure of the informatin provision function, we have framed a layered model (Figure 4). This model has six layers:

0

organizational

infrastructure

F

functional

T

technical

D

data infrastructure

I

technical

P

physical infrastructure

system architecture system architecture infrastructure

Within these layers are coded engineering which the designer (or “systems architect”)

01 Organizational inframtructura

Figure 4. Layered

207

02

Operational organization User organization

areas on may take

208

G. P. A. J. Delen and D. B. B. Rijsenbrij

J. SYSTEMS SOFTWARE 1992; 17:205-217

actions to improve quality. In so doing, he or she should realize that the actions taken in the foundation layers, i.e., in the physical and technical infrastructure, limit what is feasible in the upper layers. For example, when there is no local area network (area IS), there is hardly an option for decentralization the information provision function (area T4). Besides these vertical relationships, there are also horizontal relationships among the engineering actions within the same layer. Specification of the engineering actions is an iterative process in which one examines repeatedly first which set of actions is needed and sufficient to meet the total list of requirements, then to what degree the quality requirements must be adjusted based on the feasibility and the cost of the engineering actions, then once more which set of actions is needed for the adjusted requirements, and so on. In the end the quality engineering actions are implemented and the desired characteristics are built into the end product in an optimal way. Characteristics:

Measure

and Compare

When the product is finished, its characteristics can be measured. Now the client reverts to the black box approach, attempting to measure the value of each quality attribute. Some quality attributes (e.g., maintainability, portability, and continuity) appear not to be measurable directly, or only after the information system has been in operation for some time. For those attributes, measuring means that the client obtains insight in how effectively the specified quality engineering actions have been implemented. Finally, the measured characteristics are compared with the normalized quality requirements of step 1 and the quality loop is closed. 3. PRODUCT SYSTEM

QUALITY

AND

DEVELOPMENT

In this section the quality loop is projected onto a system development life cycle. We chose the SDM [l l] which is the most widely used method in the Netherlands. However, since the previously described approach to product quality is independent of any particular system development method, other elaborations are possible as well. Figure 5 shows the development of product quality (starting from the quality requirements) and the development of functionality (starting from the functional requirements) as interrelated tracks through the life cycle. This figure will be discussed below. Information

Systems

Planning

In the information strategy the client gives a strategic view on the role and the value of information for the

company (quality dimension IV). The information strategy also has to result in a model for the total information provision function of the company. Within this model, different information systems may exist concurrently, based on the same physical, technical, data, and organizational infrastructure. Figure 4 indicates the position of an arbitrary information system in such a constellation (hatched area). As may be seen in the figure, one can, within the context of any specific information system, only take quality engineering actions in the upper layers (the architectural layers) of the model; all the actions in the infrastructure layers are already specified in the information strategy. Rijsenbrij and Helders [3] call these the strategic implementation guidelines. In these guidelines the client states, for example, whether there is, or should be, a local area network, whether the company has an exclusive relationship with a specific hardware supplier, whether all information systems are to be based on a corporate data model, the company’s security concept, the conditions imposed by the user organization, and so on.

Definition

Study

In the definition study, rough quality requirements are formulated in addition to the functional requirements for a specific information system based on the role and importance of that information system for the company.

J. SYSTEMS SOFTWARE 1992; 17:205-217

Information Systems Quality This means that one discusses with the user the importance of every quality aspect of product dimensions II and III. The levels of importance may be determined in terms such as not applicable relevant important crucial. Important and crucial requirements should be explained as much as possible, with an indication as to their backgrounds. At this stage, those backgrounds are more relevant than any attempt to quantify the requirements. For the most crucial quality requirements one could already suggest some engineering actions in the display of alternatives, accompanied by a cost/benefit overview. Of course the suggested actions should fit the information strategy. For an ordinary application system this implies that the freedom of choice is limited to the functional (F) and technical (T) architecttiral layers of Figure 4.

System Design

In the system design phase, quality requirements are translated into engineering actions, in the course of which the requirements may be adjusted (while preserving conformity of the adjusted quality requirements to the rough requirements in the definition study). The engineering actions should also respect the conditions set by the information strategy. This translation is an iterative process between quality requirements and engineering actions that finally results in a three-part product supplementing the unctions s~cification: Quality requirements are detailed in mutual relationship down to the attribute level. Some requirements may be fo~ula~ now in measurable terms; for other quality requirements the acceptance criterion will be whether sufficient engineering actions have been taken to cover the requirement. Qua&y engineering actions are specified. The engineering actions that are necessary and sufficient to cover the quality requirements are specified for all layers in the structure of the information provision function (Figure 4) in mutual relationship. In practice, the s~ification of the quality actions will intertwine with Ihe functional specification; the designer should add a separate index of all the quality engineering actions specified in order to keep good overview. The relation between quality requirements and engi-

209

neering actions is essentially a many-to-many relationship; therefore a cross-reference is necessary to document the translation from the product quality requirements to the specification of engineering actions within the product architecture. Figure 6 gives a rough and general overview of the possible relations according to the model of Deutsch and Willis [ 121. It also shows that the engineering actions needed Coobtain quality on some aspect, e.g., reliability (III. l), may hurt another aspect, e.g. economy (111.3~).Note that the relations in a specific situation for a specific information system may differ from the general ones of Figure 6. As a consequence, the systems architect has to reexamine for every information system which set of engineering actions forms the best solution for covering the quality requirements of that information system. Detailed System Design In the detailed system design phase, the requirements recede in favor of the specified engineering actions. As we have seen, most quality requirements have been translated in the system design to engineering actions, which are no longer distinguished from the other parts of the detailed specification. Sometimes those engineering actions may even take the shape of separate supportive subsystems, such as a security shell, dialogue monitor, and so on. However, it remains necessary that the designer understand the original requirements and remain conscious of the purpose of the engineering actions he or she is elaborating, so that he or she may give feedback on the quality requirements when complications become apparent. Implementation In the implementation phase the quality engineering actions are implemented, which means that the desired characteristics are built into the information system. The system test compares the info~ation system with the detailed design and for that reason is concerned primarily with the specified engineering actions. It is only in the acceptance test that the requirements become explicit again. In this activity, different departments will measure the characteristics of the information system on behalf of the client and compare them with the functional specification and the quality requirements. These departments are the users, who examine the functionality and the general efficiency and effectiveness of the information system (quality aspects III.3 and 111.4), with emphasis on the attributes of speed and user-friendliness;

210

l

l

l

G. P. A. J. Delen and D. B. B. Rijsenbrij

J. SYSTEMS SOFTWARE 1992; 17:205-217

the computer center, which examines the continuity (aspect 111.2)and the machine efficiency, focusing on the (machine-)economy (attribute 111.3~)of the information system;

ronment, with tools for reproduction of test input, for dialogue simulation, output comparison, and so on, as an investment in economical execution of the extra testing effort.

the application management group, who examine the static aspects of the information system (dimension II): flexibility, maintainability, and so on;

Execute test scenarios that approach realistic production circumstances in volume and duration. The best way to do this is to have the new information system run shadow operation for some time, while the old information system is still in use.

the internal accountant, who examines the correctness, completeness, authorization, and timeliness of the information provision function (quality aspect III. 1).

Measure the testing process itself: how much time does it take to test specific parts of the info~ation system? From this measurement statistical data can be obtained that give the value of some attributes. The most trivial example is the testability attribute (11.3).

These departments should be aware that the measurement of quality characteristics requires extra effort and, therefore, additional budget above the budget for an acceptance test on functionality only. If they are not prepared to spend that extra effort, it is useless to state extra quality requirements because there will be no proof that the extra quality is actually implemented. To measure the characteristics involved, some (if not all) of the following additional efforts may be necessary: l

l

Organize a mockup department for users where the manual procedures may be tested in

Extend the test and inspection program (as a part of the project plan) with extra test scenarios. Organize and integrated and independent test envi-

asp*cts areas 4)

Enginmmring (see figure

of

produet

II static 1234567

quality

Operational

02

User

03

Hanual

organiz.

organization procedures

Pl

P~OCL?BB

F2

Functional

model

F3

History

Tl

Ueer

T2

Technical

T3

Standardization

security

III dynamic 4

T4

Decentralization

T5

Hodularization

O

Oata

environmeni

operational

PI

Physical

P2

Housing

Legenda:

.+..++. . . . . . . . . . .

. . . .++ ++. -.

environm

security

+

The

.

.

..--.

..+-+. .

.

.

. . . . . . . .++..+.

+.+-++ +. . .

.

.

++..+..

**

* -

.++. . *. . ..++.+

. . . . . +* + + .

. . . . . * . . . . , . . .

++ * . . . . +. +

have

poeitiva

II

have

little

II

have

negative

++

effect effect effect

-

-

on

-

+

*

actions

+ +

.

.

.

IV information It/m4 5 N.A.

.

.*.

.

2)

+

.

.+.+++.

r..

infrastructure

I1 Development 12/5

++ + + . f ++. . . . +. . . +.

+. . . . . . +.

security

Ii'\ c d&a

. . . . . . . * . . . . . . . . . . I . .

1

interface

figure

12 a&b

01

(we

-

the

ing actions and aspects of product quality (normative example).

. . +

quality I, rt

Figure 6. Relation between quality engineer-

aspect

Information

J. SYSTEMS SOFTWARE 1992; 17:205-217

Systems Quality

211

tion on the flexibility and maintainability of the system (it is recommended that this kind of measurement continue in the operational stage of the information system).

how to state the requirements of the quality attribute in the specification document in such a way that they can be verified. This is closely related to the way of measuring the characteristic later on;

At some point one reaches an economic limit to direct measurement by testing. For the characteristics not yet completely measurable, this implies that the client has to revert to examination of the actions which have been taken to cover the requirements.

an overview of the most relevant engineering actions. Because the same actions may recur with different quality attributes, the actions are not elaborated here, but reference is made to the second part of the guide, which describes all the quality engineering actions;

Installation

the unit in which the characteristic is expressed, for example, man hours, function points, in hours or as a percentage;

During installation there will be no more change in product quality. In installation as a process, however, the client should instruct and inform the end users so that the information system will be effectively used. This is because the ultimate effectiveness of an information system depends as much on the quality of the information system itself as on the degree of acceptance by the end users.

in what way and to what extent the characteristic may be measured. We distinguish the following standard situations:

Explicit direct measurements.

The attribute may be measured in the system or acceptance test using direct test input. This applies, for example, to speed (attribute 111.3a).

Implicit direct measurement. Operation

and Control

Some characteristics (e.g., maintainability and portability) will become apparent in this stage. It makes good sense to carry out a systems evaluation after the first year of operation, if only to extend the understanding of the relationships between quality requirements and actions, for the benefit of information systems to be developed afterwards. The conclusions of such an evaluation might also lead to an adjustment of the information strategy.

4. PRACTICAL APPLICATION BY A SOFTWARE HOUSE: THE “GUIDE ON PRODUCT QUALITY”

by Quality

measurement. The attribute may only be measured directly after the information system has been in operation for some time. During the test stage, only an indirect measurement can examine the specified quality engineering actions. This applies to attributes like portability or continuity.

In practical situations one often finds, depending on the circumstances, a combination of the situations described, as in the example shown in Appendix A. of Quality

Engineering

Actions

Appendix B shows the description of the technical security actions (engineering area T2 from Figure 4), which are referred from the restorability attribute. For every type of engineering action, the positive, negative, and (more or less) neutral effects on the quality attributes (indicated with the symbols + , -/- and o) have been summed up, in accordance with the first part of the guide. These indications are general; in specific situations the effects might be a bit different.

Attribute

Appendix A shows the description of attribute III2c, “restorability.” The example shows that the following items are described for every attribute: l

Indirect

Description

To apply this approach efficiently, one needs a comprehensive list containing all (or most) types of applicable engineering actions and their effects on the quality attributes, and an overview of the measurability by attribute. One such list is the “Guide on Product Quality” [13]. It consists of two parts: 1) a description by quality attribute, and 2) a description of quality engineering actions.

Description

The attribute may be measured via its processes, e.g., the test process, the problem report procedure, and so on. The most trivial example is the testability itself (aspect 11.3).

the definition (Appendix all attributes);

C shows the definitions

of

5. EXPERIENCES FROM PRACTICE This guide on product following purposes:

quality

can be used

for the

1. as a directive to specify the quality requirements an information system;

on

212

G. P. A. 3. Delen and D. B. B. Rijsenbrij

J. SYSTEMS SOFTWARE 1992; 1?:205-217

2. as a checklist for (product) auditing; 3. as an aid to organizing an acceptance test.

Specification

of Quality Requirements

For the specification of quality requirements one may choose between a “hard” approach, in which the software house assumes product responsibility, and a “soft” approach, in which the software house only assumes an effort responsibility. Product responsibility arises when quality requirements are formulated in terms of numerical values of the quality attributes. Most software houses these days still find it hard to estimate the cost of meeting such quantified requirements, and besides they have too little influence on environmental factors to be able to accept product responsibility. For that reason, requirements are formulated in terms of effort responsibility. This implies that one agrees on a selected set of quality attributes that are to be emphasized, a set of engineering actions designed to achieve good quality for those attributes, and the amount of effort to be spent on those actions. Of course many variations of these two approaches are possible.

Checklist for (Product) Auditing The guide’s description of quality attributes and engineering actions can be used as a checklist when performing an audit on the quality of tire deliverables of system development. This is the “product audit“ described by Rij~nbrij [4]. When elaborating this approach to the engineering and measurement of product quality, a striking resemblance is apparent with the manner in which professional EDP auditors examine the information provision function in large organizations [14, 151.

Organization

for their explanation of the 8pprO8Ch and the tools of EDP auditors, and our colleagues J. J. P. van H&en, H. J. Kouwenhoven, and G. Vlasblom for their contribution to the compilation of the “Guide on Product Quality.”

of the Acceptance

Test

By combining the descriptions of measurability by attribute for all those attributes which the client wishes to emphasize, the client may at an early stage gain insight into all those things needed for organizing the acceptance test so that it can test the qudity requirements. This will help the client estimate the effort required and plan the acceptance test. The description of the acceptance test in Section 3 indicates some of the things to be expected here.

REFERENCES’ 1. IS0 9001, Quality Systems-M~el

for Quality Assurance in D~sign/Development, Pr~uction, Ins~llation, and Servicing, Internation~ Organization for Standardization, 1987. 2. IS0 DIS 9000-3, Guidelines for the Application of IS0 9001 to the Development, Supply, and Maintenance of Software, International Organization for Standardization, 1990. 3. D. B. B. Rijsenbrij and B. J. Helders, Modern Information Policy, CGS Technical Newsletter 5, 1988. 4. D. B. B. Rijsenbrij, A Quality System for Software Development (An Implementation Model of IS0 90011, CGS Systems review, 1989. 5. G. P. A. J. Delen, and D. B. 8. Rijsenbrij, Quality attributes of EDP projects and info~ation systems, CGS Technical Newsletter 7 (1989). 6. El. W. Boehm, Characteristics of Software Quality, North Holland, 1978. 7. E. Veldhuizen, Kwaliteit van informatiesystemen, in Kwaliteit en kwaliteitszorg van moderne informatiesystemen, Erasmus University, Rotterdam, 1986 (in Dutch). 8. J. Verhaeven, Kwaliteitsbeheer van informatiesystemen, Beleidsinformatietijdschrift 13, Leuven, 1987 (in Dutch). International Organiza9. IS0 8402, Quality-Vocabulary, tion for Standardization, 1987. 10. R. P. De Moe1 and J. J. P. Van Hulzen, De weg van eisen naar eigenschap~n van info~atievoorz~ening, Informatie 30-2 (1988) (in Dutch). 11. W. S. Turner, R. P. Langerhorst SDM: System Development Methodology, Cap Gemini Publishing, 1989. 12. M. S. Deutsch and R. R. Willis, Software Quality Engineering, Prentice-Hall, Englewood Cliffs, New Jersey, 1988. 13. G. P. A. J. Delen, H. J. Kouwenhoven, and D. B. B. Rijsenbrij, Kwaliteit van produkten (Guide on Product Quality), Cap Gemini Publishing, Rijswijk, 1991 (in Dutch). 14. J. A. Saers, Risico-analyse en de EDP-auditor, in Automat~~ering onder ~ontrole, Kluwer/Moret EDP audit (1989) (in Dutch). 15. P. L. A. M. van Kessel, EDP-audit van informatiesystemen, in Automatisering onder controle, Kluwer/Moret EDP audit (in Dutch). 16. D. B. B. Rijsenbrij and A. H. Bauer, Project Diagnosis, a Proper Start is Half the Battle, CGS Systems review, 1990.

ACKNOWLEDGMENTS We thank J. A. Saers and P. L. A. M. van Kessel from Moret & Limper9 aCCOUnt8ntS (part Of Arthur Young intern8tjon81)

’ The internal publications of the CGS (Gemini Sogeti group) may be obtained directly from the authors.

x6SMRAELLITY

and

r@uuirements

information

as

as

supply

fast

(MTTR) and continuity,

after

repair

preparation Technical security

*

beat

backup-

and

engineering

ppO3 a fact

(eection

guWan+ee

recovery-procedures

to

fire-brigade area oection

action

of a company overview by

or

MTTR

consists

of

the

time

needed

far

fHTTR)

the

maximum

time

of

medeUrement for the restorability.

time

the

the

PQO3

solving

the

disturbance

nain-

action

(the

is another

important

unit

of

again mean

repair

hours,

T2.5)

is the

databane

on the II.2

provision those

@tart up the information processing (the 'recovery-time'). Beaides the

interruption

time) and the time needed after that to from the point where it we.e interrupted

The

in

eubeyetem.

Restore

reports

recovery,

of

ie dependent see attribute

inquiries

Time To Restore (IT'L'R), this is the inem time, expressed in during which the information supply is down after e disturbance.

sirri-t. The HeM

of

known

maintenance a comprehensive

The

for

implementation

See

keeping ready a backup machine Physical security actions (P2)

*

(D)

repair eystern,

unforeseen

use of utilitiee for analyeie and roorqanization in the oparational environment (12 t/m IS)

Actions

infrastructure

*

data

Actions

*

the

To

each

Time far

layer of the information for repairability in all

('recovery')

personnel

the time it takes for technical tainability of the information

of utilitiea for actions (T2)

instruction plan for backup FunCtiOnal security actions (F2)

*

in

maximum

separately

the

actions to be able to find and repair the cause possible ('repairability') and actions to start up

Mean Time To Restore other attributes of

the

Because disturbances may occur in every function, one has to take engineering actions area*. we give one example for every layer: * organizational actions (01, 02, 03)

the

the the

1991-08-30

QA/PQO2.30

the information system after e disturbdnce.

charactixristiceby attribute

fall in two claeees:

a disturbance

again

of

Q-8 The actions

One e.pccifies relation with

gL?ecificationof

The ease and speed of restoring

E4Lfinition

XIX.ZC

: Requirements

SECTION

QUALITY

: PRODUCT

ASSUL%NCE

: QUALITY

VOLUME

GUIDE

: Requirementa and

QUALITY

: PRODUCT

ASSURANCE

[I 31

: QUALITY

Description

characterietics

by

attribute

continue

(typea

to

cover

the

restorability.

duration of the shadow operation has to revert to examination of

taken

and one

enough

of)

have

occurred

to

bane

regietrating the shadow-operation

have to be scaled down the engineerrng actions,

for other reasons, which have been

it volume

1991-08-30

QAjPQO2.31

low rate of interruptions an earlier time. When the

dieturbbncee

upon. Only in case of a very stop the shadow production at

until

reliable statistics seems justified to

should

Measurability The HTTR may be measured by running shadow operation while In principle the restoration-time after every disturbance.

SECTION

:UIDE

VOLUME

APPENDIX A. Example Quality Attribute

E

TECENICN, SECURITY ACTIONS to

3:

+

+

Effects:

+

documcante

the

the

or

proceseae.

stores

could

a manual

ths

mutual

(attributes

III.la

and

b)

procedure.

of

and

of

can

of the data

coneietency

signaling of deviations. the correctness and completeness

of

authorization

02.4)

IIL.3a

type

of the involved more than eny action

(attribute

(eee

ordered the combination

data/documents

have which

certain

friendlinees

checking

to

u#er

procedure

stores

periodic

data

for

stores and the Helps to control

programe

and

compared

speed

authorization

111.3b) when

Raises

manual

Checking

of

which

that combination specifies,

Raises the authorizedneea and verifiability proceseee (attribute 111.1~ and IV.5) much

authorize

development

tvoe

must

the prnlanent data

The

Action

Effecta:

persons

only after all persona of a given system to do so. A protected table

syetem

by

be proceseed information

Electronic authorization the automated information

Enfoceement

Rction true 2:

iction Woe 1: Acceee control Phe denial of access to certain data and/or information funetione and/or #orkstatione to pereons who are not authorized. Phe elaboration of this action type implies that all ueere have to check in with rheir identification and paesword, before entering the information eyatem. Then rhe information system consults come tablea to decide which information functi3"s and 80 on are accessible to that user. The least acceeeible functions of the information eystem should be the functions which control the 'authorization :ablea' and the 'paseword tablee'; these functions are raetricted to the 3ecurity officers. It is recommended to place the access control in a separate %uppartive subsystem, which comptiee control function6 for the tables and xccesa-control modules (see aleo TS Hodularitation'). The complexity of such a subeyetem will depend on the complexity of the acceae rules; ae long aa the requirements are not extreme, one may fairly well use package software, like for axample RACF. Raiees the authorizedneae and verifiability (attribute 111.1~ Effecte: + resp. IV.5) -/Hurts the epeed and economy (attribute III.3a and c) of the automated part of the information eystx?m.

meant

1991-08-30

Lhis category encompasees a wide range of actions, which are primarily LBBWO the reliability and continuity of the information prOceeSing.

r2

actions

tvoe

4:

Transaction

of

consistent

state

to

another.

0

+

Effects:

the

of

0

-/-

In terme

of

be

the

quality

cannot

transfer

which

which

data

disturbances,

which

involve

only

one

or

e few

trans-

of

the

A cheap DBMS.

action

aesluming

use

can

be

the

type

A cheap

of

action.

facilities of the DBMS.

such.

prevent

coherent backupe, unless

assuming

uee

can

provisions that

additional

be

to

of the

made made

are

provi24 hour6 to make

transactiona, if need should be. premise for‘ getting the information

for:

facilities

programs

of

the data stores backup copies

and

made

sion function resltorable (attribute 111.2~) The information ayatem cannot be available 7 daye of every week, because it ehould be frozen periodically

of all logged is a neceeeary

saving of a coherent eet of copies of all transaction8 starting from the last

manual procedures

that

This raiaee etrangly the rabuetnese of the information system (attribute 111.2b).

of small actions.

reprocessing This +

logging

the periodic

marked. in the data.

Tcaneactione

1991-08-30

QAjPQO3.19

attributes: it raises the correctness of the data processing and helps to control the completenese (attribute III.la and b) Aewres that the information processing can Continue in spite

Sackuoand recoverv-svstem pction tvoe 5: A backupand recovery system consists of

Effects:

one

actions

and rollback into transactions.

completely are 'rolled back' and Prevents inconsistencies +

from

proceesed

stores

[I 31

engineering

nrotection

quality

Decomposition of the dataproceasing

Rction

: Catalogus

SECTION

of quality

engineering

: catalogue

;ECTION

Actions

ASSURANCE

: PRODUCT QUALITY

QUALITY

:

QA/PQ03.18 WIDE

QUALITY ASSURANCE VOLUHE

Engineering

: PRODUCT QUALITY

of Quality

:

Description

XJIDE

B. Example

fOLUt4E

APPENDIX

Information Systems Quality APPENDIX Attributes

C. Definition

J. SYSTEMS SOFTWARE 1992; 17:205-217

of Quality Aspects and

In this description we follow the structure and numbering of the quality tree shown in Figure 2. Dimension I: The Development

Process

The quality of the development process depends on the quality of the development organization as well as on the contribution by the client’s organization, so this dimension is divided into IA.

contribution of the developers

IB.

contribution of the client’s organization and the users

The type of contract determines which of these is most important. In a fixed price/fixed time/fixed quality project, the emphasis falls on the quality of the developer, while in the case “body shopping” by the software house, much more is expected from the client’s organization. It is never possible, however, to develop a good info~ation system without a subs~ntial contribution from the users. For definitions of the 2 x 5 = 10 aspects of process quality, we refer the reader to Delen and Rijsenbrij [5]. Dimension II: The STATIC Properties of the Information System

The static quality aspects of the information system itself are most relevant to the developers and the maintenance and control crews. All aspects refer to the ease of adapting the system to changing demands from its environment. They are treated here in order of increasing seriousness of the adaptation. 11.1 Flexibility: the degree to which the user may make extensions to or variations on the information system, without adapting the programs. In other words, this is the extent of the changes the user control organization may execute without being dependent on the EDP department for maintenance. II.2 Maintainability: the ease of adapting the programs by EDP professionais to new demands of the user (perfective maintenance), as well as to restore bugs (corrective maintenance). 11.3 Testability: the ease and speed of testing the ~nctionali~ and performance of the system (and again after each adaptation). 11.4 Portability: the ease of transferring the infor-

215

mation system from one specific hardware and software environment to another. IISa External connectivity: the ease of realizing interfaces with other info~ation systems. IISb Internal connectivity: the quality of the interfaces between the various parts of the information system, and the ease of adapting them. 11.6 Reusability: the degree to which parts of the information system, or its design, may be used again for the development of other applications. II.7 Fitness of the infrastructure: the fitness of the hardware, the datacom network, the system software, and the DBMS for the application concerned, and the degree to which these infrastructure elements conform to each other.

Dimension Ill: The Functioning System

of the Information

The dynamic quality aspects of the information system are important for the user of the system. We follow the aspect list of the NIVRA (the Dutch organization of register accountants). III.1 Reliability of the information provision function: the reasonable certainty that the processing of the data is performed correctly, completely, by authorized persons, and in a timely manner. The total reliability of an info~ation function is the product of the reliability of: l

the manual collecting and preparation of data

l

the automated data processing

l

the manual postprocessing of the data

in which it should be realized that the manual steps are the most vulnerable. The definition of reliability already contains the four attributes, which are described below. III.la Correctness: the degree to which the system processes the input stream (including mutations) correctly according to the specifications into consistent data collections, even if one tries intentionally to have the system do something different. IH.l& Completeness: the certainty that the system processes all input and mutations completely, so that no data is missing or repeated in the data collections.

216

J. SYSTEMS SOFTWARE 1992; 17:205-217

IZI.Zc Authorization: the degree to which it can be guaranteed that the data and the coding can be changed or referenced only by authorized persons. ZZZ.ld Timeliness: the degree to which the information becomes available in time to take the actions and decisions which the information was meant to support. By this definition, the norm for timeliness is completely dependent on the process, which is supported by the information supply. It is evident that the timeliness requirement for a flight reservation system is an order of magnitude different from the requirement for a bookkeeping application. III.2 Continuity of the information supply: the reasonable certainty that the data processing is continuous and undisturbed, or in case of serious disturbances, is resumed within a reasonable period of time. Continuity is divided into attributes in order of seriousness of the disturbance. ZZZ.2a Uninterruptedness: the degree to which the data processing remains free of disturbances. 111.26 Robustness: the degree to which normal data processing can continue in spite of a disturbance. 111.2~ Restorability: the ease and speed of restoring the information supply after a disturbance. ZZZ.2d Degradation possibility: the ease of continuing the main line of information supply when part of it has broken down. One speaks about “graceful degradation” when, during calamities, the system can fall back stepwise on ever simpler forms of information supply (the so-called emergency procedures) until, in the worst case, only manual administration remains. ZZZ.2eDiversion possibility: the ease of transferring (part of) the information supply to another location. III.3 Efficiency of the information supply: the ease with which the system supplies the information (considered independently from the relevance of the information). ZZZ.3a Speed: the speed of processing transactions. The speed of interactive processing is divided into the internal speed and the total speed. The internal speed can be measured as the mean response time, which is the mean period of time between the user sending a question or command screen and receiving an answer screen from the automated system. More interesting than the internal speed is the total speed, which

G. P. A. J. Delen and D. B. B. Rijsenbrij is the total time needed by the automated system and the user together to process a transaction completely. This time is composed of 1) the time needed by the user to formulate a command or question to the system; 2) the system response time; and 3) the time the user needs to understand the system’s answer and to react accordingly. It is evident that the time needed by the user depends strongly on the friendliness (attribute 111.3b) of the user interface. The speed of batch processing is composed only of the internal speed and can be measured as the batch turnaround time. ZZZ.3b User friendliness: the ease of communication with the automated system for the end users. The effort of communication with the system consists of a one-time effort-the learning time, and a permanent effort-the time which is spend at every usage to understand the system. 111.3~ Economy: the ratio between the performance of the system (to be expressed in the transaction volume and the total speed) and the amount of resources consumed, such as CPU cycles, IO time, storage capacity, man hours, and so on. ZZZ.3d Match with manual procedures: the degree to which the use of the automated information system fits well in the manual administration of the business process. ZZZ.3e Workability of the manual procedures: the workability of the manual procedures themselves, as far as they belong to the information provision function. III.4 Effectiviness of the information provision function: the degree to which the information provision contributes to achieving the company’s business goals, as recognized and selected in the information strategy. An effective information system results in greater efficiency of the business processes it supports. ZZZ.4a Coverage of business processes: the number of the business processes that are supported by the information provision compared to the list of processes selected in the system definition, and weighted to the frequency and importance of the processes. ZZZ.4bAvailability: the degree to which the information supply is available for every user when and where he or she needs it. 111.4~ Usability: the degree to which the information supply is tuned to the user organization and to the

Information Systems Quality profile of the intended end users. It happens sometimes that information systems cannot be installed successfully because they are too difficult for the users, Infamous in this respect are systems using metaconcepts, which in themselves are the ideal solution for high flexibility requirements (see attribute II.l), but at the same time surpass the understanding of the end users by an order of magnitude. 111.4d Decision support: the value of the information supply as background and foundation for policy and decision making. This attribute is preeminently important for decision support systems and management information systems. Z114e End-user support: the degree to which the information system takes over routine jobs so that the user’s creativity may be directed toward more important aspects of the job. Dimension IV: The Value of the Information Provided for the Company

The information provided by the information system is generally spoken of as a projection of some aspects of the real world. The value of that projection for a company is determined by I the relevance of the aspects selected for projection l

the selection made when defining the information system

J. SYSTEMS SOFTWARE 1992: 17:205-217

l

the actual reliability of the info~ation function (aspect III. 1)

217 provision

IV.1 Correctness: the degree to which the information provided accurately projects the real world. IV.2 Completeness: the degree to which the information provided gives a complete projection of the defined aspects of the real world. Both the lack of information and redundant information (like variations in the spelling of names) are manifestations of incompleteness. In combination with the IV.1, the question here is whether the information projects “the truth, the whole truth, and nothing but the truth.” IV.3 Currency: the degree to which the information projects the real world as it was at the time the information was produced. IV.4 Accuracy: the degree of detail of the information provided. Requirements on accuracy and currency (considered as accuracy in time) make no sense unless correctness and completeness can be taken for granted. The norm for accuracy should be related to the goal of the information supply. IV.5 Verifiability: the ease of verifying the correctness, completeness, and autho~zation of the information, and of tracing and reconstructing the various steps of the data processing.