- Email: [email protected]
MDSE advisor: knowledge-based techniques applied to software design assessment
MDSE Advisor: knowledgebased techniques applied to software design assessment David Budgen and Mustafa Marashi
An overview of progress in the design and development of the Advisor subsystem during the first two years of the MASCOT Design Support Environment project is given. The Advisor is a software tool which makes use of a set of design metrics in applying knowledge-based techniques to the task of assessing a design, where the design is expressed using the MASCOT design representation. The experimental basis of this work is described, and the principles upon which the Advisor is based are outlined. Keywords: software tools, knowledge-based techniques, software design, assessment, MDSE Advisor
A major problem in the development of large softwarebased systems is that many of the problems that originate from the short-comings, or incompleteness, of design are only detected at the construction and testing stages of the software life-cycle. To some extent, this is inherent in the abstract nature of the design process and its products, since some classes of problem will only emerge when a design is implemented, and are consequences of the way that a design is transformed during this process. However, there are also many issues of quality that could be recognized at the design stage by using a mix of previous experience and design assessment practices. Since the economic benefits of detecting weaknesses or errors in a design (which may, of course, be design choices that will eventually make it hard to modify or update a system) at an early stage are considerable, a tool named Advisor is being developed, which will seek to provide a designer with some form of reasoned assessment about the static qualities of a design. Because the rules of design assessment are based upon a mixed set of criteria, the use of knowledge-based techniques has been regarded as essential. The rationale behind the Advisor, and the progress that has been made to date Department of Computing Science, University of Stifling, Stirling FK9 4LA, UK
with its development, is described in the later sections of this paper.
ADVISOR'S ROLE The Advisor is a tool which forms a component of the MASCOT Design Support Environment (MDSE), which in turn is a support environment geared to the support of the design process. The Advisor is primarily concerned with evaluating the static description of a design which is expressed using the Modular Approach to Software Construction, Operation and Test (MASCOT) representation'. The idea behind Advisor originally arose from the experiences of teaching system design using MASCOT, and of assessing the designs which were produced by classes of advanced undergraduate and postgraduate students. On the basis that an assessment of design quality seemed to be a practical exercise when performed by a human, it therefore seemed at least possible that it could be performed by some form of expert system as well. It was originally envisaged that the main users of Advisor would be either: • inexperienced designers, who would use it to check their designs, and to help them to identify where the consequences of particular choices might lead to later problems; or • experienced designers, who would use it to perform consistency checking and 'what if" exercises, in order to explore ideas about the form of a design. Since these are tasks that also need to be performed at various points during the development of a design, this further emphasises the need to use expert system techniques, as it is also necessary for the Advisor to be used to perform analyses of incomplete (and possibly inconsistent) designs. As the development of Advisor progressed, it also became evident that a third role for it was also likely to be adopted, which was its use by project managers as part of the acceptance testing for a system (while there are likely to be some significant
0950-7051/88/040235--05 $03.00 © 1988 Butterworth & Co (Publishers) Ltd Vol 1 No 4 September 1988
235
pitfalls in this type of use, it is only realistic to recognize that it will be very likely to occur). While the above points have been considered in the context of the development process for a system, they are also very relevant to the maintenance process. Indeed, the maintenance of a system is often performed by staff who are not familiar with all the thinking of the original design team, and hence will be unaware of some of the possible side-effects of the changes they might consider making. For this reason alone, the availability of a means of assessing the consequences of changes should prove invaluable for a maintenance team. (There is of course the inevitable corollary to this, in that some means of ensuring that the design description continues to match the implemented system will be required, and needs to be enforced.) There are obviously some significant issues surrounding the criteria that are used by Advisor in making its assessments about the quality of a design. In the rest of this paper we seek to describe how these are chosen, and the ways in which we have sought to minimize the subjective aspects of design assessment. MASCOT'S ROLE The Advisor subsystem has been based upon the MASCOT design representation for two principal reasons: MASCOT provides a well-defined design representation which is 'standard' within its particular user community, and which is not subject to significant local variations. There exists a substantial user community which has experience with implementing large systems with MASCOT, and which therefore can provide both a source of expert knowledge, as well as a source of opinions, about design quality measures and the weighting that should be ascribed to each of these for particular problem domains. MASCOT itself is oriented towards supporting the design and implementation of real-time systems. Its main components are the graphical design representation (the ACP diagram*), a set of constructional forms and the specification of a real-time executive. Together, these can be considered as providing a virtual machine environment that forms a layer of abstraction between the application system and the physical machine. The MASCOT formalism has undergone some changes, and the MASCOT 2 representation has been considerably developed and extended in producing the MASCOT 3 standard. However, the underlying virtual machine concepts are largely unchanged by these developments. This is an important point, since the relative newness of the MASCOT 3 standard has meant that the expert system aspects of Advisor have been based mainly upon the experiences that have been gained with the use of MASCOT 2. While the MASCOT user community is not particularly large, it is also concentrated mainly in one sector of the computing community, and so is relatively homogeneous. In the longer term it is hoped that the ideas *The ACP diagram describes a network based upon the three MASCOT system elements: activities, channels, pools (ACP).
236
incorporated into Advisor can be extended to handle other forms of design representation. DESIGN M E T R I C S The idea of metrics (forms of measurement) as applied to software-based systems originated largely with the work of the late Maurice Halstead, and in particular with his ideas about software science2. While the basis for much of this work has subsequently been questioned 3'4, it has nonetheless spawned a considerable volume of work in the area of metrics as applied to software. Most of this work has concentrated upon assessing code rather than design. There are good enough reasons for this, since a programming language possesses a relatively well-defined syntax and semantics, and so its form can be quantified and analysed. Various metrics have been developed, with some of the more successful being McCabe's Cyclomatic Complexity Measure 5'~ and the Information Flow ideas of Henry and KafuraL Unfortunately, the application of metrics to design representations has been less successful. There are a number of reasons why this has proved so, including: • the lack of any well-defined syntax and semantics for most design representations; • the lack of concensus about how to interpret such metrics as do exist, and the difficulty of assessing them by means of experiments; and • the lack of any good models of quality on which to base new metrics. However, the benefits which can be obtained from being able to detect likely weaknesses in a system at the relatively early stages of design, has meant that it has been well worth seeking to find ways of bettering our knowledge in this area. In developing Advisor, existing metrics were examined (used for both code and/or design assessment), and ways of quantifying existing thoughts about design quality were looked at. This is a large and complex topic, and is to be the subject of a separate survey document. Where possible, the ideas generated from this work are being incorporated into the design of Advisor. In particular, it should be emphasised that Advisor is only concerned with assessing the static qualities of a design. (Within the MDSE a separate set of tools are used for predicting performance and resource requirements.) The Advisor is also primarily concerned with assessing the logical form of a design, and does not concern itself with the process by which this was produced. So, it is not linked to any design method, nor are any particular design practices assumed in the metrics themselves.
EXPERIMENTAL F O U N D A T I O N S The relative lack of any strong theoretical foundation that can be used for identifying suitable measures of design quality requires that some form of experimental evidence is needed in order to support the construction of a system such as Advisor. Because of the nature of the problem there are several features that make the gathering of such data difficult. These include: Knowledge-Based Systems
• the relatively large scale and complexity of many of the systems of interest; • the lack of systematic record-keeping for many existing designs (including records of changes made during implementation); and • the lack of comparative design studies. Within the MDSE project it is considered that there are two useful sets of experiments that could be performed in order to help with the creation and validation of the rule-set needed in Advisor: 1 Obtaining a number of designs for a single problem, with these being produced by a mix of experienced and inexperienced designers. The designers would also be asked to provide details of their intermediate workings, and the reasons for any decisions that were involved in these. 2 Obtaining a number of designs for systems which have actually been implemented (and ideally, have also been subsequently modified or upgraded). These would provide a means of confirming the predictions of Advisor. The first set of experiments were begun at an early stage of the MDSE project, and were based upon two sample problems: first, an Air Traffic Control system (ATC); and second, a bank Auto-Teller Machine (ATM) network. A set of ten designs have now been obtained for the first of these problems, and an additional four designs for the second problem. These designs have been analysed extensively, and some of the conclusions are described below. The second set of experiments have yet to be performed, since they require the use of Advisor, and its rule-set is still only in a very prototype form. However, they are seen as an important part of the validation process. DESIGN M O D E L In the analysis of the design experiments described above, we sought to examine both the design process, and also the product of that process (the structure of the design that resulted from the process). The conclusions of this analysis will be the subject of other papers, as well as being a major input into the M A S C O T Designer's Guide which is being produced as a part of the MDSE study, and so they will only be outlined in this section. The MASCOT representation is very powerful, but it is also incomplete in terms of its ability to describe a design which is based upon the MASCOT virtual machine. Therefore, one major task was to examine the experimental data, and to extract from this the set of additional factors that were required in order to be able to model a design in full. Because what has emerged is essentially a scheme of classification for the features of a MASCOT design, we have chosen to term it as
a design taxonomy. The taxonomy is largely based upon the MASCOT system elements (activities, channels and pools), although it also places considerable emphasis upon the nature of the information that flows around a MASCOT system (and which has been loosely termed 'message types' for this purpose). It attempts to encapsulate the main features that a designer will manipulate in an Vol 1 No 4 September 1988
abstract fashion when thinking about a design, and when mentally modelling its operation s . The main purpose of producing the taxonomy has been to provide us with a means of describing the logical structure of a design in a suitably abstract form, and allowing this to be extracted from a description of the physical structure of a design. So the taxonomy provides a framework, within which we can seek to describe our ideas about the 'design rules' (see below). D E S I G N R U L E S AND C O N S T R A I N T S The role of the 'design rules' is to provide a way of applying a set of quality measures (metrics) to the logical description of a MASCOT design, as represented using the design taxonomy. The rules themselves can be classified in a number of ways, but we have found that the following provides a useful way of grouping the rules according to their origins and purpose. 1 Rules which are based upon MASCOT syntax and semantics. 2 Rules which are based upon design theory and upon ideas about quality metrics. 3 Heuristic rules that are based upon experience with the use of MASCOT, rather than upon any logical model ('rules of thumb'). We have preferred to term these constraints, for reasons that will be explained later. Before going on to describe the three classes of rule a little more fully, it may also be useful to consider the sources of material for these rules. These sources include: • the metrics study performed as part of MDSE; • the analyses of the design experiments; • discussions with experienced MASCOT designers; • our own experiences; • analyses of student design exercises; • comments received on our preliminary findings; and • initial experiences with the prototype Advisor. Obviously, this list is not complete, and it is also likely that the influence of the last item will increase as we begin to develop the Advisor subsystem beyond its prototype form. The rules which are based upon MASCOT syntax and semantics are in many ways the most direct and simple rules, since they are mostly 'grammar-based'. One example of a syntax-based rule would be: 'Activities may only be connected to channels, pools and device servers.' A rule of this type can be, and is, enforced by the graphics editor used within MDSE, and so such pure grammar rules are not currently included in the Advisor rule-base. However, there are a number of other features that can easily complicate even this class of rules, and which are included in the Advisor. One of these is the idea of 'normality', as expressed in the rule: 'An activity will normally take input from only one channel.' (this is concerned with the question of activation criteria). This type of rule is quite useful for picking up the kind of error that is easily made by an inexperienced designer, but it is really a rule which is based upon the idea of 'rules of form' rather than being based upon 237
the MASCOT grammar itself. As with all rules of form, there are occasions when it is appropriate to break such rules (although far less often than the inexperienced designer might think!), unlike those rules which are based solely upon the MASCOT grammar itself. The rules which are based upon design theory and quality metrics express ideas about such qualities as complexity, cohesion and coupling, information hiding, dependencies and parallelism, and because they are based upon fairly abstract concepts, these have proved difficult to quantify. While these rules will be the key component of Advisor, they are still evolving and will continue to do so indefinitely. They are also difficult to express, since a rule which is used to determine (e.g.) the extent of the data coupling occurring between an activity and the rest of a subsystem, may involve terms dealing with message types, IDA types, as well as the frequency and type of operations that are performed by the activity upon the different messages. As a first step we have tried to adapt some of those metrics which have already demonstrated their value for the assessment of code. A particular example is McCabe's Cyclomatic Complexity Measure v ( G ) s. This is based upon graph-theoretic modelling, and provides an assessment of the complexity of the flow of control within a segment of a program. Hall and Preiser 9 have also mapped this to a system of concurrent processes, and overall it is generally recognized as being a relatively well-established and useful measure. The basic measure is expressed as v (G) = e - n + p where e is the number of edges, n is the number of vertices, and p is the number of connected components. This measure was mapped to the MASCOT representation, and was used by 'hand' to assess a number of systems available, from which we have been encouraged to incorporate it into the prototype system in order to allow further experiment. Some extensions to the original form of interpretation have been needed in this, and the resulting form makes use of a wide range of information contained in the basic design taxonomy. The last group of rules have been described as constraints, since they are largely based upon observation and experience, and may be strongly influenced by local programming and design practices. An example of such a rule might be: 'The ratio of activities to channels should be Pl -bdl.' Since bothpl and dl may vary according to local practice or the type of application, these will need to be parameters of the Advisor rather than fixed values. An important point about the constraints is that their main effect is to limit the acceptable solution space for a design, rather than to provide any real assessment of the quality of the design. Because it is likely that a design will be developed in a piecemeal fashion, and over a number of iterative cycles, it is impractical to use the graphical editor to alert the user of any 'violations' of these constraint rules. This is partly because the editor may only have a small section of the design available at any time, and also because such warnings are not desirable while the design is still only partly complete. More significantly, these rules are included in Advisor because it is desirable to link their output with the output from the metric rules when the assessment of a design is finally presented to 238
the user. We expect that the output from the constraints rules may well prove to be of more interest to the project manager than to the designer, in many cases. DEVELOPMENTS
TO DATE
The MDSE project started in January 1986, and in January 1988 a prototype Advisor (containing a very elementary rule-set) was finally integrated with the other components. The complexity of the task of the Advisor means that it is really a subsystem containing a number of tools, although at present only two of these are operational to any extent. The first of these tools is a menu-based program which is used to help a designer to describe their intentions in terms of the design taxonomy. It has so far been termed the Formatter, but this name is now considered as no longer really appropriate and will need to be changed. The second tool is the rule-based assessment tool, the Assessor, which is based upon the Inference ART expert system shell. The initial rule-set being used in this is quite small and simple, but over the third year of the project it is expected that this will be expanded considerably. We will also be attempting to verify this tool, by such means as assessing actual systems that have been produced from MASCOT designs, as well as by using the results of the design experiments conducted earlier in the project. An important aspect of using such a system as the Advisor is to be able to interpret the results that are produced by using the Assessor. Because different users will have different requirements, including the ability to make an assessment of the overall quality of a design, and comparisons between the effects of two options in a design, there will almost certainly need to be at least two further tools to provide for the analysis of the output from the Assessor. However, the detailed design of these is awaiting greater experience with the use of the Assessor, and the development of a fuller set of rules. (The need to await experience with using the Assessor reflects the influence of the process of design. A tool of this nature needs to be able to fit into this, but in due course it may also be an influence that changes the actual process itself.) As can be seen, the Advisor subsystem is still evolving, and it involves us in pulling together a number of ideas about such issues as design quality, design metrics, the design process, and about the nature of the MASCOT virtual machine itself. The prototype Advisor is therefore not only the product of an experimental process, but is also an experimental tool in itself, and one that we expect to use extensively for the remaining part of the MDSE project. ACKNOWLEDGEMENTS The MDSE project is funded by the Alvey Directorate, and the partners are Ferranti Computer Systems Ltd., YARD Ltd., LOGSYS and the University of Stirling, UK.
REFERENCES 1 l E E Software Eng. J. Special issue on MASCOT 3,
Vol 1 (May 1986) Knowledge-Based Systems
2 Halstend, M Elements of Software Science Elsevier, The Netherlands, (1977) 3 Coulter, N S 'Software science and cognitive psychology' IEEE Trans. Software Eng. Vol SE-9 (March 1983)pp 166-171 4 Conte, S D, Dunsmore, H E and Shen, V Y Software Engineering Metrics and Models Benjamin/Cummings, USA (1986) 5 MeCabe, T J 'A complexity measure' IEEE Trans. Software Eng. Vol SE-2 (December 1976) pp 308-320 6 Myers, G J 'An extension to the cylomatic measure
Vol 1 No 4 September 1988
of program complexity' ACM SIGPLAN Notices (October 1977) pp 61-64 7 Henry, S and Kafura, D 'The evaluation of software systems structure using quantitative software metrics' Software Pract. & Exper. Vol 14 (June 1984) pp 561-573 8 Adelson, B and Soloway, E 'The role of domain experience in software design' 1EEE Trans. Software Eng. Vol SE-11 (November 1985) pp 1351-1360 9 Hall, N R and Preiser, S 'Combined network complexity measures' IBM J. Res. & Dev. Vol 28 (January 1984)
239
An overview of progress in the design and development of the Advisor subsystem during the first two years of the MASCOT Design Support Environment project is given. The Advisor is a software tool which makes use of a set of design metrics in applying knowledge-based techniques to the task of assessing a design, where the design is expressed using the MASCOT design representation. The experimental basis of this work is described, and the principles upon which the Advisor is based are outlined. Keywords: software tools, knowledge-based techniques, software design, assessment, MDSE Advisor
A major problem in the development of large softwarebased systems is that many of the problems that originate from the short-comings, or incompleteness, of design are only detected at the construction and testing stages of the software life-cycle. To some extent, this is inherent in the abstract nature of the design process and its products, since some classes of problem will only emerge when a design is implemented, and are consequences of the way that a design is transformed during this process. However, there are also many issues of quality that could be recognized at the design stage by using a mix of previous experience and design assessment practices. Since the economic benefits of detecting weaknesses or errors in a design (which may, of course, be design choices that will eventually make it hard to modify or update a system) at an early stage are considerable, a tool named Advisor is being developed, which will seek to provide a designer with some form of reasoned assessment about the static qualities of a design. Because the rules of design assessment are based upon a mixed set of criteria, the use of knowledge-based techniques has been regarded as essential. The rationale behind the Advisor, and the progress that has been made to date Department of Computing Science, University of Stifling, Stirling FK9 4LA, UK
with its development, is described in the later sections of this paper.
ADVISOR'S ROLE The Advisor is a tool which forms a component of the MASCOT Design Support Environment (MDSE), which in turn is a support environment geared to the support of the design process. The Advisor is primarily concerned with evaluating the static description of a design which is expressed using the Modular Approach to Software Construction, Operation and Test (MASCOT) representation'. The idea behind Advisor originally arose from the experiences of teaching system design using MASCOT, and of assessing the designs which were produced by classes of advanced undergraduate and postgraduate students. On the basis that an assessment of design quality seemed to be a practical exercise when performed by a human, it therefore seemed at least possible that it could be performed by some form of expert system as well. It was originally envisaged that the main users of Advisor would be either: • inexperienced designers, who would use it to check their designs, and to help them to identify where the consequences of particular choices might lead to later problems; or • experienced designers, who would use it to perform consistency checking and 'what if" exercises, in order to explore ideas about the form of a design. Since these are tasks that also need to be performed at various points during the development of a design, this further emphasises the need to use expert system techniques, as it is also necessary for the Advisor to be used to perform analyses of incomplete (and possibly inconsistent) designs. As the development of Advisor progressed, it also became evident that a third role for it was also likely to be adopted, which was its use by project managers as part of the acceptance testing for a system (while there are likely to be some significant
0950-7051/88/040235--05 $03.00 © 1988 Butterworth & Co (Publishers) Ltd Vol 1 No 4 September 1988
235
pitfalls in this type of use, it is only realistic to recognize that it will be very likely to occur). While the above points have been considered in the context of the development process for a system, they are also very relevant to the maintenance process. Indeed, the maintenance of a system is often performed by staff who are not familiar with all the thinking of the original design team, and hence will be unaware of some of the possible side-effects of the changes they might consider making. For this reason alone, the availability of a means of assessing the consequences of changes should prove invaluable for a maintenance team. (There is of course the inevitable corollary to this, in that some means of ensuring that the design description continues to match the implemented system will be required, and needs to be enforced.) There are obviously some significant issues surrounding the criteria that are used by Advisor in making its assessments about the quality of a design. In the rest of this paper we seek to describe how these are chosen, and the ways in which we have sought to minimize the subjective aspects of design assessment. MASCOT'S ROLE The Advisor subsystem has been based upon the MASCOT design representation for two principal reasons: MASCOT provides a well-defined design representation which is 'standard' within its particular user community, and which is not subject to significant local variations. There exists a substantial user community which has experience with implementing large systems with MASCOT, and which therefore can provide both a source of expert knowledge, as well as a source of opinions, about design quality measures and the weighting that should be ascribed to each of these for particular problem domains. MASCOT itself is oriented towards supporting the design and implementation of real-time systems. Its main components are the graphical design representation (the ACP diagram*), a set of constructional forms and the specification of a real-time executive. Together, these can be considered as providing a virtual machine environment that forms a layer of abstraction between the application system and the physical machine. The MASCOT formalism has undergone some changes, and the MASCOT 2 representation has been considerably developed and extended in producing the MASCOT 3 standard. However, the underlying virtual machine concepts are largely unchanged by these developments. This is an important point, since the relative newness of the MASCOT 3 standard has meant that the expert system aspects of Advisor have been based mainly upon the experiences that have been gained with the use of MASCOT 2. While the MASCOT user community is not particularly large, it is also concentrated mainly in one sector of the computing community, and so is relatively homogeneous. In the longer term it is hoped that the ideas *The ACP diagram describes a network based upon the three MASCOT system elements: activities, channels, pools (ACP).
236
incorporated into Advisor can be extended to handle other forms of design representation. DESIGN M E T R I C S The idea of metrics (forms of measurement) as applied to software-based systems originated largely with the work of the late Maurice Halstead, and in particular with his ideas about software science2. While the basis for much of this work has subsequently been questioned 3'4, it has nonetheless spawned a considerable volume of work in the area of metrics as applied to software. Most of this work has concentrated upon assessing code rather than design. There are good enough reasons for this, since a programming language possesses a relatively well-defined syntax and semantics, and so its form can be quantified and analysed. Various metrics have been developed, with some of the more successful being McCabe's Cyclomatic Complexity Measure 5'~ and the Information Flow ideas of Henry and KafuraL Unfortunately, the application of metrics to design representations has been less successful. There are a number of reasons why this has proved so, including: • the lack of any well-defined syntax and semantics for most design representations; • the lack of concensus about how to interpret such metrics as do exist, and the difficulty of assessing them by means of experiments; and • the lack of any good models of quality on which to base new metrics. However, the benefits which can be obtained from being able to detect likely weaknesses in a system at the relatively early stages of design, has meant that it has been well worth seeking to find ways of bettering our knowledge in this area. In developing Advisor, existing metrics were examined (used for both code and/or design assessment), and ways of quantifying existing thoughts about design quality were looked at. This is a large and complex topic, and is to be the subject of a separate survey document. Where possible, the ideas generated from this work are being incorporated into the design of Advisor. In particular, it should be emphasised that Advisor is only concerned with assessing the static qualities of a design. (Within the MDSE a separate set of tools are used for predicting performance and resource requirements.) The Advisor is also primarily concerned with assessing the logical form of a design, and does not concern itself with the process by which this was produced. So, it is not linked to any design method, nor are any particular design practices assumed in the metrics themselves.
EXPERIMENTAL F O U N D A T I O N S The relative lack of any strong theoretical foundation that can be used for identifying suitable measures of design quality requires that some form of experimental evidence is needed in order to support the construction of a system such as Advisor. Because of the nature of the problem there are several features that make the gathering of such data difficult. These include: Knowledge-Based Systems
• the relatively large scale and complexity of many of the systems of interest; • the lack of systematic record-keeping for many existing designs (including records of changes made during implementation); and • the lack of comparative design studies. Within the MDSE project it is considered that there are two useful sets of experiments that could be performed in order to help with the creation and validation of the rule-set needed in Advisor: 1 Obtaining a number of designs for a single problem, with these being produced by a mix of experienced and inexperienced designers. The designers would also be asked to provide details of their intermediate workings, and the reasons for any decisions that were involved in these. 2 Obtaining a number of designs for systems which have actually been implemented (and ideally, have also been subsequently modified or upgraded). These would provide a means of confirming the predictions of Advisor. The first set of experiments were begun at an early stage of the MDSE project, and were based upon two sample problems: first, an Air Traffic Control system (ATC); and second, a bank Auto-Teller Machine (ATM) network. A set of ten designs have now been obtained for the first of these problems, and an additional four designs for the second problem. These designs have been analysed extensively, and some of the conclusions are described below. The second set of experiments have yet to be performed, since they require the use of Advisor, and its rule-set is still only in a very prototype form. However, they are seen as an important part of the validation process. DESIGN M O D E L In the analysis of the design experiments described above, we sought to examine both the design process, and also the product of that process (the structure of the design that resulted from the process). The conclusions of this analysis will be the subject of other papers, as well as being a major input into the M A S C O T Designer's Guide which is being produced as a part of the MDSE study, and so they will only be outlined in this section. The MASCOT representation is very powerful, but it is also incomplete in terms of its ability to describe a design which is based upon the MASCOT virtual machine. Therefore, one major task was to examine the experimental data, and to extract from this the set of additional factors that were required in order to be able to model a design in full. Because what has emerged is essentially a scheme of classification for the features of a MASCOT design, we have chosen to term it as
a design taxonomy. The taxonomy is largely based upon the MASCOT system elements (activities, channels and pools), although it also places considerable emphasis upon the nature of the information that flows around a MASCOT system (and which has been loosely termed 'message types' for this purpose). It attempts to encapsulate the main features that a designer will manipulate in an Vol 1 No 4 September 1988
abstract fashion when thinking about a design, and when mentally modelling its operation s . The main purpose of producing the taxonomy has been to provide us with a means of describing the logical structure of a design in a suitably abstract form, and allowing this to be extracted from a description of the physical structure of a design. So the taxonomy provides a framework, within which we can seek to describe our ideas about the 'design rules' (see below). D E S I G N R U L E S AND C O N S T R A I N T S The role of the 'design rules' is to provide a way of applying a set of quality measures (metrics) to the logical description of a MASCOT design, as represented using the design taxonomy. The rules themselves can be classified in a number of ways, but we have found that the following provides a useful way of grouping the rules according to their origins and purpose. 1 Rules which are based upon MASCOT syntax and semantics. 2 Rules which are based upon design theory and upon ideas about quality metrics. 3 Heuristic rules that are based upon experience with the use of MASCOT, rather than upon any logical model ('rules of thumb'). We have preferred to term these constraints, for reasons that will be explained later. Before going on to describe the three classes of rule a little more fully, it may also be useful to consider the sources of material for these rules. These sources include: • the metrics study performed as part of MDSE; • the analyses of the design experiments; • discussions with experienced MASCOT designers; • our own experiences; • analyses of student design exercises; • comments received on our preliminary findings; and • initial experiences with the prototype Advisor. Obviously, this list is not complete, and it is also likely that the influence of the last item will increase as we begin to develop the Advisor subsystem beyond its prototype form. The rules which are based upon MASCOT syntax and semantics are in many ways the most direct and simple rules, since they are mostly 'grammar-based'. One example of a syntax-based rule would be: 'Activities may only be connected to channels, pools and device servers.' A rule of this type can be, and is, enforced by the graphics editor used within MDSE, and so such pure grammar rules are not currently included in the Advisor rule-base. However, there are a number of other features that can easily complicate even this class of rules, and which are included in the Advisor. One of these is the idea of 'normality', as expressed in the rule: 'An activity will normally take input from only one channel.' (this is concerned with the question of activation criteria). This type of rule is quite useful for picking up the kind of error that is easily made by an inexperienced designer, but it is really a rule which is based upon the idea of 'rules of form' rather than being based upon 237
the MASCOT grammar itself. As with all rules of form, there are occasions when it is appropriate to break such rules (although far less often than the inexperienced designer might think!), unlike those rules which are based solely upon the MASCOT grammar itself. The rules which are based upon design theory and quality metrics express ideas about such qualities as complexity, cohesion and coupling, information hiding, dependencies and parallelism, and because they are based upon fairly abstract concepts, these have proved difficult to quantify. While these rules will be the key component of Advisor, they are still evolving and will continue to do so indefinitely. They are also difficult to express, since a rule which is used to determine (e.g.) the extent of the data coupling occurring between an activity and the rest of a subsystem, may involve terms dealing with message types, IDA types, as well as the frequency and type of operations that are performed by the activity upon the different messages. As a first step we have tried to adapt some of those metrics which have already demonstrated their value for the assessment of code. A particular example is McCabe's Cyclomatic Complexity Measure v ( G ) s. This is based upon graph-theoretic modelling, and provides an assessment of the complexity of the flow of control within a segment of a program. Hall and Preiser 9 have also mapped this to a system of concurrent processes, and overall it is generally recognized as being a relatively well-established and useful measure. The basic measure is expressed as v (G) = e - n + p where e is the number of edges, n is the number of vertices, and p is the number of connected components. This measure was mapped to the MASCOT representation, and was used by 'hand' to assess a number of systems available, from which we have been encouraged to incorporate it into the prototype system in order to allow further experiment. Some extensions to the original form of interpretation have been needed in this, and the resulting form makes use of a wide range of information contained in the basic design taxonomy. The last group of rules have been described as constraints, since they are largely based upon observation and experience, and may be strongly influenced by local programming and design practices. An example of such a rule might be: 'The ratio of activities to channels should be Pl -bdl.' Since bothpl and dl may vary according to local practice or the type of application, these will need to be parameters of the Advisor rather than fixed values. An important point about the constraints is that their main effect is to limit the acceptable solution space for a design, rather than to provide any real assessment of the quality of the design. Because it is likely that a design will be developed in a piecemeal fashion, and over a number of iterative cycles, it is impractical to use the graphical editor to alert the user of any 'violations' of these constraint rules. This is partly because the editor may only have a small section of the design available at any time, and also because such warnings are not desirable while the design is still only partly complete. More significantly, these rules are included in Advisor because it is desirable to link their output with the output from the metric rules when the assessment of a design is finally presented to 238
the user. We expect that the output from the constraints rules may well prove to be of more interest to the project manager than to the designer, in many cases. DEVELOPMENTS
TO DATE
The MDSE project started in January 1986, and in January 1988 a prototype Advisor (containing a very elementary rule-set) was finally integrated with the other components. The complexity of the task of the Advisor means that it is really a subsystem containing a number of tools, although at present only two of these are operational to any extent. The first of these tools is a menu-based program which is used to help a designer to describe their intentions in terms of the design taxonomy. It has so far been termed the Formatter, but this name is now considered as no longer really appropriate and will need to be changed. The second tool is the rule-based assessment tool, the Assessor, which is based upon the Inference ART expert system shell. The initial rule-set being used in this is quite small and simple, but over the third year of the project it is expected that this will be expanded considerably. We will also be attempting to verify this tool, by such means as assessing actual systems that have been produced from MASCOT designs, as well as by using the results of the design experiments conducted earlier in the project. An important aspect of using such a system as the Advisor is to be able to interpret the results that are produced by using the Assessor. Because different users will have different requirements, including the ability to make an assessment of the overall quality of a design, and comparisons between the effects of two options in a design, there will almost certainly need to be at least two further tools to provide for the analysis of the output from the Assessor. However, the detailed design of these is awaiting greater experience with the use of the Assessor, and the development of a fuller set of rules. (The need to await experience with using the Assessor reflects the influence of the process of design. A tool of this nature needs to be able to fit into this, but in due course it may also be an influence that changes the actual process itself.) As can be seen, the Advisor subsystem is still evolving, and it involves us in pulling together a number of ideas about such issues as design quality, design metrics, the design process, and about the nature of the MASCOT virtual machine itself. The prototype Advisor is therefore not only the product of an experimental process, but is also an experimental tool in itself, and one that we expect to use extensively for the remaining part of the MDSE project. ACKNOWLEDGEMENTS The MDSE project is funded by the Alvey Directorate, and the partners are Ferranti Computer Systems Ltd., YARD Ltd., LOGSYS and the University of Stirling, UK.
REFERENCES 1 l E E Software Eng. J. Special issue on MASCOT 3,
Vol 1 (May 1986) Knowledge-Based Systems
2 Halstend, M Elements of Software Science Elsevier, The Netherlands, (1977) 3 Coulter, N S 'Software science and cognitive psychology' IEEE Trans. Software Eng. Vol SE-9 (March 1983)pp 166-171 4 Conte, S D, Dunsmore, H E and Shen, V Y Software Engineering Metrics and Models Benjamin/Cummings, USA (1986) 5 MeCabe, T J 'A complexity measure' IEEE Trans. Software Eng. Vol SE-2 (December 1976) pp 308-320 6 Myers, G J 'An extension to the cylomatic measure
Vol 1 No 4 September 1988
of program complexity' ACM SIGPLAN Notices (October 1977) pp 61-64 7 Henry, S and Kafura, D 'The evaluation of software systems structure using quantitative software metrics' Software Pract. & Exper. Vol 14 (June 1984) pp 561-573 8 Adelson, B and Soloway, E 'The role of domain experience in software design' 1EEE Trans. Software Eng. Vol SE-11 (November 1985) pp 1351-1360 9 Hall, N R and Preiser, S 'Combined network complexity measures' IBM J. Res. & Dev. Vol 28 (January 1984)
239