A tool for visual and formal modelling of software designs

JID:SCICO AID:1763 /FLA

[m3G; v 1.134; Prn:26/05/2014; 16:00] P.1 (1-28)

Science of Computer Programming ••• (••••) •••–•••

Contents lists available at ScienceDirect

Science of Computer Programming www.elsevier.com/locate/scico

A tool for visual and formal modelling of software designs Nuno Amálio a,∗ , Christian Glodt b a b

Escuela Politécnica Superior, Engeniería Informática, Universidad Autónoma de Madrid, 28049 Madrid, Spain Faculty of Science, Technology and Communication, University of Luxembourg, L-1359 Luxembourg

h i g h l i g h t s • • • • •

The VCL VCL The The

paper presents the Visual Contract Builder (VCB) tool supporting the Visual Contract Language (VCL). is a graphical language for describing software designs formally. and VCB have been applied to several case studies. paper evaluates VCB based on a survey carried out in the context of a controlled experiment. paper includes several reflections on strengths and weaknesses of VCB and lessons learnt.

a r t i c l e

i n f o

Article history: Received 20 November 2012 Received in revised form 5 April 2014 Accepted 7 May 2014 Available online xxxx Keywords: Visual modelling languages MDE Formal methods Tool-support Empirical evaluation

a b s t r a c t Diagrams are ubiquitous in software engineering and widely used for software modelling. The visual contract language (VCL) enables an approach to software design modelling that is entirely graphical and has a mathematical basis. VCL’s main novelties lie in its capacity to describe predicates visually and in its graphical front-end to formal modelling. VCL is brought to life in the visual contract builder (VCB) tool presented in this paper. VCB provides diagram editors for the whole VCL suite, it type-checks diagrams and generates Z formal specifications from them; the Z specification enables formal verification and validation using Z theorem provers. The paper evaluates VCB based on the results of a survey carried out in the context of a controlled experiment. The work presented here is a contribution to the area of visual design modelling: the paper presents a state of the art tool supporting the novel VCL language and concrete empirical results on the usefulness and effectiveness of tool and language in particular, suggesting benefits of visual modelling in general. © 2014 Elsevier B.V. All rights reserved.

1. Introduction

The prevalence of visual notations in software engineering (SE) can be explained by certain properties of diagrams that benefit cognition [1,2]. In the broad field of SE and in the specific field of software modelling, the “visual” is the most widespread representation [2]. This SE trend is entirely consistent with the ubiquity and importance of visual representations in traditional engineering [3,4]. Despite this prominence, mainstream visual languages for software design modelling that are driven by SE practice, such as the industry standard UML [5,6] and its predecessors, OMT [7], Booch [8] and others [9], have certain drawbacks:

*

Corresponding author. E-mail addresses: [email protected] (N. Amálio), [email protected] (C. Glodt).

http://dx.doi.org/10.1016/j.scico.2014.05.002 0167-6423/© 2014 Elsevier B.V. All rights reserved.

JID:SCICO AID:1763 /FLA

[m3G; v 1.134; Prn:26/05/2014; 16:00] P.2 (1-28)

N. Amálio, C. Glodt / Science of Computer Programming ••• (••••) •••–•••

2

• UML and similar languages have not been designed to be cognitively effective [2]. UML, for instance, is known to breach many principles of visual language design [2].

• UML and similar languages have several semantic issues. The semantics of UML, its accompanying constraint language OCL, and similar languages that preceded UML, have been, over the years, criticised for vagueness, ambiguity and imprecision [10,9,11–19]. This is attributed to the definitions of semantics in these languages, which are known to lack precision and formality [10,9,17,19]. There have been many formalisations of UML to date [20–27], but they only cover subsets of UML; no formalisation or formal framework covering the whole UML has been produced to date. This suggests that UML-like languages are appropriate for sketching, but not satisfactory for precise and rigorous modelling, due to their many semantic interpretations and lack of firm mathematical foundations. A consequence of this, is that consistency defects are a serious problem [28–30]. Furthermore, the task of model analysis is severely impaired as these languages do not provide means for analysing a model mechanically and exhaustively, using theorem-proving or model-checking, because their semantics is not formally-defined. • UML and similar languages cannot describe all properties graphically. UML, for instance, uses the textual OCL to describe contracts and invariants. • There is some dissatisfaction with the diagrams provided by UML-like languages for describing behaviour [15–17,31]. Descriptions of behaviour provided by UML collaboration and sequence diagrams are partial as they describe scenarios through interactions between instances; UML state and activity diagrams provide total descriptions, but their emphasis on explicitly defined states and transitions between these states may be appropriate for reactive systems, but they fall short of meeting the requirements of more general information systems. There is another group of SE modelling languages that puts a strong emphasis on semantics. The so-called formal methods (FMs) [32,33] are rigorous, precise and have sound semantic definitions; their mathematical-based semantics enables mechanical and exhaustive verification and validation. FMs, however, are not without shortcomings:

• FMs are seen as difficult to use as they require a considerable amount of expertise; their entry costs are seen as a barrier for their adoption in industry [33,34].

• Despite a considerable number of success stories, FMs have not seen a widespread adoption in terms of industrial practice [33]. Although there has been substantial progress in recent years, the onus of formality does not appear to justify their general uptake [35,34]; the effort and expertise that they require appears to be justified only in domains where cost of software error is very high, such as the safety-critical niche [36,34,33]. • From a market perspective, current FMs do not appear to offer a business advantage in today’s competitive software industry where time to market is a main competing factor [37,38]. The Visual Contract Language (VCL) [39–41], a general-purpose object-oriented (OO) modelling language for the abstract graphical description of software designs with a formal semantics, tries to address the above-mentioned drawbacks of existing SE modelling languages. On the one hand, VCL aims at improving mainstream UML-like languages: (a) it introduces rigour, precision and means for mechanical and exhaustive model analysis, (b) it extends the realm of what can be described visually, and (c) it improves existing graphical representations for the description of behaviour. On the other hand, VCL aims at improving the practicality of FMs by using visual descriptions as a front-end for sound mathematical approaches; this is done to (a) move the onus of formality from the user (the engineer) to the designer of the modelling language, (b) tap into knowledge and education of SE practitioners on OO modelling, (c) mitigate the need for expertise in formal methods and (d) enable engineers with diverse backgrounds to participate in a common modelling effort. VCL tries, therefore, to contribute to both model-driven engineering (MDE) [42] using visual languages and FMs. It addresses the MDE side of the equation by proposing an approach to modelling that is entirely visual providing novel diagram types that do graphically what UML does textually with OCL. The FMs side of the equation is addressed by enabling the generation of formal specifications from VCL models, which can be subject to exhaustive verification and validation. Like UML, VCL follows an OO style of description, expressing a software system as a collection of data and associated behaviour. It supports, however, a more abstract approach to modelling that is closer to set theory and is inspired in formal modelling languages based on set theory, such as Z [43,44], B [45] and Alloy [46]; this is done without breaching the compatibility with UML. VCL’s distinguishing feature lies in its capacity to express predicates visually, which enables a graphical approach to behavioural modelling based on design by contract [47]. In addition, VCL tackles the modelling of large and complex systems with an approach to separate concerns based on coarse-grained modularity mechanisms to enable the definition of modules to represent requirements and design concerns. VCL builds up on previous work that combines UML and Z [26,27,48] to introduce rigour in UML-based development and is compatible with UML-based development processes. This paper presents the tool that brings VCL to life: the visual contract builder (VCB).1 Together, VCL and VCB, establish a symbiotic relationship that is important to address the needs of software modelling: better modelling techniques and tools to support them [49]. A prototype version of VCB, which only supported two diagram types of VCL (structural and assertion

1

http://vcl.gforge.uni.lu.

JID:SCICO AID:1763 /FLA

[m3G; v 1.134; Prn:26/05/2014; 16:00] P.3 (1-28)

N. Amálio, C. Glodt / Science of Computer Programming ••• (••••) •••–•••

3

diagrams), appears in [40]. This paper presents a more complete version of VCB, supporting the whole VCL modelling suite (comprising five diagram types) and evaluates VCB based on a user survey conducted in the context of a controlled experiment to compare VCL against UML and OCL [50]. 1.1. Contributions This paper’s contributions are as follows:

• The VCB tool supporting a novel software design language that is entirely visual and has a formal basis. VCB provides diagram editing facilities, consistency-checking through type-checking to give instant feedback to users on model errors and generation of Z specifications that can be processed independently with the purpose of formal verification and validation using theorem proving. To our knowledge, no other software design modelling tool achieves this level of support for such a design modelling language that expresses predicates graphically. • An evaluation of the VCB Eclipse-based tool through a comparison with a popular UML Eclipse-based tool based on a user survey done in the context of a controlled experiment. This is, to our knowledge, the first evaluation of visual design modelling tools involving rigorous statistical testing of hypothesis. Although the results of this evaluation only apply to the specific evaluated tools and the specific versions used in this evaluation, they tell us something about the quality of both VCB and VCL in particular, and graphical approaches to modelling in general. 1.2. Paper outline The remainder of this paper starts by giving some background on VCL (Section 2). Then, it introduces VCB and its functionalities (Section 3), gives some instructions on how to download the VCB tool and how to get started with VCL using VCB (Section 4), describes the actual implementation (Section 5) and discusses the practical application that both VCL and VCB have seen so far (Section 6). Finally, the paper evaluates VCB based on a user survey (Section 7), discusses the presented results (Section 8), makes a comparison with related work (Section 9) and concludes with a summary of what is presented (Section 10). 2. Background: the visual contract language (VCL) VCL [39,41,40] is a language to model software designs visually.2 VCL’s designs are formal because they have an underlying mathematical semantics expressed in the formal specification language Z [43,44]. VCL evolved from its predecessor, the UML+Z framework [26,27,48] for rigorous software design modelling, which combines UML and Z. VCL and UML+Z have the same semantic domain: ZOO [25,27], an approach to write OO models in Z. Both VCL and UML+Z define the semantics of diagrams through a mapping into ZOO. A significant difference is that VCL describes in graphical form invariants and operations that are expressed textually using Z in UML+Z. Appendix A gives more detailed background on VCL using the model of the secure simple bank case study from [54]. The following gives a brief description of VCL’s diagram types. 2.1. VCL diagram types A VCL model is made up of a set of interrelated diagrams of different types; each diagram describes one specific portion of the overall system. Several specimens of the different diagram types of VCL are given in Appendix A. VCL’s diagram suite comprises the following types:

• Package diagrams (PDs) define packages, coarse-grained modules, and their dependencies with other packages (e.g. PDs of Fig. A.10). Packages are VCL’s highest-level structures and main coarse-grained unit of separation of concerns; a VCL model is organised around them. A VCL package comprises one PD, one structural diagram (SD), one behaviour diagram (BD), and several assertion and contract diagrams. VCL’s package composition mechanisms enable the construction of larger packages from smaller ones. • SDs, akin to UML class and entity-relationship diagrams, define the state space of a package (e.g. SDs of Fig. A.11). They provide the big picture over the structural elements of a package. An SD contains state structures representing concepts, relations between concepts, and invariants that constrain the state space. The invariants included in an SD are explicit references to their definitions in separate assertion diagrams (ADs). • BDs identify the operations of a package (e.g. BDs of Fig. A.12), providing a big picture over the units that make a package’s overall behaviour. Each operation included in a BD is a reference to a separate definition in an AD or contract diagram (CD), depending on whether the operation is a query or a mutator (an operation that changes state).

2

Further information on VCL is available at http://vcl.gforge.uni.lu/.

JID:SCICO AID:1763 /FLA

[m3G; v 1.134; Prn:26/05/2014; 16:00] P.4 (1-28)

N. Amálio, C. Glodt / Science of Computer Programming ••• (••••) •••–•••

4

Fig. 1. Screenshot of VCB Eclipse plugin highlighting editor of assertion diagrams.

• ADs define predicates over a single state. They are compositional: it is possible to build larger ADs from smaller ones. ADs are used to define invariants and query operations (e.g. ADs of Figs. A.13 and A.14).

• CDs describe operations that change state as contracts made up of a pre- and a post-condition (e.g CDs of Fig. A.15), which are predicates over a before and an after state. Like ADs, CDs are also compositional: it is possible to build CDs from other ADs and CDs. CDs are used to describe operations that change state (mutators). 3. The visual contract builder tool The visual contract builder (VCB) is VCL’s Eclipse-based tool. It provides the following functionalities: (a) editing of VCL diagrams, (b) type-checking of diagrams for checking well-formedness and meaningfulness, and (c) generation of Z specifications from VCL diagrams. A screenshot of VCB is given in Fig. 1. 3.1. Diagram editing VCB’s diagram editors enable the construction of VCL diagrams using typical actions of graphical user interfaces, such as drag and drop, mouse hovering and point and click. All VCL diagrams of Appendix A were drawn using VCB. Screenshot of Fig. 1 highlights the diagram editor for ADs. 3.2. Type-checking VCB type-checks VCL diagrams to help users in building consistent and meaningful models. This is based on VCL’s type system defined in [55], which allows errors to be expressed in terms of diagrams providing valuable feedback to the user. Fig. 1 illustrates type-checking error reporting in VCB; AD highlighted in Fig. 1 is a modified version of Fig. A.13d with an error that is signalled to the user with error markers. By hovering the mouse over the marker, the user gets further information on the signalled error.

JID:SCICO AID:1763 /FLA

[m3G; v 1.134; Prn:26/05/2014; 16:00] P.5 (1-28)

N. Amálio, C. Glodt / Science of Computer Programming ••• (••••) •••–•••

5

Fig. 2. Architecture of the visual contract builder tool. (Each box represents a component; Double-sided arrows represent interaction.)

3.3. Z generation VCB generates Z specifications from VCL diagrams. An excerpt of the generated Z for the secure simple bank VCL model presented in Appendix A is given in Appendix B; the complete Z specification can be found in [54]. 4. Installation instructions and getting started VCB is available for download from the VCL web-site.3 Detailed download instructions can be obtained by clicking the downloads tab of the web-site.4 VCB is installed on the Eclipse platform like any other Eclipse plug-in. It has been tried on the Indigo, Juno and Kepler versions of Eclipse. The VCL web-site includes several artifacts to help the user getting started with both VCL and VCB:

• The video demo provides a quick introduction to VCB. • The VCL/VCB tutorials, available from the Tutorials tab, gives detailed instructions on how to build VCL models using VCB.

• Sample VCL models of toy case studies, which are available as ‘.zip’ files from the tutorial web pages. Once downloaded, the sample models can be imported into Eclipse. 5. Implementation VCB is an Eclipse-based tool that provides an environment to build and manipulate VCL diagrams. Fig. 2 presents VCB’s overall architecture highlighting its major components that interact to provide the different VCL related functionalities. Next sections overview VCB as a whole and describe its different components. 5.1. Overview VCB uses two representations of VCL diagrams: (a) one based on eclipse modelling framework (EMF) metamodels and (b) another based on text whose form is described using grammars. VCL is essentially a diagrammatic language, but there are several VCL constructions that are textual and, hence, require textual parsing; their form is better described as grammars. VCL’s type-checking is defined on grammars. As metamodel and grammar representations are equivalent, it is trivial to go from one representation to the other. VCB’s main components (Fig. 2) reflect this dual representation:

• DiagramEditing is responsible for the construction and manipulation of VCL diagrams. • VCLProcessor processes the textual representation of VCL diagrams in order to type-check VCL diagrams and generate Z from them.

• DiagTxtMediator mediates the interaction between DiagramEditing and VCLProcessor.

3 4

http://vcl.gforge.uni.lu. http://vcl.gforge.uni.lu/download.html.

JID:SCICO AID:1763 /FLA

[m3G; v 1.134; Prn:26/05/2014; 16:00] P.6 (1-28)

N. Amálio, C. Glodt / Science of Computer Programming ••• (••••) •••–•••

6

Table 1 Total lines of Java code in VCB at time of writing (counted by sloccount tool). Diagram editing

DiagTxtMediator

VCLProcessor

Generated editors Custom frameworks Total DiagTxtMediator Total Generated parsers Custom code of Type-Checker and Z Generator Total

270 090 20 815 290 905 6154 6154 7405 14 181

Total VCB

21 586 318 645

Fig. 3. Interaction between components involved in the live error-feedback funtionality of the Visual Contract Builder tool.

Table 1 presents the number of lines of Java code for each VCB component and the whole VCB. The counts were produced using the sloccount tool. Most Java code of VCB is automatically generated (diagram editors and generated parsers), making a total of 87% of generated code. Diagram of Fig. 3 describes how the different VCB components interact to bring about VCB’s live error-feedback functionality to notify users of errors as they edit diagrams. The interaction is as follows: 1. As the user edits diagrams, a basic validation is performed at the level of the GMF diagrams. The detected errors are reported to the user. 2. If no errors are detected during basic validation, the diagrams are then transformed into text by component DiagTxtMediator. 3. The text representing the diagrams is parsed in component VCL Processor; an abstract syntax tree representation is built; any parsing errors are reported to the user. 4. Finally, if there are no parsing errors, diagrams are type-checked. Errors found during type-checking are reported to the user through the DiagTxtMediator component. 5.2. Diagram editing VCB’s diagram editing facility consists of two main sub-components: Generated Editors and Custom Frame-

works. 5.2.1. Generated editors This consists of VCB’s diagram editors that are generated from VCL’s metamodels [55] using GMF. Each diagram type has its own metamodel; the implementation stores metamodels as instances of Ecore; the graphical editors are generated from these Ecore metamodels. 5.2.2. Custom frameworks This component provide low-level base classes that observe the state of each graphical editor, following the observer pattern [56]. Built upon these base classes, there are higher-level abstractions allowing the specification of, for instance, conditional visibility of diagram elements, double-clicking behaviour, custom synchronisation between model state and diagram state, migration of instances of previous versions of metamodels to newer ones, and so on. The customisations for each VCL diagram type are grouped into a separate plugin.

JID:SCICO AID:1763 /FLA

[m3G; v 1.134; Prn:26/05/2014; 16:00] P.7 (1-28)

N. Amálio, C. Glodt / Science of Computer Programming ••• (••••) •••–•••

7

Table 2 Some data about releases, downloads and usage of the VCB tool. 1st release

February 2011

Number of downloads since 20.02.2012 Number of students that, to our knowledge, used VCL and VCB for practical work

220 52

Table 3 Summary data of VCL case studies. (No of structural modelling units gives total number of modelling units inside all the model’s packages.)

No of packages No of structural modelling units No of developers Built using VCB

CCCMS [41]

PM [51]

SSB [54]

bCMS [53]

42 415 3 No

17 178 2 Yes

12 76 1 Yes

11 105 1 Yes

5.3. DiagTxtMediator This component is responsible for: (a) producing the textual representation of VCL diagrams to feed the VCL Processor component, and (b) taking the errors produced by type-checking in the VCL Processor and assigning them to relevant diagram elements so that they can be presented to the user. The transformation from metamodels to text is done following a variation of the visitor pattern [56]. Every node of the diagram is visited to produce the corresponding text. 5.4. VCL processor The VCL processor component is further divided into sub-components: Generated Parsers, Type-Checker and Z Generator. 5.4.1. Generated parsers VCL Processor uses a textual representation of VCL diagrams whose form is described using grammars [55]. Component Generated Parsers provides textual parsers to process these textual representations. The parsing results in an abstract syntax tree (AST), from which VCB does both type-checking and Z generation. The Java code for the parsers of each VCL diagram type are generated from grammars using the JFlex scanner generator5 and CUP parser generator6 technologies. 5.4.2. Type checking The type-checker implements the type inference rules defined in [55]. It is in fact a refinement of the VCL type system of [55]. The defined type system has a partial semantics: if a phrase is well-typed, the rules are able to determine so; otherwise there is a point when no rules can be applied and the type derivation is undefined. The implementation removes this undefinedness by considering that this situation is an error; VCL expressions are assigned the type Null when it is not possible to assign a valid type to them. The implementation follows the visitor pattern [56]. It visits every node in the AST to determine its type (if any) and detects errors. When an error is found, a well-formedness error is created; this indicates where the error has been detected and includes a message describing the problem. 5.4.3. Z generator The Z generator also follows the visitor pattern [56]. It visits every node in the AST to produce the corresponding Z text, using type information derived by the type-checker along the way. Once the generation is complete, the Z is type-checked using the CZT [57]7 type-checker. This is a mere sanity check used for debugging purposes because all type errors ought to be captured during VCL type-checking. 6. VCL and VCB in practice VCL and VCB have had several applications. Experimentation with the language in order to obtain feedback so that both language and tool could be improved has been an integral part of the process of designing and developing both VCL and VCB. Table 2 presents some data on releases, tool downloads and usage of VCL and VCB. VCL and VCB have been applied to several case studies and used in the context of student projects. Some of the case studies to which VCL and VCB have been applied are challenges proposed by research communities. Table 3 presents summary data concerning the main case studies to which VCL has been applied. The case studies are as follows:

5 6 7

http://jflex.de/. http://www2.cs.tum.edu/projects/cup/. http://czt.sourceforge.net/.

JID:SCICO AID:1763 /FLA

[m3G; v 1.134; Prn:26/05/2014; 16:00] P.8 (1-28)

N. Amálio, C. Glodt / Science of Computer Programming ••• (••••) •••–•••

8

• The simple bank case study coming from UML+Z [27,26] has been used throughout the development of VCL and VCB. It includes constraints that are challenging to represent graphically. It is used as a VCB tutorial.8

• The secure simple Bank (SSB) case study used in this paper extends simple bank with security concerns, such as •

• • •

authentication and access control. It is our toy case study for experimenting with VCL’s coarse-grained modularity mechanisms. The VCL model of SSB is given in [54]. SSB is also used as a VCL/VCB tutorial. The first large case study to which VCL has been applied is the car-crash crisis management system (CCCMS) [41]. CCCMS was proposed as a challenge for aspect-oriented modelling approaches. It was in the context of the VCL effort associated with CCCMS that VCL’s mechanisms for coarse-grained separation of concerns were developed. VCB did not exist when CCCMS was modelled in VCL: the diagrams were drawn using a drawing tool. The Pacemaker (PM) is an industrial case study of a pacemaker medical device that was released into the public domain by Boston Scientific. It is a benchmark case study of the software verification challenge [33]. The VCL model, constructed using VCB, is presented in [52,51]. The Barbados crisis management system (bCMS) is a case study that was proposed to compare modelling approaches. It is a reduced and a more focused version of CCCMS. The VCL model of bCMS, constructed using VCB, is given in [53]. In collaboration with the Henri Tudor applied research centre, we have built a VCL model using VCB of a tangible user interface (TUI) application [58]. TUIs use real-world objects placed on a table to interact with the digital world.

7. Evaluation The adequacy of VCL and VCB to model software designs has been demonstrated by their application to several problems (Section 6). This section presents an empirical evaluation of VCB based on a user survey. VCB’s goals are to fully support VCL’s graphical modelling approach. We have undertaken a controlled experiment to compare VCL against the industry standards UML and OCL [50]. As part of this experiment, we conducted a survey to compare the tools used in the experiment: VCB and Papyrus UML.9 This is the basis of the VCB evaluation presented here. 7.1. The controlled experiment The controlled experiment [50] was designed to evaluate the effectiveness of VCL for improving the user performance on a set of common tasks associated with software design modelling when compared to UML and OCL. This took into account two different perspectives: construction of models and model usage. The experiment asked students of software engineering and computer science degrees to perform several design modelling related tasks using both VCL and UML+OCL. The modelling construction tasks comprised: (a) construction of a structural model, (b) construction of a behavioural model and (c) a problem comprehension questionnaire based on the modelling activity. The model usage tasks comprised: (a) defect detection and (c) a model comprehension questionnaire based on a given model. Subjects were given two to three hours training in VCL and UML+OCL with the supporting tool (4 to 6 hours training in total). The time spent on the experiment’s tasks amounted to 8 to 9 hours. 7.2. Recruitment and experiment execution The experiment was executed in four different venues: (a) Faculty of Sciences of the University of Lisbon (Portugal), (b) Faculty of Science and Technology of the New University of Lisbon (Portugal), (c) University of Luxembourg and (d) University of York (UK). Software engineering and computer science students were recruited from student mailing lists of these universities; university lecturers advertised the experiment in relevant classes. Students were rewarded with A C50 vouchers for taking part in the experiment. The goal was to involve computer scientists and software engineers; subjects were required to have completed or be in the process of completing a course on object-oriented modelling using UML. The experiment’s training intended to complement the existing education of the participants to ensure that they would be able to carry out the tasks of the experiment competently. There were 43 participants in total from all four universities. 7.3. The survey One week after the experiments’ tasks were complete, subjects were asked to complete two surveys: (a) a debriefing survey concerning the actual experiment comparing the languages VCL against UML and OCL, and (b) a VCB survey. It is the results of the latter survey that we document here. In total, 42 of the 43 subjects completed the VCB survey. The VCB survey evaluates the effectiveness of VCB in supporting VCL and the design modelling activity. It consisted, mostly, of a comparison with Papyrus, the UML tool used in the experiment. Papyrus is a popular Eclipse plug-in, integrated in the Eclipse distribution and available from Eclipse marketplace; it serves as a benchmark for the purpose of comparison and evaluation. In the experiment, both VCB and Papyrus were running on the Juno version of Eclipse.

8 9

Available at http://vcl.gforge.uni.lu/tutorials.html. http://www.eclipse.org/papyrus/.

JID:SCICO AID:1763 /FLA

[m3G; v 1.134; Prn:26/05/2014; 16:00] P.9 (1-28)

N. Amálio, C. Glodt / Science of Computer Programming ••• (••••) •••–•••

9

Table 4 Sections of the VCB survey questionnaire, and corresponding research questions and dependent variables. Section

RQ

Description

Usefulness

RQ1

Eleven questions for each tool on a seven-level Likert scale from 1 (strongly agree) to 7 (strongly disagree). Dependent variable: U (Usefulness).

Ease of Use

RQ2

Seventeen questions for each tool on a seven-level Likert scale from 1 (strongly agree) to 7 (strongly disagree). Dependent variable: EoU (Ease of Use).

Usability

RQ3

Closed questions corresponding to usability criteria with alternatives: VCB, Papyrus UML and No Preference. Dependent variables: W (writing or sentence construction), R (reading), N (navigation through model artifacts), E (model editing), O (maps and overviews), LEC (live error checking), LaF (look and feel), C (cohesion between parts), CU (clarity and understandability), CS (comfort/satisfaction), Eng (engagement), Doc (documentation), Rel (reliability), OU (overall usability).

Support for modelling

RQ4

Closed questions with alternatives: VCB, Papyrus UML and No Preference. Dependent variables: MS (modelling the state space), MI (modelling invariants), MO (modelling operations), MU (model usage tasks) and PT (Preferred tool).

Pros and cons

RQ5

Two open questions (free text) asking users to list three positive and three negative aspects of VCB, respectively. Dependent variables: Pos (positive aspects), Neg (negative aspects).

The design of the survey followed standard techniques for phrasing subjective questions and designing survey questionnaires [59] to avoid bias and to increase the reliability of the responses. The survey is driven by the following research questions (RQs):

• • • • •

RQ1. Is VCB perceived as more useful than Papyrus UML? RQ2. Is VCB perceived as more easy to use than Papyrus UML? RQ3. Is VCB perceived as being better than Papyrus UML at different usability criteria relevant to such design modelling tools? RQ4. Is VCB perceived as providing better support for software design modelling tasks than Papyrus UML? RQ5. What are VCB’s perceived positive and negative aspects?

RQ1 and RQ2 above are based on the notions of perceived usefulness and ease of use [60]. Perceived usefulness is the degree to which a person believes that using a particular system would enhance his or her performance in some task. Perceived ease of use is the degree to which a person believes that using a particular system would be free of effort. Both RQ1 and RQ2 are evaluated based on an aggregate score. RQ3 evaluates individually each usability criterion, such as reading and sentence construction. Table 4 outlines the structure of the survey questionnaire. It relates the structure of the survey questionnaire with corresponding RQs and underlying dependent variables; each RQ has its own section in the questionnaire. The questionnaire proper is available at http://bit.ly/1b5CM0i. 7.4. Variables and hypothesis Table 4 gives the dependent variables under scrutiny in the survey and the corresponding research question. Research questions RQ1 to RQ4 are to be analysed quantitatively using hypothesis testing techniques. For each of the 21 variables associated with research questions RQ1 to RQ4 of Table 4, we state the null (H 0 ) and alternative hypothesis (H a ). The null hypothesis states that there is no difference in the evaluated modelling tools; the alternative hypothesis says that VCB is better than Papyrus. Due to the nature of the data, the hypothesis for RQ1 and RQ2 (numerical data) are slightly different from those of RQ3 and RQ4 (nominal or categorical data). For RQ1 and RQ2, given a variable V representing either U or EoU (Table 4), the hypothesis are as follows:

H i0 :

V (VCB) = V (Papyrus)

H ai :

V (VCB) > V (Papyrus)

The dependent variables of RQ3 and RQ4 (see Table 4) have three values: no preference, VCB and Papyrus. The null hypothesis is that all these values have the same frequencies:

H i0 :

V (VCB) = V (Papyrus) = V (No_Preference)

The alternative hypothesis is that the VCB alternative has a higher frequency than the other two:

H ai :

V (VCB) > V (Papyrus) ∧ V (VCB) > V (No_Preference)

JID:SCICO AID:1763 /FLA

10

[m3G; v 1.134; Prn:26/05/2014; 16:00] P.10 (1-28)

N. Amálio, C. Glodt / Science of Computer Programming ••• (••••) •••–•••

Table 5 Results of hypothesis testing. Cell colours: confirmed alternative hypothesis (shaded; green in the web version), null or other hypothesis (white). Statistical significance: * significant with α = .05, ** α = .01, *** α = .005. Variables: U = Usefulness, EoU = Ease of use, W = writing, R = Reading, N = Navigation, E = Editing, O = Maps/Overviews, LEC = Live error checking, LaF = Look and Feel, C = Cohesion, CU = Clarity/Understandability, CS = Comfort/Satisfaction, Eng = Engagement, Doc = Documentation, Rel = Reliability, OU = Overall Usability, MS = Modelling the state space, MI = modelling of invariants, MO = Modelling of operations, MU = model usage, PT = overall preferred tool.

7.5. Results The results of hypothesis testing are summarised in Table 5. The alternative hypothesis that have been confirmed, indicating a significant difference in favour of VCB, are coloured in green. The p-values that are used to confirm or reject the hypothesis of Table 5 were calculated using either Wilcoxon or Chi-squared tests; Wilcoxon is preferred to the paired t-test as it is non-parametric, and, hence, not restricted by the t-test assumptions. Some results of the Chi-squared tests indicated a significant difference not in favour of VCB (non-green rows with *s in Table 5), but favouring the no preference alternative. The statistical analysis was carried out using the R statistical software [61]. The following discusses the results for each RQ. 7.5.1. RQ1. Is VCB perceived as more useful than Papyrus UML? Fig. 4a presents two sample questions enquiring on usefulness of VCB; there would be two corresponding questions for Papyrus. From the responses to the usefulness questions, we computed a score from 1 to 7 to reflect the Likert scale; the weight 7 corresponds to strongly agree if the question is positive or strongly disagree if the question is negative. The actual values of the usefulness variable (U ) are score rates (total score divided by max score). The usefulness survey results are summarised in the two leftmost box plots of Fig. 5. The mean of VCB’s usefulness rate was 71.8% (55.3/77, standard deviation = .1331) and that of Papyrus was 69.1% (53.2/77, sd = .1550) with a mean of differences (U (VCB) − U (Papyrus)) of about 2.7%. This difference is not significant (Wilcoxon test p-value = .2237, t-test p-value = .2764). 7.5.2. RQ2. Is VCB perceived as being more easy to use than Papyrus UML? Fig. 4b presents two sample questions enquiring on ease-of-use of Papyrus; there would be two corresponding questions for VCB. From the responses to the ease-of-use (EoU) questions, we computed a score from 1 to 7 to reflect the Likert scale (as the one for usefulness, see above). The results are summarised in the two rightmost box plots of Fig. 5. The mean of VCB’s ease-of-use score rate was 62.5% (74.3/119, sd = .1101) and that of Papyrus was 56.9% (67.6/119, sd = .1318), with a mean of differences of about 5.7%. This difference was found to be significant (Wilcoxon test p-value = .01512 < .05, t-test p-value = 0.0309 < .05). 7.5.3. RQ3. Is VCB perceived as being better than Papyrus UML at different usability criteria relevant to such design modelling tools? Fig. 6 summarises the results of the usability section in the survey questionnaire, comparing perceived usability of VCB against Papyrus. Respondents had three options for each usability criterion: no preference, VCB or Papyrus.

JID:SCICO AID:1763 /FLA

[m3G; v 1.134; Prn:26/05/2014; 16:00] P.11 (1-28)

N. Amálio, C. Glodt / Science of Computer Programming ••• (••••) •••–•••

11

Fig. 4. Sample questions of the VCB questionnaire.

Fig. 5. Box plots describing distributions of usefulness and ease of use (* = mean, o = outlier, middle bar in boxes = median).

Papyrus was the most voted option for writing (W), or sentence construction, but without a significant difference (χ 2 ’s test p-value = .7515 > .05). This was the only criterion in which Papyrus outranked the remaining options of the questionnaire.

JID:SCICO AID:1763 /FLA

12

[m3G; v 1.134; Prn:26/05/2014; 16:00] P.12 (1-28)

N. Amálio, C. Glodt / Science of Computer Programming ••• (••••) •••–•••

Fig. 6. Number of people that selected either No Preference, Papyrus or VCB for each usability criteria of the VCB survey questionnaire. (W = writing, R = Reading, N = Navigation, E = Editing, O = Maps/Overviews, LEC = Live error checking, LaF = Look and Feel, C = Cohesion, CU = Clarity/Understandability, CS = Comfort/Satisfaction, Eng = Engagement, Doc = Documentation, Rel = Reliability, OU = Overall Usability.)

Fig. 7. Frequency of choices No Preference, Papyrus or VCB in the questions of the modelling section of the VCB questionnaire.

VCB was the most voted option for reading (R), navigation (N), editing (E), maps and overviews (O), live error checking (LEC), look and feel (LaF), cohesion (C), clarity and understanding (CU) and comfort/satisfaction (CS). Of these, R, N, LEC and LaF were found to be significant.10 The alternative no preference was the most voted for engagement (Eng), documentation (Doc), reliability (Rel) and overall usability (OU). The differences for Eng, Doc and Rel were found to be significant.11 7.5.4. RQ4. Is VCB perceived as providing better support for software design modelling tasks than Papyrus UML? Fig. 7 summarises the results of the modelling section in the survey questionnaire. VCB was the most voted alternative for variables MS (modelling the state space), MI (modelling state invariants), MO (modelling of operations using pre- and post-conditions), and PT (preferred modelling tool). The differences for MI and MO were significant.12 The difference in favour of the no preference option for MU (model usage) is also significant. 7.5.5. RQ5. What are VCB’s perceived positive and negative aspects? Having demonstrated VCB’s benefits in comparison to Papyrus quantitatively, we now turn to a qualitative analysis of VCB’s perceived positive and negative aspects. The free texts with the respondents answers to these questions were examined and categorised. To avoid introducing bias in the interpretation of the qualitative data, the classification of answers into categories was done in a conservative fashion: answers were classified according to closely matching categories even if that meant having some redundancy in the categories. This was done to avoid biased liberal interpretations; categories were introduced freely whenever needed. Fig. 8 summarises the results on the perceived positive aspects. The most listed positive aspect was understandability (Un, 15), followed by eases modelling (EM, 9) and error detection (ED, 9). Other positive aspects identified by respondents were readability (R, 8), visualisation (V, 8), modelling of invariants and constraints (MIC, 5), modelling of operations (MO, 5), 10 Results of χ 2 tests are as follows: R (p = 4.876E−5 < .005), N (p = .008348 < .01), E (p = 0.07116 > .05), O (p = 0.07116 > .05), LEC (p = .0007905 < .005), LaF (p = .001303 < .005), C (p = .07643 > .05), CU (p = .2231 > .05), CS (p = 0.4244 > .05). 11 Results of χ 2 tests are as follows: Eng (p = 0.004389 < .005), Doc (p = .01705 < .05), Rel (p = .004389 < .005), OU (p = .1353 > .05). 12 The results of the χ 2 test are as follows: MS (p = .1353 > .05), MI (p = 2.754E−5 < .005), MO (p = .004389 < .005), PT (p = .09773 > .05), MU (p = .01281 < .05).

JID:SCICO AID:1763 /FLA

[m3G; v 1.134; Prn:26/05/2014; 16:00] P.13 (1-28)

N. Amálio, C. Glodt / Science of Computer Programming ••• (••••) •••–•••

13

Fig. 8. Frequency of VCB’s perceived positive aspects. (A = Accuracy, CM = Closeness to Mathematics, Ch = Cohesion, Cf = Comfort, EM = Eases Modelling, ED = Error Detection, I = Intuitive, L = Learnability, LaF = Look and feel, MU = Model Usage, MIC = Modelling of Invariants and Constraints, MO = Modelling of Operations, MS = Modelling of State Space, N = Navigation, Ov = Overviews, R = Readability, Soc = Separation of Concerns, Un = Understandability, Us = Usability, V = Visualisation, O = Other.)

Fig. 9. Frequency of perceived negative aspects in VCB. (GUI = aspects of GUI interface, CM = Closeness to Mathematics, Cf = Comfort, Cu = Cumbersome, DE = Diagram Editing, Doc = Documentation, EM = Error Messages, GD = Graphic Design, LoG = Lack of Guidance, Le = Learnability, MO = Modelling of Operations, N = Navigation, NR = Novel Representation/New Symbols, Or = Organisation, Tf = Too flexible, TmD = Too many Diagrams, TI = Tool Immaturity, UA = Unclear Advantages, Un = Understandability, Up = Unproductive, Us = Usability.)

cohesion (Ch, 5), navigation (N, 4), overviews (Ov, 4), usability (Us, 4), accuracy (A, 3), learnability (L, 3), look and feel (LaF, 3), model usage (MU, 3), closeness to mathematics (CM, 2), comfort (Cf, 2), intuitiveness (I, 2), modelling of state space (MS, 2) and separation of concerns (SoC, 2). The value other (O, 6) in the chart of Fig. 8 aggregates the aspects that were listed once, namely: usefulness, productivity, unambiguous, stability, interactivity and eclipse platform. Fig. 9 summarises the results on the perceived negative aspects. The most listed negative aspect was learnability (L, 22). This was followed by diagram editing (DE, 16), documentation (Doc, 9), novel representations/new symbols (NR, 5), cumbersome (CU, 5), usability (Us, 4), closeness to mathematics (CM, 3), aspects of GUI interface (GUI, 2), error messages (EM, 2), lack of guidance (LG, 2), navigation (N, 2), too many diagrams (TmD, 2), tool immaturity (TI, 2), unclear advantages (UA, 2), unproductive (Up, 2), comfort (Cf, 1), graphic design (GD, 1), modelling of operations (MO, 1), too flexible (TF, 1) and understandability (Un, 1). 7.6. Survey limitations and validity A number of limitations may influence the validity and generalizability of the survey results presented here. There is a confounding factor between tool and modelling language that affects the construct validity of the survey. Although the questions are clearly formulated in terms of the used tools, many respondents did not make such a clear distinction in their replies. Nevertheless, the results do reflect the perceived level of satisfaction of the examined tools in supporting their corresponding modelling languages. Internal validity may have been affected by questionnaire design and recruitment issues. As always with questionnaires there is the possibility that their design may introduce bias, which affects the outcome. This is aggravated by the fact that the developers of VCL and VCB designed the questionnaire. This was mitigated by carefully designing questions according to standard techniques and guidelines [59]; the questions are as objective and symmetric as possible to avoid bias to either one of the sides. Another issue has to do with the recruitment based on volunteers who are usually more motivated and suited for a new task than the whole population. However, the participants covered a large range of people from different cultures, educational levels and educational settings: we had participants studying in three different European countries, from four different universities, whose educational levels ranged from undergraduate students in their second years to students undertaking more advanced studies, such as PhD. The main weakness of the survey and associated controlled experiment lies in the external validity. Most participants are students whose experience may be limited when compared to experienced UML and OCL modelers who may feel less need for VCL’s visualisations. Nevertheless, the participants included one software developer, and several PhD and masters students with some experience in the software industry. Another problem may have been the artificial nature of the case studies used in the controlled experiment. Although, the case studies were designed to be realistic, the fact is that they are still considerably smaller than a realistic industrial case study.

JID:SCICO AID:1763 /FLA

[m3G; v 1.134; Prn:26/05/2014; 16:00] P.14 (1-28)

N. Amálio, C. Glodt / Science of Computer Programming ••• (••••) •••–•••

14

Table 6 Strengths and weaknesses of the VCB tool. Strengths

Weaknesses

Usability

+ + + +

– Diagram editing – Documentation – Learnability

Generation of artifacts

+ Generation of Z specifications

– No support for code-generation – No generation of formal specifications for languages other than Z

Verification and validation

+ Amenability to formal verification and validation via

– No direct support for verification and validation in VCB

MDE

+ + + +

Good support for VCL’s visual approach Live-error checking Navigation Modelling of invariants and operations

Z using theorem proving – Only support for Z-based verification; no support for model-checking. Tackling research challenges in MDE Focus on usability Good empirical results Support for novel diagram types (VCL assertion and contract diagrams) for software design modelling + Support widespread OO design methods.

– Introduces new representations – No study so far on socio-cultural impact of VCL/VCB – No integration with popular behavioural diagram types of UML, such as use-case and sequence diagrams, and statecharts

8. Discussion The case studies and projects to which VCL has been applied (Section 6) together with the presented survey results (Section 7) suggest that VCL is an adequate OO design modelling language and that VCB does a good job in supporting VCL’s visual modelling approach. The fact that, overall, VCB was perceived as being better than the popular Papyrus UML Eclipse-based tool, tells us something about the quality of both VCB and VCL. VCL and VCB are but an attempt to improve both model-driven engineering (MDE) and formal methods. There are, however, several limitations in VCL and VCB, and ways in which they could be improved. The next sections discuss the following: (i) VCL’s purpose and suitability, (ii) VCB’s strengths and weaknesses, (iii) several implementation and design issues, and (iv) lessons learnt while developing, using and evaluating VCB. 8.1. VCL’s purpose and suitability VCL’s main purpose is to facilitate formal and OO design modelling through a graphical front-end. VCL is, therefore, suited for tasks that involve OO modelling. VCL and VCB could be used for teaching to introduce OO design modelling with set-theoretic concepts and to give an intuition on the connection between object-orientation and set theory. VCL has several limitations in terms of what it can express and ways to circumvent some of those limitations:

• VCL does not include explicit time primitives, but it can express properties that involve time, as evidenced by the VCL model of [51].

• VCL can express properties that are non-deterministic, but it cannot describe properties that are probabilistic. • VCL may not be suitable for all possible software modelling needs. For instance, if one needs to model goals and objectives of a system, software architectures, or business-oriented descriptions, then other modelling languages may be more appropriate. VCL is most suitable to model systems with a rich-state space that need to provide some internal behaviour in order to provide value to the outside world. It is less suitable to model interactions between systems, as is typical in concurrent systems, because VCL does not provide primitives to describe such interactions. VCL can be used in more rigorous settings, such as critical systems, because, from VCL, it is possible to obtain a formal and rigorous Z specification that can be subject to independent verification and validation using theorem proving [62]. Through theorem proving, a modeller can verify both structural and dynamic properties; although dynamic properties are typically harder to prove. It is also possible to analyse the model in order to gain insights into the intricacies of the modelled system; [24,27] develops a technique to analyse models with diagrams (involving both structural and dynamic properties) that we intend to incorporate in both VCL and VCB in the future. 8.2. Strengths and weaknesses Table 6 summarises VCB’s strengths and weaknesses, which are elaborated in the next sections. 8.2.1. Usability VCL supports an approach to software design modelling that is entirely visual. It has been designed in this way for practical reasons: graphical representations are known to facilitate reasoning for certain tasks [1], they have proved valuable in engineering [3], they are the favoured means of expression by practitioners for the task of software modelling, and they

JID:SCICO AID:1763 /FLA

[m3G; v 1.134; Prn:26/05/2014; 16:00] P.15 (1-28)

N. Amálio, C. Glodt / Science of Computer Programming ••• (••••) •••–•••

15

facilitate communication with end-users and customers as they convey information more effectively to non-technical people than text [2]. This paper’s empirical results endorse the claimed benefits of VCL’s graphical approach. VCB outranked Papyrus UML with a significant difference (Section 7.5) for ease of use, several usability criteria (reading, navigation, live error-checking, and look and feel) and two types of modelling tasks — modelling of constraints and invariants, and modelling of operations. In large part, this is due to VCB’s support of VCL’s visual modelling approach, which appears to benefit usability. VCB’s most selected positive aspect (Fig. 8), understanding, and the related aspects of reading, visualisation, eases modelling and intuitiveness, also reflect this positive effect of visualisation on usability. Perhaps, the most prominent features of VCB, not directly related to VCL, that positively affect usability are navigation and live-error checking; navigation facilitates usage and exploration of a VCL model and its composing diagrams; live-error checking gives instant feedback to users on model errors. VCB’s major weaknesses are writing or sentence construction, and diagram editing. Sentence construction was the only usability criterion that gave Papyrus an advantage over VCB (Fig. 6, writing) albeit with no significant difference; diagram editing was the second most listed negative aspect (Fig. 9). This suggests that text may be a better medium for constructing sentences than the point and click approach of VCB’s diagram editors. We believe that other negative aspects identified by the survey respondents (Fig. 9), such as cumbersome and unproductive, are related to this problem. It is interesting to comment on the most listed negative aspect of VCB (Fig. 9): learnability. Although, there may be a VCL/VCB factor in this choice, even because respondents identified novel representations as a negative aspect (Fig. 9), we believe that this has more to do with the difficulty of design modelling in general. In the VCL versus UML+OCL experiment [50], participants received the same training in VCL and UML+OCL; as UML+OCL is more widely known this should favour the performance of subjects in the UML+OCL tasks, but this was not the case. In the end, listing learnability as a negative aspect appears to be the respondents’ way of acknowledging that design modelling is not a trivial exercise. Although VCL and VCB appear to help with understanding (Fig. 8), they are not a panacea; there is still a considerable learning curve to surmount to become a proficient VCL modeller using VCB. 8.2.2. Generation of artifacts VCB generates Z specifications from VCL models to enable formal verification and validation (V &V ) at the level of the Z specification. This may be important in the development of critical applications. In the future, however, we would like to be able to generate code from VCL models, and partial or total formal specifications that can be fed into more powerful theorem provers and model-checkers. The Z generation is one of the features of VCB that has been used the least so far. Many of VCB’s users seem to be indifferent to it. The Z generation feature was not part of the VCL vs UML+OCL experiment. In the survey, however, the participants listed the closeness to mathematics aspect, which is related, as both a positive (Fig. 8) and a negative (Fig. 9) aspect. 8.2.3. Verification and validation (V &V ) As said above, the generated Z specification can be subject to independent V &V using Z theorem provers, such as Z/Eves. In the future, we want to enhance VCB with a graphical approach to V &V that is based on the snapshot analysis technique developed in [24,27] and similar to the model validation approach of the USE tool [63]. V &V was not part of the VCL vs. UML+OCL experiment and was not evaluated in the survey (Section 7). 8.2.4. Model-driven engineering (MDE) VCL tries to improve the state of the art in software design modelling and VCB is as tool to support VCL. In doing so, we are addressing two MDE challenges of [64], modelling and separation of concerns, and three challenges of [65], capability, scalability and usability. VCL and VCB are also aligned to recent research results on the application of MDE in industry [66, 49], which acknowledge the importance of better modelling techniques — in particular, those that improve usability, which is seen as key in facilitating uptake — and tools to support such techniques. Despite the good empirical results coming from the application of VCL/VCB to several case studies (Section 6) and the VCB survey (Section 7), we have not, so far, considered in our research the socio-cultural impact of introducing such a technology, a major uptake factor [66,65,49]. VCL is entirely compatible, from a conceptual point of view, with methods of OO analysis and design based on UML. For instance, [67] presents a popular OO method that is widely taught in university courses of OO analysis and design. [67] uses operation contracts and describes them using natural language; these contracts can be described graphically using VCL. VCL and VCB support, therefore, these widespread and widely taught OO design methods. VCL introduces novel modelling diagram types that are not expressible in UML alone and that take the role of OCL, doing graphically what OCL does textually. Assertion and contract diagrams, which express in graphical form predicates of invariants and operations, try to improve the state of the art in diagrammatic representations of behaviour: (a) they provide total definitions of behaviour, unlike scenario-based UML collaboration and sequence diagrams, and (b) contract diagrams and their focus on changes of object state and behavioural composition lift the restriction of having behaviour defined on explicitly declared states and their transitions (such as UML state and activity diagrams). VCB provides full support for these novel diagram types of VCL in terms of editors, type-checkers and Z translators. The respondents of the VCB survey presented in Section 7, recognised the effectiveness of VCB in supporting the modelling of invariants and operations: VCB was perceived as being better than Papyrus for these tasks with a significant difference (Section 7.5.4), these two tasks were

JID:SCICO AID:1763 /FLA

[m3G; v 1.134; Prn:26/05/2014; 16:00] P.16 (1-28)

N. Amálio, C. Glodt / Science of Computer Programming ••• (••••) •••–•••

16

identified as positive aspects of VCB (Fig. 8, MIC and MO). On the other hand, the survey respondents also listed novel representations as a negative aspect (Fig. 9, NR) acknowledging the cognitive effort involved in learning them. Although VCL’s novel diagram types appear to have a positive impact, there is no integration with popular behavioural diagram types of UML, such as use-case and sequence diagrams, and statecharts; such integration, is left for future work.13 8.3. Implementation issues We now discuss several design decisions taken during the development of VCB. 8.3.1. Diagrams and text VCB keeps a dual representation of VCL models: one based on EMF (Eclipse Modelling Framework) metamodels and another based on textual grammars. This was done for the following reasons:

• VCL is essentially a diagrammatic language. However, there are several constructions that are textual: they require textual parsing and their form is better described using grammars.

• VCB checks the well-formedness of models using type-checking. The type rules that govern this system are defined on VCL’s grammars, following the standard approach to define a type system inductively based on the tree structure that is induced by the language’s grammar; metamodels do not induce such tree structures. • It is convenient to have a textual representation of VCL diagrams. As discussed above (Section 8.2.1), text appears to be better a representation for editing. This dual approach may enable us to develop in the future a way of generating diagrams from textual descriptions. 8.3.2. Customisation of the generated editors VCL’s diagram editors were built using Eclipse’s GMF, however, the crude GMF-generated editors could not satisfy VCL’s diagram editing needs. VCL’s editing semantics could have been implemented through customisations that introduce modifications to the generated code. This is supported by EMF, which allows the user to tag certain Java code elements that are not overwritten by the process of code generation. We have found, however, that this approach is cumbersome: (a) there is a substantial amount of redundancy and inefficiency in the generated code; (b) the code customisations induce high-coupling, which brings maintenance problems, because they are too tied to the generated code. A central tenet of VCB’s design philosophy is that invasive changes to the generated code should be kept to a minimum (these should only be done in certain limited and well-defined circumstances). Instead, to achieve low-coupling between the customisations and the generated code, VCB performs its diagram-editing-specific behaviour using simple customisation frameworks that observe the state of each VCL diagram editor; this enables the implementation of VCB’s specific editing requirements. 8.4. Lessons learnt VCB confirms the importance of having languages supported by tools. The first relevant VCL model development [41] was done without VCB using a drawing tool: the development was slow, the models, due to their size, were difficult to manage, and language learners were deprived of feedback, unsure about the validity of their diagrams. This changed substantially from the moment we started using VCB, which enhanced learning of VCL. An important lesson we learnt while using and evaluating VCB is that constructing diagrams using a point-and-click graphical approach can be cumbersome at times. This was one of VCB’s main dissatisfying aspects identified in the survey (Fig. 9, diagram editing). Diagram editing appears to be effective for building something like UML class diagrams (or VCL SDs) made of boxes and the lines that connect them. However, things become more complex when building more complex and intricate graphical constructions, as it often happens with VCL predicates. This suggests an approach to construct diagrams from text. Graphical parsers, however, appear to have an advantage: interactivity. Users can learn how to build sentences by playing with the graphical parser; once some syntactical construct is built, it is possible to know what else can be added. Although there may be some complexities involved in building diagrams, the results of the VCB survey (Section 7) confirm their usefulness once they are constructed. The VCB development taught us much about the interplay that exists between graphics and text. Advanced graphical languages, such as VCL, inevitably resort to a mixture of text and graphics. Being purely graphical often entails clutter in diagrams expressing more complex properties. Inevitably, supporting tools need to resort to both graphical and textual parsers. VCB’s usage by students highlighted the importance of typing and model consistency-checking. In our experience, users tend to produce better models when the tool gives them immediate feedback on model errors. The users themselves recognised the importance of VCB’s live error-checking (LEC) in the survey (Section 7), which was perceived as being better than

13

The integration of statecharts in VCL’s predecessor, the UML+Z framework, has already been explored [24,26,27].

JID:SCICO AID:1763 /FLA

[m3G; v 1.134; Prn:26/05/2014; 16:00] P.17 (1-28)

N. Amálio, C. Glodt / Science of Computer Programming ••• (••••) •••–•••

17

that of Papyrus UML with a significant difference (see Fig. 6) and it was the second most listed positive aspect (Fig. 8, error detection). The usage of VCL in practice suggests that VCL is doing well in achieving some of its design goals. One of VCL’s goals was to allow people with different skills to take part in the same modelling effort; another goal was to produce mathematical specifications from diagrams. In the case studies to which VCL and VCB have been applied (Section 6), only one developer is a Z expert, yet Z specifications can be obtained from the developed models. By using VCB and following the VCL tutorials, users, after some effort, can build meaningful VCL models from which a mathematical specification can be obtained; learning of Z seems to require more effort. This suggests that VCL and VCB are providing an improvement over traditional formal methods that require expertise. A striking lesson of the three years in which VCB has been used is the reduced usage of the Z generation feature of VCB. Most of our users only care about producing consistent and valid VCL models, overlooking most of the times the fact that VCB can actually generate Z from the diagrams they produce. Z generation has not been listed as a positive aspect of VCB by any survey respondent (Fig. 8), although respondents listed accuracy, closeness to mathematics and unambiguous as positive aspects. This suggests that some users find it reassuring that a mathematical description can be produced from their VCL diagrams even if they don’t use it. 9. Related work An early version of VCB is presented in [40]. This only supported part of VCL — structural and assertion diagrams only. This paper presents a version of VCB that supports all diagram types of VCL together with a tool evaluation. The next sections discuss related work, but focussing on the following topics: tool support for languages that are entirely visual, related modelling tools, and empirical evaluations of visual modelling tools. 9.1. Entirely visual design languages and their tools There are other visual design languages that, like VCL, have a formal basis and are capable of expressing predicates visually, such as augmented constraint diagrams (ACD) [68] and Visual OCL (VOCL) [69,70]. However, tool support for these languages does not go beyond proof of concept mock-ups. In a comparative study focused on practical application [71,72], VCL outperformed ACD and VOCL; VCB and its capacity to provide live feedback on model errors was a major factor in VCL’s perceived advantages. Lohmann et al. [73] translates, using graph transformation rules, UML class diagrams and contracts to Java skeletons and JML assertions. The approach is supported by a tool, the Visual Contract Workbench [74], an Eclipse plug-in. The definitions produced by this approach are partial as only skeletons are produced, and, unlike VCB, there is no support for writing predicates of invariants, which have to be written in the code. There are other approaches for describing predicates of invariants and predicates of operations using contracts in the field of model transformations [75,76]. These approaches are based on graph transformations; their tool support has been reported. However, they are designed for model-transformation; VCL and VCB tackle general-purpose OO design. 9.2. Related design modelling tools The AutoZ tool [77] generates Z specifications expressed in the ZOO style from UML class (similar to VCL SDs) and state diagrams, following UML+Z [27,26]. Predicates of invariants and operations are expressed textually in Z. AutoZ uses Z type-checking to check well-formedness of diagrams, which complicates error-reporting. The UML-B tool [78] supports an integration between UML and the B method, where the semantics is defined as a translation from UML class and state diagrams to a B specification. Unlike VCL, the UML-B profile supports formal model refinement. UML-B does, however, lack an approach to write predicates graphically; predicates of operations and invariants are written textually in terms of the generated B specification; well-formedness is also checked at the level of the resulting B specification. It is interesting to compare VCL against UML tools, such as Papyrus14 and USE [63]. The main difference with respect to VCB is that they express predicates of invariants and operations textually using OCL; VCB does it graphically using VCL. Table 7 compares these two UML tools and VCB taking several tool features into account. VCB and Papyrus are both Eclipse plugins and so they provide support for Eclipse, unlike USE which provides its own environment. VCB and Papyrus provide facilities for diagram editing, unlike USE which builds UML diagrams from user constructed textual specifications. Papyrus, as of the Juno version of Eclipse, does not support model consistency-checking, and, hence, no support for live feedback on model errors; VCB and USE do perform consistency-checking of design models, but only VCB supports a live environment with instant feedback, where errors are reported as diagrams are constructed. USE is the only tool that does model validation. Papyrus is the only tool that generates code into several programming languages, such as Java and ADA.

14

http://www.eclipse.org/papyrus/.

JID:SCICO AID:1763 /FLA

[m3G; v 1.134; Prn:26/05/2014; 16:00] P.18 (1-28)

N. Amálio, C. Glodt / Science of Computer Programming ••• (••••) •••–•••

18

Table 7 Comparison between tools Papyrus UML, USE and VCB.

9.3. Empirical evaluations of visual modelling tools This is, to our knowledge, the first empirical evaluation of design modelling tools that focus on usability and that involves rigorous statistical testing of hypothesis. There are empirical studies of general CASE (Computer Assisted Software Engineering) tools, but there is no direct comparison between such tools:

• Iivari [79] surveys the adoption of CASE tools in industry with an analysis that involves statistical testing of hypothesis. Unlike our survey, there is no comparison of modelling tools. An interesting outcome of this study is how perceived complexity affects the acceptance of a technology; [79] suggests the investment in techniques that improve the usability of technologies, which is what we are trying to do with VCL. • Post et al. [80] compare different CASE tools to elicit their most important features from a user perspective. Usability is considered, but only from a broad perspective. [80] concludes that there are two distinct uses of case tools, design and code generation or prototyping, and that CASE tools tend to emphasise one or the other; according to [80], code generation or prototyping is seen as more important. VCB emphasises the design activity. • Post and Kagan [81] evaluate the Rational Rose modelling tool and its support for UML. This is just an evaluation of this particular tool; there is no comparison with other UML tools. There are evaluations of visual modelling tools based on broad criteria categories, but that do not go as deep as the study presented here on the specific subject of usability. These evaluations have some validity issues as they do not go beyond ranking of the evaluated approaches, not providing a rigorous statistical analysis based on hypothesis testing:

• Tobias et al. [71,72] compared modelling languages that describe predicates visually and their tools; in this comparison the pair VCL/VCB outranked the other two competitors mainly due to its superior tool support. This was done based on a problem that was modelled by a single person using the evaluated approaches; based on this modelling exercise, the approaches were ranked for each individual criterion. • A similar ranking approach is followed in [82] to evaluate different UML tools. [83] and [84] compare different UML tools based on broad criteria categories, but there is no raking or statistical analysis. Bobkowska and Reszke [85] study the usability of different modelling tools by investigating the performance of subjects in doing tasks related to modelling. This covered tool support for UML use-case, class, sequence and state diagrams. In this evaluation, subjects are not asked to actually solve modelling tasks, but to draw some given diagram with some modelling tool. Based on the performance of the subjects, some usability conclusions are drawn. Although some quantitative results are taken based on the performance of subjects using descriptive statistics, there is no statistical testing of hypothesis to check the significance of the results. 10. Conclusions This paper presents and evaluates the visual contract builder (VCB) tool.15 VCB supports the visual contract language (VCL), which takes an approach to software design modelling that is entirely diagrammatic. VCB provides diagram editors for all of VCL’s diagram types, supporting VCL’s novel capabilities to express predicates and system dynamics (through preand post-conditions) graphically. In addition, VCB provides a facility for live error checking as models are built based on type-checking, and a generator to produce Z specifications from VCL models that can be subject to independent verification and validation using theorem proving. VCL and VCB have already shown to be adequate for software design modelling through their application in several student projects and real-world case studies. Some of the case studies are challenges and benchmarks put forward by research communities. These applications of VCL and VCB have involved people with a diverse range of skills. The paper evaluates VCB based on a survey that was conducted in the context of a controlled experiment. The results of this evaluation suggest that VCB is effective in supporting VCL’s graphical approach to software design modelling and in

15

http://vcl.gforge.uni.lu.

JID:SCICO AID:1763 /FLA

[m3G; v 1.134; Prn:26/05/2014; 16:00] P.19 (1-28)

N. Amálio, C. Glodt / Science of Computer Programming ••• (••••) •••–•••

19

helping model users with their modelling tasks; the usability results of this evaluation, in particular, are very positive: VCB was, for many comparative criteria, perceived as being more usable than Papyrus, a popular Eclipse-based UML tool, with a significant difference, meaning that it cannot be attributed to chance alone. The most relevant contribution of the work presented here is the VCB tool supporting an approach to software design modelling that is entirely graphical. To our knowledge, there is no other tool that achieves the level of sophistication of VCB for such an entirely graphical language. Another contribution, is an empirical evaluation of VCB. To our knowledge, this is the first time that tool support for graphical design modelling is compared empirically based on a survey whose results are subject to rigorous statistical testing of hypothesis. The empirical results presented here strongly suggest that VCB does a good job in supporting VCL and that VCB is able to compete with a popular UML-based tool. Although the results of the evaluation only apply to the specific evaluated tools and their specific versions, they tell us something about the quality of VCB and VCL, raising interesting research questions on the benefits of graphical approaches to software design modelling. Acknowledgements Most of the research work presented in this paper was done during Amálio’s post-doctoral research work at the University of Luxembourg. We would like to thank Professor Pierre Kelsen of University of Luxembourg for his support of both VCL and VCB. The empirical evaluation presented in this paper was done in the context of a controlled experiment to evaluate VCL against UML and OCL, which was done in collaboration with professor Lionel Briand of University of Luxembourg. Pierre Kelsen and Lionel Briand financed the VCL vs UML+OCL controlled experiment, which made the empirical evaluation presented here possible; we are grateful to both of them. We would like to thank all participants of the VCL vs UML+OCL controlled experiment at the Faculty of Sciences of the University of Lisbon (FCUL), Faculty of Sciences and Technology of the New University of Lisbon (FCT/UNL), University of Luxembourg and University of York (UY). Several people at the host institutions made the controlled experiment possible, namely, Isabel Nunes (FCUL), João Araújo (FCT/UNL), Fiona Polack (UY), Antónia Lopes (FCUL) and Ana Moreira (FCT/UNL). Appendix A. Modelling in VCL This appendix gives an overview of VCL using the secure simple bank case study [54]. All diagrams presented here have been drawn using the VCB tool. A tutorial on how to build the complete model of this case study in VCB is given in http://vcl.gforge.uni.lu/tutorials.html. The next sections present VCL’s visual primitives and diagram types; each diagram type is illustrated using the case study. A.1. Visual primitives A VCL model is organised around packages, represented as clouds to allude to a world of their own. VCL packages encapsulate structure and behaviour. In VCL, it is possible to construct larger packages from smaller ones using incorporation (or embedding), which is represented in VCL as enclosure. In Fig. A.10f, Authorisation incorporates the enclosed packages, which means that Authorisation is the composition of the incorporated packages plus something else. VCL represents sets as labelled rounded contours. Topological notion of enclosure can either denote the subset relation (to the left, custsSav! is subset of custsCurr!) or, when accompanied by symbol , define a set in terms of the enclosed sets and objects (to the left, CustType is defined by enumerating its elements). Objects are represented as rectangles; they denote an element of some set (e.g. corporate and personal to the left). Edges describe relations between sets and objects. The property edge (labelled arrow) to the left describes a predicate. In Fig. A.11b, the property edges are definitional, introducing properties of sets. Assertions (labelled hexagons) represent some state constraint or observe operation (a query). They refer to a single system state (e.g. assertion CorporateHaveNoSavings to the left). Contracts (labelled double-lined hexagons) represent operations that change state; hence, they are double-lined as opposed to single-lined assertions (e.g. OpenAccount, left).

A.2. VCL diagram types A VCL model comprises a collection of packages. Each package comprises one package diagram (PD), one structural diagram (SD), one behavioural diagram (BD) and several assertion and contract diagrams (ADs and CDs). The following sections briefly describe each diagram type of VCL.

JID:SCICO AID:1763 /FLA

[m3G; v 1.134; Prn:26/05/2014; 16:00] P.20 (1-28)

N. Amálio, C. Glodt / Science of Computer Programming ••• (••••) •••–•••

20

Fig. A.10. Sample package diagrams of VCL model of secure simple bank.

A.3. Package diagrams Package diagrams (PDs) define VCL packages. They include the package being defined (the current package) and its dependencies with other packages. Fig. A.10 presents sample PDs of secure simple bank. PDs are as follows:

• The package being defined or the current package is represented with a bold line. All other packages in a PD are foreign. The symbol , which can be attached to some package, assertion or contract, has an expansion meaning:

•

• • •

•

double-clicking opens the definition. In VCB, double-clicking on packages of a PD gives the user a choice to navigate to the package’s package, structure or behaviour diagrams. In Figs. A.10b and A.10h, current packages are, respectively, Bank and SecBank. The current package can be defined as container (symbol ) or ensemble (symbol ). The former act as mere containers for sets and their local operations; containers do not constitute a functional unit, they do not have a global identity and so they cannot include relation edges and global operations. Ensemble packages, on the other hand, constitute a functional unit on their own, and so they have a global identity, comprising relation edges and global operations. In Fig. A.10, packages CommonTypes and RolesAndTasksBank are containers; all others are ensembles. Uses edges, represented as hollow-headed arrows, indicate that the current package may use elements defined in other packages. In Fig. A.10b, package Bank uses package CommonTypes, which is given alias CT. In PDs, enclosure denotes incorporation. Incorporation means that the structures of the enclosed packages become part of the current package. In Fig. A.10h, SecBank incorporates packages SecForBank, BankWithJI and TransactionsSwitch. Incorporated packages are connected with edges to resolve conflicts. There are two kinds of such edges: overrides and merges. Override edges say that certain sets of the source package override those with the same name of the target package in the current package; only sets with no local properties may be overridden. Merge edges say that certain specified sets with the same name from the linked packages are to be merged. Fig. A.10f says that set User of package Authentication overrides User of AccessControl. The current package may extend incorporated sets. This is specified using an extends list. In Fig. A.10g, package SecForBank extends incorporated set User.

A.4. Structural diagrams Structural diagrams (SDs) define the structures that, together, make the state space of a package. Fig. A.11 gives sample SDs of packages that are part of the VCL model of secure simple bank. The SDs are as follows:

JID:SCICO AID:1763 /FLA

[m3G; v 1.134; Prn:26/05/2014; 16:00] P.21 (1-28)

N. Amálio, C. Glodt / Science of Computer Programming ••• (••••) •••–•••

21

Fig. A.11. Sample structural diagrams of VCL model of secure simple bank.

• An SD defines two kinds of primitive sets: value and class. Class sets (or, simply, classes) define a set of objects

•

•

• •

16

with identity; value sets define a set of values (objects without identity). Classes are represented with a bold-line; they include a label CLASS on the top-right part of the contour. Value sets are not bold-lined and they include the label VALUE. In SDs of Fig. A.11, Customer, Account, User and Session are class sets; all others are value sets. Sets whose name are preceded by symbol  are defined in terms of what they have inside them. The  means that the set is closed from a definitional point of view because it is defined by what it has inside only. Sets that only include the  are definitional. Sets that include symbol  followed by symbol ↔ and that contain the label DERIVED are derived sets; they are defined from primitive entities of the model. In Figs. A.11b and A.11c, sets CustType, AccType, LoginResult and UserStatus are definitional value sets describing enumerations, defined by enumerating their possible values. In Fig. A.11a, TimeNat is a derived set defined from Nat, the set of natural numbers.16 An SD can contain constants of sets and scalars. Constants have their labels preceded by symbol ©. Scalar constants are represented as objects (rectangles); set constants as rounded contours. When placed inside sets, constants objects are used to name values or objects of the enclosing set; this is a common idiom to define enumerations in VCL (e.g. sets CustType, AccType, LoginResult and UserStatus of SDs of Fig. A.11 are defined this way). An SD can also include constants with a type designator; such constants are local when attached to some set and global when they stand alone. In Fig. A.11c, maxPwMisses defines a scalar constant of type Nat that is local to set User. Reference sets contain symbol ↑ and label REFERENCE; they are used to refer to sets defined in used packages as described in the PD. Reference sets of Figs. A.11b and A.11c refer to Name and TimeNat defined in package CommonTypes (alias CT). Edges with circled labels are relation edges; they define binary relations between sets and they include multiplicity constraints on each edge’s extremity, which are represented both numerically and using lines and cir-

This defines time as set of time points that are isomorphic to the natural numbers.

JID:SCICO AID:1763 /FLA

[m3G; v 1.134; Prn:26/05/2014; 16:00] P.22 (1-28)

N. Amálio, C. Glodt / Science of Computer Programming ••• (••••) •••–•••

22

Fig. A.12. Sample behaviour diagrams of VCL model secure simple bank.

cles. A circle on the edge means multiplicity 0, a single line means multiplicity 1, and a line-end with a crow’s foot means multiplicity many. In Figs. A.11b and A.11c, Holds and HasSession are relation-edges defining, respectively, multiplicities one-to-many and one-to-optional (? means optional or the 0..1 UML multiplicity). • There are two alternative ways of representing state properties (attributes or fields) possessed by all objects of some set: (i) either using directed edges (outgoing arrows) as in SD of Fig. A.11b (e.g. name) from the set to which they belong to the set that gives their type, where the edge’s label is the property’s name, or (ii) listed in the set’s properties compartment (according to the syntax, ‘name : type’) in a style that is akin to UML class diagrams, as in Fig. A.11c (e.g. uid). • Assertions, represented as single-lined elongated hexagons, identify invariants, which are separately defined in ADs. Assertions connected to some set are local; those standing-alone are global. In VCB, double-clicking on an assertion takes the user to its AD (symbol ). In Figs. A.11b and A.11c, CorporateHaveNoSavings, HasCurrentBefSavings and HasSessionIffLoggedIn are global, SavingsArePositive and MaxPwMissesInv are local. A.5. Behaviour diagrams A behaviour diagram (BD) defines a map over the behavioural units of a package; it identifies these units without actually defining them. A BD may include: (a) references to package operations to be separately specified in ADs and CDs or (b) operation compositions. Fig. A.12 presents sample BDs of secure simple bank; the operations included in them are as follows:

JID:SCICO AID:1763 /FLA

[m3G; v 1.134; Prn:26/05/2014; 16:00] P.23 (1-28)

N. Amálio, C. Glodt / Science of Computer Programming ••• (••••) •••–•••

23

• There are two kinds of specified operations: update (or modifiers) and observe (or queries). Queries are represented as assertions, modifiers as contracts (double-lined elongated hexagons). In Fig. A.12b, operations UserIsLoggedIn, GetUserGivenId and IsLoggedIn are queries; all others are modifiers. • Specified operations may be local or global: local when they are inside some set and global otherwise. The global operations of a package define the behaviour that the package offers to the outside world. Each specified operation needs to be defined in an AD or CD; double-clicking takes the user to its definition (symbol ). In BDs of Fig. A.12, the operations inside the contours Account, Customer, Session and User are local; all others are global. To support coarse-grained separation of concerns, VCL is equipped with composition mechanisms to put together behaviours from different packages. VCL’s composition mechanisms build-up on ideas surrounding aspect-orientation. BDs support three kinds of operation compositions [41]: integral, merge and join extension. Such compositions are defined in boxes; each box is named after the kind of composition. The operation compositions of Fig. A.12 are as follows:

• Operations of imported packages are visible within the package, but not visible to the outside world. Integral extension promotes operations from incorporated packages to package operations to make them visible to the outside world. The promoted operation is made available in the new package unaltered (hence name integral). Operations to be integrally extended are represented inside an integral extension box (there is at most one); they are represented visually as normal operations connected to the package where they come from. Special operation All refers to all operations of a package and it may include a list of operations to exclude. BD of Fig. A.12c says that all operations of package TransactionsSwitch with the exception of IsSuspended and Init are to be integrally extended; likewise for all operations of package SecForBank except UserIsLoggedInAndHasPerm and Init. • Merge extension is a form of composition that merges or joins separate behaviours coming from different packages. Merge extensions are specified in a merge box; all separate operations that are included in the merge box with the same name are merged into a new package operation joining the separate behaviours. Semantically, merged operations are combined using an operator that is akin to logical conjunction. BD of Fig. A.12c includes a merge box saying that Init of SecForBank is to be merged with Init of TransactionsSwitch. • Join extension is VCL’s aspect-oriented like mechanism. It inserts a certain extra behaviour into a group of operations. Join extensions involve placing the group of operations to extend inside a join box; the extra behaviours to insert are specified as join contracts, comprising a pre- and a post-condition, that is connected to the box through a fork edge. BD of Fig. A.12c includes a join extension that says that all operations of BankWithJI are to be joined with contract AuthACJoin. A.6. Assertion diagrams Assertion diagrams (ADs) describe predicates over a single state of the modelled system. They are used to describe invariants and observe operations (queries). Sample ADs are given in Figs. A.13 and A.14; they are as follows:

• An AD is made of two compartments: declarations (top) and predicate (bottom). This is illustrated in ADs of Figs. A.13 and A.14.

• An AD may have a global or a local scope. Global ADs include names of package and assertion; local ADs include names of package, set and assertion. ADs of Figs. A.13c, A.13a and A.14a are local; all others are global.

• The declarations compartment may include variables, which are either scalar (represented as objects) or collections (represented as sets). Each variable has a name and a type. Variables can denote either inputs (name suffixed by ‘?’) or outputs (name suffixed by ‘!’). Fig. A.14d declares output set accs!; Figs. A.14a, A.14b and A.14c declare several input and output objects. • The declarations compartment may include imported assertions, either standing alone or combined in logical formulas. Double-clicking on an imported assertion takes the user to its AD definition (symbol ). An assertion import comprises an optional up arrow symbol ↑, name of imported assertion with optional origin qualifier and an optional rename list. ↑ symbol indicates that the import is total (variables and predicate are imported); when not present the import is partial (only the predicate is imported). Rename list indicates variables of imported assertion that are to be renamed (e.g. [a!/a?] says that a! replaces a?). Fig. A.14b has two assertion imports: one total and one partial. Total assertion import says that assertion GetBalance is to be called on Account object a!; as import is total, variable bal! defined in Account.GetBalance (Fig. A.14a) is also defined in AccGetBalance. Partial import of GetAccountGivenAccNo means that output a! is visible in AccGetBalance, but not to the outside world; input aNo? from GetAccountGivenAccNo is visible to the outside world as it is also declared in AccGetBalance. • The predicate compartment includes visual formulas made up of set expressions, predicates and propositional logic operators. Fig. A.14a expresses an equality predicate using a predicate property edge to say that bal! is to hold value Account.balance. Fig. A.13a expresses an implication formula to say that if the property aType of Account has value savings then the property balance must be greater or equal than 0. Fig. A.14d outputs the set of account numbers whose balances are negative; this builds a set using a set definition construction (symbol ) by constraining set Account using predicate property edges (arrows); the constructed set (accs) is then used to say, using a quan-

JID:SCICO AID:1763 /FLA

[m3G; v 1.134; Prn:26/05/2014; 16:00] P.24 (1-28)

N. Amálio, C. Glodt / Science of Computer Programming ••• (••••) •••–•••

24

Fig. A.13. Sample ADs of invariants of VCL packages Bank and Authentication of secure simple bank.

tifier, that the output set accNos! contains all account numbers of the accounts in accs. Fig. A.14c expresses a set membership predicate using a predicate property edge to say that output a! belongs to accounts with property accNo equal to aNo? (there is at least one). Fig. A.13d comprises a set formula that defines a set by constraining the relation Holds, using property edge modifiers with operators domain (
) and range restriction (), to give the set of tuples made of corporate customers and savings accounts; outer shading (reinforced with symbol ∅) then says that this set must be empty. A.7. Contract diagrams Contract diagrams (CDs) describe system dynamics. They comprise a pair of predicates corresponding to pre- and postconditions. The pre-condition describes what holds before the operation is executed. The post-condition describes the effect

JID:SCICO AID:1763 /FLA

[m3G; v 1.134; Prn:26/05/2014; 16:00] P.25 (1-28)

N. Amálio, C. Glodt / Science of Computer Programming ••• (••••) •••–•••

25

Fig. A.14. Sample ADs of operations of VCL packages Bank and Authentication of secure simple bank.

of the operation, saying what holds after execution. CDs are used to describe operations that change the state of the modelled system. Sample CDs are given in Fig. A.15; they are as follows:

• Like ADs, CDs are made of two principal compartments for declarations and predicate. In CDs, the predicate compartment is subdivided in two for pre-condition (left) and post-condition (right). This is illustrated in CDs of Fig. A.15. • CDs are similar to ADs in terms of what can be included in the declarations compartment. The only difference is that CDs can include both imported assertions and contracts. This is illustrated in CDs of Fig. A.15. In Fig. A.15e, import of contract HasSessionAddNew includes a renaming. In Fig. A.15f, the two imported contracts are combined using a disjunction to say that a login operation is either successful (LoginOk, Fig. A.15e) or not (LoginNotOk, Fig. A.15d). • In CDs, pre- and post-condition compartments are made of the same sort of visual formulas used in the predicate compartment of ADs. In the post-condition compartment, the variables that refer to the after state are bold-lined. In Figs. A.15a and A.15c, pre- and post-conditions compartments are made of arrows formulas stating equality predicates; the post-state variable status in Fig. A.15c is bold-lined in the post-condition compartment to say that its value changes from loggedIn (precondition) to loggedOut (post-condition).

JID:SCICO AID:1763 /FLA

26

[m3G; v 1.134; Prn:26/05/2014; 16:00] P.26 (1-28)

N. Amálio, C. Glodt / Science of Computer Programming ••• (••••) •••–•••

Fig. A.15. Sample CDs of package Authentication of secure simple bank.

JID:SCICO AID:1763 /FLA

[m3G; v 1.134; Prn:26/05/2014; 16:00] P.27 (1-28)

N. Amálio, C. Glodt / Science of Computer Programming ••• (••••) •••–•••

27

Appendix B. Excerpt of Z specification This appendix gives an excerpt of the generated Z of the VCL model of secure simple bank presented in Appendix A. The Z for the set User defined in VCL package Authentication (SD of Fig. A.11c) is as follows:

Above, the Z axiomatic definition UsermaxPwMisses comes from the constant maxPwMisses of set User in SD of Fig. A.11c. Z schema User0 corresponds to definition of class set User in SD of Fig. A.11c. Z schema UserMaxPwMissesInv comes from the AD of Fig. A.13c. Schema User is the final definition of the state of set User taking the local invariant into account. Finally, schema UserLoginOk comes from the CD of Fig. A.15a. References [1] J.H. Larkin, H.A. Simon, Why a diagram is (sometimes) worth ten thousand words, Cogn. Sci. 11 (1987) 65–99. [2] D.L. Moody, The “physics” of notations: toward a scientific basis for constructing visual notations in software engineering, IEEE TSE 6 (35) (2009) 756–779. [3] E.S. Ferguson, The mind’s eye: nonverbal thought in technology, Science 197 (1977) 4306. [4] E.S. Ferguson, Engineering and the Mind’s Eye, MIT Press, 1992. [5] Object Management Group, Unified Modeling Language: infrastructure (ver. 2.4.1), 2011. [6] G. Booch, J. Rumbaugh, I. Jacobson, The Unified Modeling Language User Guide, Addison-Wesley, 1999. [7] J. Rumbaugh, M. Blaha, W. Permerlani, F. Eddy, W. Lorensen, Object Oriented Modelling and Design, Prentice-Hall, 1991. [8] G. Booch, Object-Oriented Analysis and Design with Applications, Addison-Wesley, 1994. [9] R. Wieringa, A survey of structured and object-oriented software specification methods and techniques, ACM Comput. Surv. 30 (4) (1998) 459–527. [10] A. Evans, R. France, K. Lano, B. Rumpe, The UML as a formal modeling notation, in: UML’98, in: LNCS, vol. 1618, Springer, 1998, pp. 336–348. [11] P. Stevens, On the interpretation of binary associations in the unified modelling language, Softw. Syst. Model. 1 (1) (2002) 68–79. [12] B. Henderson-Sellers, F. Barbier, Black and white diamonds, in: UML’99, in: LNCS, vol. 1723, Springer, 1999, pp. 550–565. [13] R. Hennicker, H. Hussmann, M. Bidoit, On the precise meaning of OCL constraints, in: Object Modeling with the OCL, in: LNCS, vol. 2263, Springer, 2002, pp. 69–84. [14] A.D. Brucker, B. Wolff, HOL-OCL: experiences, consequences and design choices, in: UML 2002, in: LNCS, vol. 2460, Springer, 2002, pp. 196–211. [15] B. Henderson-Sellers, UML – the good, the bad or the ugly? Perspectives from a panel of experts, Softw. Syst. Model. 3 (1) (2005) 4–13. [16] R. France, S. Ghosh, T. Dinh-Trong, Model-driven development using UML 2.0: promises and pitfalls, IEEE Comput. 39 (2) (2006) 59–66. [17] M.V.C. Manfred Broy, UML formal semantics: lessons learned, Softw. Syst. Model. 10 (4) (2011) 441–446. [18] B. Rumpe, R. France, Variability in UML language and semantics, Softw. Syst. Model. 10 (4) (2011) 439–440. [19] Z. Micskei, H. Waeselynck, The many meanings of UML 2 sequence diagrams: a survey, Softw. Syst. Model. 10 (4) (2011) 489–514. [20] M. Richters, M. Gogolla, On formalizing the UML object constraint language OCL, in: ER ’98, in: LNCS, vol. 1507, Springer, 1998, pp. 449–464. [21] M. Richters, A precise approach to validating UML models and OCL constraints, Ph.D. thesis, Universität Bremen, 2001. [22] D. Varró, A formal semantics of UML statecharts by model transition systems, in: ICGT 2002, LNCS, Springer, 2002, pp. 378–392. [23] H. Störrle, Semantics of interactions in UML 2.0, in: Human Centric Computing Languages and Environments, HCC 2003, IEEE, 2003, pp. 129–136. [24] N. Amálio, S. Stepney, F. Polack, Formal proof from UML models, in: Proc. ICFEM 2004, in: LNCS, vol. 3308, Springer, 2004, pp. 418–433. [25] N. Amálio, F. Polack, S. Stepney, An object-oriented structuring for Z based on views, in: ZB 2005, in: LNCS, vol. 3455, 2005, pp. 262–278. [26] N. Amálio, F. Polack, S. Stepney, UML+Z: augmenting UML with Z, in: H. Abrias, M. Frappier (Eds.), Software Specification Methods, ISTE, 2006, pp. 81–102. [27] N. Amálio, Generative frameworks for rigorous model-driven development, Ph.D. thesis, Dept. Computer Science, Univ. of York, 2007. [28] C.F. Lange, M.R. Chaudron, Effects of defects in UML models – an experimental investigation, in: ICSE 2006, IEEE, 2006, pp. 401–410. [29] A. Reder, A. Egyed, Incremental consistency checking for complex design rules and larger model changes, in: MODELS 2012, in: LNCS, vol. 7590, Springer, 2012, pp. 202–218. [30] K. Farias, A. Garcia, J. Whittle, Assessing the impact of aspects on model composition effort, in: AOSD 2010, ACM, 2012, pp. 73–84.

JID:SCICO AID:1763 /FLA

28

[31] [32] [33] [34] [35] [36] [37] [38] [39] [40] [41] [42] [43] [44] [45] [46] [47] [48] [49] [50] [51] [52] [53] [54] [55] [56] [57] [58] [59] [60] [61] [62] [63] [64] [65] [66] [67] [68] [69] [70] [71] [72] [73] [74] [75] [76] [77] [78] [79] [80] [81] [82] [83] [84] [85]

[m3G; v 1.134; Prn:26/05/2014; 16:00] P.28 (1-28)

N. Amálio, C. Glodt / Science of Computer Programming ••• (••••) •••–•••

B. Dobing, J. Parsons, How UML is used, Commun. ACM 49 (5) (2006) 109–113. E.M. Clarke, J.M. Wing, Formal methods: state of the art and future directions, ACM Comput. Surv. 28 (4) (1996) 626–643. J. Woodcock, P.G. Larsen, J. Bicarregui, J. Fitzgerald, Formal methods: practice and experience, ACM Comput. Surv. 41 (4) (2009) 19–36. C. Snook, R. Harrison, Practitioners’ views on the use of formal methods: an industrial survey by structured interview, Inf. Softw. Technol. 43 (2001) 275–283. D.L. Parnas, Formal methods technology transfer will fail, J. Syst. Softw. 40 (3) (1998) 195–198. G. Cleland, D. MacKenzie, Inhibiting factors, market structure and industrial uptake of formal methods, in: WIFT’95, IEEE, 1995, pp. 46–60. R. Baskerville, J. Pries-Heje, Short cycle time systems development, Inf. Syst. J. 14 (2004) 237–264. K. Kautz, S. Madsen, J. Nørberg, Persistent problems and practices in information systems development, Inf. Syst. J. 17 (2007) 217–239. N. Amálio, P. Kelsen, Modular design by contract visually and formally using VCL, in: VL/HCC 2010, IEEE, 2010, pp. 227–234. N. Amálio, C. Glodt, P. Kelsen, Building VCL models and automatically generating Z specifications from them, in: FM 2011, in: LNCS, vol. 6664, Springer, 2011, pp. 149–153. N. Amálio, P. Kelsen, Q. Ma, C. Glodt, Using VCL as an aspect-oriented approach to requirements modelling, in: TAOSD VII, in: LNCS, 2010, pp. 151–199. D.C. Schmidt, Model-driven engineering, IEEE Comput. 39 (2) (2006) 25–31. J.M. Spivey, The Z Notation: A Reference Manual, Prentice-Hall, 1992. ISO, Information technology—Z formal specification notation—syntax, type system and semantics, ISO/IEC 13568:2002, Int. Standard, 2002. J.-R. Abrial, The B Book: Assigning Meaning to Programs, Cambridge University Press, 1996. D. Jackson, Software Abstractions: Logic, Language, and Analysis, MIT Press, 2006. B. Meyer, Applying “design by contract”, Computer 25 (10) (1992) 40–51. N. Amálio, F. Polack, S. Stepney, Frameworks based on templates for rigorous model-driven development, Electron. Notes Theor. Comput. Sci. 191 (2007) 3–23. J. Whittle, J. Hutchinson, M. Roucefield, H. Burden, R. Heldal, Industrial adoption of model-driven engineering: are the tools really the problem?, in: MODELS 2013, in: LNCS, vol. 8107, Springer, 2013, pp. 1–17. N. Amálio, L. Briand, An empirical evaluation of visualisation in software design modelling: the VCL vs UML+OCL experiment, Tech. Rep. TR-LASSY13-05, LASSY, Univ. of Luxembourg, 2013, http://bit.ly/19ui1NP. J. Leemans, N. Amálio, A VCL model of a cardiac pacemaker, Tech. Rep. TR-LASSY-12-04, Univ. of Luxembourg, 2012, http://bit.ly/xiob5d. J. Leemans, N. Amálio, Modelling a cardiac pacemaker visually and formally, in: VL/HCC 2012, IEEE, 2012, pp. 257–258. N. Amálio, The VCL model of the Barbados crisis management system, Tech. Rep. TR-LASSY-12-09, Univ. of Luxembourg, 2012, http://bit.ly/W5C8ZY. N. Amálio, VCL model of the secure simple bank case study, Tech. Rep. TR-LASSY-11-05, Univ. of Luxembourg, 2011, http://bit.ly/q1LrPj. N. Amálio, The type system of VCL, Tech. Rep. TR-LASSY-12-10, Univ. of Luxembourg, 2012, http://bit.ly/gu7wv5. E. Gamma, R. Helm, R. Johnson, J. Vlissides, Design Patterns, Addison-Wesley, 1995. P. Malik, M. Utting, CZT: a framework for Z tools, in: ZB 2005, in: LNCS, vol. 3455, Springer, 2005, pp. 65–84. E. Tobias, Visual modelling of and on tangible user interfaces, Master’s thesis, Faculty of Science, Technology and Communication, University of Luxembourg, 2012, http://bit.ly/17keLxR. A.N. Oppenheim, Questionnaire Design, Interviewing and Attitude Measurement, Continuum, 1996. F.D. Davis, Perceived usefulness, perceived ease of use, and user acceptance of information technology, Manag. Inf. Syst. Q. 13 (3) (1989) 319–340. R Core Team, R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing, Vienna, Austria, 2013, http://www.R-project.org/. J. Rushby, Formal methods and their role in the certification of critical systems, Tech. Rep. SRI-CSL-95-1, Comp. Science Lab., SRI Int., 1995. M. Gogolla, F. Büttner, M. Richters, USE: a UML-based specification environment for validating UML and OCL, Sci. Comput. Program. 69 (2007) 27–34. R. France, B. Rumpe, Model-driven development of complex software: a research roadmap, in: Future of Software Engineering, IEEE, 2007, pp. 37–54. B. Selic, What will it take? a view on adoption of model-based methods in practice, Softw. Syst. Model. 11 (4) (2012) 513–526. J. Hutchinson, M. Rouncefield, J. Whittle, Model-driven engineering practices in industry, in: ICSE 2011, IEEE, 2011, pp. 633–642. C. Larman, Applying UML and Patterns, Prentice-Hall, 2005. A. Fish, J. Flower, J. Howse, The semantics of augmented constraint diagrams, J. Vis. Lang. Comput. 16 (2005) 541–573. P. Bottoni, M. Koch, F. Parisi-Presicce, G. Taentzer, A visualisation of OCL using collaborations, in: UML 2001, in: LNCS, vol. 2185, Springer, 2001, pp. 257–271. K. Ehrig, J. Winkelmann, Model transformation from visual OCL to OCL using graph transformation, Electron. Notes Theor. Comput. Sci. 152 (2006) 23–37. E. Tobias, E. Ras, N. Amálio, Suitability of visual modelling languages for modelling tangible user interface applications, in: VL/HCC 2012, IEEE, 2012, pp. 269–270. E. Tobias, E. Ras, N. Amálio, VML usability for modelling TUI scenarios — a comparative study, Tech. Rep. TR-LASSY-12-06, University of Luxembourg, LASSY, 2012, available at http://bit.ly/LMQTJJ. M. Lohmann, S. Sauer, G. Engels, Executable visual contracts, in: IEEE Symposium on Visual Languages and Human-Centric Computing, 2005, pp. 63–70. M. Lohmann, G. Engels, S. Sauer, Model-driven monitoring: generating assertions from visual contracts, in: ASE’06, IEEE, 2006, pp. 355–356. E. Guerra, J. de Lara, M. Wimmer, G. Kappel, A. Kusel, W. Retschitzegger, J. Schönböck, W. Schwinger, Automated verification of model transformations based on visual contracts, Autom. Softw. Eng. 20 (1) (2013) 5–46. R. Grønmo, B. Møller-Pedersen, G.K. Olsen, Comparison of three model transformation languages, in: ECMDA-FA 2009, in: LNCS, vol. 5562, Springer, 2009, pp. 2–17. J. Williams, F. Polack, Automated formalisation for verification of diagrammatic models, Electron. Notes Theor. Comput. Sci. 263 (2010) 211–226. C. Snook, M. Butler, UML-B: formal modeling and design aided by UML, ACM Trans. Softw. Eng. Methodol. 15 (1) (2006) 92–122. J. Iivari, Why are CASE tools not used?, Commun. ACM 39 (10) (1996) 94–103. G. Post, A. Kagan, R.T. Keim, A comparative evaluation of CASE tools, J. Syst. 44 (1998) 87–96. G. Post, A. Kagan, OO-CASE tools: an evaluation of rose, Inf. Softw. Technol. 42 (6) (2000) 383–388. S.M. Thomas, Evaluation of UML tools using an end-to-end application, Master’s thesis, Faculty of Engineering and the Built Environment, University of the Witwatersrand, 2003. L. Khaled, A comparison between UML tools, in: ICECS’09, IEEE, 2009, pp. 111–114. Heena Ranjna, A comparative study of UML tools, in: ACAI 2011, 2011. A. Bobkowska, K. Reszke, Usability of UML modeling tools, in: Software Engineering: Evolution and Emerging Technologies, IOS Press, 2005, pp. 75–86.