How do practitioners use conceptual modeling in practice?

Data & Knowledge Engineering 58 (2006) 358–380 www.elsevier.com/locate/datak

How do practitioners use conceptual modeling in practice? Islay Davies a, Peter Green b, Michael Rosemann Marta Indulska b, Stan Gallo b a

a,*

,

Faculty of Information Technology, Queensland University of Technology, 126 Margaret Street, Brisbane, Queensland 4000, Australia b UQ Business School, University of Queensland Ipswich, 4305, Australia Available online 25 July 2005

Abstract Much research has been devoted over the years to investigating and advancing the techniques and tools used by analysts when they model. As opposed to what academics, software providers and their resellers promote as should be happening, the aim of this research was to determine whether practitioners still embraced conceptual modeling seriously. In addition, what are the most popular techniques and tools used for conceptual modeling? What are the major purposes for which conceptual modeling is used? The study found that the top six most frequently used modeling techniques and methods were ER diagramming, data flow diagramming, systems flowcharting, workflow modeling, UML, and structured charts. Modeling technique use was found to decrease significantly from smaller to medium-sized organizations, but then to increase significantly in larger organizations (proxying for large, complex projects). Technique use was also found to significantly follow an inverted U-shaped curve, contrary to some prior explanations. Additionally, an important contribution of this study was the identification of the factors that uniquely influence the decision of analysts to continue to use modeling, viz., communication (using diagrams) to/from stakeholders, internal knowledge (lack of) of techniques, user expectations management, understanding modelsÕ integration into the business, and tool/software deficiencies. The highest ranked purposes for which modeling was undertaken were database design and management, business process documentation, business process improvement, and software development.  2005 Elsevier B.V. All rights reserved.

*

Corresponding author. Tel.: +738 649 473; fax: +738 649 390. E-mail addresses: [email protected] (I. Davies), [email protected] (P. Green), [email protected] (M. Rosemann), [email protected] (M. Indulska), [email protected] (S. Gallo). 0169-023X/$ - see front matter  2005 Elsevier B.V. All rights reserved. doi:10.1016/j.datak.2005.07.007

I. Davies et al. / Data & Knowledge Engineering 58 (2006) 358–380

359

Keywords: Conceptual modeling; Business systems analysis; Modeling techniques; Modeling tools; Project size; Modeler experience

1. Introduction The areas of business systems analysis, requirements analysis, and conceptual modeling are well-established research directions in academic circles. Comprehensive analytical work has been conducted on topics such as data modeling, process modeling, meta modeling, model quality, and the like. A range of frameworks and categorizations of modeling techniques have been proposed (e.g. [9,12,18]). However, these various frameworks and categorizations appear to lack extensive field study testing in practice. Thus, it is difficult to provide solid statements on the importance and potential impact of related research on the actual practice of conceptual modeling. More recently, Wand and Weber [23, p. 364] assume ‘‘the importance of conceptual modeling’’ and they state that ‘‘Practitioners report that conceptual modeling is difficult and that it often falls into disuse within their organizations.’’ Unfortunately, anecdotal feedback to us from information systems (IS) practitioners confirmed largely the assertion of Wand and Weber [23]. Accordingly, as researchers involved in attempting to advance the theory of conceptual modeling in organizations, we were concerned to determine that practitioners still found conceptual modeling useful and that they were indeed still performing conceptual modeling as part of their business systems analysis processes. Moreover, if practitioners still found modeling useful, why did they find it useful and what were the major factors that inhibited the wider use of modeling in their projects. In this way, the research that we were performing would be relevant for the practice of information systems development (see the IS Relevance debate on ISWorld, February 2001). For this research, we use the aspects of conceptual modeling as defined by Wand and Weber [23] to guide our work. Conceptual models are developed and used during the requirements analysis phase of information systems development. Such models are mostly graphic, and they are used to represent both static (e.g., entities) and dynamic phenomena (e.g., processes) in some domain. Conceptual modeling consists of a grammar (i.e., a set of constructs and rules to combine those constructs), a method (i.e., procedures by which the grammar can be used), a script (i.e., the product of the modeling process), and a context (i.e., the setting in which the modeling occurs). Accordingly, we are interested in the use of grammars like entity–relationship, data flow, and workflow. In a wider sense, we are also interested in the use of the methods or methodologies that apply one or more of these grammars, like rapid application development (RAD) or joint application development (JAD). Hence, the research in this paper is motivated in several ways. First, we want to obtain empirical data that conceptual modeling is indeed being performed now and into the foreseeable future in IS practice in Australia. Second, we want to find out what are the principal tools, techniques, and purposes for which conceptual modeling is performed currently in Australia. In this way, researchers can obtain valuable information to help them direct their research towards aspects of conceptual modeling that contribute most to practice. In particular, in line with this motivation, we are keen to examine how the conceptual modeling is performed and indeed if there are differences due to the size of the organization (i.e., the size of projects) and the years of experience

360

I. Davies et al. / Data & Knowledge Engineering 58 (2006) 358–380

of the modeler(s). We were particularly interested in the impact of these two factors. Organization size often proxies for sophistication of use and budgetary capability to experiment with relatively expensive tools and techniques (e.g., [2,5,14]). Moreover, larger organizations tend to be involved with development projects that are relatively larger in size and complexity (e.g., [1,5,14]). Years of experience in modeling is a proxy for the novice-expert dichotomy. Hence, it is reported commonly in the literature as being a significant driver in the choice and use of techniques and tools in modeling (e.g., [7,13,14]). Finally, we were motivated to perform this study so that we could gather and analyze data on major issues and benefits specific to the task of conceptual modeling in practice. So, this research aims to provide current insights into actual modeling practice. The underlying research question is ‘‘How do practitioners actually use conceptual modeling in practice?’’ The derived and more detailed sub-questions are: 1. What are the popular tools and techniques used for conceptual modeling in Australia? Is the selection and use of these tools and techniques markedly different because of the size of the organization (i.e., project) using them or the years of experience in modeling of the people using them? 2. What are the purposes of modeling? 3. What are major problems and benefits specific to conceptual modeling? In order to provide answers for these questions, an empirical study using a web-based survey was conducted. In summary, we found that the current state of usage of business systems/conceptual modeling in Australia is: ER diagramming, data flow diagramming, systems flowcharting, and workflow modeling being most frequently used for database design and management, software development, documenting and improving business processes. The use of these techniques decreases significantly as we move from smaller to medium-sized organizations. They increase significantly in use as we move from medium to larger organizations. Tool use follows a similar pattern but not significantly so. Modeling technique use was found to follow significantly an inverted U-shaped pattern as modelers gained experience in modeling. Another contribution of this study is the analysis of textual data concerning problems and benefits in the continued use of conceptual modeling. Clearly, relative advantage (disadvantage)/usefulness from the perspective of the analyst was the major driving factor influencing the decision to continue (discontinue) modeling. Furthermore, the modeling work of respondents is supported in most cases by the use of Visio (in some version) as an automated tool. Visio is more heavily used proportionately by respondents in larger organizations than those in smaller organizations. The remainder of the paper unfolds in the following manner. The next section reviews the background of conceptual modeling practice, and it develops a series of hypotheses around the influence of organization size (i.e., project size and complexity) and years of modeling experience on the use of modeling techniques and tools. The third section explains briefly the instrument and methodology used. Then, major quantitative results of the survey are reviewed with particular emphasis on differences driven by organization size and years of modeling experience. The fifth section presents the dominant results of the analysis of the textual data on the problems and benefits of modeling. The last section concludes and gives an indication of further work planned.

I. Davies et al. / Data & Knowledge Engineering 58 (2006) 358–380

361

2. Background and hypotheses Over the years, much work has been done on how to do modeling—the quality, correctness, completeness, goodness of representation, understandability, differences between novice and expert modelers, and many other aspects (e.g., [16]). Comparatively little empirical work, however, has been undertaken on modeling in practice. Floyd [8] and Necco et al. [17] conducted comprehensive empirical work into the use of modeling techniques in practice but that work is now considerably dated. Batra and Marakas [3] attempted to address this problem of a lack of current evidence from practice however their work focused on comparing the perspectives of the academic and practitioner communities regarding the applications of conceptual data modeling. Indeed, these authors simply reviewed the academic and practitioner literatures without actually collecting primary data on the issue. Moreover, their work is now dated. However, it is interesting that they (p. 189) observe that ‘‘there is a general lack of any substantive evidence, anecdotal or empirical, to suggest that the concepts are being widely used in the applied design environment.’’ Batra and Marakas [3, p. 190], state that ‘‘Researchers have not attempted to conduct case or field studies to gauge the cost-benefits of enterprise-wide conceptual data modeling (CDM).’’ This research has attempted to address the problems alluded to by Batra and Marakas [3]. Iivari [10] provided some data on these questions in a Finnish study of the perceptions of effectiveness of CASE tools. However, he found the adoption rate of CASE tools by developers in organizations very low (and presumably the extent of conceptual modeling to be low as well). Gorla et al. [9] empirically investigated the area of systems analysis practice but their study was limited to tools and, in particular, process tools. Curtis et al. [5] found that the productivity of the developers and quality of developed systems were affected by limited domain knowledge, fluctuating requirements and communication breakdowns. The focus of their work was developersÕ productivity and resultant system quality. They were not concerned with what techniques and tools were used, nor for what purposes they were used. Fitzgerald [7] also investigated empirically the adoption of various methodologies and techniques for systems development. However, his work was more concerned with how these methodologies and techniques were used to develop the systems. In particular, he found that different methods are used in a pragmatic way resulting in a unique instantiation of the method for each development project. More recently, Persson and Stirna [19] noted the problem of lack of empirical investigation, however, their work was limited in that it was only an exploratory study into practice. Most recently, Chang et al. [4] conducted 11 interviews with experienced consultants in order to explore the perceived advantages and disadvantages of business process modeling. This descriptive study did not, however, investigate the critical success factors of process modeling. Sedera et al. [20] have conducted three case studies to determine a process modeling success model, however they have not yet reported on a planned empirical study to test this model. Furthermore, the studies by Chang et al. [4] and Sedera et al. [20] are limited to the area of process modeling. 2.1. Organization size Large organizations are involved typically with large systems and large system development [5]. Prior studies have alluded to the differences encountered in such developments in terms of

362

I. Davies et al. / Data & Knowledge Engineering 58 (2006) 358–380

complexity and sophisticated interfaces [2,5,14]. Indeed, Avison and Fitzgerald [1] suggest that developersÕ backlash to modeling and methodologies (and the tools they employ) arise for several reasons. The most common is disappointing productivity. However, another major reason is that the methodologies, techniques and tools are overly complex, usually designed for the largest and most complex development projects. Indeed, strict adherence to the technique may result in developing requirements to the ultimate degree, often over and above what is legitimately required. Technical skills required by developers may be difficult and expensive to acquire, and the associated tools may be costly to implement and difficult to use. In view of these issues, we would expect that generally as the size and complexity of the projects increased significantly, we would see an increase in the use of modeling techniques (as they are used in methodologies) and their associated tools. However, this situation does not imply that modeling does not occur in smaller organizations involved in smaller, less complex projects. Indeed, we expect a certain level of modeling and simple tool use for requirements analysis, however, this level of use will drop off as organizations (and projects) increase in size and complexity. This increase will occur until a threshold point of size and complexity is reached that allows organizations to justify the expense, level of difficulty, and time involved in the use of the detailed techniques and tools to acquire comprehensively the requirements of large, complex projects. Accordingly, Proposition 1. The use of modeling techniques and tools by organizations will decrease with their size until a threshold point of project size and complexity is reached when the use of the techniques and tools will increase with the size of organization. H1a. The use of modeling techniques will decrease significantly as organizations move from smaller to medium size. H1b. The use of modeling techniques will increase significantly as organizations move from medium to larger in size. H1c. The use of modeling tools will decrease significantly as organizations move from smaller to medium size. H1d. The use of modeling tools will increase significantly as organizations move from medium to larger size. 2.2. Years of modeling experience Fitzgerald [6] conducted a series of in-depth interviews with two developers from eight organizations ranging in size from 50 employees to 16,000 employees. Based on the evidence from the interviews, he determined that a U-shaped curve seemed to capture the relationship between developer experience and methodology use. In other words, he believed that inexperienced developers adhered strongly to methods, techniques and tools when attempting to elicit requirements and develop a system. This adherence could be due to their previous educational exposure or to the fact that inexperienced developers require a useful template for the development process. However, as experience is gained, the use of techniques and tools falls away dramatically as the developers become disenchanted with the lack of productivity from the use of such techniques and tools. As developers become highly experienced (or expert), Fitzgerald [6] posits that their use

I. Davies et al. / Data & Knowledge Engineering 58 (2006) 358–380

363

soars as they derive a sensible tailored set of methods, techniques, and tools at an appropriate level of granularity for their environment. Lee and Truex [15] find this U-shaped curve explanation of use appealing when they explain that this curve may be a collection of small oscillating curves depicting smaller cycles of cognitive loosening and tightening of the IS developerÕs thinking as they gain experience in systems development. Indeed, they posit that by shortening the smaller cycles of cognitive change, we may be able to more quickly help developers reach the ‘‘mature phase’’ of informed methods use [p. 362]. By contrast, Kautz et al. [14] find that the developersÕ experience with regard to the development process and the application domain influences method utilization. Indeed, experienced developers use their domain knowledge in many situations instead of the prescribed methods and they develop systems without going through formal step by step guides. On the other hand, less experienced developers express a need for explicated methodologies that can help them learn the companyÕs development practices. Accordingly, we expect that the use of techniques and their associated tools will be high with inexperienced developers/modelers but then as they gain experience, we expect the use of formal techniques and tools to decrease significantly. Indeed, rather than a U-shaped curve hypothesized by Fitzgerald [6] and Lee and Truex [15], we expect to see an inverted U-shaped curve representing the use of techniques and tools by modelers as they move from inexperienced (novice) to highly experienced (expert) developers. Accordingly, Proposition 2. The use of techniques and tools by developers will increase initially with the gaining of experience until a point is reached when the developers can use their experience to overview the application domain and analyze systems without having to go through formal step by step guides. H2a. The use of modeling techniques will increase significantly as years of modeling experience increase to the point of becoming an ‘‘expert’’. H2b. The use of modeling techniques will decrease significantly as years of modeling experience increase from the point of becoming an ‘‘expert’’. H2c. The use of modeling tools will increase significantly as years of modeling experience increase to the point of becoming an ‘‘expert’’. H2d. The use of modeling tools will decrease significantly as years of modeling experience increase from the point of becoming an ‘‘expert’’.

3. Methodology This study was conducted in the form of a web-based survey issued with the assistance of the Australian Computer Society (ACS) to its members. The survey consisted of seven pages.1 The first page explained the objectives of our study. It also highlighted the available incentive, i.e., free participation in one of five workshops on business process modeling. The second page asked for the purpose of the modeling activities. In total, 17 purposes (e.g., database design

1 A copy of the survey pages is available from the authors on request. Appendix A demonstrates the page that gathered the data on the techniques used, the third page of the survey.

364

I. Davies et al. / Data & Knowledge Engineering 58 (2006) 358–380

and management, software development) were made available. The respondents were asked to evaluate the relevance of each of these purposes using a five-point Likert scale ranging from 1 (not relevant) to 5 (highly relevant). The third page (shown in Appendix A) asked for the modeling techniques2 used by the respondent. It provided a list of 18 different modeling techniques ranging from data flow diagram and ER diagrams, to the various IDEF standards, up to UML. For each modeling technique, the participants had to provide information about the past, current and future use of the modeling technique. It was possible to differentiate between infrequent and frequent use. Furthermore, participants could indicate whether they knew the technique or did not use it at all. It was possible also to add further modeling techniques that they used. The fourth page was related to the modeling tools. Following the same structure as for the modeling technique, a list of 24 modeling tools was provided. A hyperlink provided a reference to the homepage of each tool provided. It was clarified also if a tool had been known under a different name (e.g., Designer2000 for the Oracle9i Developer Suite). The fifth page explored qualitative issues. Participants were asked to list major problems and issues they had experienced with modeling as well as perceived key success factors. On the sixth page, demographic data was collected. In particular, years of experience in business systems analysis and modeling and the size of the organization in which the respondent works were collected on this page. The seventh page allowed contact details for the summarized results of the study and the free workshop to be entered. The instrument was piloted with 25 members of two research centers as well as with a selected group of practitioners. Minor changes were made based on the experiences within this pilot. 3.1. Step 1 of the analysis of problems and benefits—Nvivo 2 A contribution of this paper is an examination of the data gathered through the fifth page of the survey. This section of the survey asked respondents to list problems and benefits for them in the use of conceptual modeling in their organizations. The phenomena that responses to these questions allowed us to investigate were why do we continue/discontinue to use a technical method (implemented using a technological tool)—conceptual modeling. To analyze these phenomena, we used the following procedure: 1. What responses confirm the factors we already know about from prior studies on the adoption of technological techniques and tools in regard to these phenomena; and 2. What responses are identifying new factors that are specific to the domain of conceptual modeling? To achieve step 1, we performed a review of the current thinking and literature in the areas of adoption and continued use of a technology to identify the major factors that have been used to explain the adoption of technological methods and tools. Then, using Nvivo 2, one researcher classified the textual comments, where relevant, according to these known factors. Independently, this first researcherÕs classification was then reviewed by a second researcher. The two researchers then met to confirm their points of agreement in the coding and to reconcile their points of 2 ÔTechniqueÕ here is used as an umbrella term referring to the constructs of the technique, their rules of construction, and the heuristics and guidelines for refinement i.e., the conceptual modeling grammar of Wand and Weber [23].

I. Davies et al. / Data & Knowledge Engineering 58 (2006) 358–380

365

difference. Nvivo 2 was used to perform this initial step of the analysis because it is a well-accepted analytical tool for textual data. In particular, it is useful when an a priori model or set of factors exists. In this case, we had obtained from the literature a comprehensive set of factors that have been used to explain the adoption of such technological phenomena. In other words, we had an a priori model with which to ‘‘code’’ up the data. Researchers can use this set of factors to ‘‘code’’ up their textual data in Nvivo. Nvivo then allows the researcher to categorize and summarize the coded results easily. 3.2. Step 2 of the analysis of problems and benefits—Leximancer After step 1, there remained textual comments that did not readily fit into one or other of the known factor categories. These unclassified responses had the potential to provide us with insight on factors unique and important to the domain of conceptual modeling. However, the question was how to derive this information in a relatively objective and unbiased manner from the textual data. We used a new state-of-the-art textual content analysis tool called Leximancer.3 Using this tool, we identified from the unclassified text five new factors specific to conceptual modeling. Subsequently, one researcher again classified the remaining responses using these newly identified factors. His classification was reviewed and confirmed by a second researcher. Finally, the relative importance of each of the new factors was determined. The Leximancer system allows its users to analyze large amounts of text quickly. The tool performs this analysis both systematically and graphically by creating a map of the constructs—the document map. The constructs are displayed in such a manner that links to related subtext may be subsequently explored. Each of the words on the document map represents a concept that was identified. The concept is placed on the map in proximity of other concepts in the map through a derived combination of the direct and indirect relationships between those concepts. Essentially, the Leximancer system is a machine-learning technique based on the Bayesian approach to prediction. The procedure used for this is a self-ordering optimization technique and does not use neural networks. Once the optimal weighted set of words is found for each concept, it is used to predict the concepts present in fragments of related text. In other words, each concept has other concepts that it attracts (or is highly associated with contextually) as well as concepts that it repels (or is highly disassociated with contextually). The relationships are measured by the weighted sum of the number of times two concepts are found in the same ÔchunkÕ. An algorithm is used to weight them and determine the confidence and relevancy of the terms to others in a specific chunk and across chunks. Leximancer was selected for this qualitative data analysis for several reasons: • Its ability to derive the main concepts within text and their relative importance using a scientific, objective algorithm. • Its ability to identify the strengths between concepts (how often they co-occur)—centrality of concepts. • Its ability to assist the researcher in applying grounded theory analysis to a textual dataset. 3 For more information on Leximancer, see http://www.leximancer.com. This site includes many case studies of the use of Leximancer on large volumes of textual data.

366

I. Davies et al. / Data & Knowledge Engineering 58 (2006) 358–380

• Its ability to assist in visually exploring textual information for related themes to create new ideas or theories; and • Its ability to assist in identifying similarities in the context in which the concepts occur—contextual similarity. In particular, Leximancer is a useful tool when a researcher is exploring the textual data to attempt to uncover important factors. In other words, it is highly useful when the researcher does not have an a priori set of factors or model by which to analyze the data. Of course, like all software solutions, Leximancer is not without its problems. Smith and Humphreys [21] report how they have used Leximancer to analyze a 50 MB sample of Usenet news postings, the novel ‘‘Pride and Prejudice’’, the King James Bible, and other large texts. They note that resource problems can be encountered during the phases of learning, indexing, and clustering. Moreover, the algorithm for the learning phase may produce arbitrary results as no suitable benchmarking training data has been found to date. However, in all case studies to date, Leximancer has been highly successful for learning and classifying from the same body of text.

4. Survey results and discussion From 645 individuals who started to fill out the survey, 312 actually completed the entire survey, which leads to a completion rate of 48.4%. Although our survey invitation email was supposed to go out to members from all over Australia, we are highly doubtful that it did, as we had to rely on the individual state branches to send out the email. Of the 12,000 members of the ACS, 1567 indicated in their most recent membership profiles that they were interested in conceptual modeling/business systems analysis. Accordingly, our 312 responses indicate a response rate of 3% if the full membership received the email invitation. However, it is more likely that the survey would only be completed by those members who had an interest in conceptual modeling/systems analysis. So, a relevant response rate of 19.9% is more likely. In any circumstances, reporting a response rate on a web-based survey is arbitrary because the researcher cannot control for people outside the sampling frame happening on the website and filling in the survey. Moreover, we offered participation in one of five seminars on business process modeling free of charge as an inducement for members to participate. This offer was accepted by 176 of the 312 respondents. Corresponding with the nature of the ACS as a professional organization, 87% of the participants were practitioners. The remaining respondents were academics (6%) and students (6.7%). It is also not a surprise that 85% of the participants characterized themselves as an IT service person while only 15% referred to themselves as a business person or end user. Fifty-three percent of the respondents indicated that they gained their knowledge in Business Systems Analysis from University. Further answers were TAFE (Technical and Further Education) (4.5%), and ACS (2.3%). Sixteen percent indicated that they did not have any formal training in Business Systems Analysis. Twenty-four percent of the respondents indicated that they have less than 4 years experience with modeling. Thirty-seven percent have between 4 and 10 years of experience. A significant proportion, 39%, has more than 10 years of experience with modeling. Forty-nine percent of respondents indicated that they worked in firms employing less than 100 people. However, 10% of the respondents worked in organizations with 100 to 1000 employees,

I. Davies et al. / Data & Knowledge Engineering 58 (2006) 358–380

367

Table 1 Top six modeling techniques most frequently used Technique

Frequent use

%

Infrequent use

%

Not used/known

%

ER diagram Data flow diagram System flowcharts Workflow modeling UML (unified modeling language) Structured charts

132 105 90 69 66 49

42 34 29 22 21 16

60 82 82 75 49 64

19 26 26 24 16 20

120 125 140 168 197 199

38 40 45 54 63 64

and 29% indicated that they worked in organizations employing more than 1000 employees. So, by Australian standards, they would be involved in software projects of reasonable size. Table 1 presents from the data the top six most frequently used modeling techniques. It describes the usage of techniques as frequently used, infrequently used (which in the survey instrument was defined as used less than five times per week), and, not known or not used. The table clearly demonstrates that the top six most frequently used (used 5 or more times a week) techniques are ER diagramming, data flow diagramming, systems flowcharting, workflow modeling (range of workflow modeling techniques), UML, and structured charts.4 It is significant to note that even though object-oriented analysis, design, and programming has been the predominant paradigm for systems development over the last decade, 63% of respondents either did not know or did not use UML. While not every conceptual modeling technique available was named in the survey, the eighteen techniques used were selected based on their popularity reported in prior literature. Moreover, while not explicitly reported in Table 1, this current situation of non-usage appears to be set to increase into the short-term future (next 12 months) as the planned frequent use of the top four techniques is expected to drop to less than half its current usage, viz., ER diagramming (19%), data flow diagramming (15%), systems flowcharting (9%), and workflow modeling (13%). Furthermore, no increase in the intention to use any of the other techniques was reported, to balance this out. Accordingly, respondents may perceive a reduction of new developmental work requiring business systems modeling in the short-term future. It may also just reflect the lack of planning of future modeling activities. Our work was also interested in what tools were used to perform the conceptual modeling work that was currently being undertaken. Table 2 presents the top six most frequently used tools when performing business systems analysis and design. The data is reported using the same legend as that used for Table 1. Again, while not every conceptual modeling tool available was named in the survey, the tools were selected based on their popularity reported in prior literature. Table 2 clearly indicates that Visio (61%—both infrequent and frequent use) is the preferred tool of choice for business systems modeling currently. This result is not surprising as the top four most frequently used techniques are well supported by Visio (in its various versions). A long way second in frequent use is Rational Rose (21%—both infrequent and frequent use) reflecting the current level of use of object-oriented 4 RAD was nominated with 23% frequent use, however it was not reported in Table 1 in line with our focus on conceptual modeling grammars or techniques.

368

I. Davies et al. / Data & Knowledge Engineering 58 (2006) 358–380

Table 2 Top six most frequently used tools Tool

Frequent use

%

Infrequent use

%

Not used/known

%

Visio Rational Rose Oracle9i Developer Suite iGrafx FlowCharter AllFusionERwin Data Modeler WorkFlow Modeler

137 34 20 17 10 5

44 11 6 5 3 2

52 31 28 42 12 2

17 10 9 13 4 1

123 247 264 253 290 305

39 79 85 81 93 97

analysis and design techniques. Again, at least 40% of respondents (approximately) do either not know or use any of the 24 tools named in the survey—even a relatively simple tool like Flowcharter or Visio. Moreover, while not explicitly reported in Table 2, into the short-term future (next 12 months), the planned frequent use of the top two tools is expected to drop significantly from their current usage levels, viz., Visio (23%) and Rational Rose (9%) with no real increase reported for planned use of other tools to compensate for this drop. Again, this trend in planned tool usage appears to reflect the fact that respondents expect a reduction in new developmental work requiring business systems modeling in the short-term future. Table 3 now presents those top six modeling techniques most frequently used broken down according to the size of the organization of the respondent. Each percentage column in the table represents the ratio of the number of respondents in that row to the total of respondents for that size category. Use of these techniques is strong in smaller organizations (<100). We observe a significant decrease in use as we move to organizations of medium size (t-stat = 5.01017). Then we observe a significant increase in usage of techniques as we move into large organizations (>1000) (tstat = 5.153408). It appears that as we originally hypothesized, if size of organization proxies for size and complexity of project, then in small to medium-sized organizations we see a decrease in the use of techniques until the size and complexity of the projects require more formal methods of requirements analysis. Accordingly, H1a and H1b appear to be supported. This support is mitigated however by the fact that there is an uneven spread of responses from participants across the various organization size categories. The situation is depicted in Fig. 1 below. Table 3 Top six most frequently used techniques by size of organization Technique

<100

%

<1000

%

P1000

%

ER diagram Data flow diagram System flowcharts Workflow modeling UML (unified modeling language) Structured charts

70 55 42 34 31 24

42 33 25 20 19 14

7 4 10 9 10 3

27 15 38 35 38 12

41 36 34 24 21 16

46 40 38 27 23 18

t-Statistic Sig.

5.01017 <0.01

5.153408 <0.01

Average of Technique Use

I. Davies et al. / Data & Knowledge Engineering 58 (2006) 358–380

369

Technique Use by Size of Organization 50 40 30 20 10 0

<100

< 1000

>=1000

Size of Organization

Fig. 1. Technique use by size of organization.

Table 4 Top six most frequently used tools by size of organization Tool

<100

%

<1000

%

P1000

%

Visio Rational Rose Oracle9i Developer Suite iGrafx FlowCharter AllFusion ERwin Data Modeler WorkFlow Modeler

45 12 9 4 3 3

27 7 5 2 2 2

36 9 5 7 4 0

47 12 6 9 5 0

52 10 6 6 3 2

58 11 7 7 3 2

t-Statistic Sig.

0.29407 ns

0.316382 ns

Furthermore, it appears from Table 3 that small organizations (<100) appear to employ greater use relatively of the well-established structured techniques of ER diagramming, data flow diagramming, and systems flowcharts. By contrast, the respondents from larger organizations (>1000) seem to make greater use proportionately of the more recent (some would argue, more complex) object-oriented techniques such as workflow modeling and UML. Table 4 shows how the top 6 most frequently used tools are employed by respondents in organizations of different sizes. While the data on the use of tools is swamped by the prevalence of the use of Visio (in some form), Table 4 does show that Visio is more heavily used proportionately by respondents in medium and larger organizations (<1000, >1000) than those respondents in smaller organizations. Also, in line with expectations, it appears that more modern, more complex tools like Rational Rose and Oracle Developer are used proportionately more by modelers in larger organizations than those in smaller organizations. While we see a decrease in the use of tools from smaller organizations to medium organizations, the difference is not significant. Similarly, as predicted, we see an increase in tool usage as we move from medium organizations to larger organizations but again the difference is not significant. Accordingly, while the directions of tool use are as predicted, H1c and H1d are not supported. Table 5 demonstrates the top six modeling techniques most frequently used stratified according to the years of modeling experience of the respondent. Table 5 shows that inexperienced developers make strong use of techniques. Indeed, there is a significant increase in usage from the 0 to 3 years level to the 4–10 years level of experience

370

I. Davies et al. / Data & Knowledge Engineering 58 (2006) 358–380

Table 5 Top six most frequently used techniques by years of modeling experience Technique

0–3 yrs

%

4–10 yrs

%

11–20 yrs

%

>20 yrs

%

ER diagram Data flow diagram System flowcharts Workflow modeling UML Structured charts

30 26 14 17 11 10

39 34 18 22 14 13

47 31 41 30 31 21

40 26 35 26 26 18

38 27 23 15 15 11

50 36 30 20 20 14

17 21 12 7 9 7

40 49 28 16 21 16

3.0846 <0.05

Average of Technique Use

t-Statistic Sig.

2.1781 <0.05

1.9834 ns

Technique Use by Experience 40 30 20 10 0 3

10

20

>20

Years of experience

Fig. 2. Technique use by years of modeling experience.

(t-stat = 3.0846). From that point however, we see a significant decrease in technique usage as practitioners move into the 11–20 years of modeling experience level (t-stat = 2.1781). Indeed this trend continues into the greater than 20 years of experience level, although the decrease is not significant (t-stat = 1.9834). Fig. 2 demonstrates the inverted U-shaped curve that emerges in technique use as modeling experience increases. These results support H2a and H2b. Indeed, these results appear to support the reasoning of Kautz et al. [14] and they appear to contradict the explanations of Fitzgerald [6] and Lee and Truex [15]. Table 6 shows how the use of the most frequently used tools is distributed across the experience groups. Table 6 Top six most frequently used tools by years of modeling experience Tool

0–3 yrs

%

4–10 yrs

%

11–20 yrs

%

>20 yrs

%

Visio Rational Rose Oracle9i Developer Suite iGrafx FlowCharter All Fusion ERwin Data Modeler WorkFlow Modeler

28 6 4 3 2 1

37 8 5 4 3 1

60 13 8 7 4 1

51 11 7 6 3 1

36 10 6 5 3 3

47 13 8 7 4 4

13 5 2 2 1 0

30 12 5 5 2 0

t-Statistic Sig.

0.8187 ns

0.4788 ns

1.1984 ns

I. Davies et al. / Data & Knowledge Engineering 58 (2006) 358–380

371

Table 6 appears to tell a similar story of use across the top 6 most frequently used tools. That is, the usage of tools appears to increase as modelers move from 0 to 3 years of experience to the 4– 10 years level, but not significantly so. Once modelers are highly experienced (>10 years), their use of modeling tools decreases, although not significantly so. While the directions of the predictions are correct, the differences in tool usage by modelers as they gain modeling experience are not significant. So, H2c and H2d are not supported. However, it would appear that those highly experienced modelers may be able to bring so much domain knowledge to bear that they do not require a computer-based tool to support their understanding and conception of the problem. This explanation is in line with that provided by Kautz et al. [14]. Business systems modeling (conceptual modeling) must be performed for some purpose. Accordingly, we were interested in obtaining data on the various purposes for which people might be undertaking modeling. Using a five-point Likert scale (where 5 indicates very frequent use), Table 7 presents (in rank order from the highest to the lowest score) the average score for purpose of use from the respondents. Table 7 indicates that database design and management remains the highest average purpose for use of modeling techniques. This fact links to the earlier result of ER diagramming being the most frequently used modeling technique. Moreover, software development as a purpose would support the high usage of data flow diagramming and ER diagramming noted earlier. Indeed, the relatively highly regarded purposes of documenting and improving business processes, and managing workflows, would support further the relatively high usage of workflow modeling and flowcharting indicated earlier. The more specialized tasks like identifying activities for activity-based costing and internal control purposes in auditing appear to be relatively infrequently used purposes for modeling. This fact however may derive from the type of population that was used for the survey, viz., members of the Australian Computer Society.

Table 7 Average use score for modeling purpose (in rank order) Purpose

Average score

Database design and management Business process documentation Improvement of internal business processes Software development Improvement of collaborative business processes Workflow management Design of enterprise architecture Change management Knowledge management End user training Software configuration Software selection Certification/quality management Human resource management Activity-based costing Auditing Simulation

3.9 3.8 3.8 3.8 3.5 3.4 3.4 3.4 3.2 3.1 3.1 2.9 2.8 2.6 2.6 2.5 2.5

372

I. Davies et al. / Data & Knowledge Engineering 58 (2006) 358–380

5. Textual analysis results and discussion In the first step of the analysis of the textual data obtained on the problems and benefits of conceptual modeling for practitioners, we identified from a survey of the literature on the adoption of technological methods and tools the factors that had been shown to influence that adoption. The results of that search are summarized and defined in Table 8. This table identifies the factors that

Table 8 Summary of factors identified for initial analysis Factor

Definition

Source(s)

Relative advantage

The degree to which adopting/using the technique is perceived as being better than using the practice it supersedes The degree to which adoption/usage of the technique is perceived to enhance ones image or status The degree to which adopting the technique is compatible with the individualÕs job responsibilities and value system The degree to which using a particular technique is free from effort The degree to which one can experiment with the technique on a limited basis before making an adoption or rejection decision The degree of perceived risk that accompanies the adoption of the technique The degree to which the technique is visible within the organization The degree to which results of adopting/using the technique are observable and communicable to others Generated by the normative beliefs that a respondent attributes to what relevant others (colleagues/peers/respected management) expect them to do with respect to adopting the technique as well as their motivation to comply with those beliefs Self-confidence in a participantÕs own ability to perform a behavior Availability of and ease of access to, technological infrastructure and support Degree to which decisions are motivated by accepting information from expert sources and integrating it into ones cognitive system Decisions resulting from feeling some bond with a likeable source Degree of influence that is produced by a powerful source having control over the respondent in the forms of rewards and punishments Degree of support for the project from middle and upper management of the organization Degree to which the decisions or attitudes were affected by communications problems between the respondents and key stakeholders within the organization

Karahanna et al. [11]

Image Compatibility

Complexity Trialability

Risk Visibility Results demonstrability Subjective norms

Self efficacy Facilitating conditions Internalizations

Identification Compliance

Top management support Communication issues

Karahanna et al. [11] Tan and Teo [22]

Karahanna et al. [11] Karahanna et al. [11]

Tan and Teo [22] Karahanna et al. [11] Karahanna et al. [11] Karahanna et al. [11]

Tan and Teo [22] Tan and Teo [22] Karahanna et al. [11]

Karahanna et al. [11] Karahanna et al. [11]

Tan and Teo [22] Karahanna et al. [11]

I. Davies et al. / Data & Knowledge Engineering 58 (2006) 358–380

373

Table 9 Results of classification by key factors influencing continued use (after phase 1) Key Relative advantage/usefulness Complexity Compatibility Internalizations Top management support Facilitating conditions Communication issues Subjective norms Self-efficacy Risk Results demonstrability Trialability Visibility Identification Compliance Image Unclassified Total (all records)

Percentage (%) 45 8 7 6 5 4 3 2 1 1 1 0 0 0 0 0 17 100.00

Totals 441 74 69 54 48 42 25 22 14 11 5 4 2 2 2 0 165 980

we already know from prior studies will influence the adoption and continued use of a technological process such as conceptual modeling. Nine hundred and eighty (980) individual comments were received across the questions on problems and benefits of modeling. Using the known factors (Table 8) influencing continued use of new technologies in firms, Table 9 shows the classification of the 980 comments after phase 1 of the analysis using Nvivo. Clearly, relative advantage (disadvantage)/usefulness from the perspective of the analyst was the major driving factor influencing the decision to continue (discontinue) modeling. Does conceptual modeling (and/or its supporting technology) take too much time, make my job easier, make my job harder, and make it easier/harder for me to elicit/confirm requirements with users? Such comments typically contributed to this factor. Furthermore, it is not surprising to see that complexity of the method and/or tool, compatibility of the method and/or tool with the responsibilities of my job, the views of ‘‘experts’’, and top management support were other major factors driving analystsÕ decisions on continued use. Prior literature had told us to expect these results, in particular, the key importance of top management support to the continued successful use of such key business planning and quality assurance mechanisms as conceptual modeling for systems. However, nearly one-fifth of the comments remained unclassified. Were there any new, important factors unique to the conceptual modeling domain contained in this data? Fig. 3 shows a document (concept) map produced by Leximancer from the unclassified comments. Five factors were identified from this map using the centrality of concepts and the relatedness of concepts to each other within identifiable ÔchunksÕ. While the resolution of the Leximancer generated concept map (Fig. 3) may be difficult to read on its own here, the concepts (terms) depicted are referred to within the discussion of the relevant factors below.

374

I. Davies et al. / Data & Knowledge Engineering 58 (2006) 358–380

Fig. 3. Concept map produced by Leximancer on the unclassified comments.

5.1. Internal knowledge (lack of) of techniques This group centered on such concepts as knowledge, techniques, information, large, easily and lack. Related concepts were work, systems, afraid, UML and leading. Accordingly, we used these concepts to identify this factor as the degree of direct/indirect knowledge (or lack of) in relation to the use of effective modeling techniques. Highlighted inadequacies raise issues of the modelerÕs skill level and questions of insufficient training. 5.2. User expectations management This group centered on such concepts as expectations, stakeholders, audience and review. Understanding, involved, logic and find were related concepts. Consequently, we used these items to identify this factor as issues arising from the need to manage the expectations of users as to what they expect conceptual modeling to do for them and to produce. In other words, the analyst must ensure that the stakeholders/audience for the outputs of conceptual modeling have a realistic understanding of what will be achieved. Continued (discontinued) use of conceptual modeling may be influenced by difficulties experienced (or expected) with users over such issues as acceptance, understanding and communication of the outcomes of the modeling techniques. 5.3. Understanding the models integration into the business This group centered on understanding, enterprise, high, details, architecture, logic, physical, implementation and prior. Accordingly, we identified a factor as the degree to which decisions are affected by stakeholder/modelerÕs perceived understanding (or lack of) in relation to the models

I. Davies et al. / Data & Knowledge Engineering 58 (2006) 358–380

375

Table 10 Relative importance of factors specific to conceptual modeling Key Communication (diagrams) to/from stakeholders Internal knowledge (lack of) of techniques User expectations management Understanding models integration into the business Tool/software deficiencies Total

Percentage (%)

Total

28 27 18 17 10

46 44 30 28 17

100

165

integration into business processes (initial and ongoing). In other words, for the user, to what extent do the current outputs of the modeling process integrate with the existing business processes and physical implementations to support the goals of the overall enterprise architecture? 5.4. Tool/software deficiencies This group was focused on such concepts as software, issues, activities, and model. Subsequently, a factor was identified as the degree to which decisions are affected by issues relating directly to the perceived lack of capability of the software and/or the tool design. 5.5. Communication (using diagrams) to/from stakeholders This final group involved such concepts as diagram, information, ease, communication, method, examples, and articulate. Related concepts were means, principals, inability, hard, audience, find, and stakeholders. From these key concepts, we deduced a factor as the degree to which diagrams can facilitate effective communication between analysts and key stakeholders in the organization. In other words, to what extent can the use of diagrams enhance (hinder) the explanation to, and understanding by, the stakeholders of the situation being modeled? Using these five new factors, we revisited the unclassified comments and, using the same dual coder process as before, we confirmed a classification for those outstanding comments easily. Table 10 presents this classification and the relative importance of those newly identified factors. As can be seen from Table 10, communication using diagrams and internal knowledge (lack of) of the modeling techniques are major issues specific to the continued use of modeling in organizations. To a lesser degree, properly managing usersÕ expectations of modeling and ensuring users understand how the outcomes of a specific modeling task support the overall enterprise systems architecture are important to the continued use of conceptual modeling. Deficiencies in software tools that support conceptual modeling frustrate the analystÕs work occasionally.

6. Conclusions and future work This paper has reported the results of a survey conducted nationally in Australia on the status of conceptual modeling. It achieved 312 responses. The study found that the top six most

376

I. Davies et al. / Data & Knowledge Engineering 58 (2006) 358–380

frequently used modeling techniques were ER diagramming, data flow diagramming, systems flowcharting, workflow modeling, UML, and structured charts. If size of organization proxies for size and complexity of project, then in small to medium-sized organizations we see a significant decrease in the use of techniques until the size and complexity of the projects require more formal methods of requirements analysis. Thus, we see a significant increase in use moving from medium to larger organizations. Accordingly, H1a and H1b appear to be supported. This study found that clearly Visio is the preferred tool of choice for business systems modeling currently. Rational Rose and Oracle Developer suite were a long way second in frequent use. Visio is more heavily used proportionately by respondents in larger organizations than those respondents in smaller organizations. In line with expectations, it appears that more modern, more complex tools like Rational Rose and Oracle Developer are used proportionately more by modelers in larger organizations than those in smaller organizations. While we see a decrease in the use of tools from smaller organizations to medium organizations, the difference is not significant. We see an increase in tool usage as we move from medium organizations to larger organizations but again the difference is not significant. Accordingly, while the directions of tool use are as predicted, H1c and H1d are not supported. There is a significant increase in usage from the 0 to 3 years level to the 4– 10 years level of modeler experience. From that point however, we see a significant decrease in technique usage as practitioners move into the 11–20 years of modeling experience level. Indeed this trend continues into the greater than 20 years of experience level. In support of the reasoning of Kautz et al. [14] and contrary to the explanations of Fitzgerald [6] and Lee and Truex [15], an inverted U-shaped curve emerges in technique use as modeling experience increases. These results support H2a and H2b. Modeling tool use follows a similar inverted U-shaped curve as modeling experience increases however the differences are not significant. H2c and H2d are not supported. Database design and management remains the highest average purpose for use of modeling techniques. This fact links to the result of ER diagramming being the most frequently used modeling technique. Moreover, software development as a purpose would support the high usage of data flow diagramming and ER diagramming. A contribution of this study is the analysis of textual data concerning problems and benefits in the continued use of conceptual modeling. Clearly, relative advantage (disadvantage)/usefulness from the perspective of the analyst was the major driving factor influencing the decision to continue (discontinue) modeling. Moreover, using a state-of-the-art textual analysis and machine-learning software package called Leximancer, this study identified five factors that uniquely influence the continued use decision of analysts, viz., communication (using diagrams) to/from stakeholders, internal knowledge (lack of) of techniques, user expectations management, understanding models integration into the business, and tool/software deficiencies. The results of this work are limited in several ways. Although every effort was taken to mitigate potential limitations, it still suffers from the usual problems with surveys, most notably, potential bias in the responses and lack of generalizability of the results to other people and settings. Also, when analyzing the results of technique and tool use by size of organization, the spread of responses to the survey was not even across the organization size categories. Furthermore, in relation to the qualitative analysis, even though a form of dual coding (with confirmation) was employed, there still remains subjectivity in the classification of comments. It would have been more objective for trained coders, independent of the researchers, to have performed the coding on the data. By having researchers perform the coding, albeit independently, they may inadvertently

I. Davies et al. / Data & Knowledge Engineering 58 (2006) 358–380

377

bring their own expectation biases to the coding. Last, while the members of the research team all participated, the identification of the factors using the Leximancer document map and the principles of relatedness and centrality remains arguable. We intend to extend this work by administering the survey in other countries (England, Sweden and the Netherlands) to address the issues of lack of generalizability in the current results and cultural differences in conceptual modeling.

Appendix A. Which of these modelling techniques do you use? Please indicate for the following modelling techniques, if you have used them in the past (within the last 3 years), if you use them currently and if you plan to use them in the near future (within 1 year). Please select ÔFÕ to indicate frequent use and ÔIÕ for infrequent (occasional) use (less than five times). Leave the cell blank otherwise. Please also indicate, if you are not aware of a modelling technique in the column ÔunknownÕ. You may use the ÔunknownÕ button to reset all other selections made. If you know of or use another modelling technique not listed, please include the name and your use of the technique at the end of this page, below ÔotherÕ.

378

I. Davies et al. / Data & Knowledge Engineering 58 (2006) 358–380

References [1] D.E. Avison, G. Fitzgerald, Where now for development methodologies? Communications of the ACM 46 (1) (2003) 78–82. [2] J. Bansler, K. Bodker, A reappraisal of structured analysis: design in an organizational context, ACM Transactions on Information Systems 11 (2) (1993) 165–193. [3] D. Batra, G.M. Marakas, Conceptual data modeling in theory and practice, European Journal of Information Systems 4 (3) (1995) 185–193. [4] S. Chang, M. Kesari, P. Seddon, A content-analytic study of the advantages and disadvantages of process modeling, in: J. Burn, C. Standing, P. Love, (Eds.), 14th Australasian Conference on Information Systems, Perth, 2003. [5] B. Curtis, H. Krasner, N. Iscoe, A field study of the software design process for large systems, Communications of the ACM 31 (11) (1998). [6] B. Fitzgerald, The use of systems development methodologies in practice: a field study, Information Systems Journal 7 (4) (1997) 121–132. [7] B. Fitzgerald, An empirical investigation into the adoption of systems development methodologies, Information and Management 34 (1998) 317–328. [8] C. Floyd, A Comparative Evaluation of System Development Methods. Information Systems Design Methodologies: Improving the Practice, North-Holland, Amsterdam, 1986, pp. 19–37. [9] N. Gorla, H.-C. Pu, W. Rom, Evaluation of process tools in systems analysis, Information and Software Technology 37 (2) (1995) 119–126. [10] J. Iivari, Factors affecting perceptions of CASE effectiveness, IEEE Software 4 (1995) 143–158. [11] E. Karahanna, D.W. Straub, N.L. Chervany, Information technology adoption across time: a cross-sectional comparison of pre-adoption and post-adoption beliefs, MIS Quarterly 23 (2) (1999) 183–213. [12] G.M. Karam, R.S. Casselman, A cataloging framework for software development methods, IEEE Computer (Feb.) (1993) 34–46. [13] K. Kautz, T. McMaster, Introducing structured methods: an undelivered promise?—A case study, Scandinavian Journal of Information Systems 6 (2) (1994) 59–78. [14] K. Kautz, B. Hansen, D. Jacobsen, The utilization of information systems development methodologies in practice, Journal of Information Technology, Cases and Applications 6 (4) (2004). [15] J. Lee, D.P. Truex, Exploring the impact of formal training in ISD methods on the cognitive structure of novice information systems developers, Information Systems Journal 10 (2000) 347–367. [16] O.I. Lindland, G. Sindre, A. Solvberg, Understanding quality in conceptual modeling, IEEE Software (March) (1994) 42–49. [17] C.R. Necco, C.L. Gordon, N.W. Tsai, Systems analysis and design: current practices, MIS Quarterly (Dec.) (1987) 461–475. [18] T.W. Olle, J. Hagelstein, I.G. Macdonald, C. Rolland, H.G. Sol, M.F.J. van Assche, A.A. Verrijn-Stuart, Information Systems Methodologies: a Framework for Understanding, Addison-Wesley, Wokingham, 1991. [19] A. Persson, J. Stirna, Why Enterprise Modeling? An Explorative Study Into Current Practice, in: 13th Conference on Advanced Information Systems Engineering, Switzerland, 2001, pp. 465–468. [20] W. Sedera, G. Gable, M. Rosemann, R. Smyth, A success model for business process modeling: findings from a multiple case study, in: T.P. Liang, Z. Zheng (Eds.), 8th Pacific Asia Conference on Information Systems (PACISÕ04). Shanghai, 2004. [21] A.E. Smith, M.S. Humphreys, Evaluation of Unsupervised Semantic Mapping of Natural Language with Leximancer Concept Mapping, Behavior Research Methods, accepted for publication. [22] M. Tan, T.S.H. Teo, Factors influencing the adoption of internet banking, Journal of the Association for Information Systems 1 (2000) 1–43. [23] Y. Wand, R. Weber, Research commentary: information systems and conceptual modeling—A research agenda, Information Systems Research 13 (4) (2002) 363–376.

I. Davies et al. / Data & Knowledge Engineering 58 (2006) 358–380

379

Islay Davies entered the Ph.D. program at QUT in August 2002 and has been working with Prof. Michael Rosemann and Prof. Peter Green on an ARC Discovery Project concerning an evaluation and comparison of the process modeling techniques within ARIS and UML, based on the Bunge-Wand-Weber (BWW) Ontology. IslayÕs Ph.D. topic ‘‘Ontological Evaluation of ARIS’’ is a subset of this project and involves the development of a BWW meta model, meta model reference methodology development, evaluation of ARIS and a comparison of this to prior evaluations of UML, and an empirical study testing the ontological issues in ARIS with modelers. In addition, her Ph.D. work also includes: the comparison of ontologies using meta models. Islay also teaches part-time at QUT in a unit on Business Systems Analysis and Design.

Peter F. Green is Professor of Electronic Commerce and Business Information Systems cluster leader in the UQ Business School at the University of Queensland. He has qualifications in Computer Science, Accounting, and a Ph.D. in Commerce (Information Systems) from the University of Queensland. Dr. Green is a Chartered Accountant and a Member of the Australian Computer Society. Dr. Green has worked during his career as the Systems Support Manager at the South-East Queensland Electricity Board (SEQEB), for a Chartered Accountancy firm, and a Queensland government department. Peter has researched, presented, and published widely on systems analysis and design, conceptual modeling, information systems auditing, and eCommerce. Dr. GreenÕs publications have appeared in such internationally refereed journals as Information Systems, Journal of Database Management, and the Australian Journal of Information Systems. Michael Rosemann has worked after his MBA (1992) for 7 years at the Department of Information Systems, University of Mu¨nster, Germany, where he received his Ph.D. in 1995. Michael is now co-leader of the Business Process Management Group at the Queensland University of Technology, Brisbane, Australia. MichaelÕs research interests are Business Process Management and Enterprise Systems. He is the Chief Investigator of a number of research projects funded by the Australian Research Council and SAP Research. Michael published more than 40 journal publications, 60 conference publications and 35 book chapters. He is the author and editor of five books and a member of the Editorial Board of seven journals.

Marta Indulska is a lecturer at the UQ Business School at the University of Queensland. She obtained her Ph.D. in Computer Science, in the research area of Information Systems, at the University of Queensland, in 2004. MartaÕs main research area is the application of ontology, specifically to analyze the completeness and clarity of various modeling grammars and interoperability standards. Her other research interests include information systems design, data warehouse design, spatial information systems, and electronic commerce. Her teaching focuses on topics in Electronic Commerce and Information Systems.

380

I. Davies et al. / Data & Knowledge Engineering 58 (2006) 358–380 Stan Gallo teaches Computer Based Information Systems and Data Management at the UQ Business School. He is a research assistant for Professor Peter Green and a Microsoft certified Original Equipment Manufacturer, having started and grown his own IT consultancy. He is a member of the Australian Computer Society and holds a degree in electronic commerce. Stan is a former Detective and 15 year veteran of the Queensland Police Service. He specialized as a senior investigator and operational controller in a diverse range of specialist areas. His research interests lie in Computer and Intrusion Forensics and Computer Security.