Use of software systems development methods An empirical study in Brunei Darussalam

AND !WTlt%ARE mMo#GY ELSEVIER

Information

and Software Technology

39 (1998) 949-963

Use of software systems development methods An empirical study in Brunei Darussalam Md. Mahbubur Rahim, Afzaal H. Seyal, Mohd. Noah Abd. Rahman Depurrment of Computing and hformation

Systems, hstitut Teknologi Brunei, Simpang 125, Jalan Muara. Bandar Seri BegaM;an 37868, Brunei Darussalam

Received

17 March 1997; revised 31 October

1997; accepted

3 November

1997

Abstract The practices of software development methods and techniques have been widely reported in the information systems (IS) literature. However, a vast majority of the literature describes solely the use of well advocated methods and techniques, while a few studies are available that investigated influence of organisational attributes on method adoption. Moreover, experiences of method use within US organisations dominate IS literature, while reports from the Asian region are non-existent. Based on these rationales, a study was undertaken to examine the method adoption pattern of the public and private sector organisations in Brunei Darussalam. Out of 100 organisations, 36 (36%) participated in the survey. Two thirds (67%) of the participating organisations reported adoption of a systematic approach to software development by embracing a method. Even though it appears satisfactory for a newly established small country like Brunei, the use of individual methods, particularly the well known ones like structured methods, is less than expected. Statistical analysis reveals that method use is dependent on the type of organisation, and varied between matured and novice organisations. The implications of these findings are discussed. These findings are also compared with those of US, UK and Australian studies, and interesting differences are highlighted. 0 1998 Published by Elsevier Science B.V. Keywords:

Method; Software systems; Software development

1. Introduction Software systems are an integral part of information technology (IT) that support not only the routine problem solving tasks at various levels of an organisation, but also promotes (IT) as the primary weapon for competitive advantage. As such, successful development of software systems is crucial for the survival of an organisation. Unfortunately, development of software systems is fraught with numerous problems. Software development projects are still characterised by major delays, cost overruns and operational problems ranging from persistent bugs through to complete disaster [ 11. In response to these problems, a wide spectrum of methods and techniques have emerged [2]. Today, there exists literally hundreds, if not thousands, of software development methods and techniques [3]. Even, there has been significant commercial interest in some of the well publicised methods [4]. Proponents claim that software systems that are built with appropriate methods require less maintenance. Moreover, benefits like better project control, higher quality, and reduced cost are often attributed to method adoption. However, despite such claims, software development methods are not practiced widely. A number

of studies [5-71 confirm low usage of software development methods across countries. A vast majority of the current studies on the use of software development methods highlight the experiences of US and the European organisations. In contrast, little is reported on how those methods are actually being practiced within a small Asian country, and how successful they have been. Against this background, a survey study was undertaken during the first quarter of 1996 to examine the adoption behaviour of software development methods within the public and private sector Bruneian organisations. Brunei Darussalam is a small sultanate located on the north-west coast of Borneo island with a total population of nearly 0.3 million [8]. Its main economic activity is dominated by the oil and gas sector. Gross domestic product. (GDP) per capita was B$23 86.5 (US$l = B$l.4) in 1992 [9]. After receiving independence in 1984, government placed considerable importance on IT. Recently, a new department known as Information Technology Division (ITD), was established in order to promote the use of IT within the public sector. Several projects, involving development of information systems (IS), procurement of hardware and software, and training of IS personnel, costing several millions of Brunei

0950-5849/98/$19.00 0 1998 Published by Elsevier Science B.V. All rights reserved PII Sn95n-5849(97~00052-9

950

Md. M. Rahim et al./lnfomation

and Sofhvare Technology 39 (1998) 949-963

dollars received approval from this division. However, the successful implementation of these projects does not depend upon finance alone, but to a greater extent on how the IS professionals apply the well known methods, techniques and tools in those projects. Sadly, no efforts have previously been undertaken to investigate the use of methods in context to the Bruneian organisations. A study was, thus, initiated to capture the views of the Bruneian IS professionals towards a set of popular well known software development methods. This paper is based on the findings of this study. It reports the adoption pattern of software development methods within the public and private sector Bnmeian enterprises, and evaluates the level of IS personnel’s satisfaction with respect to the adopted methods. Furthermore, the problems and benefits arising from method adoption are also identified. Lastly, the profile of those organisations that adopted a method is discussed. In this paper, the term software has been used in the generic sense. Consequently, software systems embrace the concepts of transaction processing systems, management reporting systems, decision support systems, expert systems, executive information systems among others. This study is useful to IS researchers and practitioners alike. IS managers of organisations currently using methods, can use the results of this study to compare their level of method usage as opposed to other organisations. They could learn what benefits are generally reaped due to the use of methods, and can compare their experiences with others. Likewise, organisations intending to introduce software development method would be benefited as well. The results of this study can serve as a guide that IS managers could use to select their own method. The findings further carry implications to the academics who could redesign their IS course contents based upon the results of this study. Researchers in the IS field can compare the results of this study with related research studies conducted in other parts of the world. This paper proceeds as follows. First, prior studies on software development practices as reported in the IS literature are reviewed. Second, a research model is proposed, and several hypotheses are postulated. Third, the research approach is described. Next, findings of the study are presented. This is followed by a discussion on the results. Lastly, some suggestions are offered, and conclusions are drawn.

2. Prior studies: a review The practices of software development methods and techniques have been widely reported in the IS literature. However, a vast majority of the literature describes solely the use of well advocated methods and techniques, while a few studies are available that focused on the influence of organisational attributes on method adoption. In general, existing literature on method adoption can be grouped into two categories. One group focused primarily on the use of

prototyping approach and examined its success, benefits and problems. The works of Langle et al. [lo], Carey and Currey [ll], Doke [ 121, Saarinen [13], Hardgrave and Wilson [14], and Selamat et al. [15] fall in this category. These studies indicate that prototyping is slowly gaining popularity even though it was not widely practiced in 1980’s. The second group of studies examined software development practices including use of development methods, techniques, and tools. A large portion of these studies were undertaken in US during the 1980s. For instance, Jenkins[ 161 provided empirical results on systems development practices from twenty-three US organisations. They reported that most projects were completed significantly over budget, even though some methods were used. Guimaraes [ 171 studied the use of application development techniques in 43 US organisations, and found that slightly over half of the organisations enforced structured design and structured programming as a development standard, while structured analysis did not appear to be acceptable by the organisations. This finding is in agreement with those of Sumner and Sitek [18] and Carey and McCleod [5] who reported low usage of structured methods in US organisations. Carey and McCleod [5] further correlated the use of software development tools with organisational budget, and found that budget levels of organisations had significant effect on tools utilisation. Recent study by Palvia and Nosek [19] found that some methods and techniques like SREM, BICS, Young/Kent methodology, Information Engineering, SADT, were not used by the US organisations. In UK, the practice of software engineering is not different from that of US. Reports from Price Waterhouse [20] and Hardy et al. [6] indicated a low usage of good software engineering practices across UK sectors. Moreover, Holt [21] reported that nearly one-third of UK organisations did not embrace any structured methods. A recent study on the IS profile of UK manufacturing firms found that smaller firms often had no central IS strategy, and had less systematic approach to software development [7]. In the Asia-Pacific region, a few attempts were made to examine software development practices. Rouse et al. [22] compared the adoption of leading software development practices between Australian and US organisations, and found that the adoption rate is slower in Australian organisations. For instance, only 56% of Australian organisations used structured method as compared to 70% of US organisations. While in Malaysia, over 80% of organisations were found to rely on a traditional systems development life cycle approach for software development. Only 8% of Malaysian organisations used SSADM to some extent [23]. The key findings of the empirical studies on software development methods as cited above are summarised in Table 1. It is obvious that experiences of the US organisations dominate the literature, while reports from the European and Pacific region are limited. Sadly, the IS literature is almost void of study that focused exclusively on the software development practices in Asian countries.

951

Md. M. Rahim et al/Information and Software Technology 39 (1998) 949-963 Table 1 Prior method studies

( 1980- 1997)

Region

Authors

Subject of study

Remarks

US

Jenkins (1984) [16]

Studied system development practices in 23 US organisations Investigated application development techniques in 43 US organisations Studied tools and methods use in US organisations Examined use of techniques, tools and methods in US organisations Surveyed use of application development productivity tools among 67 US organisations Examined use of system development techniques among 65 US organisations Surveyed software engineering practice within 50 UK organisations Studied customisation of structured methods among 102 UK organisations Surveyed systems development practices among 92 UK manufacturing companies Compared system development methodology with project success in 43 development projects among 21 large Finnish organisations Compared software development practices between Australian and US organisations Investigated use of CASE tools and associated methods in 40 Malaysian organisations

Software projects were found to exceed budget even if methods were applied Structured analysis was found to be unpopular

Guimaraes

( 1985) [ 171

Summer and Sitek (1986) [18] Carey and McCleod (1988) [S] Doke (1989) [12]

Europe

Palvia and Nosek (1993) [19] Holt (1997) [21] Hardy et al. (1995) [6] Waste11 and Sewards (1995) [7] Saarinen (1990) [ 131

Asia-Pacific

Rouse et al. (1995) [22] Selamat et al. (1994) [15]

Table 1 further reveals that even though method utilisation has been studied widely, empirical research investigating the pattern of method adoption with organisational parameters is lacking. In this paper, it is argued that organisational factors are likely to have an impact on the method adoption process of an organisation. This is an additional rationale that initiated this study to examine the method adoption pattern of the public and private sector organisations in Brunei Darussalam. In particular, this study has sought to establish whether a relationship exists between the use of software development methods and organisational parameters.

3. Research

model and hypotheses

This study has addressed four related but distinct research objectives. The first objective is to identify the existing software development methods, and their level of use within Bruneian organisations. The second objective is to identify the profile of those organisations that adopted one or more method(s) to develop their software applications. The third objective is to evaluate the degree of benefits and problems that were experienced by the IS personnel while using methods. The last objective is to assess the influence of organisational and IS departmental attributes on the adoption of those methods. More specifically, to compare the organisational features (e.g. organisation size, type of industry etc.) of those organisations that adopted methods and those that did not.

Low use of structured

method was reported

Correlated organisational budget with tool and method use Compared popularity versus importance of application development productivity tools Information Engineering and SREM were found to be least used Nearly one-third (3 1%) organisations did not use any structured method Low usage of good software engineering practices across UK sectors Larger companies made more use of a method as compared to smaller companies Success of projects was not affected by the use of any methodology Australian organisations’ adoption rate of prominent practices is slower than that of US organisations Only 8% of organisations were found to use SSADM

This study did not employ any particular theory of method adoption, because there is none. A normative model as shown in Fig. 1, is however developed on the basis of the existing IS literature, and is used for the purpose of the study. This is a one-stage model that relates independent and dependent variables, without any intermediate variables. Furthermore, the relationships as shown in the model should be considered as associative rather than casual in nature.The normative model as presented in Fig. 1, has included six factors that are likely to explain the method adoption pattern of organisations. These factors are grouped into two categories: organisational attributes and IS profile. These two categories of variables are expected to exert considerable influence on the adoption of software development methods. Organisational attributes include type of organisation, nature of business sector, and size of organisation, while IS profile includes existence of an independent IS department, number of IS personnel working within organisation, and age of IS function of organisation. A detailed discussion on each of the factors are provided below: Type of Organisation: Organisations are categorised in terms of their belonging to either the private or the public sector. Delineating the two sectors is important, as differences exist in the ‘mission statements’ of the private and the public sector organisations. Moreover, the public sector organisations are not accountable to shareholders like the private sector firms [24]. Thus, the use and role of IS in the public sector is likely to be different from those of the private sector firms. Furthermore, companies in the private sector operate under acute time constraints as to meet

952

Md. M. Rahim et al./Infonnation and Sofhvare Technology 39 (1998) 949-963 IS Proflle

Organisational Attributes

Fig. 1. Research

model.

customers demand and deadline. As such, IS people working in those companies do not have enough time to apply methods for application development. In contrast, IS employees within the public sector are more likely to use methods. One reason is the existence of government imposed standards. Another reason is that IS employees in the public sector organisations experience less constraints like their counterparts do in the private sector. Lastly, IS department in the public sector are less likely to emulate the behaviour of their private sector counterparts, because of the traditional public sector echo of service to the community. These differences lead to the first hypothesis. Hl: Adoption of software development method in the public sector is different from that of the private sector.

Organisation Size: Organisation size has been widely researched in terms of its relationship with technology adoption [27]. A number of researchers suggested that organisation size is a significant factor [28-301. Empirical studies involving this factor has however been inconsistent [31]. Ein-Dor and Segev [30] found no relationship between organisation size and the degree of centralisation of IS activities. On the contrary, Moth and Morse [32] reported a positive relationship between technology adoption and size of an organisation. This study argues that resource availability in large organisations would be greater. As a result, large organisations are more likely to take a methodological approach to software development. Hence the third hypothesis is: H3: Organisational size will be positively related to method adoption.

Nature of Business Sector: In addition to dividing organisations into public and private sectors, organisations can also be classified on the basis of the sector in which they operate. Classifying organisations into different business sectors is relevant, because different business sectors will have different experiences of method adoption, and it is necessary to identify the benefits and problems experienced by each business sector. This view is supported by Currie [25] who suggested that business types have considerable influence on the structure of IS activities of an organisation. Furthermore, Premkumar [26] claimed that certain types of organisations, particularly in the finance sector, have greater stake in IS operations. These organisations are major investors in IS, and use IS as a strategic device. As such, organisations in the finance sector are expected to invest more on methods and tools to develop their own applications, as compared to other business organisations. Thus, the second hypothesis is: H2: Method adoption will be different across business sectors.

In addition to organisational parameters, IS profile is likely to influence the adoption of IS method within an organisation. This study views that IS profile of an organisation includes three factors such as: existence of an IS department, number of IS personnel and age of IS function. Existence of IS Department: It is argued that organisations with an independent IS department are more likely to undertake a systematic approach to software systems development than those without an IS department. It is because, existence of an IS department helps formulation of corporate IS plan that are more aligned with the objectives of organisations. Such plan includes identification of priority projects, development of those priority fiojects, purchase of necessary software and hardware tools, and establishment of methods among others. Thus, existence of an IS department is a reflection of the maturity of IS function of an organisation. This view has been supported by Gibson and Nolan [33]. This is also in line with other authors like Waste11 and Sewards [7] who used existence of IS department as one of the indicators of IS sophistication in UK companies. Thus,

Md. M. Rahim et alAnformation and Software Technology 39 (I 998) 949-963

the fourth hypothesis is: H4: Existence of an independent IS department influences adoption of software development method. Age of IS Function: Some authors like Jones and Arnett [34] claimed that the age of IS department is important in the adoption of IS practices within an organisation. This notion is further supported by Franz and Robey [35] who suggested that the older an IS department, the more useful the systems are developed. This study, however, argues that an organisation may operate and continue to develop software systems without a separately established IS department. Thus, while some organisations may have an independent IS department, others may have assigned a group of people to look after the information systems function of an organisation. Thus, it is more appropriate to consider age of IS function, rather than age of IS department. It is hypothesised that at the early stages, IS people may develop software systems quickly to address the immediate needs of organisations, bypassing well established techniques and methods. Such systems often contain bugs, fail to meet user needs, and are eventually abandoned. Thus, with the lessons learned from the unsuccessful projects, IS people (whether working within an independent IS department or not) gain experience, and begin to appreciate the need and role of methods to develop future IS applications. This leads to the next hypothesis: H5: Experience of IS function would be positively related to software development method adoption. Number of IS Personnel: This study considers that organisations with fewer number of IS personnel are less likely to be engaged in the development of software systems. In contrast, organisations with greater number of IS people are expected to be more matured with technology usage, and hence are less likely to feel intimated by any new technology like method adoption. Furthermore, with the expansion of computing in organisations, the number of IS employee will grow. Many of them would be entrusted for the development of new applications to cater for the emerging needs of organisation, some others would be involved in the maintenance of existing systems, and while the rest would look after day-to-day IS operations. As the number of IS employees increases, many of them would be keen to apply prominent methods for software systems development. This view is supported by Nolan and Gibson [33] who cited ‘the number of IS personnel’ as one of the indicators of IS maturity of an organisation. Thus, the last hypothesis takes the following form: H6: Number of IS personnel employed by an organisation has influence on the adoption of software development method. In this study, the dependent variable is the ‘adoption of software development method’. The definition of method as used in this study has been adopted from Cline et al. [36]. A

953

method is an organised and systematic approach for handling the system development life cycle or its parts. It specifies individual tasks and their sequence. Some of the popular methods as cited in the literature includes various forms of traditional structured methods like SSADM, Yourdon’s structured design, DeMarco’s structured analysis, information engineering methods, object oriented methods as well as emerging alternate methods like prototyping and spiral methods. A review of the innovation literature reveals that the most common form of measurement of technology adoption is through a dichotomous measure (adoption versus nonadoption). In this study, the dependent variable, ‘adoption of method’ is, however, measured on a Likert scale that varied from 0 to 4, where 0 indicates ‘never used’, and 4 indicates ‘always used’. Organisations are broadly grouped into two categories: public and private enterprises. A variety of business sectors are identified including: finance, communication, petroleum, computer vendors, trading, public services among others. The size of an organisation is measured by the total number of employees who are on the payroll of the organisation. Size is classified into four intervals: small (99 employees and less), intermediate (between 100 to 499 employees), large (between 500 to 1000 employees), and very large (above 1000 employees). Annual turnover, which is often used as a measure of organisational size, was not convenient for the purpose of this study, as it is not appropriate for the public service organisations. The information systems activities of an organisation are generally carried out by a group of personnel who are generally placed under a separate department. However, sometimes an organisation, especially if it is a small or intermediate one, may not establish a separate IS department. Thus, the existence of an IS department is measured by a binary value: ‘yes’ or ‘no’. The number of IS personnel working within an organisation often varies widely depending on the type and size of an organisation. Five classes of IS size are identified: 9 employees and less, between 10 to 19 employees, between 20 to 49 employees, between 50 to 100 employees, and over 100 employees. Age of IS function of an organisation is defined as the years of usage of computer based information systems within an organisation. It is expressed into three classes: novice (l-5 years), moderate (6-10 years), and matured (over 10 years). In summary, the concepts that are used in the model and their measurement scales are outlined in Table 2.

4. Research approach This study adopted a survey research approach to investigate the use of software development methods within Bruneian organisations. Based on the elements of the research model as introduced in the previous section, a questionnaire was developed. The questionnaire was basically

954

Md. M. Rahim et al./lnfonnation and So&are

Technology 39 (1998) 949-963

Table 2 Definition of terms used in the research model Concept

Definition

Measurement

Method

A method is an organised and systematic approach for handling the system development life cycle or its parts. It specifies individual tasks and their sequence The degree of use of a method by the IS personnel within an organisation Organisations were grouped into two broad types depending on their ‘mission statements’ Types of business sectors in which organisations operate The size of an organisation was measured by the total number of employees who were on the payroll of the organisation Whether an organisation has an independent IS department or not The number of years of usage of computer based information systems within an organisation The number of IS personnel working in an organisation

Categorical

Method adoption Organisation type Type of business sector Organisation size Existence of IS department Age of IS function Number of IS personnel

structured in nature, and consisted of two parts. Part A asked respondents for the background information about their organisations, namely organisation type, nature of business, and organisation size. Respondents were further asked to indicate if there was a separate IS department in their organisation, age of IS function, and the number of IS personnel employed within their organisations. While Part B included three questions. The first question asked respondents to indicate whether their dominant approach to software development involved structured methods, or the organisation’s own in-house method, or no explicit method, and the degree of use of those methods. The second and third questions collected information about the extent of problems, and benefits that were experienced as a result of the use of methods. The appropriateness of the questionnaire was evaluated in two stages. First, a group of academics reviewed the questionnaire, and examined how the questionnaire fullfiled the research objectives. Second, the questionnaire was subject to a pilot study in which it was hand delivered to six individuals who were actively involved in managing software development projects within their particular organisations. In addition to written comments received, those involved in the pilot study were also contacted personally to discuss any problems which they believed were present in the questionnaire. These provided valuable feedback on the design of the questionnaire which subsequently led to some changes before the final version was distributed. One hundred organisations were selected randomly from three published sources, namely, list of government agencies from the recent telephone directory, and the directories of Malay Chamber of Commerce, and Chinese Chamber of Commerce respectively. The selected organisations were telephoned prior to mailing the questionnaire in order to identify the appropriate individual to whom the questionnaire should be sent, and to gain a commitment to returning the questionnaire, and therefore encouraging a high response rate. The final questionnaire, with a cover letter assuring the respondent’s confidentiality, were sent to the IS managers/executives of the selected organisations between

scale

Likert Dichotomous Categorical Categorical Dichotomous Categorical Categorical

January 1996 and April 1996. In order to further boost response rates, follow-up telephone calls were used.

5. Results Results obtained from the survey are presented using descriptive analysis, tabulations, and statistical analysis like chi-square (x2), Mann-Whitney tests, and two-sample t-tests as appropriate. The purpose of statistical testing was to screen out spurious results. A statistical software package, Minitab Release 10.5, was used for statistical analysis of the data tabulated from the questionnaire feedback. The results of the survey are presented in five sub-sections: profile of surveyed organisations, method popularity and use, profile of organisations that adopted method(s) for software development, analysis of benefits and problems arising from method use, and statistical investigations. 5.1. Surveyed organisations: projile Out of 100 organisations, 36 responses were received, yielding a response rate of 36%. An analysis of response rates in enterpreneurship research [37] reveals that response rates vary widely depending on the sources of address and on additional contacts. Since the addresses used in this study were obtained from published sources, and additional contacts with the respondents were made, the response rate is not unsatisfactory. Out of the responding 36 organisations, nearly two-thirds (64%) were government agencies, while the remaining 36% came from the private sector. This is reflected in Table 3. The last column of Table 3 highlights the proportion of organisations that participated from each type of organisation. It can be observed that satisfactory responses (58%) were received from the public sector, while responses from the private sector (22%) were indeed disappointing. Several telephonic reminders were given to the non-responding organisations. However, most of them expressed their inability to participate due to excessive work load at that

Md. M. Rahim et al./lnfonnation and Software Technology 39 (1998) 949-963 Table 3 Survey responses Organisation

by organisation

types

types

Questionnaire

Questionnaire

sent out

Number Public Private Total

955

Number

(a)

40 60 100

received

40 60 100

Organisation

wise

@‘o)

23 13 36

64 36 loo

58 22 36

Others t ,s

Administrative Services

I ‘C g b z

Petroleum TT! Con-puter Comrunication

B !=

Finance 0

5

10

15

20

25

30

35

40

45

50

Percentage Fig. 2. Business profile of surveyed organisations.

even though they initially expressed interest to participate in the survey. The organisations participating in the survey were involved in a variety of businesses including finance, administrative services, education, communications, IT and petroleum. This is reflected in Fig. 2. The dominance of administrative services can be clearly noticed, as they constitute over half of the surveyed sample. This is followed by the organisations from the finance, communication and IT sectors. The size of the participating organisations also varied considerably. A majority (70%) of the questionnaires were returned from organisations with employees between 100 and 499. However, some very large organisations with time,

above 1000 employees also participated in the survey. The distribution of the size of organisations is shown in Fig. 3. The IS profile of the participating organisations is provided in Table 4. It is obvious that nearly half of the organisations (47%) had a small IS team that comprised of less than ten people. While only 6% organisations had large IS team that exceeded fifty IS people. Interestingly, not a single organisation was found with over 100 IS personnel. Table 4 further reveals that nearly two-thirds of the organisations (61%) were new users of IS, as they had fewer than 6 years of experience. Matured organisations with over 10 years of IS usage were very limited. This study found that three-fourths of the surveyed organisations retained a separate IS department. All the financial organisations had a separate IS department. Interestingly,

abova 1000 employees

Table 4 IS profile of surveyed organisations Response category

Number

(RI

Cumulative @J)

500 to 1000 employees 11%

100 to 499 employees 42% Fig. 3. Distribution

of size of surveyed organisations.

Number of IS personnel l-9 people lo-19 people 20-49 people SO- 100 people Over 100 people IS experience l-5 years 6- 10 years Over 10 years Existence of IS unit Yes No

17 14 3 2 0

47 39 8 6 0

47 86 94 100 100

22 8 6

61 22 13

61 83 100

24 12

7.5 2.5

75 100

956

Md. M. Rahimet al./Infonnationand SoftwareTechnology39 (1998) 949-963

organisations that did not have a separate included only public services.

IS department

5.2. Method popularity and use Nine different methods were found to be used by the surveyed organisations. A popularity index of each method was computed, and is listed in Table 5. The popularity index was derived by dividing the number of organisations that used a particular method by the total number of participating organisations (i.e. N = 36). Several important observations can be made on the basis of Table 5. First, no single method enjoyed an overwhelming popularity. In other words, no single method emerged as being a defacto standard. Second, in-house methods were found to be quite common. Third, software prototyping approach was found to be used by one-third of the participating organisations. Fourth, the adoption of widely advocated methods like SSADM, information engineering or object oriented design is far from satisfactory. Fifth, structured methods like Yourdon’s design did not appear to be widely accepted. Sixth, less than one-tenth of the organisations were using structured analysis and Jackson system design. Lastly, cleanroom approach was never used. Survey results revealed that 24 (67%) out of 36 participating organisations adopted a systematic approach for software development. Thus, a disturbing observation is that still one third of the surveyed organisations used no method in the development of their software applications. Those twenty four organisations (N = 24) that applied a systematic approach for software development were asked to indicate their level of method use on a Likert scale that varied from 0 to 4, where 0 means ‘never used’ and 4 implies ‘always used’. Based on their responses, a mean rating for each method was calculated. This is reflected in Table 6. Once again, in-house methods received the highest rating (2.93), which implies that in-house methods were not only used by many organisations (as shown in Table 5), but were used very frequently by those organisations which adopted them. Likewise, those organisations that adopted SSADM, used it on a frequent basis. Surprisingly, prototyping which was

Table 5 Popularity

of software development

methods

Method

Number

Index Rank (N = 36)

In-house method Software prototyping SSADM Object oriented design Information engineering method Structured design by Yourdon Structured analysis by DeMarco Systems requirements engineering Jackson systems design Cleanroom approach

14 12 7 6 5 5 2 2

0.39 0.33 0.19 0.16 0.13 0.13 0.05 0.05 0.02 0.00

method

I 0

1 2 3 4 5 5 6 6 7 8

Table 6 Use of software development

methods

Method

Mean rating (N = 24)

Rank

In-house method Structured design by Yourdon SSADM Jackson systems design Software prototyping Object oriented design Information engineering method Structured analysis by DeMarco Systems requirements engineering

2.93 2.20 2.00 2.00 1.83

1 2 3 3 4 4 5 6 7

1.83 1.60 1.50 1.oo

method

identified as the second most popular method, received low rating (1.83). This implies that even though prototyping is practised by one third of the participating organisations (as indicated in Table 5), it is used on an occasional basis. Table 6 further confirms the low use of DeMarco’s structured analysis and Information Engineering Method. 5.3. Profile of method adopters Table 7 presents a brief profile of those 24 organisations (hereafter known as method adopters) that adopted a systematic approach to develop their software systems. It can be observed that the majority of the method adopters were government agencies, and were primarily engaged in administrative services and finance sectors. Moreover, these method adopters were large organisations, and had an independent IS department or unit. Furthermore, a slight

Table 7 Features of method adopters Response category

Organisation type Public Private Business sector Administrative Services Finance Petroleum Transport IT vendors Others Organisational size Small Large Age of IS function Novice (1-5 years) Matured (Over 5 years) Number of IS personnel Small (l-20) Large (over 20) Existence of IS unit Yes No

Method adopters (N = 24)

(%)

15 9

63 37

13 5 2

2

54 21 8 4 4 8

5 19

21 79

14 10

58 42

19 5

79 21

19 5

79 21

1 1

Md. M. Rahim et alAnformation and So&are

Technology 39 (I 998) 949-963

951

majority (58%) of the method adopters were novice in terms of their IS experience as they had less than 5 years of experience in developing software applications.

Table 9 Problems in method application Problems

Mean rating

5.4. Bene$ts

Time consuming to learn method Use of method increase project duration Method does not cover the full life cycle

3.3 3.1 3.1

and problems

analysis

Proponents of software development method claim that a wide range benefits are obtained from the use of a systematic approach to software development. A question, was thus, asked that required the respondents to rate a series of benefits (resulted from method adoption) on a five point Likert scale, where I stands for very unsatisfactory, and 5 means very satisfactory. The responses were compiled, and an average rating for each benefit was computed. These are shown in Table 8. It is revealed that method adopters indeed experienced benefits, but the degree of benefit varied across organisations. The top four benefits that were identified (with rating 4.0 and above) includes: improved productivity, better communication with users, increased user involvement, and fulfilment of user requirements. It is, thus, visible that methods provided an opportunity to increase user involvement in the software development project. Such user participation reduced communication gap between developers and end users. This in turn allowed developers to identify the true requirements of users. Thus, the resultant systems fulfilled user requirements, and required less maintenance. Additionally, adoption of methods produced quality documents as well. However, contrary to the current belief, use of a method did not provide substantial assistance to control software development projects. The respondents were further provided with a list of problems that were commonly encountered by them while developing software projects using methods. They were asked to indicate their level of agreement-disagreement on a Likert scale. The responses were aggregated and are presented in descending order in Table 9. The difficulty to learn the mechanics of method was identified as the most pressing problem. Other reported problems include the inability of the methods to cover the whole life cycle of software projects and the failure of the methods to reduce project completion time. Several respondents even cited that software development projects built with a recognised method actually increased project schedule. Table 8 Benefits due to method adoption Benefit

Mean rating

Improved productivity Better communication with users Increased user involvement Fulfilment of user requirements Better communication among developers Reduction in maintenance costs Fewer errors in design Better quality documents produced More project control

4.30 4.09 4.04 4.00 3.90 3.83 3.82 3.13 3.51

A slightly higher proportion of private organisations (69%) adopted method as compared to that of the public sector organisations (65%). Chi-square tests however found no significant relationship between method use and type of organisation (x2 = 0.06, P = 0.8, df= 1). Additionally, the degree of method use was examined between the public and private organisations. Interestingly, even though less proportion of public sector organisations adopted a methodological approach for software development, the degree of method use was found to be greater in public sector organisations. Statistical tests like two-sample t-test and Mann-Whitney test confirmed this difference to be statistically significant at 5% level This is reflected in Table 10. The proportion of organisations that represented various business sectors are shown in Fig. 2. A clear dominance of administrative services can be observed as they constitute half of the surveyed organisations. It is followed by finance, communication and IT sectors. Chi-square analysis indicates the existence of a relationship between type of business sectors and use of method (x’ = 1.696, P = 0.0, df= 5) at 5% significance level. In order to assess the impact of organisation size on method adoption, the sample was divided in two broad groups: a small organisation subgroup (with less than 100 employees) and a large organisation suhgroup (with 100 employees and over). Ten organisations fell into the first subgroup, and the remaining 26 in the second. A greater proportion of large organisations (73%) embraced a separate method as opposed to that of small organisations (50%). Chi-square tests however found no significant relationship between size of organisations and method use (x’ = 1.73 1, P = 0.189, df = 1). Furthermore, two-sample t-tests and Mann-Whitney tests confirm that there is no significant difference in method use between large and small organisations. This is reflected in Table 10. Three fourths of the surveyed organisations was found to have a separate IS department. Out of these, 70% adopted a systematic methodological approach for software development, while 56% of those organisations that did not have a separate IS department adopted a method. Chi-square test however found no significant relationship between these two variables (x2 = 0.66, P = 0.41, dJ‘=: 1). Furthermore. both two-sample t-tests and Mann-Whitney tests report the difference in method use between organisations having IS department and not having an IS department, to be statistically insignificant at 5% level. This is reflected in Table 10.

958

Md. M. Rahim et al./Informarion and Software Technology 39 (1998) 949-963

Table 10 Results of two-sample

r-test, and Mann-Whitney

testa

Hypothesis

Mean value

f-value

Adoption of software development method in the public sector is different from that of the private sector Organisational size will be positively related to method adoption Experience of IS function would be positively related to software development method adoption Number of IS personnel employed by an organisation has influence on the adoption of software development method Existence of an independent IS department influences adoption of software development method

Public: 3.2, Private: 2.4

aTwenty-four

organisations

P-value

Mann-Whitney P-value

Remarks

2.11

0.04

0.05

Test is significant

Large: 2.5, Small: 3.0

- 0.71

0.52

0.44

Test is insignificant

Novice: 2.3, Matured: 3.2 Small: 2.5, Large: 3.0

- 2.79

0.01

0.01

Test is significant

- 0.94

0.38

0.35

Test is insignificant

0.24

0.10

Test is insignificant

1.35

Having: 2.7, Not having: 2.0

were found to adopt a method, thus constituting

The surveyed sample was grouped into two categories (novice versus matured) to examine the impact of age of IS function of an organisation upon method use. Chi-square tests indicate a lack of significant relationship between these two variables (x2 = 0.5 1, P = 0.47, df = 1). In contrast, both two-sample t-tests and Mann-Whitney tests found the difference in the degree of method use between novice and matured organisations to be statistically significant at 5% level. This is reflected in Table 10. The impact of IS personnel size within an organisation was examined using Chi-square, two-sample t-tests, and Mann-Whitney tests. Out of 36 surveyed organisations, a greater proportion of organisations (86%) were found to have a small IS personnel ranging between 1 to 20. Chisquare tests indicated a lack of relationship between size of IS personnel and method use (x2 = 2.9, P = 0.08, df= 1). Likewise, both two-sample t-tests, and Mann-Whitney tests confirmed the difference in method use between organisations with small and large IS personnel, to be statistically insignificant at 5% level. This is reflected in Table 10.

a sample size 24 for the sake of statistical analysis.

to standardise method in public sector, among others. Further studies are however required to examine the reasons for the low use of methods. An important finding of this study is that all together half of the participating organisations reported using some variety of recognised structured methods. This is close to that of UK, where the proportion of organisations adopting structured methods increased from 44% [6] to 48% [21] in recent years. On the other hand, structured methods are used by a greater proportion of organisations in Australia and USA [22]. A comparison on the use of structured methods across different countries are shown in Table 11. The adoption of widely known methods within Bruneian organisations are compared with those of Malaysia [23], UK [6,21], and USA [19]. These are reflected in Table 12. The dominance of in-house methods can be observed in both Brunei and UK. In-house methods are by definition peculiar to particular organisations, and there is no specific as to what constitutes such a method. SSADM appears to be Table 11 Use of structured

6. Discussions One aim of this study was to provide baseline information on the prevalence of methods for software development projects in Brunei Darussalam. It was found that overall, two thirds (67%) of the participating organisations embraced a particular method, and thus adopted a systematic approach for software development. Even though the overall use of software development methods appears to be satisfactory for a newly established small country like Brunei, the use of individual methods, particularly the well known ones like SSADM, information engineering and structured methods is less than expected. This is surprising, given that these methodologies are now well-established approaches. The low use of well known methods could be attributed to several reasons such as lack of appreciation for method, lack of vendor enthusiasm to promote software tools to support method, and lack of government initiative

methods

Country

% of organisations

UK USA Australia Brunei

48 70 56 50

Table 12 A comparison

of method use

Methods

Brunei

Malaysia

USA

UK

In-house SSADM Information engineering Yourdon’s structured design Object oriented design Jackson Systems Design

39 19 13 13 16 2

N/A 8 3 N/A N/A N/A

N/A N/A N/A N/A N/A 1.5

38 24 4 4 21 5

Note: N/A stands for not available.

Md. M. Rahim et al./l~formation and Sof?ware Technology 39 (1998) 949-963 Table 13 Comparison

prototyping

adoption

Authors

Country

o/c

Carey and McCleod [5] Doke [12] Palvia and Nosek [ 191 Selamat et al. [23] Hardgrave [ 141 Rouse et al. [22] This study

USA USA USA Malaysia

49 33 32 8 71 71 33

USA USA Brunei

used by a considerable proportion of organisations in UK and Brunei. Its use is however limited (8%) in Malaysia. A larger proportion of Bruneian organisations was found to use and information engineering as compared to that of UK [6]. Object oriented method, a promising software development method, has received considerable attention from UK organisations-as 21% of UK organisations are now using this method as compared to 5% a few years ago 1211. In comparison, only 16% of Bruneian organisations are using this method. Lastly, Jackson Systems Design (JSD) which was made popular by Michael Jackson [38], was one of the least used methods in Brunei (only 2%). This finding is in agreement with that of Palvia and Nosek [19] who reported 1.5% of JSD use in US organisations. In contrast, Hardy et al. [6] reported a slightly greater (5%) use of JSD among UK organisations. During the 1980s one third of the US organisations used software prototyping approach. In recent years, the use of prototyping has however increased dramatically. Evidences from Hardgrave and Wilson [ 141 and Rouse et al. [22] confirmed that between 70 and 82% of US organisations are currently using prototyping. In contrast, only one-third of the surveyed Bruenian organisations was found to use prototyping. Thus, the current status of prototyping in Brunei is similar to the situation that prevailed in the US during the early 1980s. However, adoption of prototyping in Brunei appears to be greater than that of Malaysia [23]. This is shown in Table 13. One goal of this study was to probe whether method adoption is influenced by the organisational parameters. Out of six factors considered in this study, only one factor, namely nature of business sector, was found to be significantly related with method use. Conversely, none of the remaining five factors as outlined in the research model were found to be significant by the survey findings. This would seem to indicate that, for the organisations surveyed, the method adoption is largely independent of most research attributes. Another important question that this study attempted to answer is whether there exists any difference between organisations that adopted a method, and those that did not. While six factors were examined, only two (type of organisation and age of IS function of an organisation) produced significant results. Survey results further reveal that those organisations that

959

applied a method in general experienced productivity and quality improvements. However, benefits related to project management and control were perceived to a lesser extent.

7. Conclusions This paper has raised the importance of studying the practice of software development methods within organisations. It has further provided an overview of some of the relevant literature on method use in conjunction with examining data from questionnaire survey. It is apparent that even though a wide range of highly publicised methods are cited in the IS literature, their use is quite low within the Bruneian organisations. It is however expected that the growth of popular methods is likely to increase with the introduction of automated tools (e.g. CASE) in the near future. This study further found that the adoption of software development methods is related with the nature of business sector in which an organisation operates. Furthermore, a clear difference in method use was observed between public and private sector organisations. Such a difference was also noted between matured and novice organisations. Interestingly, most of the factors as believed to affect method use could not be supported by the survey results. This indicates that new thinking is needed to further enhance the research model in order to explain method adoption pattern. Three possible factors that were overlooked by this study are proposed for inclusion in the future studies. These are: (a) technological readiness; (b) financial readiness; and (c) people readiness. Technological readiness is concerned with the level of sophistication of IS usage among management within an organisation. Financial readiness refers to the availability of financial resources for method adoption. It includes costs of method training, hiring of method gurus and where possible automated software to implement method. People readiness refers to the degree of familiarity and keenness of the IS people with to use a method. The results of this study are useful to the IS managers of organisations intending to adopt a systematic approach for their software development. They would know the difficulties and benefits experienced by other organisations while applying methods for software development. The findings further carry implications to the Bruneian IS academics who could tune the contents of their IS course to meet the existing needs of Bruneian organisations. This study solicited the views of senior IS professionals who were in charge of IS departments or units within the surveyed organisations. Future studies should, however, focus on the views of those individual IS personnel who work in software development projects. Their opinions could differ considerably from that of their senior managers, and such differences should be identified and examined thoroughly as they may provide new light on the method adoption phenomenon. Furthermore, this study did not investigate the adoption process of software development

960

Md. M. Rahim et al./lnformation and Sofiware Technology 39 (1998) 949-963

methods within surveyed organisations. For instance, issues like who selected a method (by a committee or an IS manager), what training was provided on method, how successful was such training, who provided such training, what was the reactions of the IS employees towards method, what types of applications are developed using such methods were not addressed. Further studies are recommended to examine these issues. Lastly, it is acknowledged that this study provided a static view on the adoption phenomenon of software development methods across Bruneian organisations, unfortunately, such static view has limitations. However, despite such limitations, the study yielded some interesting findings which are of interest to the IS community. Appendix

Acknowledgements

The authors are deeply grateful to Hj. Abu Hanifah bin Hj Mohd. Salleh, Director, Institut Teknologi Brunei for his kind patronage and continuous encouragement in undertaking this study. We further express our gratitude to Hj. Awg Yussof bin Hj Awg Mohammad, Head, Dept. of Computing and IS, Institut Teknologi Brunei, for his moral support, and to Dy. Sophiana Chua Abdullah for her valuable contribution during the early stage of this study. Lastly, we are thankful to the anonymous reviewers whose comments were of great help.

A Survey instrument Appendix: Part 1: Orgaaisational

Profile

Please put a tick (4) at the appropriate

1.

Survey Instrument

box to indicate your answer.

Please indicate the type of your organisation []

Government

[ ] Private

2.

Please describe the main business of your organisation. [] [] [] [] [] []

3.

[]

[]

[]

1-99 100-499 500-999 Over 1000

Please indicate the total number of IS people employed in your organisation. []

None

II

I-9 10-19 20-49 50-99 Over100

[] [] [] [] 6.

1 to5years 6tolOyears Over 10 years

Please indicate the total number of people employed in your organisation. [] [] []

5.

Insurance

Please indicate the IT/IS experience of your organisation.

[]

4.

Administrative Services Banking, Finance, Investment, Communication Petroleum IT Vendor Others (Please specify) :

Please indicate if there exists an IT/IS unit/section/centre/department. [I

Yes

11 No

Md. M. Rahim et al./lnfomation and Sofmare Technology 39 (1998) 949-963 Part II:

Software Development

Practices

Please put a tick 4 to indicate your answer. 7.

Please indicate whether your organisation develop software application.

[I [I

has adopted and practiced

any method to

Yes No

If your response is ‘No’ please do not proceed and return the questionaire, proceed to question 8.

8.

Please indicate how often the IT/IS personnel within your organisation following software development methods. (Multiple answers are possible).

specify)

9.

I

otherwise

use the

/

Please indicate which of the following benefits due to the adoption of software development method are perceived within your organisation. (Multiple answers are possible).

with

Better

users

communication

among developers

More

project

control

961

962

Md. M. Rahim et al./Infonnation

and Software Technology 39 (1998) 949-963

10. Please indicate which of the following problems are encountered personnel while using software development method. (Multiple possible).

Problems

Strongly Disagree

Disagree

(1)

(2)

Not Applicable (3)

by the IT/IS answers are

Agree

Strongly Agree

(4)

(5)

Too much time consuming to learn the method The method is complicated The method does not cover the full life cycle The use of method increase project duration Others (Please specify)

11. Please indicate to what extent the IT/IS personnel are satisfied with the methods that are used within your organisation. (Multiple answers are possible). Somewhat Satisfied (2)

Moderately Satisfied (3)

Very Satisfied (4)

Methods

Not Satisfied (1)

SSADM Information Engineering Method Object Oriented Design Software Prototyping Jackson Systems Design Structures Analysis by De Marco Structured Design by Yourdon Systems Requirements Engineering Methodology Formal Specification In-House Method Others (Please specify) I

I

I

I

I

I

Thank you very much for your cooperation

References [l] A. Poulymenakou, A. Holmes, A contingency framework for the investigation of IS failure, European Journal of Information Systems 5 (1996) 34-46. [2] D. Batra, G.M. Ma&as, Conceptual data modelling in theory and practice, European Journal of Information Systems 4 (1995) 185- 193. [3] J.A. Bubenko, Information systems methodologies-a research view, In: T.W. OlIie, H.G. Sol, A.A. Verrijn-Stuart(Eds.), Information Systems Methodologies: Improving the Practice, North Holland, Amsterdam, 1986, pp. 289-318. [4] M.C. Winter, D.H. Brown, P.B. Checkland, A role for soft systems methodology in information systems development, European Journal of Information Systems 4 ( 1995) 130- 142. [5] J.M. Carey, R. McCleod, Use of systems development methodology and tools, Journal of Systems Management March (1988)30-35. [6] C.J. Hardy, J.B. Thompson, H.M. Edwards, The use, limitations and customisation of structured systems development methods on the United Kingdom, Information and Software Technology 37 (9) (1995) 467-477. [7] D. Wastell, A. Sewards, An information system profile of the UK manufacturing sector, Journal of Information Technology 10 (1995) 179-189. [8] Brunei Darussalam Statistical Yearbook, Statistic Review Economy Planning Unit, Ministry of Finance, 1993.

[9] Country profile, Malaysia and Brunei 1994-95, The Economist Intelligence Unit, 1995. [lo] G.B. Langle, R.L. Leitheiser, J.D. Naumann, A survey of applications systems: prototyping in industry, Information and- Management 7 (1984) 273-284. 111 J.M. Carey, J.D. Currey, The prototyping conundrum, Datamation June (1989) 25-33. 121 E.R. Doke, Application development productivity tools: A survey of importance and popularity, Journal of Information Systems Management Winter (1989) 37-43. [ 131 T. Saarinen, System development methodology and project success, Information and Management 19 (1990) 183-193. [14] B.C. Hardgrave, R.L. Wilson, An investigation of guidelines for selecting a prototyping strategy, Journal of Systems Management April (1994) 28-35. [ 151 M.H. Selamat, M.M. Rahim, M.N. Maizura, Perceptions of selected Malaysian information systems practitioners towards software prototyping: An exploratory study, Malaysian Journal of Computer Science 9 (2) (1996) 14-28. [16] A.M. Jenkins, Empirical investigation of systems development practices and results, Information and Management 7 (1984) 73-82. [17] T. Guimaraes, A study of application program development techniques?, Communications of the ACM 28 (I 985) 494-499. [ 181 M. Sumner, J. Sitek, Are structured methods for systems analysis and

963

Md. M. Rahim et alAnformation and Software Technology 39 (1998) 949-963 design being used?, Journal of Systems Management June (1986) 1827. [ 191 P. Palvia, J.T. Nosek, A field examination of system life cycle techniques and methodologies?, Information and Management 25 (1993) 73-84. 1201 Price Waterhouse, Software Quality Standards: The Costs and Benefits, Price Waterhouse Management Consultants, London, 1988. [21] J. Holt, Current practice in software engineering: A survey, Computer and Control Journal August (1997) 167- 172. [22] A. Rouse, D. Watson, A. Gilbert, R. Dilnutt, Systems development practices: A comparison of the experience of Australians and United States, In: Proc. of 1995 Pan Pacific Conference on Information Systems, June 29-July 2, Singapore, 1995, pp. 46-56. [23] M.H. Selamat, M.M. Rahim, A.T. Othman, Managing implementation of CASE tools, In: Proceedings of the Third Conference on IT and its Applications 94, April 2-3, Leicester, UK. 1994, pp. 47-55. [24] L. Willcocks, Evaluating information technology investments: research findings and reappraisal, Journal of Information Systems 2 (1992) 243-268. [25] W.L. Currie, Organisational structure and the use of information technology: preliminary findings of s survey in the private and public sector, International Journal of Information Management 16 (1) (1996) 51-64. [26] G. Premkumar, An empirical study of IS planning characteristics at common industries, OMEGA 20 ( 1992) 6 1 l-629. [27] J. Child, R. Mansfield, Technology size and organisation structure, Sociology ( 1972) 369-393.

[28] J.D. Aaron, Information systems in perspective, December (1969) 213-236.

Computing

Surveys

[29] GM. West, MIS in small companies, Journal of Systems Management 14 (1975) 10-13. 1301 P. Ein-Dor, E. Segev, Organisation context and the success of MIS, Management Science 24 ( 1978) 1064- 1077. [31] V. Grover, J.T.C. Teng, An examination of DBhlS adoption and success in American organisations. Information and Management 23 (5) (1992) 239-248. [32] M.K. Moth, E.V. Morse, Size, centralisation and organisational adoption of innovations, American Sociological Review 42 (1977) 716725. [33] C.F. Gibson, R.L. Nolan, Managing the four stages of EDP growth, Harvard Business Review 58 (1) (1974) 76-88. [34] M.C. Jones, K.P. Amett, Current practices in management information systems, Information and Management 24 ( 1993) 6 l-69. [35] C.R. Franz, D. Robey, Organisational context, user involvement and the usefulness of information systems, Decision Science 17 (1986) 329-356. [36] J.E. Cline, M.A. Ketabchi, J. Link, P. Palvia, T.W. Olle, IRM and the systems life cycle, Database Engineering June (1986) 43-48. 1371 C. Allen, T.C.D. Schewe, G. Wijk, Foot technique in the pre-call/mail survey setting, Journal of Marketing Research XVII (1980) 489-502. [38] M.A. Jackson, System Development, Prentice Hall, Englewood Cliffs, New Jersey, USA, 1983.