- Email: [email protected]
Proposing an information flow analysis model to measure memory load for software development
Information and Software Technology 42 (2000) 743±753
www.elsevier.nl/locate/infsof
Proposing an information ¯ow analysis model to measure memory load for software development T.C. Du*, C.J. Lin, C.-G. Liu Department of Industrial Engineering, Chung Yuan Christian University, Chung Li 32023, Taiwan Received 16 November 1998; received in revised form 20 March 2000; accepted 3 April 2000
Abstract System analysis is used to develop application software. However, the conventional approach has dif®culty to ensure that the software has high user-satisfaction. This research uses the user's memory load to measure the degree of satisfaction of application software. An information ¯ow analysis model is proposed for analyzing the degree of memory load. Ten types of operations, including six internal operations and four external operations, are adopted to measure the memory load. The conventional data ¯ow diagram (DFD) and state transition diagram (STD) are modi®ed by adding more functional components to estimate the memory load. The information ¯ow analysis is demonstrated by building a qualitative analysis software which requests heavier mental works but has simple software architecture. q 2000 Elsevier Science B.V. All rights reserved. Keywords: Memory load; Software development; Qualitative analysis
1. Introduction Information technology has progressed remarkably recently in both hardware and software. This progress decreases the hardware cost and increases the software requests. Therefore, the importance of software will become greater and greater, and the software functions will become more complicated. Unfortunately, the user's satisfaction in software quality does not increase in proportion to the increase of demand. This research proposes a methodology to measure the user's satisfaction level before the software has actually been developed. This research proposes a model, called an information ¯ow analysis model, to estimate the degree of a user's memory load while using a given software. The assumption is that the higher the memory requested in operating a software the lower the satisfaction will be. The information ¯ow analysis model uses six internal operations and four external operations to depict the user operations. To analyze the software design in the analysis and design stage, the data ¯ow diagram (DFD) and the state transition diagram (STD) will be modi®ed into an extended data ¯ow diagram (EDFD) and an extended state transition diagram (ESTD). Both EDFD and ESTD will have the functional components * Corresponding author. Tel.: 1 886-3-4563171; fax: 4563171. E-mail address: [email protected] (T.C. Du).
1 886-3-
to measure the memory load. The earlier diagnosis will prevent the building of an unfriendly system and can also provide a base for comparing the different design approaches of a system. The study relates to two issues: system development and human information processing. 1.1. Software development The quality of software is very dif®cult to measure before the software is actually developed. MaCall [1] listed eleven software quality factors of which the designer should be aware. The factors are maintainability, ¯exibility, testability, correctness, reliability, ef®ciency, integrity, usability, portability, reusability, and interoperability. To achieve a better software quality, a designer normally adopts a systematic approach to develop a software. The development process includes ®ve steps: system requirement study, system analysis, system design, system development, and system implementation. These ®ve steps are also called the system development life cycle (SDLC). Each step uses certain tools to analyze designated functions. For example, in the system analysis stage, the DFD, data dictionary, and process description are used to depict the data manipulation process. The STD is used to design user interfaces in the system design stage [2]. In the SDLC, the success in the analysis and design stages is the key point of developing a good performance software. Therefore, this research will focus on the analysis and design stages.
0950-5849/00/$ - see front matter q 2000 Elsevier Science B.V. All rights reserved. PII: S 0950-584 9(00)00117-8
744
T.C. Du et al. / Information and Software Technology 42 (2000) 743±753
There are several researches focused on developing quality software. For example, Alavi [3] added data modeling as a preliminary step to give prototyping more structure and ef®ciency. Zelkowitz [4] used Information Technology Engineering and Measurement (ITEM) model to depict the information used in an organization for developing integrated software engineering environment. In this study, both DFD and STD will be modi®ed to include the functions for measuring the memory load. The DFD is a structured analysis technique which describes a system as processes, inputs, outputs, and ®les [5]. Two types of approaches are used in designing DFD, the expansion approach [6] and the explosion approach [7]. This study adopts the explosion approach. The STD is a screen-sequencing variation on program ¯ow charts and hierarchy charts, and has the constructs of sequence, selection, repetition, and exit [5]. 1.2. Human information processing model Mental models are crucial in information processing because they make knowledge and information processing more ef®cient by making it unnecessary to construct understanding from scratch [8]. There are signi®cant relationships between mental models and software development. Vandenbosh interviewed 36 users to distinguish two types of leaning: mental model maintenance and mental building for designing executive support system [9]. Cant focused on the programming process instead of program structure while developing software [10]. Kul'ba used aggregated Petri net to data processing system in management information system [11]. This study focuses on the resource sharing (attention and memory) of human information processing. The human information processing model describes the critical stages of information processing involved in human performance. Generally speaking, there are four stages of human information processing [12]: 1. Sensory processing. This stage simply receives the shortterm sensory store (STSS) from external objects using human perception. The receptors include visualization, audition, etc. 2. Perception. Perception compares the stimuli with the data in long-term memory and working memory to identify the message from stimuli and distributes the results to the next stage. 3. Decision and response selection. This stage generates the decision based on the message received from perception. 4. Response execution. This stage executes the decision. When more than one job needs to be processed, the resource has to be shared. In this study, the two types of situations for resource sharing are the AND relationship and the OR relationship. If the information loading of multiple operations can be added up, an AND relationship exists. For example, if job A required 2% of memory resource and job B requests 10% of memory resource, the processing of both jobs consumes 12% of memory resource. If only one of the
jobs needs to be done, it can be considered as an OR relationship among operations. It is reasonable to assume that the memory resource of dealing OR relationship among jobs is the maximum value of the memory requested of each individual job. Section 2 will describe the EDFD and the ESTD which have the components to depict the memory load of internal information ¯ow and external interfaces. Both internal operations and external operations are used to measure the memory load, and their characteristic functions will be covered, and the information ¯ow analysis procedure will be presented. To demonstrate the information ¯ow analysis model, a software for analyzing qualitative data is built in Section 3. Section 4 designs an experiment to validate the model. Four different designs of the qualitative analysis software are actually built and tested by reviewers. The questionnaires for acquiring the reviewer's opinions are given after the testing. 2. Information ¯ow analysis model Most system analysis tools emphasize on the functional design to produce high satisfaction software. The factors that affect the degree of user satisfaction are various. However, both the system response time and the memory load have signi®cant in¯uence. Fortunately, following instructions of human±computer interface design, e.g. GOMS [13], or interface design principles, a better design in term of shorter response time can be found. However, lack of a design tool to construct a lighter memory load software is still a problem. (Note that the memory load describes the degree of memory utilized while using the software). However, it will be too late to ®nd out if a software has heavy memory load if the software has already been developed. This research develops a methodology, the information ¯ow analysis model, to estimate the user memory load in the early stage of system analysis process. The information ¯ow analysis model is modi®ed from the conventional SDLC by using an EDFD and an ESTD instead of DFD and STD. 2.1. Internal operations and external operations The operations of software can be distinguished into external operations and internal operations. The external operations focus on human perceptions such as mouse operation, key-in, visualization, and audition. The internal operations focus on the intra-brain activities such as creation, comparison, categorization, decision, selection, and search. As discussed earlier, the DFD is a technique for analyzing information ¯ow while the STD is designated for user interface design. Therefore, the internal operations are the major components of DFD. Similarly, STD is concentrated on the external operations. Both the internal and external
T.C. Du et al. / Information and Software Technology 42 (2000) 743±753
operations describe the behavior of a system and they are correlated. For example, a user can make a selection by using the mouse or hot-key. Therefore, it is important to quantify the user's operations so that different designs can be compared. Using characteristic functions to describe the operations is a good solution for quantifying the memory load of a design. These operations are explained as follows. 2.1.1. The characteristic functions of internal operations The internal operations include creation, comparison, categorization, decision, selection, and search. This study does not intend to determine the accurate memory load functions for either internal or external operations. However, for demonstrating the procedure of information ¯ow analysis model, reasonable characteristics functions of operations are assumed. This assumption also provides a thinking of the behaviors of both internal and external operations. The characteristic function of an operation is illustrated by a mathematical expression, where the data volume or data item is the independent variable (X) and the degree of memory load is the dependent variable (Y). I1-creation: The creation operation is the activity of creating objects from scratch. The characteristic function of creation is assumed as a linear function, Y aX 1 b: That is, creating many objects at a time can be treated as creating an object many times. I2-comparison: The comparison operation compares the attributes of two or more objects, e.g. colors, sizes, or meanings of objects. The larger data volume for comparison will cause a heavier memory load, and the relationship can be seen as the exponential function, Y a ebX : This is because when objects increase, the number of comparisons required increases even faster. I3-categorization: The categorization operation separates objects into different categories. It can also be measured by the exponential function, Y a ebX : One difference is that the X variable represents the data items rather than the data volume in the comparison operation. I4-decision: The decision operation makes judgment. The more decisions that need to be made, the heavier the memory load required. The function is Y a ebX : I5-selection: The selection operation chooses from options. The choice can be single or multiple. The more options that can be chosen from, the heavier the memory load is expected to be. The operation can be measured in Y a ebX : I6-search: The search operation looks for answers among objects. The more the objects to search from, the heavier the memory load is needed. The operation can be measured in Y a ebX : 2.1.2. The characteristic functions of external operations The characteristic functions of external operations for measuring the memory load can also be represented in mathematical functions:
745
E1-mouse operation: The mouse operation includes moves, clicks, double clicks, and drag-and-drops. The more mouse operations required, the higher the memory load that can be expected. The memory load function can be measured in Y a ebX : E2- key-in: To key-in data can use a single key or a compound key. The data volume needed to be keyed in, will not affect the degree of memory load. The relationship is constant, Y C: E3- visualization: The visualization operation needs to catch the signals on the screen. The more attention points on a screen needed to be analyzed the heavier the memory load required. The function is measured in Y a ebX : E4- audition: The sound system is used to inform or alert users. There are several types of sound messages such as alerting, entertaining, and warning. The more sounds that appear at a time, the more the user needs to pay more attentions to identifying their meaning. If the number of messages is over the limit that the user can handle, they are unidenti®able. The memory load function is measured in Y 1= 1 1 e2aX : Since ®nding the exact characteristic functions of the operations is not the focal point in this study, the functions are simpli®ed for easy illustration. For example, Y a ebX is simpli®ed as Y X; Y C is simpli®ed as Y 1; Y 1= 1 1 e2aX becomes Y X; and Y aX 1 b is Y X: Lindsay [14] suggested that the attention resources are shared while processing more than one job at a time. That is, if one person needs to process more than one operation at a time, his/her memory load is accumulated. In this research, the accumulation of the memory load functions are in an AND or OR relationship between individual jobs. When more than one job needs to be processed simultaneously, the memory load of a user is accumulated as an AND relationship, symbolized as ( p). The memory load of individual jobs in an AND operation will be added together. For example, if job A requests 2 units of memory load and job B requests 10, the memory load is accumulated as 2 p 10 2 1 10 12 for processing both jobs at the same time. The OR operation is used when only some jobs are needed to be processed. The symbol for OR operation is (1) and the rule is to adopt the largest value of the memory load of individual jobs. For example, if the memory load of job A is 2 and B is 10, the total memory load is 2 110 max 2; 10 10 for processing either one job. It is understandable that the memory load of internal operations is accumulated as an AND relationship when more than one operation need to be processed. That is, the memory load is I1 ( p) I2 I1 1 I2 when both operations are internal. However, the external operations can be either an AND or OR relationship when only some operations are needed. For example, if the memory load of external operations are E1, E2, and E3, and the relationship is (E1(1)E2) ( p) E3, the accumulated memory load is calculated by max(E1, E2) 1 E3.
746
T.C. Du et al. / Information and Software Technology 42 (2000) 743±753
Table 1 The functions of extended data ¯ow diagram (EDFD) and extended state transition diagram (ESTD) Name
Symbol
Explanation
DFD/STD
Data Flow
Ð
A line indicate the direction of information ¯ow A circle indicates the action performed
DFD
Data store
A log indicate the database table
DFD
External agent
A block indicate external agents in the system
DFD
Process and display
A display symbol on the left upper corner of process indicate a display interface required
No
Internal operation
A dashed line indicate the internal operations required
No
Non-accumulated display interface (NDI)
A rectangular block indicates interface for showing message
STD
A line with arrow indicates the direction of state transition A gray triangular mark on the left upper corner of NDI indicates the main interface for users
STD
External operation
Description of external operations
No
Exclusive operator
A circle with cross line indicates the constraint of entry and exit of interface
No
Process
Task/process or state change Accumulated display interface (ADI)
Ð
DFD
No
T.C. Du et al. / Information and Software Technology 42 (2000) 743±753
2.2. Extended data ¯ow diagram and extended state transition diagram The information ¯ow analysis analyzes the quality of software in the early stage by means of measuring the memory load. The information ¯ow analysis model has two stages: (1) developing an EDFD and (2) developing an ESTD. The EDFD is used for acquiring the memory load of internal operations while the memory load of external operations is obtained in ESTD. The EDFD is modi®ed from the data ¯ow diagram by adding Process and Display and Internal Operation Flow as shown in Table 1. In addition to the conventional DFD development guideline, the Process and Displayis used to represent a display interface of a process component. Note that interfaces will be created for each Process and Display function and will be used in ESTD later. The Internal Operation Flow, drawn in a bold dashed line, represents the information ¯ow between users and Process and Display via internal operations (to be discussed later). The volume of information ¯ow is the essential factor that causes high memory load. The second stage is to develop an ESTD which is altered from STD by adding three functions: Accumulated Display Interface (ADI), External Operation Flow, and Exclusive Operator (please refer to Table 1). The ADI is an interface accumulated memory load of Non-Accumulated Display Interface (NDI), which is just a simple interface. The External Operationevaluates the user memory loads of both the ADI and NDI. The External Operations of each interface will be shown within each interface block. The Exclusive Operatorfunction is dedicated to constraining the looping of user operations by limiting the entry and exit following the same interface so that the users will not get lost in the loop. 2.3. The information ¯ow analysis procedure The information ¯ow analysis model is modi®ed from the SDLC by revising system analysis and system design tools. (An example can be seen in Section 3.2) The procedure is: 1. Design database for storing data: In this stage, one can use a data conceptual modeling technique such as an entity-relationship model, to design the database structure. 2. Depict EDFD: This stage uses processes, data ¯ow, and external agents to describe the information ¯ow of the data stores de®ned in the ®rst stage. As discussed earlier, the processes include Process operations and Process and Display operations, depending on whether or not the display interfaces are required. Note that the information ¯ow is the internal operation. 3. De®ne data dictionary after completing the EDFD analysis. 4. Design user interfaces based on the Process and Display operation in EDFD analysis.
747
5. Depict ESTD: Identify the user interfaces obtained from the previous stage into ADIs and NDIs and add a gray triangle on the left upper corner for ADIs. Note that the ADIs are the main interfaces that cause the memory load. The NDIs are only for displaying messages, and their memory load will be accumulated into their corresponding ADIs. 6. Estimate the volume of information ¯ow in EDFD and compute the memory load. After the EDFD and ESTD design are completed, a computation procedure for obtaining total memory load is activated. First of all, a designer estimates the volume of information ¯ow of each internal operation in EDFD. In the design process, the volume of information ¯ow is estimated by experienced system analysts, experts, or simulation results. Fortunately, the estimate used to calculate the memory load and does not have to be absolutely accurate since it is used for comparing models. It is also bene®cial that the estimate becomes a push force to the designer to evaluate the information volume of each operation. The estimated information volume of the internal operation is substituted into the memory load functions and their corresponding relationships. Again, only AND operands appear in EDFD (internal operations) and both AND and OR operands are used in ESTD. 7. Estimate the usage frequency of external operations and compute the memory load. Similar to the procedure in EDFD, the usage frequency of each external operation has to be estimated. This estimate can also come from either experts or past experiences. To calculate the score of interfaces, one uses the usage frequencies as the weights to multiply the memory load from external operations, which substitutes the information volume from EDFD into the independent variable X of the characteristic functions, and then applies the AND/OR relationship rule to combine the external operations.It has to be noticed that some designs expect users, while using the software, to provide extra memory-aided tools, such as a pencil and paper, from outside the system. To compare different designs fairly, this type of external memory-aided tools also needs to be included. The solution is to add one extra component (interface) to represent the tool. 8. Compare designs and choose the design with the lowest score. The lower-scored design means that the user needs a smaller memory load to use the software. Form the perspective of this research, the software with the smaller memory load has better performance. 3. Designing a qualitative analysis software using the information ¯ow analysis model This research develops a qualitative research software as an example to demonstrate the information ¯ow analysis model. The reasons for using such software are: (1) the
748
T.C. Du et al. / Information and Software Technology 42 (2000) 743±753
Fig. 1. An example of extended data ¯ow diagram of the qualitative analysis software.
qualitative analysis requires a signi®cant amount of memory load; (2) the system complexity is low; and (3) the system architecture is simple. 3.1. The system requirements Five types of approaches for doing qualitative analysis are text retrievers, textbase managers, code-and-retrieve programs, code-based theory builders, and conceptual network builders [15]. The main streams of qualitative analysis software provide either the code-based theory builders or conceptual network builders. This research uses the code-based theory builder approach as an example. That is, the analysis results are constructed in the process of code building. Until now, qualitative analysis software was only used by humans to assist the data analysis since the analysis work is still too much to a computer [16]. Therefore, software should provide functions such as text editing, data coding, doing memo, data linking, data searching, and data presentation. The software developed in this research will provide the functions of data editing, coding, data searching, data linking, and data presentation. 3.2. Information ¯ow analysis model of the qualitative analysis software This section demonstrates the procedure of developing a qualitative analysis software step by step as shown in Section 2.3. 1. Design database. To process qualitative analysis, software has to store texts, memos, indices, and project information. Therefore, four schema struts are created: project table, index table, memo table, and code table.
2. Depict EDFD. The data ¯ow diagram describes the information among processes, agents, and data stores. Fig. 1 shows the EDFD for the qualitative analysis software. Several marked elements are brie¯y explained as follows: (1) Process elements. The action of the process is described in the circle. The process element can be explored for details if necessary. (2) Information ¯ow. The information ¯ow describes the information from one entity to another entity. The name of the information is shown on the line. (3) Data store. The data stores are designed in the previous step. While doing the qualitative analysis, users create a project database before editing or analyzing texts, and then store the text (or data) related to the project. (4) Process with display element. The process will become an interface for displaying messages to users. The interface number is shown at the left corner. These interfaces become ADI or NDI of ESTD in the later design. (5) Information ¯ow of the internal operations. The operations describe the types of the user's memory load from interfaces. For example, when adding data, a user need to proceed the operations, such as (I4), selection (I5), and search (I6) in the ªdisplay interface 3º. Note that the relationship between internal operators is an AND relationship, represented in ( p), such as I4 ( p) I5 ( p) I6 in the ®gure. 3. The data dictionary de®nes the information about data types, relationship, etc. Since it is not the focal point in this study, the dictionary is not shown. 4. Design user interfaces for Process and Displays. Interfaces are used for communicating between users and
T.C. Du et al. / Information and Software Technology 42 (2000) 743±753
749
Fig. 2. An example of extended state transition diagram of the qualitative analysis software.
software, it is important to get users involved in the earlier design stage. A rapid application development (RAD) tool can be used to develop the interfaces. Each Process and Display Interface should have one interface design designated to it. 5. Depict ESTD. The state transition diagram describes the transition between user interfaces. Fig. 2 shows an example of the qualitative analysis software. Some marked elements are explained as follows:
(1) Accumulated Display Interface. There are ®ve ADI in this example. The ADI accumulates relative memory load and is the major concern in the ESTD. (2) Non-accumulated Display Interface. The NDI generates a memory load because of external operations. However, the memory load of NDIs will be accumulated into their corresponding ADIs. (3) State Change process shows how an interface can switch to another.
750
T.C. Du et al. / Information and Software Technology 42 (2000) 743±753
Table 2 The memory load of an qualitative analysis design ADI Main screen 1 Open Prj New Prj Main screen 2
Main screen 3
Total: 5
NDI
Add data Tree management Del data Add index Tree management Del code Edit code Add code Add memo Del memo Edit memo Del index Edit index Open data Link to data Collect data
Usage frequency 1 1 1 1 50 100 10 50 300 100 20 50 100 100 10 40 50 100 50 50 30
Weight 1/54 1/54 1/54 1/54
50/54
Score 0 6 2 5675600 15000 5660000 600 11913700 6020000 5660000 1600 15000 40000 30000 300 4800 5000 30000 15000 20000 72000
Total: 54
0 0.11 0.04 105103.70
11031203.7
11136307.6 Average
(4) A set of External Operations. The external operations within a display interface describe the types of operations and the memory load of the interface. For example, operations of mouse operations (E1), visualization (E3), and audition (E4) are the external operations for the open project interface. The relationship of these operations is E1( p) E3( 1 )E4. (5) Exclusive Operator. The exclusive operator limits the user to entry the tree management function from Main Screen 2 and to exit from the same interface. This design prevents users from getting lost in the loop. 6. Estimate the information volume in EDFD and compute the memory load. The information volume is used for calculating the degree of memory load of a speci®c system design. As it has been discussed, the volume can be estimated by system analysts, experts, or simulation results. The importance of the volume is on the relative ratio rather than on the absolute values. For example, if a user ªopen a projectº once, he/she could ªadd a ®leº about 50 times, ªdelete a ®leº about 10 times, ªadd codesº about 100 times, ªdelete codesº about 100 times, ªadd indicesº about 300 times, ªdelete indicesº about 50 times, ªlink to other ®lesº about 50 times, etc. After estimating the volume of information ¯ow, the internal memory load can be obtained through the characteristic functions of internal operators and their relationships. For example, the Process and Display ªAdd Dataº requires the memory load of internal operations I4
Weighted score
2227261.5
( p) I5 ( p) I6 50 1 50 1 50 50; and ªDel Dataº requires I4 ( p) I5 ( p) I6 10 1 10 1 10 30: 7. Estimate interface usage frequency and compute memory load. Like the estimation made in EDFD, the usage frequency can also be obtained from experts, analysts, or simulations. Again, the relative values of the interfaces are more important than the absolute values of each interface. As an example of the qualitative analysis software, the usage frequencies are estimated. A user therefore uses ªmain screen 1º once, will use ªopen projectº once, ªadd dataº about 50 times, ªdelete dataº about 10 times, ªadd indexº about 300 times, ªdelete indexº about 50 times, ªadd memoº about 100 times, ªdelete memoº about 10 times, etc. Table 2 shows the computed memory load for a speci®c design. The weights are obtained by dividing the usage frequency of each ADI by the summation of all ADIs for balancing the impact from interfaces. The score of ADI is obtained by adding the score of its corresponding NDIs. Use ªmain screen 2º as an example: the ADI ªmain screen 2º accumulates three NDIs, ªAdd Dataº, ªDel Dataº and ªTree Managementº. The score of an NDI is obtained from calculating the memory load of external operations. For example, the NDI ªAdd Dataº has external operations E1 ( p) E3 ( 1 ) E4. Transforming the relationship of external operations becomes X ( p) X 1X 2X: The X values is the internal information volume obtained from stage 6, which is 150 for ªAdd Data.º Therefore, the score of ªAdd dataº is equal to the usage frequency times external operations, i.e.
T.C. Du et al. / Information and Software Technology 42 (2000) 743±753
751
Table 3 Evaluation scores and the calculated scores based on the proposed model Software
I(A±B)
II(A±b)
III(a±B)
IV(a±b)
Calculated score Evaluation score
2227262 3.816
2528631 3.539
4519902 3.386
4907994 3.225
50 p 2X 50 p 2 p 150 15000: Similarly, the NDIs of ªDel Dataº is 600 and ªTree Managementº is 5660000. Adding up the corresponding NDIs can obtain the score of ADI. For example, adding 15000, 600, and 5660000 is the accumulated score of main screen 2. The total weighted score of the design is 11083900. To compare the score between designs, the total weighted score has to be averaged by the number of ADIs. Otherwise, the design with more ADIs will tend to generate a higher score. It is also worth noticing that the values of ªAdd Index,º ªTree Management,º ªLink to Data,º and ªCollect Dataº have higher scores when compared with other interfaces. This observation can remind the designer to design these types of interface carefully since the user accumulate a signi®cant amount of memory load on them. That is the source from which the low satisfaction comes. For example, adding memory-added interfaces, e.g. historical notebooks or code cards, can signi®cantly increase the software's performance. 4. Validation of the information ¯ow analysis model An experiment was used to identify the degree of user satisfaction. Four different qualitative analysis software programs were developed. These four designs were based on the different software design conditions that will be described shortly. The software programs were given to seven reviewers for evaluation. The reviewers were selected from the faculty and graduate students in a department of education. The reviewers were all working on the ªqualitative researchº at that time, and were familiarized with the intent of the evaluation experiment before the evaluation took place. The interviewers needed to answer a questionnaire immediately after using each software program. After completing four software evaluations, a personal information data sheet was given to the reviewers to ensure that the reviewers have similar backgrounds. Each questionnaire is Table 4 ANOVA result of the evaluation experiment Factor
Test d.o.f
F-test
Probability
Signi®cance
Process Question Setup Subject Process p Question Process p Setup Question p Setup
(1,243) (9,243) (1,243) (6,243) (9,243) (1,243) (9,243)
5.025 14.352 21.739 15.999 0.817 0.201 0.407
0.026 0.000 0.000 0.000 0.601 0.654 0.931
* * * *
composed of 16 questions, two questions for an internal operation and one for an external operation. Each question asks how the reviewer feels about a speci®c operation after using the software. For each question, the reviewers had to rate the level of satisfaction for the software program they had just used. The satisfaction rating scale is a 5-point scale. Since the information analysis model has six internal operations (creation, comparison, categorization, decision, and selection) and four external operations (mouse operations, key-in, visualization, and audition), 10 types of questions were generated for the 10 operations. The implementation of a qualitative analysis software has two stages: analysis environment setup and data process. Each design includes two levels: menu/hot-key-control A and command-line-control a for the environment setup stage, and multiple-data-in-a-single-window B and multiple-data-in-multiple-window b for the processing stage. Four combinations of the software, i.e. software I, II, III, and IV were developed as A±B, A±b, a±B, and a±b. The evaluation result was ®rst tested whether the satisfaction was different on the four different versions of the software programs. The satisfaction index of each software was obtained from averaging the satisfaction level on each question. That is, ((question 1 1 question 2)/2 1 ¼ 1 (question 11 1 question 12)/2 1 question 13 1 question 14 1 question 15 1 question 16)/10 The evaluation scores and the theoretical calculations are presented in Table 3. Based on the evaluation results, the satisfaction levels were ranked as: software I . software II . software III . software IV Kruskal±Wallis test showed that there was signi®cant difference in user satisfaction on the four software programs X 2 14:056; d:o:f 3; a 0:05: As observed from the data, the ranking of the satisfaction index was reverse to the memory load from the information analysis model. The results imply that the software with the higher memory load causes lower user-satisfaction. That means that if the system analyst estimates the memory load of designs, he/she can prevent from choosing the lower user-satisfaction design in the ®rst place. Analysis of variance (ANOVA) was performed on the data with the following model: Yijkl m 1 ai 1 bj 1 gk 1 dl 1 abij 1 agik 1 bgjk 1 1ijkl
752
T.C. Du et al. / Information and Software Technology 42 (2000) 743±753
where a is the question factor, i 1; ¼; 10; b the setup factor, j 1; 2; g the process factor, k 1; 2; d the blocking factor, l 1; ¼; n; ab the interaction between question and setup factors; a g the interaction between question and process factors; b g the interaction between setup and process factors; and 1 the errors. The result of the ANOVA is presented in Table 4. There are several observations: 1. The question factor is signi®cant. Both the internal and external operations corresponding to the sixteen questions have a signi®cant in¯uence on user-satisfaction. If we further identify the individual operations, I1 (creation), E1 (mouse operation), E3 (visualization), and E4 (audition) have higher rating, implying that the better interface designs are please to users and the users dislike heavier memory-loading operations. 2. The setup factor is signi®cant. The implication here is that there is a signi®cant difference between menu control (mouse control) and command control: operating the menu requires less memory than operating the command control. 3. The process factor is signi®cant. The multiple-data-in-asingle-window approach is preferred to the multipledata-in-multiple-window approach. This is because users have to jump between windows for searching information in the multiple-data-in-multiple-windows. If a single window is designed for collecting the related data while processing qualitative analysis, the users can process analysis effectively. That is, this design requires fewer memory load. 4. All the interaction factors are not signi®cant. That is, there is no signi®cant impact between the question and setup factors, the question and process factors, and the setup and process factors.
5. Conclusions This research uses the degree of the user's memory load to indicate the degree of user-satisfaction on software designs. An information ¯ow analysis model is proposed for analyzing the degree of memory load. Ten categories of operations, including six internal operations and four external operations, are adopted to measure the degree of memory load. The DFD and STD of the system development life cycle are modi®ed into an EDFD and ESTD by adding more functional components. Besides the conventional system analysis concern, the EDFD and ESTD have extra components to measure the memory load. The information ¯ow analysis model is demonstrated by building a qualitative analysis software which requests heavier mental works but has a simple software architecture. This software can be used to edit, review, categorize and analyze data. For validating the model, an experiment is designed for obtaining the degree of user-satisfaction, and
then four different designs of the qualitative analysis software are built accordingly. The software is tested by interviewers, and questionnaires are given to obtain their opinions. Based on the opinions, some observations can be concluded: 1. The same conclusions of the interviewers and the information ¯ow analysis model imply that the proposed approach can be used to estimate the user-satisfaction in the early stage. 2. The Kruskal±Wallis test rejects the null hypothesis on the difference of designs, and concludes that user-satisfaction is different for the four designs. 3. The ANOVA shows that both internal and external operations have an in¯uence on the degree of usersatisfaction. 4. The ANOVA show that the different designs on the system setup have an in¯uence on user-satisfaction, and the menu-control design gives better satisfaction than the command-control design. 5. The ANOVA shows that the different designs on data process have an in¯uence on user-satisfaction, and single-window design gives more satisfaction than the multiple-window design. Observations show that the information ¯ow analysis model can be used to estimate the degree of user-satisfaction during the design stage. Furthermore, following the procedure, a design can diagnose the interfaces with heavier memory load and revise accordingly to produce a better performing software. References [1] J.A. McCall, An Introduction to Software Quality Metrics, Software Quality Management, McGraw-Hill, New York, 1979. [2] Y.W. Shih, S.W. Margaret, System Analysis and Design, West, St. Paul, 1994. [3] M. Alavi, J. Wetherbe, Mixing prototyping and data modeling for information system design, IEEE Software May (1991) 86±91. [4] M. Zelkowitz, B. Cuthill, Application of an information technology model to software engineering environment, J. Systems Software 37 (1997) 27±40. [5] J. Whitten, L. Bentley, V. Barlow, System Analysis and Design Methods, 3rd ed., Richard D. Irwin, Inc, Butt Ridge, IL, 1994. [6] C. Gane, T. Sarson, Structured System Analysis: Tools and Techniques, Prentice-Hall, Englewood Cliffs, NJ, 1978. [7] T. DeMarco, Structured Analysis and System Speci®cation, PrenticeHall, Englewood Cliffs, NJ, 1978. [8] J.P. Walsh, Selectivity and selective perception: an investigation managers belief structures and information processing, Acad. Mgmt J. 31 (4) (1988) 873±896. [9] B. Vandenbosch, C. Higgins, Information acquisition and mental models: an investigation into the relationship between behavior and learning, Inf. System Res. 7 (2) (1996) 198±214. [10] S. Cant, D. Jeffery, B. Henderson-Sellers, A conceptual model of cognitive complexity of elements of the programming process, Inf. Software Technol. 37 (7) (1995) 351±362. [11] V.V. Kul'ba, A.P. Shvetsov, Methods for analysis and design of systems maintaining certainty of information in MIS, Automation Remote Control 55 (7) (1994) 1074±1083.
T.C. Du et al. / Information and Software Technology 42 (2000) 743±753 [12] C. Wickens, Engineering Psychology and Human Performance, 2nd ed., Harper Collins, New York, 1992. [13] S.K. Card, T.P. Moran, A. Newell, The Psychology of Human Computer Interaction, Lawrence Erlbaum Associates, Hillsdale, NJ, 1983. [14] P.H. Lindsay, D.A. Norman, Human Information Processing: an Introduction to Psychology, Academic Press, New York, 1977.
753
[15] E.A. Weitzman, M.B. Miles, A Software Sourcebook: Computer Programs for Qualitative Data Analysis, Sage, Beverly Hills, CA, 1995. [16] R. Tesch, Qualitative Research: Analysis Types and Software Tools, Falmer, New York, 1990.
www.elsevier.nl/locate/infsof
Proposing an information ¯ow analysis model to measure memory load for software development T.C. Du*, C.J. Lin, C.-G. Liu Department of Industrial Engineering, Chung Yuan Christian University, Chung Li 32023, Taiwan Received 16 November 1998; received in revised form 20 March 2000; accepted 3 April 2000
Abstract System analysis is used to develop application software. However, the conventional approach has dif®culty to ensure that the software has high user-satisfaction. This research uses the user's memory load to measure the degree of satisfaction of application software. An information ¯ow analysis model is proposed for analyzing the degree of memory load. Ten types of operations, including six internal operations and four external operations, are adopted to measure the memory load. The conventional data ¯ow diagram (DFD) and state transition diagram (STD) are modi®ed by adding more functional components to estimate the memory load. The information ¯ow analysis is demonstrated by building a qualitative analysis software which requests heavier mental works but has simple software architecture. q 2000 Elsevier Science B.V. All rights reserved. Keywords: Memory load; Software development; Qualitative analysis
1. Introduction Information technology has progressed remarkably recently in both hardware and software. This progress decreases the hardware cost and increases the software requests. Therefore, the importance of software will become greater and greater, and the software functions will become more complicated. Unfortunately, the user's satisfaction in software quality does not increase in proportion to the increase of demand. This research proposes a methodology to measure the user's satisfaction level before the software has actually been developed. This research proposes a model, called an information ¯ow analysis model, to estimate the degree of a user's memory load while using a given software. The assumption is that the higher the memory requested in operating a software the lower the satisfaction will be. The information ¯ow analysis model uses six internal operations and four external operations to depict the user operations. To analyze the software design in the analysis and design stage, the data ¯ow diagram (DFD) and the state transition diagram (STD) will be modi®ed into an extended data ¯ow diagram (EDFD) and an extended state transition diagram (ESTD). Both EDFD and ESTD will have the functional components * Corresponding author. Tel.: 1 886-3-4563171; fax: 4563171. E-mail address: [email protected] (T.C. Du).
1 886-3-
to measure the memory load. The earlier diagnosis will prevent the building of an unfriendly system and can also provide a base for comparing the different design approaches of a system. The study relates to two issues: system development and human information processing. 1.1. Software development The quality of software is very dif®cult to measure before the software is actually developed. MaCall [1] listed eleven software quality factors of which the designer should be aware. The factors are maintainability, ¯exibility, testability, correctness, reliability, ef®ciency, integrity, usability, portability, reusability, and interoperability. To achieve a better software quality, a designer normally adopts a systematic approach to develop a software. The development process includes ®ve steps: system requirement study, system analysis, system design, system development, and system implementation. These ®ve steps are also called the system development life cycle (SDLC). Each step uses certain tools to analyze designated functions. For example, in the system analysis stage, the DFD, data dictionary, and process description are used to depict the data manipulation process. The STD is used to design user interfaces in the system design stage [2]. In the SDLC, the success in the analysis and design stages is the key point of developing a good performance software. Therefore, this research will focus on the analysis and design stages.
0950-5849/00/$ - see front matter q 2000 Elsevier Science B.V. All rights reserved. PII: S 0950-584 9(00)00117-8
744
T.C. Du et al. / Information and Software Technology 42 (2000) 743±753
There are several researches focused on developing quality software. For example, Alavi [3] added data modeling as a preliminary step to give prototyping more structure and ef®ciency. Zelkowitz [4] used Information Technology Engineering and Measurement (ITEM) model to depict the information used in an organization for developing integrated software engineering environment. In this study, both DFD and STD will be modi®ed to include the functions for measuring the memory load. The DFD is a structured analysis technique which describes a system as processes, inputs, outputs, and ®les [5]. Two types of approaches are used in designing DFD, the expansion approach [6] and the explosion approach [7]. This study adopts the explosion approach. The STD is a screen-sequencing variation on program ¯ow charts and hierarchy charts, and has the constructs of sequence, selection, repetition, and exit [5]. 1.2. Human information processing model Mental models are crucial in information processing because they make knowledge and information processing more ef®cient by making it unnecessary to construct understanding from scratch [8]. There are signi®cant relationships between mental models and software development. Vandenbosh interviewed 36 users to distinguish two types of leaning: mental model maintenance and mental building for designing executive support system [9]. Cant focused on the programming process instead of program structure while developing software [10]. Kul'ba used aggregated Petri net to data processing system in management information system [11]. This study focuses on the resource sharing (attention and memory) of human information processing. The human information processing model describes the critical stages of information processing involved in human performance. Generally speaking, there are four stages of human information processing [12]: 1. Sensory processing. This stage simply receives the shortterm sensory store (STSS) from external objects using human perception. The receptors include visualization, audition, etc. 2. Perception. Perception compares the stimuli with the data in long-term memory and working memory to identify the message from stimuli and distributes the results to the next stage. 3. Decision and response selection. This stage generates the decision based on the message received from perception. 4. Response execution. This stage executes the decision. When more than one job needs to be processed, the resource has to be shared. In this study, the two types of situations for resource sharing are the AND relationship and the OR relationship. If the information loading of multiple operations can be added up, an AND relationship exists. For example, if job A required 2% of memory resource and job B requests 10% of memory resource, the processing of both jobs consumes 12% of memory resource. If only one of the
jobs needs to be done, it can be considered as an OR relationship among operations. It is reasonable to assume that the memory resource of dealing OR relationship among jobs is the maximum value of the memory requested of each individual job. Section 2 will describe the EDFD and the ESTD which have the components to depict the memory load of internal information ¯ow and external interfaces. Both internal operations and external operations are used to measure the memory load, and their characteristic functions will be covered, and the information ¯ow analysis procedure will be presented. To demonstrate the information ¯ow analysis model, a software for analyzing qualitative data is built in Section 3. Section 4 designs an experiment to validate the model. Four different designs of the qualitative analysis software are actually built and tested by reviewers. The questionnaires for acquiring the reviewer's opinions are given after the testing. 2. Information ¯ow analysis model Most system analysis tools emphasize on the functional design to produce high satisfaction software. The factors that affect the degree of user satisfaction are various. However, both the system response time and the memory load have signi®cant in¯uence. Fortunately, following instructions of human±computer interface design, e.g. GOMS [13], or interface design principles, a better design in term of shorter response time can be found. However, lack of a design tool to construct a lighter memory load software is still a problem. (Note that the memory load describes the degree of memory utilized while using the software). However, it will be too late to ®nd out if a software has heavy memory load if the software has already been developed. This research develops a methodology, the information ¯ow analysis model, to estimate the user memory load in the early stage of system analysis process. The information ¯ow analysis model is modi®ed from the conventional SDLC by using an EDFD and an ESTD instead of DFD and STD. 2.1. Internal operations and external operations The operations of software can be distinguished into external operations and internal operations. The external operations focus on human perceptions such as mouse operation, key-in, visualization, and audition. The internal operations focus on the intra-brain activities such as creation, comparison, categorization, decision, selection, and search. As discussed earlier, the DFD is a technique for analyzing information ¯ow while the STD is designated for user interface design. Therefore, the internal operations are the major components of DFD. Similarly, STD is concentrated on the external operations. Both the internal and external
T.C. Du et al. / Information and Software Technology 42 (2000) 743±753
operations describe the behavior of a system and they are correlated. For example, a user can make a selection by using the mouse or hot-key. Therefore, it is important to quantify the user's operations so that different designs can be compared. Using characteristic functions to describe the operations is a good solution for quantifying the memory load of a design. These operations are explained as follows. 2.1.1. The characteristic functions of internal operations The internal operations include creation, comparison, categorization, decision, selection, and search. This study does not intend to determine the accurate memory load functions for either internal or external operations. However, for demonstrating the procedure of information ¯ow analysis model, reasonable characteristics functions of operations are assumed. This assumption also provides a thinking of the behaviors of both internal and external operations. The characteristic function of an operation is illustrated by a mathematical expression, where the data volume or data item is the independent variable (X) and the degree of memory load is the dependent variable (Y). I1-creation: The creation operation is the activity of creating objects from scratch. The characteristic function of creation is assumed as a linear function, Y aX 1 b: That is, creating many objects at a time can be treated as creating an object many times. I2-comparison: The comparison operation compares the attributes of two or more objects, e.g. colors, sizes, or meanings of objects. The larger data volume for comparison will cause a heavier memory load, and the relationship can be seen as the exponential function, Y a ebX : This is because when objects increase, the number of comparisons required increases even faster. I3-categorization: The categorization operation separates objects into different categories. It can also be measured by the exponential function, Y a ebX : One difference is that the X variable represents the data items rather than the data volume in the comparison operation. I4-decision: The decision operation makes judgment. The more decisions that need to be made, the heavier the memory load required. The function is Y a ebX : I5-selection: The selection operation chooses from options. The choice can be single or multiple. The more options that can be chosen from, the heavier the memory load is expected to be. The operation can be measured in Y a ebX : I6-search: The search operation looks for answers among objects. The more the objects to search from, the heavier the memory load is needed. The operation can be measured in Y a ebX : 2.1.2. The characteristic functions of external operations The characteristic functions of external operations for measuring the memory load can also be represented in mathematical functions:
745
E1-mouse operation: The mouse operation includes moves, clicks, double clicks, and drag-and-drops. The more mouse operations required, the higher the memory load that can be expected. The memory load function can be measured in Y a ebX : E2- key-in: To key-in data can use a single key or a compound key. The data volume needed to be keyed in, will not affect the degree of memory load. The relationship is constant, Y C: E3- visualization: The visualization operation needs to catch the signals on the screen. The more attention points on a screen needed to be analyzed the heavier the memory load required. The function is measured in Y a ebX : E4- audition: The sound system is used to inform or alert users. There are several types of sound messages such as alerting, entertaining, and warning. The more sounds that appear at a time, the more the user needs to pay more attentions to identifying their meaning. If the number of messages is over the limit that the user can handle, they are unidenti®able. The memory load function is measured in Y 1= 1 1 e2aX : Since ®nding the exact characteristic functions of the operations is not the focal point in this study, the functions are simpli®ed for easy illustration. For example, Y a ebX is simpli®ed as Y X; Y C is simpli®ed as Y 1; Y 1= 1 1 e2aX becomes Y X; and Y aX 1 b is Y X: Lindsay [14] suggested that the attention resources are shared while processing more than one job at a time. That is, if one person needs to process more than one operation at a time, his/her memory load is accumulated. In this research, the accumulation of the memory load functions are in an AND or OR relationship between individual jobs. When more than one job needs to be processed simultaneously, the memory load of a user is accumulated as an AND relationship, symbolized as ( p). The memory load of individual jobs in an AND operation will be added together. For example, if job A requests 2 units of memory load and job B requests 10, the memory load is accumulated as 2 p 10 2 1 10 12 for processing both jobs at the same time. The OR operation is used when only some jobs are needed to be processed. The symbol for OR operation is (1) and the rule is to adopt the largest value of the memory load of individual jobs. For example, if the memory load of job A is 2 and B is 10, the total memory load is 2 110 max 2; 10 10 for processing either one job. It is understandable that the memory load of internal operations is accumulated as an AND relationship when more than one operation need to be processed. That is, the memory load is I1 ( p) I2 I1 1 I2 when both operations are internal. However, the external operations can be either an AND or OR relationship when only some operations are needed. For example, if the memory load of external operations are E1, E2, and E3, and the relationship is (E1(1)E2) ( p) E3, the accumulated memory load is calculated by max(E1, E2) 1 E3.
746
T.C. Du et al. / Information and Software Technology 42 (2000) 743±753
Table 1 The functions of extended data ¯ow diagram (EDFD) and extended state transition diagram (ESTD) Name
Symbol
Explanation
DFD/STD
Data Flow
Ð
A line indicate the direction of information ¯ow A circle indicates the action performed
DFD
Data store
A log indicate the database table
DFD
External agent
A block indicate external agents in the system
DFD
Process and display
A display symbol on the left upper corner of process indicate a display interface required
No
Internal operation
A dashed line indicate the internal operations required
No
Non-accumulated display interface (NDI)
A rectangular block indicates interface for showing message
STD
A line with arrow indicates the direction of state transition A gray triangular mark on the left upper corner of NDI indicates the main interface for users
STD
External operation
Description of external operations
No
Exclusive operator
A circle with cross line indicates the constraint of entry and exit of interface
No
Process
Task/process or state change Accumulated display interface (ADI)
Ð
DFD
No
T.C. Du et al. / Information and Software Technology 42 (2000) 743±753
2.2. Extended data ¯ow diagram and extended state transition diagram The information ¯ow analysis analyzes the quality of software in the early stage by means of measuring the memory load. The information ¯ow analysis model has two stages: (1) developing an EDFD and (2) developing an ESTD. The EDFD is used for acquiring the memory load of internal operations while the memory load of external operations is obtained in ESTD. The EDFD is modi®ed from the data ¯ow diagram by adding Process and Display and Internal Operation Flow as shown in Table 1. In addition to the conventional DFD development guideline, the Process and Displayis used to represent a display interface of a process component. Note that interfaces will be created for each Process and Display function and will be used in ESTD later. The Internal Operation Flow, drawn in a bold dashed line, represents the information ¯ow between users and Process and Display via internal operations (to be discussed later). The volume of information ¯ow is the essential factor that causes high memory load. The second stage is to develop an ESTD which is altered from STD by adding three functions: Accumulated Display Interface (ADI), External Operation Flow, and Exclusive Operator (please refer to Table 1). The ADI is an interface accumulated memory load of Non-Accumulated Display Interface (NDI), which is just a simple interface. The External Operationevaluates the user memory loads of both the ADI and NDI. The External Operations of each interface will be shown within each interface block. The Exclusive Operatorfunction is dedicated to constraining the looping of user operations by limiting the entry and exit following the same interface so that the users will not get lost in the loop. 2.3. The information ¯ow analysis procedure The information ¯ow analysis model is modi®ed from the SDLC by revising system analysis and system design tools. (An example can be seen in Section 3.2) The procedure is: 1. Design database for storing data: In this stage, one can use a data conceptual modeling technique such as an entity-relationship model, to design the database structure. 2. Depict EDFD: This stage uses processes, data ¯ow, and external agents to describe the information ¯ow of the data stores de®ned in the ®rst stage. As discussed earlier, the processes include Process operations and Process and Display operations, depending on whether or not the display interfaces are required. Note that the information ¯ow is the internal operation. 3. De®ne data dictionary after completing the EDFD analysis. 4. Design user interfaces based on the Process and Display operation in EDFD analysis.
747
5. Depict ESTD: Identify the user interfaces obtained from the previous stage into ADIs and NDIs and add a gray triangle on the left upper corner for ADIs. Note that the ADIs are the main interfaces that cause the memory load. The NDIs are only for displaying messages, and their memory load will be accumulated into their corresponding ADIs. 6. Estimate the volume of information ¯ow in EDFD and compute the memory load. After the EDFD and ESTD design are completed, a computation procedure for obtaining total memory load is activated. First of all, a designer estimates the volume of information ¯ow of each internal operation in EDFD. In the design process, the volume of information ¯ow is estimated by experienced system analysts, experts, or simulation results. Fortunately, the estimate used to calculate the memory load and does not have to be absolutely accurate since it is used for comparing models. It is also bene®cial that the estimate becomes a push force to the designer to evaluate the information volume of each operation. The estimated information volume of the internal operation is substituted into the memory load functions and their corresponding relationships. Again, only AND operands appear in EDFD (internal operations) and both AND and OR operands are used in ESTD. 7. Estimate the usage frequency of external operations and compute the memory load. Similar to the procedure in EDFD, the usage frequency of each external operation has to be estimated. This estimate can also come from either experts or past experiences. To calculate the score of interfaces, one uses the usage frequencies as the weights to multiply the memory load from external operations, which substitutes the information volume from EDFD into the independent variable X of the characteristic functions, and then applies the AND/OR relationship rule to combine the external operations.It has to be noticed that some designs expect users, while using the software, to provide extra memory-aided tools, such as a pencil and paper, from outside the system. To compare different designs fairly, this type of external memory-aided tools also needs to be included. The solution is to add one extra component (interface) to represent the tool. 8. Compare designs and choose the design with the lowest score. The lower-scored design means that the user needs a smaller memory load to use the software. Form the perspective of this research, the software with the smaller memory load has better performance. 3. Designing a qualitative analysis software using the information ¯ow analysis model This research develops a qualitative research software as an example to demonstrate the information ¯ow analysis model. The reasons for using such software are: (1) the
748
T.C. Du et al. / Information and Software Technology 42 (2000) 743±753
Fig. 1. An example of extended data ¯ow diagram of the qualitative analysis software.
qualitative analysis requires a signi®cant amount of memory load; (2) the system complexity is low; and (3) the system architecture is simple. 3.1. The system requirements Five types of approaches for doing qualitative analysis are text retrievers, textbase managers, code-and-retrieve programs, code-based theory builders, and conceptual network builders [15]. The main streams of qualitative analysis software provide either the code-based theory builders or conceptual network builders. This research uses the code-based theory builder approach as an example. That is, the analysis results are constructed in the process of code building. Until now, qualitative analysis software was only used by humans to assist the data analysis since the analysis work is still too much to a computer [16]. Therefore, software should provide functions such as text editing, data coding, doing memo, data linking, data searching, and data presentation. The software developed in this research will provide the functions of data editing, coding, data searching, data linking, and data presentation. 3.2. Information ¯ow analysis model of the qualitative analysis software This section demonstrates the procedure of developing a qualitative analysis software step by step as shown in Section 2.3. 1. Design database. To process qualitative analysis, software has to store texts, memos, indices, and project information. Therefore, four schema struts are created: project table, index table, memo table, and code table.
2. Depict EDFD. The data ¯ow diagram describes the information among processes, agents, and data stores. Fig. 1 shows the EDFD for the qualitative analysis software. Several marked elements are brie¯y explained as follows: (1) Process elements. The action of the process is described in the circle. The process element can be explored for details if necessary. (2) Information ¯ow. The information ¯ow describes the information from one entity to another entity. The name of the information is shown on the line. (3) Data store. The data stores are designed in the previous step. While doing the qualitative analysis, users create a project database before editing or analyzing texts, and then store the text (or data) related to the project. (4) Process with display element. The process will become an interface for displaying messages to users. The interface number is shown at the left corner. These interfaces become ADI or NDI of ESTD in the later design. (5) Information ¯ow of the internal operations. The operations describe the types of the user's memory load from interfaces. For example, when adding data, a user need to proceed the operations, such as (I4), selection (I5), and search (I6) in the ªdisplay interface 3º. Note that the relationship between internal operators is an AND relationship, represented in ( p), such as I4 ( p) I5 ( p) I6 in the ®gure. 3. The data dictionary de®nes the information about data types, relationship, etc. Since it is not the focal point in this study, the dictionary is not shown. 4. Design user interfaces for Process and Displays. Interfaces are used for communicating between users and
T.C. Du et al. / Information and Software Technology 42 (2000) 743±753
749
Fig. 2. An example of extended state transition diagram of the qualitative analysis software.
software, it is important to get users involved in the earlier design stage. A rapid application development (RAD) tool can be used to develop the interfaces. Each Process and Display Interface should have one interface design designated to it. 5. Depict ESTD. The state transition diagram describes the transition between user interfaces. Fig. 2 shows an example of the qualitative analysis software. Some marked elements are explained as follows:
(1) Accumulated Display Interface. There are ®ve ADI in this example. The ADI accumulates relative memory load and is the major concern in the ESTD. (2) Non-accumulated Display Interface. The NDI generates a memory load because of external operations. However, the memory load of NDIs will be accumulated into their corresponding ADIs. (3) State Change process shows how an interface can switch to another.
750
T.C. Du et al. / Information and Software Technology 42 (2000) 743±753
Table 2 The memory load of an qualitative analysis design ADI Main screen 1 Open Prj New Prj Main screen 2
Main screen 3
Total: 5
NDI
Add data Tree management Del data Add index Tree management Del code Edit code Add code Add memo Del memo Edit memo Del index Edit index Open data Link to data Collect data
Usage frequency 1 1 1 1 50 100 10 50 300 100 20 50 100 100 10 40 50 100 50 50 30
Weight 1/54 1/54 1/54 1/54
50/54
Score 0 6 2 5675600 15000 5660000 600 11913700 6020000 5660000 1600 15000 40000 30000 300 4800 5000 30000 15000 20000 72000
Total: 54
0 0.11 0.04 105103.70
11031203.7
11136307.6 Average
(4) A set of External Operations. The external operations within a display interface describe the types of operations and the memory load of the interface. For example, operations of mouse operations (E1), visualization (E3), and audition (E4) are the external operations for the open project interface. The relationship of these operations is E1( p) E3( 1 )E4. (5) Exclusive Operator. The exclusive operator limits the user to entry the tree management function from Main Screen 2 and to exit from the same interface. This design prevents users from getting lost in the loop. 6. Estimate the information volume in EDFD and compute the memory load. The information volume is used for calculating the degree of memory load of a speci®c system design. As it has been discussed, the volume can be estimated by system analysts, experts, or simulation results. The importance of the volume is on the relative ratio rather than on the absolute values. For example, if a user ªopen a projectº once, he/she could ªadd a ®leº about 50 times, ªdelete a ®leº about 10 times, ªadd codesº about 100 times, ªdelete codesº about 100 times, ªadd indicesº about 300 times, ªdelete indicesº about 50 times, ªlink to other ®lesº about 50 times, etc. After estimating the volume of information ¯ow, the internal memory load can be obtained through the characteristic functions of internal operators and their relationships. For example, the Process and Display ªAdd Dataº requires the memory load of internal operations I4
Weighted score
2227261.5
( p) I5 ( p) I6 50 1 50 1 50 50; and ªDel Dataº requires I4 ( p) I5 ( p) I6 10 1 10 1 10 30: 7. Estimate interface usage frequency and compute memory load. Like the estimation made in EDFD, the usage frequency can also be obtained from experts, analysts, or simulations. Again, the relative values of the interfaces are more important than the absolute values of each interface. As an example of the qualitative analysis software, the usage frequencies are estimated. A user therefore uses ªmain screen 1º once, will use ªopen projectº once, ªadd dataº about 50 times, ªdelete dataº about 10 times, ªadd indexº about 300 times, ªdelete indexº about 50 times, ªadd memoº about 100 times, ªdelete memoº about 10 times, etc. Table 2 shows the computed memory load for a speci®c design. The weights are obtained by dividing the usage frequency of each ADI by the summation of all ADIs for balancing the impact from interfaces. The score of ADI is obtained by adding the score of its corresponding NDIs. Use ªmain screen 2º as an example: the ADI ªmain screen 2º accumulates three NDIs, ªAdd Dataº, ªDel Dataº and ªTree Managementº. The score of an NDI is obtained from calculating the memory load of external operations. For example, the NDI ªAdd Dataº has external operations E1 ( p) E3 ( 1 ) E4. Transforming the relationship of external operations becomes X ( p) X 1X 2X: The X values is the internal information volume obtained from stage 6, which is 150 for ªAdd Data.º Therefore, the score of ªAdd dataº is equal to the usage frequency times external operations, i.e.
T.C. Du et al. / Information and Software Technology 42 (2000) 743±753
751
Table 3 Evaluation scores and the calculated scores based on the proposed model Software
I(A±B)
II(A±b)
III(a±B)
IV(a±b)
Calculated score Evaluation score
2227262 3.816
2528631 3.539
4519902 3.386
4907994 3.225
50 p 2X 50 p 2 p 150 15000: Similarly, the NDIs of ªDel Dataº is 600 and ªTree Managementº is 5660000. Adding up the corresponding NDIs can obtain the score of ADI. For example, adding 15000, 600, and 5660000 is the accumulated score of main screen 2. The total weighted score of the design is 11083900. To compare the score between designs, the total weighted score has to be averaged by the number of ADIs. Otherwise, the design with more ADIs will tend to generate a higher score. It is also worth noticing that the values of ªAdd Index,º ªTree Management,º ªLink to Data,º and ªCollect Dataº have higher scores when compared with other interfaces. This observation can remind the designer to design these types of interface carefully since the user accumulate a signi®cant amount of memory load on them. That is the source from which the low satisfaction comes. For example, adding memory-added interfaces, e.g. historical notebooks or code cards, can signi®cantly increase the software's performance. 4. Validation of the information ¯ow analysis model An experiment was used to identify the degree of user satisfaction. Four different qualitative analysis software programs were developed. These four designs were based on the different software design conditions that will be described shortly. The software programs were given to seven reviewers for evaluation. The reviewers were selected from the faculty and graduate students in a department of education. The reviewers were all working on the ªqualitative researchº at that time, and were familiarized with the intent of the evaluation experiment before the evaluation took place. The interviewers needed to answer a questionnaire immediately after using each software program. After completing four software evaluations, a personal information data sheet was given to the reviewers to ensure that the reviewers have similar backgrounds. Each questionnaire is Table 4 ANOVA result of the evaluation experiment Factor
Test d.o.f
F-test
Probability
Signi®cance
Process Question Setup Subject Process p Question Process p Setup Question p Setup
(1,243) (9,243) (1,243) (6,243) (9,243) (1,243) (9,243)
5.025 14.352 21.739 15.999 0.817 0.201 0.407
0.026 0.000 0.000 0.000 0.601 0.654 0.931
* * * *
composed of 16 questions, two questions for an internal operation and one for an external operation. Each question asks how the reviewer feels about a speci®c operation after using the software. For each question, the reviewers had to rate the level of satisfaction for the software program they had just used. The satisfaction rating scale is a 5-point scale. Since the information analysis model has six internal operations (creation, comparison, categorization, decision, and selection) and four external operations (mouse operations, key-in, visualization, and audition), 10 types of questions were generated for the 10 operations. The implementation of a qualitative analysis software has two stages: analysis environment setup and data process. Each design includes two levels: menu/hot-key-control A and command-line-control a for the environment setup stage, and multiple-data-in-a-single-window B and multiple-data-in-multiple-window b for the processing stage. Four combinations of the software, i.e. software I, II, III, and IV were developed as A±B, A±b, a±B, and a±b. The evaluation result was ®rst tested whether the satisfaction was different on the four different versions of the software programs. The satisfaction index of each software was obtained from averaging the satisfaction level on each question. That is, ((question 1 1 question 2)/2 1 ¼ 1 (question 11 1 question 12)/2 1 question 13 1 question 14 1 question 15 1 question 16)/10 The evaluation scores and the theoretical calculations are presented in Table 3. Based on the evaluation results, the satisfaction levels were ranked as: software I . software II . software III . software IV Kruskal±Wallis test showed that there was signi®cant difference in user satisfaction on the four software programs X 2 14:056; d:o:f 3; a 0:05: As observed from the data, the ranking of the satisfaction index was reverse to the memory load from the information analysis model. The results imply that the software with the higher memory load causes lower user-satisfaction. That means that if the system analyst estimates the memory load of designs, he/she can prevent from choosing the lower user-satisfaction design in the ®rst place. Analysis of variance (ANOVA) was performed on the data with the following model: Yijkl m 1 ai 1 bj 1 gk 1 dl 1 abij 1 agik 1 bgjk 1 1ijkl
752
T.C. Du et al. / Information and Software Technology 42 (2000) 743±753
where a is the question factor, i 1; ¼; 10; b the setup factor, j 1; 2; g the process factor, k 1; 2; d the blocking factor, l 1; ¼; n; ab the interaction between question and setup factors; a g the interaction between question and process factors; b g the interaction between setup and process factors; and 1 the errors. The result of the ANOVA is presented in Table 4. There are several observations: 1. The question factor is signi®cant. Both the internal and external operations corresponding to the sixteen questions have a signi®cant in¯uence on user-satisfaction. If we further identify the individual operations, I1 (creation), E1 (mouse operation), E3 (visualization), and E4 (audition) have higher rating, implying that the better interface designs are please to users and the users dislike heavier memory-loading operations. 2. The setup factor is signi®cant. The implication here is that there is a signi®cant difference between menu control (mouse control) and command control: operating the menu requires less memory than operating the command control. 3. The process factor is signi®cant. The multiple-data-in-asingle-window approach is preferred to the multipledata-in-multiple-window approach. This is because users have to jump between windows for searching information in the multiple-data-in-multiple-windows. If a single window is designed for collecting the related data while processing qualitative analysis, the users can process analysis effectively. That is, this design requires fewer memory load. 4. All the interaction factors are not signi®cant. That is, there is no signi®cant impact between the question and setup factors, the question and process factors, and the setup and process factors.
5. Conclusions This research uses the degree of the user's memory load to indicate the degree of user-satisfaction on software designs. An information ¯ow analysis model is proposed for analyzing the degree of memory load. Ten categories of operations, including six internal operations and four external operations, are adopted to measure the degree of memory load. The DFD and STD of the system development life cycle are modi®ed into an EDFD and ESTD by adding more functional components. Besides the conventional system analysis concern, the EDFD and ESTD have extra components to measure the memory load. The information ¯ow analysis model is demonstrated by building a qualitative analysis software which requests heavier mental works but has a simple software architecture. This software can be used to edit, review, categorize and analyze data. For validating the model, an experiment is designed for obtaining the degree of user-satisfaction, and
then four different designs of the qualitative analysis software are built accordingly. The software is tested by interviewers, and questionnaires are given to obtain their opinions. Based on the opinions, some observations can be concluded: 1. The same conclusions of the interviewers and the information ¯ow analysis model imply that the proposed approach can be used to estimate the user-satisfaction in the early stage. 2. The Kruskal±Wallis test rejects the null hypothesis on the difference of designs, and concludes that user-satisfaction is different for the four designs. 3. The ANOVA shows that both internal and external operations have an in¯uence on the degree of usersatisfaction. 4. The ANOVA show that the different designs on the system setup have an in¯uence on user-satisfaction, and the menu-control design gives better satisfaction than the command-control design. 5. The ANOVA shows that the different designs on data process have an in¯uence on user-satisfaction, and single-window design gives more satisfaction than the multiple-window design. Observations show that the information ¯ow analysis model can be used to estimate the degree of user-satisfaction during the design stage. Furthermore, following the procedure, a design can diagnose the interfaces with heavier memory load and revise accordingly to produce a better performing software. References [1] J.A. McCall, An Introduction to Software Quality Metrics, Software Quality Management, McGraw-Hill, New York, 1979. [2] Y.W. Shih, S.W. Margaret, System Analysis and Design, West, St. Paul, 1994. [3] M. Alavi, J. Wetherbe, Mixing prototyping and data modeling for information system design, IEEE Software May (1991) 86±91. [4] M. Zelkowitz, B. Cuthill, Application of an information technology model to software engineering environment, J. Systems Software 37 (1997) 27±40. [5] J. Whitten, L. Bentley, V. Barlow, System Analysis and Design Methods, 3rd ed., Richard D. Irwin, Inc, Butt Ridge, IL, 1994. [6] C. Gane, T. Sarson, Structured System Analysis: Tools and Techniques, Prentice-Hall, Englewood Cliffs, NJ, 1978. [7] T. DeMarco, Structured Analysis and System Speci®cation, PrenticeHall, Englewood Cliffs, NJ, 1978. [8] J.P. Walsh, Selectivity and selective perception: an investigation managers belief structures and information processing, Acad. Mgmt J. 31 (4) (1988) 873±896. [9] B. Vandenbosch, C. Higgins, Information acquisition and mental models: an investigation into the relationship between behavior and learning, Inf. System Res. 7 (2) (1996) 198±214. [10] S. Cant, D. Jeffery, B. Henderson-Sellers, A conceptual model of cognitive complexity of elements of the programming process, Inf. Software Technol. 37 (7) (1995) 351±362. [11] V.V. Kul'ba, A.P. Shvetsov, Methods for analysis and design of systems maintaining certainty of information in MIS, Automation Remote Control 55 (7) (1994) 1074±1083.
T.C. Du et al. / Information and Software Technology 42 (2000) 743±753 [12] C. Wickens, Engineering Psychology and Human Performance, 2nd ed., Harper Collins, New York, 1992. [13] S.K. Card, T.P. Moran, A. Newell, The Psychology of Human Computer Interaction, Lawrence Erlbaum Associates, Hillsdale, NJ, 1983. [14] P.H. Lindsay, D.A. Norman, Human Information Processing: an Introduction to Psychology, Academic Press, New York, 1977.
753
[15] E.A. Weitzman, M.B. Miles, A Software Sourcebook: Computer Programs for Qualitative Data Analysis, Sage, Beverly Hills, CA, 1995. [16] R. Tesch, Qualitative Research: Analysis Types and Software Tools, Falmer, New York, 1990.