On the role of metaphor and language in design of third party payments in eBanking: Usability and quality

ARTICLE IN PRESS

Int. J. Human-Computer Studies 64 (2006) 770–784 www.elsevier.com/locate/ijhcs

On the role of metaphor and language in design of third party payments in eBanking: Usability and quality Catherine S. Weir, James N. Anderson, Mervyn A. Jack School of Engineering and Electronics, Centre for Communication Interface Research (CCIR), The University of Edinburgh, Alexander Graham Bell Building, The King’s Buildings, Edinburgh EH9 3JL, UK Received 23 June 2005; received in revised form 21 March 2006; accepted 23 March 2006 Communicated by M. Atwood

Abstract This paper describes results of a usability study of contrasting user-interface designs for Internet Banking (eBanking). Two specific interface metaphors were compared in the first experiment, linear form filling and array editing interaction modes. Terminology in the interaction dialogue was compared in the second experiment, using typical banking language and a generic, plain language interface. This research aimed to perform usability evaluation and comparison of the alternative interface designs to illuminate the development of new eBanking services. This research involved sixty-one participants (Internet users and customers of the involved Bank) exploring the designs in controlled experiments involving hands-on experience. Banks are interested in ensuring their eBanking services are highly customer-centric and that the interface matches customer expectations in order to drive customers towards this lower cost channel. The results of the first experiment (N ¼ 32, where N indicates the number of participants in the cohort) concluded that the simple form-filling metaphor, taken from the traditional paper-based procedure, was generally more usable than a Spreadsheet metaphor. In the second experiment (N ¼ 29), it was found that although banking terminology was not completely understood across the cohort, the instructional language changes did not impact significantly on usability. r 2006 Elsevier Ltd. All rights reserved. Keywords: Usability; User-interface; Design; Internet Banking; Experiment

1. Introduction Internet Banking (eBanking) services exist as Web-based user-interfaces that allow customers to use remote access to manage their bank accounts and transactions. Studies of usability issues relating to eBanking services are not yet as widely documented as those for Graphical User Interface (GUI) design or Internet (Web) interface design, mainly because of the commercially sensitive nature of some eBanking usability studies and the fact that the field is still relatively new. A number of design issues relating to eBanking usability have already been defined in the literature: customers tend to be very private about their finances and are concerned about eBanking security (Furnell, 2004); error prevention and recovery are very Corresponding author. Tel.: +44 131 650 2801; fax: +44 131 650 2784.

E-mail address: [email protected] (C.S. Weir). 1071-5819/$ - see front matter r 2006 Elsevier Ltd. All rights reserved. doi:10.1016/j.ijhcs.2006.03.003

important with regard to self-service account management (Liao and Cheung, 2002); consumer trust in eBanking services has an impact on levels of adoption (Aladwani, 2001). Of these, the issue of trust is the most common focus of published research (Kim and Moon, 1998; Suh and Han, 2002; Grabner-Kra¨uter and Kaluscha, 2003; Yousafzai et al., 2003). Other published research in the area focuses on customer requirements for eBanking (Jayawardhena and Foley, 2000), and several studies into eBanking adoption (Tan and Teo, 2000; Centeno, 2004; Lai and Li, 2005). However, there is a lack of published studies addressing eBanking interface designs in hands-on usability testing, we will examine this issue here. eBanking has much in common with, but has yet to become a true eCommerce application: banking customers cannot use an online service from another bank in the same way that shoppers can use many competing eCommerce Websites for researching and purchasing products and

ARTICLE IN PRESS C.S. Weir et al. / Int. J. Human-Computer Studies 64 (2006) 770–784

services. To use another eBanking service the customer must switch banks, a practise that is still rare (Hudson, 2002). However, eBanking customers are increasingly demanding better usability and a recent survey suggests more are prepared to switch (BBC News, 2005). Banks are interested in the Internet to reduce costs by migrating customers to self-service channels (Jayawardhena and Foley, 2000; Sohail and Shanmugham, 2003); the Internet channel being more cost-effective than other customer contact channels (Gopalakrisnan et al., 2003). Usability is a growing concern in eBanking as banks rollout increased self-service functionality. In such environments, greater demands are put on the customer to understand financial transactions. Banking has traditionally been undertaken by trained staff (i.e. in the branch), and banking transactions are not as commonly conducted as shopping tasks. Thus, usability engineering has an important role to play in the design of eBanking functionality. The aim of the research reported here is to examine different design metaphors for implementing funds transfer and payment functionality within an eBanking service, and to evaluate the resulting user interfaces in terms of usability. The two main considerations investigated in the study are firstly the interaction metaphor and secondly the style of language used in the human–computer interaction dialogue. Both of these design artefacts were hypothesized to have a direct impact on usability. Three contrasting user-interface design variants were compared in the experimental work described here. The first took the ‘Form’ metaphor that derived from traditional paperbased banking; the second took the ‘Spreadsheet’ metaphor from computerized numerical work. The third interface matched the look and feel of the ‘Form’ interface whilst employing simpler, ‘plain’ language for on-screen instructions, transaction terms and dialogue. This style was an alternative to the more formal banking terminology (jargon) used throughout the ‘Form’ and ‘Spreadsheet’ interface. This paper presents the results of two usability experiments involving these three interfaces, conducted under controlled conditions. In both experiments, two alternative interface designs for making payments in eBanking were compared by a representative cohort of customers. For the first experiment, prototype interfaces were developed to represent the two different metaphors for the transactions: simple, linear form-fill (matching the paper-based mode) and an array-edit mode (as used in Spreadsheets). These two interface metaphors were compared using a cohort of 32 customers. In the second experiment, two alternative payment interfaces were created using the same interaction metaphor (simple, linear form-fill), but this time investigating the use of formal banking terminology compared to the use of plain language. These two interface dialogue styles were compared using a separate cohort of 29 customers. The participants in the experiment were recruited as being representative of current and potential members of

771

the target market for the sponsoring Bank’s eBanking service; they were customers of the Bank, who were ‘Internet-savvy’. The participants were therefore a random sample of current and potential users of the eBanking service with a wide range of experience with the Internet and eBanking services. By recruiting Internet-savvy participants, it was ensured that the usability experiment sessions focused on the competing interface designs, rather than the usability of the Internet in general. In addition, these participants represented the target audience for the eBanking functionality being proposed. All undertook novel tasks on the prototype services for the experiment, which were not available on the ‘live’ banking site. None of the participants had previous experience of using either of the payment interfaces. 1.1. Usability Usability is defined as ‘the efficiency, effectiveness and satisfaction with which specified users can achieve specified goals in particular environments’ (ISO, 1998). Many other definitions of usability exist; all reinforce the idea that several aspects of system design will influence interaction (Gould, 1988). A profile of the intended user, their task and their setting are a key part of the definition. Usability engineering is a process that aims to ensure quality in use for the intended user of a finished product. The focus is on the user-interface, where direct experience of the technology takes place (Shneiderman and Plaisant, 2005). As the ISO definition suggests, usability is a multi-dimensional concept. There are often compromises to be made, trading off different goals to achieve a usable product. Efficiency, effectiveness and satisfaction are independent qualities of the system (Frøkjær et al., 2000). Several measures of the customer experience are usually combined in an empirical measurement of usability. Usability engineering is now an established practice in the software development cycle. There has been much research, method, theory and practical application published in this area (for example, Nielsen, 1993; Faulkner, 2000). Efficiency is concerned with the amount of effort required in usage. It is typically measured as the time taken or the number of clicks to complete a task. Effectiveness is indicative of application robustness and transparency, task completion, accuracy, prevention and easy recovery from errors are typical measures. Satisfaction relates to the degree to which users react positively to their experience whilst completing tasks. It may concern perceived usefulness, attractiveness and other emotional responses to the system. 1.2. Web usability Web usability is the focus of much published research (for example, Spool et al., 1997; Nielsen, 2000). Web usability also aims to ensure efficient, effective and satisfying experiences, but is specifically interested in use

ARTICLE IN PRESS 772

C.S. Weir et al. / Int. J. Human-Computer Studies 64 (2006) 770–784

of the Internet. In particular, it has been noted that satisfied users may spend longer at a Web site, may revisit the Web site later, and may recommend it to others, issues particularly relevant to eCommerce and eBanking (Zhang and von Dran, 2000). For self-service banking, efficiency is a key driver of customers to the online service (Wright, 2002; Dandapani, 2004). eBanking interfaces must therefore be quick and easy to use in order to meet the needs of eBankers. Finally, an eBanking interface must be effective, carrying out the desired transactions in a way that avoids errors, which may then be costly to resolve, or worse, go unnoticed by the customer. In the experiments described here, the usability of each interface was measured using a combination of task completion, number of errors and recovery of errors (effectiveness). Task completion was recorded when participants completed a task correctly with no errors. When errors were made and easily recovered, this also counted as completion of the task. Such errors and their recovery were noted by researchers and classified, for example, as missing data or a typographic slip. Other errors were noted by researchers and were classified as catastrophic errors when incorrectly completed banking tasks went unrecognized by the user. Such errors would have caused problems in real self-service banking applications, perhaps resulting in additional calls to humanoperated, costly channels to remedy such mistakes. Where numbers of errors were high, interface effectiveness was called into question. Usability was also measured by attitude questionnaires as measures of satisfaction (Dutton et al., 1993; Preece et al., 1994). An inventory of 21 statements was chosen relating to common usability and Web issues, including aspects specific to eBanking. It concentrated on visual artefacts and Web interaction: the size of text, navigation, information organization, control, page layout, appearance, readability, quality and enjoyment. These attributes relate to many Website and eCommerce usability issues. eBanking specific questions were also measured in the questionnaire: helpfulness, understanding, amending transaction details (specific to tasks in the first experiment), and the language (specific to the second experiment). In this way individual items were investigated in detail, along with an overall measure of interface usability. The results of the experiments include an overall usability score for all usability attributes with the potential to evaluate the interface and uncover specific areas where usability can be improved. The data concerning specific experiment variables (such as data input methods and dialogue language in this case) assist in further understanding the differences between the alternative designs. High, positive attitudes to usability attributes indicated high satisfaction with the usability of the interface. In the experiments, participants were not timed for efficiency, but instead were encouraged to use the ‘think aloud’ protocol for eliciting qualitative remarks about the interfaces they were using. The use of a discussive

protocol has drawbacks as some participants find it more difficult than others do, and it interferes with precise timing (Karat, 1988; Shneiderman and Plaisant, 2005). The data collected is beneficial in understanding how people approach eBanking tasks and interfaces, which provides important feedback for the user-centred design cycle. Controlled experiments are used to provide accurate and balanced assessments of usability issues, independent of the tastes and preferences of any individual (Gould, 1995). Statistical analysis, such as hypothesis significance testing using the analysis of variance (ANOVA), can be used to determine what the data collected shows. This allows inferences to be made about how various controlled factors have attributed to performance and satisfaction measures (Landauer, 1988). The number of participants needed depends upon the amount of segmentation required in the population. For robust statistical testing, larger numbers are needed in each key demographic group, i.e. of differing ages or genders (Landauer, 1988) to lessen the impact of individual differences. These studies transfer well to banking applications, where tasks can be specific and presented in well-defined scenarios.

1.3. Metaphors Metaphors are widely used to aid the design and usability of human computer interactions, for example, the desktop for an office or home computer screen, where a wastebasket icon for deleted files is now in common use by a range of operating systems (Alty et al., 2000). A metaphor offers a model that allows a user’s experience, knowledge and behaviour to be transferred from a familiar, real-world entity to, for example, a new computer interface. The metaphor, when well conceived, reduces the burden of grasping new concepts. However, in reproducing aspects of a real-world process, the user-interface designer is not guaranteed to produce innovative or usable interfaces, but can instead fall into the trap of replicating inefficient procedures. A formal task analysis can usually identify any problems with current procedures and can be helpful in identifying possible improvements for a new system. The process of creating a metaphor usually involves, as a first step, understanding what has to be done by the system by analysing the task domain (Erickson, 1995). The resulting metaphor or metaphors must subsequently be tested with representative users to ensure it is being effective in improving the interaction design. In designing an interface for eBanking payments, the first version opted to digitalize the branch banking forms, completed by customers and given to staff for actioning on their behalf. The second version of the payment interface was derived from the common Spreadsheet metaphor, widely used for numerical and financial purposes.

ARTICLE IN PRESS C.S. Weir et al. / Int. J. Human-Computer Studies 64 (2006) 770–784

1.4. The problem of third party payments in eBanking Third party payments (P3P) are defined as funds being transferred from a bank customer’s account to an account either held by them at a different bank or by another person at the same or a different bank. As such, any payments made beyond the sphere of a customer’s own accounts, but with the same bank, are also termed third party payments. These typically fall into two basic categories: bill payments (paying utility, credit card and other formal bills) and miscellaneous payments (paying smaller companies and private individuals as recipients). One challenge for the deployment of effective eBanking interfaces is to migrate regular bill payments from other channels (such as telephone banking) onto the bank’s Internet service. Initially, eBanking services inherited the payment arrangements from other channels. When functionality for creating payment instructions online was first introduced, the evolution of eBanking from a passive device for viewing account details to an interactive service was complete. This study focuses on usability experiments exploring interface design options for making various types of payment, and how they could be incorporated into the eBanking service. 2. Interface designs The original design of the eBanking service considered in this research employed a standard interaction mode used on the Internet: linear form filling. The user interface metaphor matched real-world branch transactions by means of on-screen versions of the standard paper form. In this design, the metaphor supported interactions with which customers were familiar from transacting in the branch and their use of various Websites on the Internet. The use of this ‘Form’ metaphor was therefore examined with respect to usability and human factors issues with a view to extending its usage into miscellaneous third party payments. 2.1. Functionality The current and proposed eBanking functionality was categorized into the following types:











Passive viewing: Activities such as checking balance information and statement items. Transactional interactions: Defined as moving money to other internal or external accounts using existing payment orders such as transfers, bill payments and other miscellaneous payments. Informational interactions: Defined as the process of inputting account details to create payment orders, but also extending to form filling for opening new accounts (savings, loans etc.) online. Spring-cleaning: Defined as deleting payment arrangements, payee account details that are no longer in use,

773

inactive direct debits and other dormant account information. Most mass-market eBanking services offer simple abilities, such as passively viewing account details, statements and the ability to do limited spring-cleaning. Some sites provide scope for transactional interactions—such as internal transfers. The addition of an external payment service to the eBanking service increases the potential for transactional and informational interactions. Banks are keen for their customers to be able to fulfil all types of banking needs using the online channel, with its associated cost-savings for the bank and convenience for the customer. In the first experiment, the difference between the two competing designs was the payment interface metaphor. The payment interface offered a list of hyperlinks to individual payment forms or a Spreadsheet array of payments. As such, the usability evaluation concentrated on tasks that highlighted the payment interface pages. Participants performed four tasks: one required deleting a future payment to a company; another required deleting an account from the list of recipients and two concerned amending amount details to pay certain accounts and utility bills. The tasks required simple data input and editing. They were designed to maximize exposure to the two alternative metaphors. In the second experiment, the difference between the designs was in the language used for instructions, terminology and tone of voice. The language changes were distributed throughout the service. Three tasks were selected that highlighted the dialogue changes between the two interface designs. One task required creating a bill payment to pay money to a company (such as a utility or telephone bill). Another task required sending money to a third party account (such as a friend or relative). A third task concerned performing internal account transfers (such as from savings to a current account or vice versa). The tasks required following instructions and performing considerable data input in forms. They were designed to maximize exposure to the alternative dialogues. 2.2. Linear form-filling metaphor There are clear advantages to using linear form filling for types of eBanking tasks such as entering details of external accounts. Several details are requested in sequence and entered into form fields by the customer. The linear formfill mode utilizes a step-by-step procedure that matches customer expectations of the Internet and the bank. A major disadvantage of this method is that form filling makes amending details and making payments a slow process. Each form pertains to a single transaction; therefore multiple tasks are not supported conveniently by this approach. The eBanking interface also requires a common security feature to complete online transactions: repeated password entry.

ARTICLE IN PRESS C.S. Weir et al. / Int. J. Human-Computer Studies 64 (2006) 770–784

774

Such password validation is not an unusual feature to see on an eBanking interface, and repeated password entry is an approach used by many UK banks. Other common practices include asking for more than one ‘password’ (or similar) for transactions. Although, there are some eBanking interfaces which do not insist on such additional security measures, eBanking security issues are constantly being exposed and recently even more rigorous measures have been researched (Nilsson et al., 2005). eBanking security procedures were not the focus of this experiment, therefore security measures were kept consistent in all three interfaces for reasons of experimental control. The Form design was characterized by an interaction mode using separate forms for each payment requiring data input, amendment or deletion of details. Payment details could be altered using the forms (Fig. 1). Each form was accessed from the overview table using hyperlinks, and contained editable details for each payment order. Action buttons offered options to make, change or delete the selected payment.

2.3. Spreadsheet metaphor An alternative metaphor was used as the basis of the design of an innovative solution for performing routine payment transactions (bills and miscellaneous payments). Having completed a linear set up process, the payment details were stored in an editable array, coined the ‘Spreadsheet’ metaphor. A number of table cells were editable and the overview table itself formed the transaction interface. Payment amounts, dates and customer reference information could be added, amended and

Fig. 1. Simple linear form in the Form design.

deleted using a single page. The Spreadsheet interface therefore allowed multiple payment tasks to be performed with single password entry. Rigorous error checking procedures were employed, with the scanning of each payment row for missing or unrecognized inputs. Errors were fed back to the customer and displayed on the screen. In the case of deletion, the dialogue prompted for confirmation whether to delete the pending payment or the entire arrangement (i.e. including the saved account details). The Spreadsheet design was characterized by the interaction mode of using an editable array (table) for data amendment and deletion whilst the functionality for creating payment arrangements was available in the standard form-fill mode. The ‘Payments and transfers’ page held a table of accounts to which money could be transferred and payments made. The bill and miscellaneous payments included fields that were directly editable, such as payment date and amount. In principle, multiple changes on multiple arrangements could be performed at one time, and the password only needed to be entered once, at the end (Fig. 2). 2.4. Dialogue style A second experiment was designed to examine customer understanding of banking terminology, investigating whether an eBanking interface should employ jargon to match the bank terminology used on other channels, or whether a more generic, simple ‘plain English’ dialogue would produce an interface with better usability. Theory suggests that interface dialogues should contain simple, natural language: the user’s language (Nielsen, 1993). Whilst customers in the branch may be familiar with bank staff using banking terms, and performing the appropriate transaction, in self-service the burden falls to the customer to choose the correct terms for different transactions. Two alternative dialogues were created based on the linear form interface design. An innovative solution was prototyped which combined all (user-initiated) transactional interaction tasks: payments, transfers and standing orders. This plain language interface used the linear set-up process, and form-fill metaphor, but combined the userinitiated transactions and described them in plain and

Fig. 2. Editable array in the Spreadsheet design.

ARTICLE IN PRESS C.S. Weir et al. / Int. J. Human-Computer Studies 64 (2006) 770–784

775

simple language—avoiding any Bank jargon (such as ‘standing order’ and ‘beneficiary’). This interface was compared with the use of standard, formal banking terminology. 2.5. Formal dialogue style The dialogue style for the formal style of interface was characterized by the use of traditional banking language and terminology. As such, each transaction was referred by its proper name: transfers indicated internal money movement within a single customers accounts at a bank; payments indicated money sent to external accounts; standing orders were regular arrangements to pay specific accounts on a habitual basis. These constitute customerinitiated transactions, typically under the customer’s control in eBanking. In contrast, direct debits are arrangements that the customer has no control over (pull transactions), as after initiation the customer can only view the details or cancel the task, with no direct control over timing and amounts. In addition to use of these terms, the dialogue used by this interface was formal, using terms such as ‘beneficiary’ to indicate a receiving account. As such, this dialogue is referred to here as the Formal style, an example instruction is shown in Fig. 3. 2.6. Plain language dialogue style A panel of user-interface developers and designers with expertise in usability engineering conducted an informal critique of the proposed interface and dialogue designs. The panel were concerned that the payment interface did not use straightforward language and recommended the creation of a plain-speaking version to compare with the proposed Formal dialogue style. This process resulted in the Plain language interface used for comparison in the second experiment. The idea of removing jargon from interfaces is not new and is a typical usability concern (Preece et al., 1994). Whether banks should swap formal terminology for plainer language is a debate that is still unresolved (Davies, 1996; BBC News, 2006).

Fig. 4. Banking terms used in the Plain Language Dialogue Style.

Fig. 5. Instructions used in the Plain Language Dialogue Style.

The Plain language dialogue style was characterized the use of simple, ‘plain English’ language in all instructions and labels. For example, instead of making a distinction between payments and standing orders in the menu and assigning alternative pages for these transactions, the terms ‘regular’ and ‘one-off’ were used to describe payments instead. The list of user-initiated payments therefore included transfers, payments and standing orders in one overview table as shown in Fig. 4. Otherwise, the interaction mode was the same as the Form metaphor and the Formal dialogue style. In addition, complicated dialogues were rewritten to promote scanning rather than reading and to give additional information regarding any choices available. The process of adding payment details to the interface involved choosing to specify account details or using predefined company accounts to pay (common) utility bills. This process was initially presented as a straight choice in the Formal dialogue style. In the Plain language style, the options were clearly distinguished and the dialogue offered further explanation of the two alternate forms. Some words were flagged as being particularly inappropriate to the audience, including the term ‘beneficiary’. The more commonly understood ‘recipient’ was used instead, as shown in Fig. 5. 3. Research questions and hypotheses

Fig. 3. Instructions used in the Formal Dialogue Style.

Each experiment compared two designs of interface and interaction for eBanking payments. In the first experiment, metaphor was the focus, and linear form filling was compared to Spreadsheet array editing. To highlight the difference in metaphor, tasks concentrated specifically on

ARTICLE IN PRESS 776

C.S. Weir et al. / Int. J. Human-Computer Studies 64 (2006) 770–784

making, amending and deleting payments. In the second experiment, language was the focus and use of generic, plain language was compared to banking jargon. The tasks concentrated specifically on setting up and making various payments and transfers, where this difference was most apparent. The aim of these studies was to provide quantitative and qualitative feedback on usability and customer perspectives on the various interface designs proposed to perform eBanking transactions. Experiment 1: An experiment comparing Form and Spreadsheet metaphors. H10. The use of different interaction metaphors for the payment interface will not result in different customer perceptions in terms of usability scores. H20. The use of different interaction metaphors for the payment interface will not result in different customer perceptions in terms of preferences. Experiment 2: An experiment comparing Plain Language and Formal banking terminology in interface dialogue style. H30. The use of alternative dialogues for the payment interface will not result in different customer perceptions in terms of usability scores. H40. The use of alternative dialogues for the payment interface will not result in different customer perceptions in terms of preferences. We predict that a measurable usability difference will be achieved through the different metaphors in Experiment 1 and through the different use of language in Experiment 2. Participant variables such as age and gender were balanced in the experimental design along with the order of experience. Each interface tested was substantially different from the next and altered in one design feature only. We collected data to test the null hypotheses and indicate whether they were accepted or rejected at the 0.05 level in a two-tailed test, and if rejected we have indicated the direction of the difference.

observed the participant’s actions and reactions, participants commented on what they were thinking and doing. After direct experience with each interface participants were asked to complete the usability questionnaires about their experiences. The usability questionnaire consisted of a set of twentytwo statements concerning the key attributes for evaluating the usability of interfaces. Each questionnaire item relates to issues that have been documented to potentially enhance or reduce usability (i.e. Shneiderman and Plaisant, 2005). The questionnaire used a Likert format 7-point scale (Likert, 1932) ranging from strongly agree (1) to strongly disagree (7). The questionnaire design followed standard practice (Coolican, 1990; Oppenheim, 1992) using an equal number of positive and negative statements, presented in a randomized order. This technique, known as counterbalancing, is a method for controlling for acquiescence response set, which is the tendency to agree rather than disagree to any question statements (Colman, 2001). For the statistical analysis, polarity was reversed for the set of positive statements; therefore a score of 7 (on the 7-point scale) was consistently indicative of a strong positive attitude. Overall usability was determined for each experiment by determining the mean of all the usability questions by all participants. Individual statements were also analysed separately to identify any specific usability issues that arose from the hands-on usage sessions. The twenty-one usability statements that were consistent between the two experiments were also analysed for the Formal language, Form design interface (used in both experiments), allowing the two cohorts from the different experiments to be compared. After experiencing both interfaces, and completing the usability questionnaires, participants made a quality rating, recorded as a value on a 30-cm linear scale labelled ‘‘Worst’’ at 0 cm and ‘‘Best’’ at 30 cm. This quality rating involved evaluating both interfaces against each other on the linear scale, and was also recoded to indicate an explicit preference for one interface or the other, or no preference. Finally, the experiment included a one-to-one structured interview where participants were given an opportunity to comment on their experiences and give their opinions on a range of related issues.

4. Usability methodology 5. The metaphor experiment The research reported here used repeated measures, within-subjects experiment designs (Robson, 1983; Creswell, 2003) for collecting usability data. The data included quantitative usability measures and qualitative remarks. Usability assessment was carried out using a combination of recorded measurements such as task completion, error rates and recoveries, attitude questionnaires, observations and participant commentary (‘think aloud’). The evaluations were based on hands–on experience with both experimental interfaces (in a balanced order of presentation), with customers performing representative tasks in well-defined scenarios. During the sessions, researchers

5.1. Experiment design The metaphor experiment investigated two alternative interface designs for eBanking services—form fill: the Form metaphor and array-edit: the Spreadsheet metaphor. A cohort of 32 participants (customers of the Bank) took part in the experiment. The cohort was balanced for age, gender and presentation order of the interfaces, the between subject factors. The dependent variables were the responses to individual items in the usability questionnaire, the quality rating and deduced preferences.

ARTICLE IN PRESS C.S. Weir et al. / Int. J. Human-Computer Studies 64 (2006) 770–784

5.2. Tasks Customers were asked to perform four tasks. Two involved making amendments to existing payment arrangements and two involved deleting information, firstly a future payment and then an entire arrangement. These tasks allowed the customers to experience and focus on several instances of the contrasting interface design metaphors. They performed a similar number and scope of tasks on the second interface; details were slightly altered to assist in engaging the customer with their second experience. 5.3. Demographics The customer cohort was balanced in terms of gender: half of the participants were under 40 years old, while the oldest participants recruited were between 50 and 59 years old. The order of experience for the two interfaces was also balanced across the participants of differing ages and genders. Participating customers made frequent use of the Internet (a recruitment criterion), accessing it mainly from home or work, with a high proportion doing both. Of the customers in the first cohort, 44% used the eBanking service weekly or more frequently. A total of 65% of all transactions performed by eBankers were examples of passive viewing; the remainder were transactional interactions where they performed self-service tasks. A total of 46% of the cohort described themselves as predominantly using Internet Banking to conduct their finances. 6. Metaphor experiment results 6.1. Usability results To investigate the impact of the differing interaction metaphors, repeated-measures ANOVA procedures were carried out using the mean responses to the twenty-two Likert usability questionnaire statements completed after each exposure to an eBanking interface. Individual attributes were also compared in the same way. Three between-subject factors were included in the analyses: age group, gender and order of experience. The mean usability scores (on the 7-point response scale) for the Form and Spreadsheet designs are presented in Table 1. The majority of participants responded to the simpler Form metaphor with higher usability scores. The difference in perceived usability was statistically significant (F ð1; 24Þ ¼ 5:57; p ¼ 0:027) in favour of the Form metaphor for the mean of the full set of usability attributes. Six of the 22 individual usability attributes were scored significantly (at the 0.05 level) more favourably for the Form metaphor over the Spreadsheet metaphor: Stressfulness, p ¼ 0:0001; Flustered, p ¼ 0:006; Degree of control,

777

p ¼ 0:029; Knew what to do next, p ¼ 0:037, Knew where I was, p ¼ 0:039 and Clarity of page layout, p ¼ 0:047. Between-subject effects showed a significant interaction between age and gender on the mean of both design variant usability scores, F ð1; 24Þ ¼ 9:672; p ¼ 0:005. T-tests on these data indicated that both male and female younger participants score the interfaces consistently (t ¼ 1:749; df ¼ 14; p ¼ 0:102). Older males score both designs significantly higher than their younger counterparts (t ¼ 2:270; df ¼ 14; p ¼ 0:040). Older women score both designs significantly lower than the younger women (t ¼ 2:384; df ¼ 14; p ¼ 0:032). Older women score both designs much lower than older men, a very highly significant result at t ¼ 3:291; df ¼ 14, p ¼ 0:005, illustrated in Table 2. Similar significant age/gender interactions were found for 12 of the 22 individual attributes. A number of order effects were also found in attitudes towards the two different interface designs for some individual questionnaire attributes. One such significant effect was for Ease of altering details, p ¼ 0:022, the data are presented in Table 3. Participants who experienced the Form metaphor first scored it highly in terms of Ease of altering details, and scored the Spreadsheet metaphor considerably lower, a t-test revealed this to be a highly significant difference, p ¼ 0:007. However, this effect was not apparent for those participants who used the Spreadsheet metaphor first. This order effect was also noted for the attributes: degree of enjoyment, p ¼ 0:009; User-friendliness, p ¼ 0:013; attitude to using the site, p ¼ 0:017; frustration with organization, p ¼ 0:011. Stressfulness showed an age interaction, p ¼ 0:011. The data indicated that the older age group had significantly different attitudes to both metaphor designs, finding the Spreadsheet design very much more stressful to use. No effects could be found due to eBanking experience or for those who used the Internet as their main banking channel. However, participants were not balanced in terms

Table 1 Mean usability scores (22 questions) for the contrasting metaphors Metaphor

Mean usability score

Std. deviation

Na

Form Spreadsheet

5.398 4.906

0.7321 0.9776

32 32

a

Where N indicates the number of participants.

Table 2 Mean scores for both designs, by age and gender

o40 40+

Male mean

Std. deviation

Std. error mean

Female mean

Std. deviation

Std. error mean

4.793 5.630

0.9208 0.3444

0.3069 0.1302

5.442 4.914

0.3657 0.4869

0.1382 0.1623

ARTICLE IN PRESS C.S. Weir et al. / Int. J. Human-Computer Studies 64 (2006) 770–784

778

Table 3 Mean usability scores to attribute ‘‘Ease of altering details’’ by order of experience of each design variant

Form then spreadsheet Spreadsheet then form

First mean

Std. deviation

N

6.19 6.19

1.047 0.544

16 16

Second mean

Std. deviation

N

4.69 6.06

1.957 0.772

16 16

Table 4 Mean quality score per design metaphor Metaphor

Mean quality score

Std. deviation

N

Form Spreadsheet

22.27 16.92

5.011 6.391

32 32

of eBanking or Internet usage across experimental conditions in this study. 6.2. Quality rating and preference results Participants used a linear 30 cm scale to rate the two design metaphors they had used in terms of quality: best to worst. These scores were collected and used to indicate both a quality rating, and a ranked preference for either interface. The same patterns were found in the quality ratings as were seen in the usability data. The distribution of quality rating scores for the two interfaces were significantly different; favouring the Form metaphor, F ð1; 24Þ ¼ 7:957; p ¼ 0:009 (see Table 4). The quality scores were also used to determine a ranked preference for one interface or the other. The preference data indicated that the Form metaphor was strongly preferred to the more complex Spreadsheet metaphor, with 75% of participants ranking the Form design higher than the Spreadsheet design. Only 25% of participants preferring the Spreadsheet design; a binomial test showed a significance level p ¼ 0.007 which is statistically highly significant. 6.3. Relationship between usability and quality It is also of interest to investigate possible correlations between participants’ expressed perceptions toward usability and their quality ratings for each metaphor. There was a strong correlation between perceived usability and quality. A Pearson correlation analysis showed a highly significant positive correlation, po0.001. These data are illustrated in Fig. 6. 6.4. Qualitative analysis Participants were interviewed about their experiences with the different eBanking interfaces, and gave their opinions on a wide range of topics.

Quality Score Difference (Form-Spreadsheet)

Order

30.0 20.0 10.0 0.0 -10.0 -20.0 R Sq Linear = 0.622

-30.0 -3.0

-2.0

-1.0 0.0 1.0 2.0 Usability Score Difference (from-Spreadsheet)

3.0

Fig. 6. Relationship between perceived usability and quality ratings for the alternate metaphors.

Participants liked several aspects of the miscellaneous payment functionality: that it was quick and easy; that it increased the functionality of eBanking, and that the designs had clear instructions when needed. Participants who liked the Form design appreciated the guidance they were given with the step-by-step process. For the Spreadsheet design, which a quarter of customers did prefer, they liked the flexibility of information being both visible and amendable in one step, therefore describing it as more convenient and efficient. However, this perception did not extend to the majority of the cohort who described it as complicated. There was also a great deal of confusion surrounding the different terms for transactions such as ‘Payments’ (external) and ‘Transfers’ (internal), ‘Standing orders’ (defined as a customer’s instruction to their bank to make a regular, fixed amount payment to a specified recipient automatically) were not understood across the board. Participants were not always aware of the differences— particularly tending to misunderstand ‘Standing orders’ and ‘Direct debits’. The majority (81%) of the 32 participants expressed an interest in performing third party payments on eBanking. They felt this service would be extremely useful. 6.5. Task performance issues Most participants completed the tasks in the experiment without problems, with a 95% success rate on tasks for the Form interface, compared to a 90% success rate with the Spreadsheet interface. The pattern follows that seen in the usability, quality and preference analyses: that overall, the Form metaphor provided the more effective interface in terms of task completion. The main errors seen erred on the cautious side: where participants were asked to delete payment arrangement details for ‘old’ accounts that they no longer used, they occasionally chose to delete any

ARTICLE IN PRESS C.S. Weir et al. / Int. J. Human-Computer Studies 64 (2006) 770–784

pending payments, and keep the arrangement details. For the purposes of task completion, such an error was noted, but did not constitute a catastrophic error (where wrong amounts, accounts or failed transactions were performed unrecognized by the user, and would have caused problems in real self-service banking applications). 6.6. Discussion The design based on the Form metaphor was considered generally usable. The design based on the Spreadsheet metaphor, although also considered broadly usable, was significantly less so than the Form design. There were no significant differences in the ways that men and women or customers of different age groups responded to the specific interfaces, although there was a tendency for older men to give the most positive attitude and quality scores, whilst older women gave significantly lower scores. The results from the group of older females showed that their scores were lower in the areas related to clarity and navigation. From the data it appeared that this group of older females were less frequent users of the Internet and this may have caused their lower attitude scores. There were also no main effects due to the order of presentation, although for some individual attributes, the Spreadsheet design scored significantly lower when experienced second. There was evidence that those who experienced the Spreadsheet metaphor first found it reasonably easy for altering their transactions, whilst those who used the systematic (step-by-step) process first found the editable array much harder when they encountered it in the second experience. Further results also highlighted this order effect, with think aloud comments and observations indicating confusion with the greater number of options presented at once when experiencing the Spreadsheet design second in sequence. The main areas of concern for the Spreadsheet design were that participants found it more stressful to use, were more flustered and had less control. They did not feel that they knew what to do next or knew where they were (orientation) in the interface. Another source of the lower mean usability score for the Spreadsheet design was participants’ less positive attitudes towards the clarity of the page layout. Customer quality ratings and derived preferences matched their attitudes towards the usability of the interfaces. There were indications of a strong relationship between usability and quality, as expected from the theoretical underpinnings of usability engineering and its associated value to the interface development cycle. The ‘think aloud’ remarks, comments and interview responses of customers helped to identify good and bad qualities of the two Internet Banking metaphors. Many participants described the Form metaphor as simple, straightforward and easy to use. Some noted that they did not need any instructions to use the Form version as the page layout guided them through their transactions

779

intuitively. Participants often mentioned that they preferred a page dedicated to each individual payment. The Spreadsheet metaphor was often described as complicated, overwhelming and confusing. Many participants complained that they, reluctantly, had to read the instructions and concentrate more to use it. However, participants who preferred the Spreadsheet metaphor liked seeing all the information at once, perceiving the tasks as quicker with less pages loaded, fewer mouse clicks. The Spreadsheets’ editable table did not seem as familiar to customers as banking forms. The Spreadsheet metaphor did not match the knowledge of a majority of participants in the cohort and therefore struggled to compete with the standard transaction form digitalized for the online environment. Participants with increased experience of financial services, Internet Banking or numerical skills might have been more familiar with the Spreadsheet metaphor. The cohort of Internet-savvy Lloyds TSB customers identified as the target audience for an eBanking service did not exhibit any trends towards these particular characteristics. Familiarity with numerical work, finance or experience with online transactions was not controlled in this experiment. Future work in this area might benefit from such comparisons. As populations of Internet users and eBankers increase, such comparisons should be feasible. Other issues that participants in the experiment commented on were concerns about the language used for banking terms on the service. Participants were unclear about the differences between the different transaction types and names. Researchers observed confusion on the part of participants when choosing between Payments & transfers, Standing orders or Direct debits from the menu. These issues were investigated in a second controlled experiment. Using the successful Form metaphor design, a second interface was created with alternate instructions, language, banking terms (jargon) including the combination of all customer instructed (push) transactions into one menu item and page. The design, results and discussion of this experiment follows. The experiment design followed as closely as possible the design of the Metaphor experiment. 7. The dialogue style experiment 7.1. Experiment design The dialogue style experiment investigated two alternative dialogue styles for improving the Form metaphor in eBanking, discussed above. Typical banking terminology was used in instructions and labels in one interface, referred to as the Formal dialogue style. Simple, ‘plain English’ language was substituted in the second experimental interface, referred to as the Plain Language dialogue style. A total of 29 participants took part in the experiment; none had been involved in the metaphor experiment. The cohort was balanced for age, gender and presentation order of the interfaces, the between-subject

ARTICLE IN PRESS 780

C.S. Weir et al. / Int. J. Human-Computer Studies 64 (2006) 770–784

factors. The dependent variables were the responses to individual items in the usability questionnaire and the quality rating and preferences.

Table 5 Mean usability scores (22 questions) for the contrasting dialogue styles

7.2. Tasks

Formal Plain Language

Participants were asked to perform one transfer, one bill payment (from set-up to paying the bill) and similarly to set-up and pay a miscellaneous payment; this focus of the tasks allowed the participant to experience the contrasting interface dialogues. They performed the same three banking tasks with the second interface, but with the specific details changed to ensure that the participant engaged with the second set of tasks.

Mean usability score

Std. deviation

N

5.2790 5.3025

0.7929 0.7137

29 29

Table 6 Mean quality score per dialogue language style Dialogue style

Mean quality score

Std. deviation

N

Formal Plain language

20.39 20.13

5.651 5.102

29 29

7.3. Demographics The participant cohort consisted of 14 females and 15 males. It was composed of just over 60% in the under 40 s age group. Again, most participants made frequent use of the Internet, accessing it mainly from home or work, with a high proportion doing both. Half of the participants used the Bank’s eBanking service weekly or more frequently. Some 95% of all transactions performed consisted passive viewing; the other transactions were examples of transactional interactions and a slightly reduced figure of 35% of this cohort described themselves as predominantly using Internet Banking to conduct their finances. 8. Dialogue style experiment results

was similar for both sequences (Formal then Plain Language and vice versa). This result indicates there may have been a general learning effect due to the similar interaction modes across the two interface designs (menu driven, hyperlinked text, with simple form-fill dialogues). It was interesting that no significant effect was found when comparing the results to the usability question pertaining to the understanding of the words and phrases used on the two interfaces. It appeared from the data that participants had not found a marked effect in this feature, even when their interaction with the designs had been focused on tasks with these altered instructions. Order of presentation also had no effect on this attribute. Both interfaces were seen to be very similar by participants so it is therefore probable that dialogue factors had little influence on usability for these simple tasks.

8.1. Usability results 8.2. Quality rating and preference results To investigate the impact of the differing dialogue styles, repeated-measures ANOVAs were carried out using the mean responses to the Likert usability questionnaire completed after each exposure to one of the eBanking dialogue styles. Individual items on the usability questionnaire were also compared in the same way. Three between-subject factors were included in the analyses as follows: age group, gender and order of experience. The mean usability scores (7-point response scale) for the Formal and Plain English styles are presented in Table 5. There was no significant difference between the overall attitudes to these two design dialogue styles for the 29participant cohort F ð1; 21Þ ¼ 0:008; p ¼ 0:931; N ¼ 29. There were also no significant differences in the ways that men and women, or participants of different age groups, responded to the two dialogue styles. There were also no main effects due to the order of presentation of the interfaces. The ANOVAs calculated on individual usability attributes showed a number of effects. There was a highly significant order effect (p ¼ 0:004) for the attribute: degree of confusion with layout, showing that the second interface experienced caused less confusion than the first. The effect

Participants used the linear 30 cm quality rating scale to rate the two interfaces from best to worst. These scores were collected and used to indicate both a quality rating, and a ranked preference for either interface. The same patterns were found in the quality ratings as were seen in the usability data. The data are shown in Table 6, the distributions of quality rating scores for the two interfaces were not significantly different. The quality scores were also used to determine a ranked preference for one interface or the other. The preference data indicates that the Formal dialogue style was preferred by 59% of the 29 participants, with the other 41% preferring the Plain Language dialogue style. A binomial test showed that the groups were not significantly different. 8.3. Relationship between usability and quality Again, possible correlations between participants’ expressed perceptions toward usability and their quality ratings were investigated for the alternate dialogue styles. There was a strong correlation between perceived usability and quality—a Pearson correlation analysis showed a

ARTICLE IN PRESS C.S. Weir et al. / Int. J. Human-Computer Studies 64 (2006) 770–784

correct usage of transactions and terms. Considering participants’ statements concerning their lack of motivation in reading on-screen instructions, the provision of such a glossary would require further studies: getting the attention and the targeting information to those who require this help, whilst managing the intrusion to others. All of the 29 participants expressed an interest in performing third party payments on eBanking. They agreed with the other cohort, feeling that this service would be extremely useful, allowing eBanking to perform similar functions to automated telephone-banking services and human-operated channels.

30.0 Quality Score Difference (Formal - Plain Language)

781

20.0 10.0 0.0 -10.0 -20.0 R sq Linear = 0.379

-30.0

8.5. Task performance issues -1.50

-1.00 -0.50 0.00 0.50 1.00 Usability Score Difference (Formal Plain Language)

1.50

highly significant positive correlation, po0.001. The correlation was similar to that seen in the metaphor experiment, and the data are illustrated in Fig. 7.

Most participants completed the tasks in the experiment without problems: The success rate on tasks for the Formal dialogue style interface was 92%, and similarly a 92% success rate was calculated for the Plain Language dialogue style interface, indicating no difference between the two in terms of task performance. The pattern follows that seen in the usability, quality and preference analyses. Participants using the alternate dialogue styles perceived no noticeable usability or quality differences.

8.4. Qualitative analysis

8.6. Discussion

Participants were interviewed about their experiences with the different dialogue styles, and gave their opinions on a wide range of topics. The major finding was that participants who used eBanking had not explored the provision to set up any payments online. The idea of allowing customers to send money to friends and other specific bank accounts was highly praised for convenience and speed. One third of the participants mentioned the language changes, commenting mainly on wording changes in the dialogue boxes. Participants often mentioned that when they read instructions on Web pages, they tended to look over them very quickly as they want to get on with using a service, not reading about it. In general, they were very happy with the amount of information they had to read, stating that it was clear, concise and simply written. The characteristics that they identified did not include the tone (formal or plain English language), or the removal of banking terms such as standing order from the site. Varying the amount and design of any written instructions, instead of altering the language tone, might have been a more illuminating experiment. Again, researchers observed some confusion regarding the different terms for banking transactions, particularly the difference between Direct Debits other transactions. Participants were also unclear as to whether ‘Transfers’ could be made to external accounts. However, almost all participants expressed the need for financial terms to be consistent across branch, telephone and Internet channels. Many suggested adding a glossary to educate them in the

Both the Formal and Plain Language dialogue styles were perceived to be generally usable. There were no significant differences between the usability scores obtained by each design. There were no differences between men, women, or customers of different age groups. There were also no main effects due to the order of presentation. It was observed that most participants did not spend much time studying or reading through the on-screen instructions, particularly in their second experience; therefore it may have been that the differences were not obvious enough to attract their attention once the basic interaction mode had been ‘learned’. In addition, the use of different tones of voice may have little effect on customers’ perceptions of quality and preference. Task completion rates were also unchanged for the two dialogue styles. The tasks may have been easy enough to understand in both versions of the service. General familiarity with banking terms and phrases was enough to make them seem invisible when operating a generally highly usable interface to perform new transactions. The responses and remarks reinforced the problem that the customers’ knowledge of banking terms (Standing Order, Direct Debit, etc.) remains a serious limitation to eBanking services. In this experiment, it became clear that customers would like to maintain these terms, as they are common to the industry and other banking channels. Therefore, fixing the English and using generic language for transactions did not work. However, we believe that presenting these results informs researchers and industry about these interface design issues and is an important step

Fig. 7. Relationship between perceived usability and quality ratings for the alternate metaphors.

ARTICLE IN PRESS C.S. Weir et al. / Int. J. Human-Computer Studies 64 (2006) 770–784

782

in illuminating the design of eBanking services into the future. Participant quality ratings and derived preferences matched their attitudes towards the usability of the interfaces. Again, the relationship between quality and usability is observed in these data. 9. Comparison of participant cohorts It was also of interest in this experiment to compare the two experiment cohorts to ascertain whether they were responding similarly to the stimuli. As twenty-one of the usability attitude questions were identical across the two experiments, and both included use of the Form design (Formal dialogue style) interface, there was opportunity to compare a portion of the two cohorts who experienced the Form design in the first instance before comparing it to either the Spreadsheet metaphor or Plain Language dialogue designs. It is important to note however, that the tasks were not identical for the metaphor and dialogue style experiments. The usability attitude questions were taken after direct experience and before exposure to the competing interface, a comparison on the mean of the matched usability questions was carried out. Comparison of quality scores was not performed, as these were taken after experience of both interfaces in each case. The univariate ANOVA was used to compare the usability scores for the first instance of the Form/Formal design for each experiment. The data are shown in Table 7. The data indicated that the two randomly selected cohorts did indeed respond similarly to the same interfaces (despite their experience being mediated through different tasks). There was no significant difference in the two usability scores: F ð1; 23Þ ¼ 1:459; p ¼ 0:239; N ¼ 31. 10. Conclusion This research has confirmed by experiment that the role of metaphor is of major importance in driving the design of usable P3P functionality in eBanking. The usability responses showed a clear significant difference with participants favouring the linear Form metaphor to the compact Spreadsheet functionality. In the usability scores, there was a highly significant difference between the Form and the Spreadsheet designs. The Form design was considered more usable than the Table 7 Mean usability scores for form metaphor, formal dialogue design (both experiments, when seen first in sequence) Dialogue style

Mean usability score (21 matched questions)

Std. deviation

N

Metaphor experiment (1) Dialogue style experiment (2)

5.339 5.102

0.5924 0.8370

16 15

innovative interaction mode offered by the Spreadsheet design. Participants preferred the simple, step-by-step nature of the Form interaction mode to the more complex Spreadsheet interactions. Perceived timesaving in task completion in the Spreadsheet design was picked up by a minority of participants. This factor did not seem to be appreciated by the majority, the attribute relating to speed not being significantly different between the two designs. It is perhaps that, although eBankers are driven by efficiency, the online service provides sufficient rapidity compared to counter service simply by avoiding a trip to the bank. In contrast, a change in the formality of the terminology and dialogues used throughout the service was not found to be significant. Participants barely noticed the differences, and obtained high effectiveness on the tasks for both versions of the service. Although qualitative remarks indicated some issues with banking terms, they were not sufficiently removed by the alterations made in this experiment. There was a consensus that banking terms needed to be consistent across all customer-contact channels. No significant effects were found from changing the dialogue style from using formal banking terminology to using simpler plain English language. In fact, observations from the participant sessions indicated a lack of interest in reading any amount of on-screen text. This finding corresponds with other studies of reading on the Web (Spool et al., 1997; Morkes and Nielsen, 1998), suggesting that users simply scan the page and do not read sentences fully. There was such a high level of effectiveness in task completion that it could be considered that the usability of the interface (with the successful form metaphor) was sufficient to negate the need for examining and reading dialogues carefully. Possible extensions to this experiment would be better to focus on different styles of instruction presentations and amounts. Concentrating on less frequent tasks, such as account opening applications—requiring detailed information to be presented and input, might benefit from further dialogue style research. The trend observed in the analysis of these experiments is that whilst making instructional and information dialogues using plain language (as plain as possible) was not of significant value to increase interface usability or quality perceptions, it was of some value. Qualitative remarks from the first experiment and expert heuristic evaluations exposed issues with the language tone and the use of extensive banking jargon in the interfaces. However, as is often the case in consumer research, the reactions during and after hands-on use of differing dialogues in the second experiment revealed a range of responses to the language but no significant usability, quality, task completion (effectiveness) or preference differences. The participants’ desire to be educated in banking terms, suggesting a glossary in many cases, would require further research, especially given the desire not to read through text on-screen. Similarly, the extension of eBanking into applications for loans or mortgages requires thought on

ARTICLE IN PRESS C.S. Weir et al. / Int. J. Human-Computer Studies 64 (2006) 770–784

how to present important information to customers unlikely to read details carefully online. The eBanking environment must pay close attention to these issues and design to comply with Financial Services Authority (FSA) regulations and procedures. Observations gathered during both experiments indicated that participants exhibited a drive to get on with the interactive part of data input and transaction. The layout of the Spreadsheet design caused more customers to pause in their interaction and read the instructions. Comparing the results from the two experiment cohorts for the Form design (consistent to both comparisons) revealed no significant difference in usability responses, even when experience was mediated through differing tasks. It was therefore concluded that both cohorts were reacting similarly and were likely to represent typical customers of the bank in question. An interesting area of future research would be to design a glossary for eBanking. Design features such as simple, scannable information on banking terminology and its use, with the ability to grab the attention of those who need it would have to be balanced with ensuring that the addition of such information does not intrude on the current high levels of usability experienced by others. eCommerce services typically use simple language in their interfaces, therefore glossary design would be of limited use to most online shops. There are other specialist applications requiring the use of consistent (cross-channel) terminology, and therefore the education of application users. This area of research is probably more relevant to eLearning environments and work-related software than a typical shopping site. In this respect, eBanking is quite different from other Web-based commerce: it is available to members of the public with no training in financial services, yet requires some specialized knowledge in order to avoid errors and confusion. Use of the eBanking channel is increasing all the time, and thus eBankers are becoming a more diverse group, with varying amounts of experience with the provided functionality. Customers who use eBanking frequently to perform multiple transactional interactions might still benefit from extensions to the linear, form-filling metaphor. General financial knowledge as well as use of Spreadsheet software at home or work might also affect the perceived usability of advanced interaction metaphors, such as the Spreadsheet. Indeed, experience of eBanking and preference for a Banking channel might also be variables to be controlled in the design of future eBanking interaction experiments. As eBanking has the primary aim of migrating (Internet using) customers to online self-service, the Form metaphor was recommended as being appropriate for this purpose. Acknowledgements The authors would like to acknowledge the financial support for this research from Lloyds TSB Bank. The work

783

has benefited greatly from helpful discussions with colleagues at the Centre for Communication Interface Research (CCIR), in particular Dr. John Foster and Dr. Fergus McInnes. References Aladwani, A.M., 2001. Online banking: a field study of drivers, development challenges, and expectations. International Journal of Information Management 21 (Issue 3), 213–225. Alty, J.L., Knott, R.P., Anderson, B., Smyth, M., 2000. A framework for engineering metaphor at the user interface. Interacting with Computers 13 (2), 301–322. BBC News, 2005. Reluctance to switch banks fades, Published 28/11/05, r BBC MMV, available online: http://news.bbc.co.uk/1/hi/business/ 4385842.stm, accessed on 19/01/2006. BBC News, 2006. Barclays bans jargon in makeover, Published 03/02/06, r BBC MMVI, available online: http://news.bbc.co.uk/1/hi/business/ 4674548.stm, accessed on 06/02/2006. Centeno, C., 2004. Adoption of Internet services in the Acceding and Candidate Countries, lessons from the Internet banking case. Telematics and Informatics 21 (Issue 4), 293–315. Colman, A.M., 2001. A Dictionary of Psychology. Oxford University Press, Oxford. Coolican, H., 1990. Research Methods and Statistics in Psychology. Hodder & Stoughton, GB. Creswell, J.W., 2003. Research design qualitative and quantitative approaches, second ed. Sage Publications, Thousand Oaks, California. Dandapani, K., 2004. Success and failure in Web-based financial services. Communications of the ACM 47 (5), 31–33. Davies, M., 1996. Image problems with financial services: some considerations for improvement. Management Decision 34 (2), 64–71. Dutton, R.T., Foster, J.C., Jack, M.A., Stentiford, F.W., 1993. Identifying usability attributes of automated telephone services. In: Proceedings of Eurospeech 93, Berlin, pp. 1335–1338. Erickson, 1995. Working with interface metaphors. In: Baecker, R.M. (Ed.), Readings in Human–Computer Interaction: Toward the Year 2000. Morgan Kaufmann, San Francisco, California, pp. 147–151. Faulkner, X., 2000. Usability Engineering. Macmillan, London, UK. Frøkjær, E., Hertzum, M., Hornbæk, K., 2000. Measuring usability: are effectiveness, efficiency and satisfaction really correlated? CHI Letters 2 (1), 345–352. Furnell, S., 2004. E-commerce security: a question of trust. Computer Fraud & Security 2004 (Issue 10), 10–14. Gopalakrisnan, S., Wischnevsky, J.D., Damanpour, F., 2003. A multilevel analysis of factors influencing the adoption of Internet Banking. IEEE Transactions on Engineering Management 50 (4), 413–426. Gould, J.D., 1988. How to design usable systems. In: Helander, M. (Ed.), Handbook of Human–Computer Interaction. North-Holland, Amsterdam, pp. 757–789. Gould, J.D., 1995. How to design usable systems. In: Baeker, R.M., Grudin, J., Buxton, W.A.S., Greenberg, S. (Eds.), Readings in Human–Computer Interaction: Toward the Year 2000, second ed. Morgan Kaufmann, USA, pp. 93–121. Grabner-Kra¨uter, S., Kaluscha, E.A., 2003. Empirical research in on-line trust: a review and critical assessment. International Journal of Human–Computer Studies 58 (Issue 6), 783–812. Hudson, W., 2002. The lost world of E-Banking. SIGCHI Bulletin 34 (5), 7. ISO 9241-11, 1998. Ergonomic requirements for office work with visual display terminals (VDTs)—Part 11: guidance on usability. Jayawardhena, C., Foley, P., 2000. Changes in the banking sector—the case of Internet Banking in the UK. Journal of Internet Research: Networking and Policy 10 (1), 19–30. Karat, J., 1988. Software evaluation methodologies. In: Helander, M. (Ed.), Handbook of Human–Computer Interaction. North-Holland, Amsterdam, pp. 891–903.

ARTICLE IN PRESS 784

C.S. Weir et al. / Int. J. Human-Computer Studies 64 (2006) 770–784

Kim, J., Moon, J.Y., 1998. Designing towards emotional usability in customer interfaces—trustworthiness of cyber-banking system interfaces. Interacting with Computers 10, 1–29. Lai, V.S., Li, H., 2005. Technology acceptance model for Internet Banking: an invariance analysis. Information and Management 42 (Issue 2), 373–386. Landauer, T.K., 1988. Research methods in human–computer interaction. In: Helander, M. (Ed.), Handbook of Human–Computer Interaction. North-Holland, Amsterdam, pp. 905–928. Liao, Z., Cheung, M.T., 2002. Internet based e-banking and consumer attitudes: an empirical study. Information and Management 39, 283–295. Likert, R., 1932. A technique for the measurement of attitudes. Archive fur Psychologie, 140. Morkes, J., Nielsen, J., 1998. Applying writing guidelines to Web Pages. In: CHI 98, Conference on Human Factors in Computing Systems. ACM Press, New York pp. 321–322. Nielsen, J., 1993. Usability Engineering. Morgan Kaufmann, San Francisco. Nielsen, J., 2000. Designing Web Usability. New Riders Publishing, Indianapolis. Nilsson, M., Adams, A., Herd, S., 2005. Building security and trust in online banking. In: CHI ’05, Extended Abstracts on Human Factors in Computing Systems (Portland, OR, USA, April 02–07, 2005). ACM Press, New York, NY, pp. 1701–1704. Oppenheim, A.N., 1992. Questionnaire Design, Interviewing and Attitude Measurement. Cassell, London.

Preece, J., Rogers, Y., Sharp, H., Benyon, D., Holland, S., Carey, T., 1994. Human–Computer Interaction. Addison-Wesley, Wokingham, UK. Robson, C., 1983. Experiment, Design and Statistics in Psychology: An Introduction. Pelican Books, Great Britain. Shneiderman, B., Plaisant, C., 2005. Designing the User Interface: Strategies for Effective Human–Computer Interaction, International edition. Pearson, Addison-Wesley. Sohail, M.S., Shanmugham, B., 2003. E-banking and customer preference in Malaysia: an empirical investigation. Information Sciences 150, 207–217. Spool, J., Scanlon, T., Snyder, C., DeAngelo, T., Schroeder, W., 1997. Web Site Usability: A Designer’s Guide. Morgan Kaufmann, San Francisco. Suh, B., Han, I., 2002. Effect of trust on customer acceptance of Internet banking. Electronic Commerce Research and Applications 1 (Issues 3–4), 247–263. Tan, M., Teo, T.S.H., 2000. Factors influencing the adoption of Internet Banking. Journal of Association for Information Systems 1 (Article 5), 1–42. Wright, A., 2002. The changing competitive landscape of retail banking in the e-commerce age. Thunderbird International Business Review 44 (Issue 1), 71–84. Yousafzai, S.Y., Pallister, J.G., Foxall, G.R., 2003. A proposed model of e-trust for electronic banking. Technovation 23 (Issue 11), 847–860. Zhang, P., von Dran, G.M., 2000. Satisfiers and dissatisfiers: a two-factor model for website design and evaluation. Journal of the American Society for Information Science 51, 1253–1268.