The role of design properties and demographic factors in soft usability problems

The role of design properties and demographic factors in soft usability problems Chajoong Kim and Henri H. C. M. Christiaans, School of Design & Human Engineering, Ulsan National Institute of Science and Technology, 50 UNIST-gil, Ulju-gun, Ulsan, 44919, South Korea User-centred design and co-design are nowadays prevalent in product design. However, the number of product returns in consumer electronic industry is continuously increasing. Most complaints are not technical in nature but have to do with non-technical or ‘soft’ problems. Our study investigates these problems with electronic devices in relation to design properties, characteristics of users and their follow-up (re)actions. The results show that people massively complain about a large variety of products, from computers to e-book readers, and from washing machines to vacuum cleaners. Soft problems are the outcome of the interaction between user characteristics and design properties. Whether users take action upon their complaints also depend on their background. The results have to be translated into a design language. Ó 2016 Elsevier Ltd. All rights reserved.

Keywords: user centred design, usability, product design, soft problems, user behaviour

S

Corresponding author: Henri H.C.M. Christiaans hchristiaans@unist. ac.kr

ince user-centred design was introduced to the electronic industry, they have increasingly adapted the paradigm in their design process to make consumer products successful in the market. Nevertheless, product return of consumer electronic products has been continuously increasing since mid 90s (Den Ouden, 2006; Koca, Karapanos, & Brombacher, 2009). Interestingly, more than half of the reasons for product returns have nothing to do with technical problems, but are related to ‘soft’ problems that cannot be traced back to a specification violation or failure (Kim, 2014; Kim & Christiaans, 2012): this is the same concept as ‘No Fault Found’ or ‘No Trouble Found’ problems in reliability engineering (Brombacher, Sander, Sonnemans, & Rouvroye, 2005; Khan, Phillips, Jennions, & Hockley, 2014). Soft problems have first been recognized explicitly within modern high-end consumer electronics industry and then especially within the mobile industry: in 2006, product returns due to soft problems cost the global mobile industry $4.5 billion (Overton, 2006). More recently consultancy firm Accenture puts the percentage for these soft problems for returned electronic consumer products in the US at 68%. The company estimates the overall cost for product returns for the US market alone to be $13.8 billion (Douthit, Flach, www.elsevier.com/locate/destud 0142-694X Design Studies 45 (2016) 268e290 http://dx.doi.org/10.1016/j.destud.2016.04.006 Ó 2016 Elsevier Ltd. All rights reserved.

268

& Agarwal, 2011). Product return must be a big threat to the electronic consumer product industry because it would end up with consumer disloyalty and a serious revenue loss. However, the unfamiliarity of industry with the nature and meaning of problems that has no technical cause could be a much bigger threat to them since they can hardly cope with such kind of consumer complaints unless they are aware of the reasons of product return. The increasing user dissatisfaction with electronic consumer products, on the other hand, has raised questions about the reasons for this increase on the user side. One might postulate that the complexity of electronic devices has increased, leading to ‘intrinsic’ cognitive load problems (see for a recent publication about cognitive load theory (De Jong, 2010)). Another reason for complaints, as mentioned in literature, is when the product doesn’t meet users’ expectations (Den Ouden, Yuan, Sonnemans, & Brombacher, 2006). For instance, consumers are likely to be attracted by the number of features when buying a product. Once they have actually worked with a product, however, usability starts to matter to them (Rust, Thompson, & Hamilton, 2006). However, the increase over the years in the number of soft problems does not mean that product quality is going down. More and more consumers nowadays are becoming more aware and react accordingly: they do not take any problem with the product for granted anymore. And when they feel dissatisfied with the device they are more willing to take action. Hence, interaction effects between manufacturers’ strategy, products’ characteristics and users’ characteristics might cause the soft problems phenomenon. Leaving the companies out of the equation, we mainly focus in this study on the influence of user characteristics, design properties and their interactions. The research questions are: 1. How do ‘soft’ problems with electronic consumer products interact with user characteristics and design properties? 2. What actions are consumers willing to take after experiencing any dissatisfaction with such a product?

1 Background 1.1 Soft problems In earlier survey studies regarding soft problems a categorisation of these problems was already proposed (Kim & Christiaans, 2012; Kim, Christiaans, & van Eijk, 2007). First, it turned out that all problems expressed by users were related to their perception of the product’s ‘instrumental quality’, i.e. the extent to which the device contributes to the performance of users or to the promotion of their goals (Hassenzahl, 2004). Three types of soft problems could be derived from the data following the quality dimensions

The interaction between usability problems, users and design properties

269

as proposed by Madureira (1991) and Dantas (2011): sensory, functional, and operational quality problems. In Figure 1, examples of the three types are shown based on our previous surveys. Functional quality problems refer to how poor the functional aim of a product is achieved. This was expressed by problems with operation and performance related to functions, and/or by complaints about the absence of functions. Sensory quality problems refer to negative experience related to sensory perception, such as the object’s structure, size, weight, noise it produces, and touch. Operational quality problems are related to user’s cognitive load required while interacting with the product. The user evaluates these quality problems mainly on the basis of ease of use and richness of feed forwards and feedbacks offered by the product itself. In the study described here we are not so much interested in the three instrumental quality dimensions of soft problems, but more concrete in the relation between user characteristics and design properties. While the user aspects will focus on demographic factors, the product aspects are all related to the interaction between person and product.

1.2

User characteristics

Earlier studies already did an extensive investigation into the significance of user characteristics related to the experience of soft problems and design properties turned out that demographic variables were more influential than personality characteristics (Kim, 2012; Kim & Christiaans, 2012). Age is a critical factor to anticipate soft problems with consumer electronic products. Ease of use is more important for the old generation. Education level also plays a significant role, sometimes in an unexpected way. High educated people complaint about problems with the operation of electronic products. They are probably so sensitive to any illogical or strange way of operations asked for, that they show their dissatisfaction with it. Culture also showed to have influence on the occurrence of particular types of soft problems. Culture was only defined in terms of differences between different continents. In the studies mentioned three countries were compared, USA, South Korea and the Netherlands. The three countries differed not only in the kind of products they complained about but also in the aforementioned qualities. Although in the previous studies a long list of personality traits was investigated for their relation with people’s complaining behaviour and type of products complained about, there was hardly any significant correlation. For that reason in the current study only demographic factors will be taken into account.

270

Design Studies Vol 45 No. PB July 2016

Figure 1 Three instrumental quality dimensions of soft problems as a result of two survey studies (Kim & Christiaans, 2012; Kim et al., 2007)

1.3

Design properties

In the study described here six ‘design properties’ were used, as they were related in earlier studies with usability issues (see Figure 2). These properties can be defined as follows:
Transparency of operation: transparency can have two meanings: (1) to be easily detected or seen-through (visibility), and (2) to be readily understood. In our study the second meaning has been used and can be detailed in the following aspects (Patrick & Mu, 2009): B Understandability: the application structure, navigation, procedures, features and terminology should be comprehensible for users. B Learnability: the usage of the application or hardware device should exhibit a gradual learning curve and encourage exploration. B Self-descriptiveness: this is also called ‘affordance’ or ‘obviousness’ and is the system’s ability to speak for itself. When the application is presented to the user, it should be intuitively obvious how the system operates and what kind of tasks can be achieved. B Feedback: this refers to whether there is an accessible, clear, and timely indication and response to user’s actions. B Metaphors: metaphors support the transfer of real world knowledge into applications.
Physical interaction density: the extent to which an electronic product has a physical interaction with the person when using the product. For example, the use of a vacuum cleaner or shaver is closer in terms of physical interaction than the use of a washing machine or toaster. We assume that people

The interaction between usability problems, users and design properties

271

Figure 2 Six design properties of a consumer electronic product













will have a stronger relationship or tie with an object, when it is closer to their body. From literature about product attachment e the strength of the emotional bond a consumer experiences to a specific product e it is known that people who are strongly attached to a product are likely to handle the product differently than when they have no relation with their product (Mugge, Schoormans, & Schifferstein, 2008). Perceived performance: the extent to which a product performs better or worse than expected. Expectations play an important role in consumer satisfaction/dissatisfaction. Consumers compare product performance with their expectations in the post-usage process and when the expectations do not match with the actual product performance consumers feel some degree of tension. This tension leads to either satisfaction or dissatisfaction, making adjustments either in expectations or in the perception of the product’s actual performance (Kim, 2012). Frequency of use: the number of times a product has been used during the life of the product. The more and/or more regularly a product is used the more likely the user will complain when that product fails in whatever way. Importance of usability: literally it represents how critical the usability of a product is considering the purchasing reasons of the product. The underlying assumption is that people who are more aware of the comfort and easyto-use characteristics of a product will be more critical about those characteristics of consumer products in general. This attitude might stem from previous bad experiences with similar products. Product importance: the degree to which an electronic consumer product is important for the user. How important a product is, is rather personal depending on either the functionality of this product for the user or on its emotional value.

1.4

Consumers’ action proneness

Although consumer dissatisfaction with products is undoubtedly based on the factors mentioned before, sellers and manufacturers of these products are often not aware of it. Reason is that non-failure-found or soft problems are

272

Design Studies Vol 45 No. PB July 2016

not taken as serious as technical failure problems or that consumers are not taking action towards the seller or manufacturer. There are quite a number of studies about consumer dissatisfaction and brand loyalty, but they are not focused on the typical problems we are discussing here. Therefore, it is very relevant to know whether consumers are nowadays more willing to take action after being confronted with a product that does not meet their expectations. For example, a number of studies show that customers in different cultures have different complaint behaviours and intentions (Hernandez, Strahle, Garcia, & Sorensen, 1991; Liu & McClure, 2001; Richins & Verhage, 1985). Their focus was on complaining behaviours in case of product and service failure in general. The question is whether this result also holds for the typical soft problems in electronic devices.

2

Method

A survey study was performed making use of a questionnaire in order to answer the research questions (1) How do soft problems about electronic consumer products interact with user characteristics and design properties? And (2) What actions are consumers willing to take after experiencing any dissatisfaction with such a product?

2.1

Participants

Participants were people from three countries, the Netherlands, South Korea and the United States. In the Netherlands the first author recruited people via several sources such as handing and mailing flyers and visit elderly communities. In South Korea and the US participants were recruited via contact persons at some local universities. These contacts were instructed to ask collaboration from local people with various demographic backgrounds, with the restriction that age should range between 20 and 80, and the number of male and female participants should be more or less equal. The total number of participants was 629, 62 of whom answered that they had no complaints in using consumer electronics. Those participants were not taken into consideration. This process generated a data set on individual user characteristics, their soft problems and the products based on a sample of 567 participants: 181 North-American (80 males and 101 females), 210 South Korean (115 males and 95 females), and 176 Dutch people (89 males and 87 females). Their age ranges from 20 to 80. Their demographic characteristics are shown in Table 1.

2.2

Equipment

A questionnaire was used to identify soft problems that participants had experienced, and for their user characteristics (i.e. the demographic (culture level), cognitive (universal level), and personality factors (individual level)). In the web questionnaire per screen only one question was presented. Only after

The interaction between usability problems, users and design properties

273

Table 1 Demographic characteristics of participants (N [ 567)

Variable

America (N ¼ 181) n

Gender Male 80 Female 101 Age at time of survey (years) 20e29 51 30e39 30 40e49 12 50e59 14 60þ 74 Educational level Below middle school e graduate Middle school graduate 16 High school graduate 87 Undergraduate degree 66 Postgraduate degree 12

South Korea (N ¼ 210)

Netherlands (N ¼ 185)

Total

%

n

%

n

%

44.2 55.8

115 95

54.8 45.2

92 93

50.6 49.4

287 50.1 289 49.9

28.2 16.6 6.6 7.7 40.9

78 40 20 33 39

37.1 19.0 9.5 15.7 10.4

50 50 26 23 36

27.0 27.0 14.1 12.4 19.5

179 120 58 70 149

31.1 20.8 10.1 12.2 25.8

27

12.9

14

7.6

41

7.1

59 61 37 26

28.1 29.0 17.6 12.4

29 21 73 48

15.7 11.9 39.5 25.9

104 169 176 86

18.1 29.8 30.6 14.9

e 8.8 48.1 36.5 6.6

n

%

giving an answer the system went to the next question. The answers given by participants were automatically saved to a database on the Internet. Participants who were not familiar with a computer, such as some elderly people, participated through a paper-based questionnaire having the same questions and format as the web-based questionnaire. This was done with the help of student assistants in each country: they visited places where (senior) citizens gather and handed out the questionnaires with instructions. The questionnaire started with two open-ended questions to discover soft problems and the causes of these problems. The first question was: ‘With what electronic household product did you feel most dissatisfied with, other than technical problems.’ In the second question, participants were asked to explain for the product mentioned in question 1, what specific dissatisfaction or complaints they had. The next questions were about: e User characteristics: cultural background (USA, South-Korea, the Netherlands), gender, age, and education (from ‘lower than high school’ to ‘PhD degree’) e (Re)actions of participants to others (the store, the company, friends). We made use of an existing Likert scale type questionnaire with 15 items (Maute & Forrester, 1993). Their questionnaire was slightly modified since they aimed at measuring consumer complaining behaviour for airline service. A six-points graphic rating scale (‘Not at All Likely’ e ‘Highly Likely’) was developed (see Table 2 for these questions).

274

Design Studies Vol 45 No. PB July 2016

Table 2 Questions to measure consumer complaining behaviour 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

I would be upset, but not respond in any way. I would go back to the shop and speak to the manager on duty. I would bring back the product and would exchange it for one from another brand. I would make negative comments about the brand to other people nearby. I would wait and hope that things improved. I would call the helpdesk of the company (or brand) and argue with the employee. I would decide to buy products from another brand next time. I would complain to a consumer organization, government agency or newspaper. I would write or send an email to the company to complain I would demand an immediate refund. I would remain loyal to the brand. I would never by any product of that brand anymore. I would remain calm and wait for the shop or company to sort things out. I would tell my friends not to buy that brand product. I would communicate the reasons for my dissatisfaction to the helpdesk employee.

The aforementioned six design properties, used in this study, were measured as follows. e Operational transparency and interaction density: products were categorised according to three levels: high, medium or low. The categorization was done by one of the authors of this article and independently by a researcher from the same Faculty of IDE TU Delft. Interrater agreement (Cronbach’s Alfa) was .90 for operational transparency and .86 for interaction density. In case of disagreement the two ‘judges’ decided together in what category the product should be placed. e Perceived product performance: a 4-points scale Likert scale, ranging from ‘worse than expected’ to ‘better than expected’, was used in order to avoid the neutral middle point of uneven scales. e Product importance and importance of usability: both properties were measured on a 4-points Likert scale, ranging from ‘very unimportant’ to ‘very important’. e Frequency of use: the (4-points) scale values were: ‘never, 1e2 times per month, 1e2 times per week, every day’.

2.3

Procedure

The questionnaire started with the introduction of the survey and with an instruction. Participants were asked to fill out the questionnaire step by step. Participants who were not familiar with a computer, such as some elderly people, participated through a paper-based questionnaire having the same questions and format as the web-based questionnaire. This was done with the help of student assistants in each country: they visited to locations where (senior) citizens meet for social activities, and handed out the questionnaires with instructions. The questionnaires were collected after they were completely filled out. In the Internet version the questions were presented on the screen

The interaction between usability problems, users and design properties

275

one at a time. Only after giving an answer the system went to the next question. The answers given by participants were automatically saved to a database on the Internet.

2.4

Data analysis

Data were analysed by several statistical methods using SPSS 20. First, frequency distributions were generated as well as means and standard deviations. Second, in those cases where we looked for the effect of any geographic variable (e.g. culture) on a dependent variable (e.g. frequency of use), excluding the effect of other geographic variables (e.g. age or gender) we performed an ANCOVA. This method was used because of the fact that some of these variables are continuous representing an interval scale and others are dichotomous and/or nominal.

3 Results 3.1 Soft problems and product type A total of 662 complaints are reported, all of which related to soft problems. A number of participants report more than one complaint. Because of our aim to correlate the soft problem mentioned with product and user characteristics, only the first complaint mentioned by a participant was put in our analysis. In their complaints a total of 76 different household electronic products are mentioned varying from mobile phone, computer, and DVD player to washing machine, iron, shaver and toaster. For a summary of these product types mentioned see Figure 3. Most complaints are about complex electronic products such as mobile phones and MP3 players while relatively simple ones such as air-conditioner and dishwasher are hardly mentioned. Among the category ‘Other’ in Figure 3 are products such as refrigerator, digital camera, hair dryer, rice cooker, toaster, earphones, electric blanket, monitor, mouse, coffee machine, tablet PC, food processor and thermostat. Each of them was not frequently mentioned. Similar to our previous studies the complaints were classified into three categories: sensory, functional, and operational quality. Complaints related to functional quality were the most reported, followed by complaints related to operational quality (Figure 4).

3.2

Design properties and user characteristics

In the following section the correlation between user characteristics e defined in terms of culture, age, gender and education, and design properties complaint about will be reported. Culture was operationalized as the participants’ national background: United States, South-Korea and the Netherlands.

276

Design Studies Vol 45 No. PB July 2016

Other 29% Mobile phone 25%

Alarm clock, 2% GPS, 2%

Computer, 16%

Printer, 2% Washing machine, 3%

Television, 3% DVD player, 4% Vacuum, 4%

Remote, 5% MP3 player, 5%

Figure 3 Percentage of complaints per consumer electronic product

Sensory quality 20%

Operational quality 31%

Consumer Electronic Products Functional quality 49% Figure 4 Percentages of soft problems in consumer electronic products

As mentioned before the results are based on ANCOVA. With this analysis it was possible to look at the effect of each user characteristic while controlling for the other user characteristics.

3.2.1

Design properties and culture

For all three countries mobile phones are most frequently reported, with computers in the second place. Among the US participants both products take more than half of the total product complaints (53%). For the South Korean and Dutch participants this percentage is much lower (both 34%), but still significant. In spite of the similarities, each country seems to have its own character in terms of products they complain about (see also Figure 5). These countrytypical products are:

The interaction between usability problems, users and design properties

277

35 US

South-Korea

Holland

30 25 20 15 10 5 0

Figure 5 Frequency of consumer electronic products complaint about (in %), per country

e US: CD player, digital picture frame, digital watch, e-book reader, external HDD, food processor, game console, car computer, hair dryer, scanner, toaster, wireless router, voice recorder, mouse, telephone. e South Korea: air-conditioner, electronic manometer, camcorder, CD player, curling iron, digital watch, dishwasher, electronic blanket, electronic dictionary, external HDD, fan, fermentation device, electric massager, humidifier, monitor, shaver, voice recorder, body steamer. e Netherlands: bread maker, bathroom scale, food processor, camcorder, bike light, electric knife, HDD recorder, thermostat, humidifier, iron, hair dryer, lamp, mouse, electric juicer, pedometer, printer server, wireless router, toaster, telephone. Regarding the relation between design properties and culture, in Table 3 the results of ANCOVA are presented. Among the six design properties perceived performance is of lower importance than the other properties. The mean scores and the F-values (at least at <.01 level) in Table 2 indicate that with three design properties cultural differences play a significant role:
Operational transparency: US participants complain more than SouthKorean and Dutch participants about products that are operationally

278

Design Studies Vol 45 No. PB July 2016

Table 3 Culture (countries) in relation to design properties: means, (standard deviations) and ANCOVA F-values

Culture

Operational transparency Interaction density Product importance Perceived performance Frequency of use Importance of usability

USA

S-Korean

Dutch

N ¼ 181

N ¼ 210

N ¼ 185

x (sd)

x (sd)

x (sd)

2.65 2.64 3.16 2.35 3.46 3.51

(.60) (.57) (.98) (.84) (.88) (.80)

2.13 2.60 3.08 2.12 3.51 3.22

(.89) (.56) (.97) (1.02) (.83) (.95)

2.20 2.52 3.06 2.17 3.23 3.40

(.81) (.67) (1.01) (.89) (.92) (.87)

F

Sig.

27.37 5.56 .55 3.04 7.59 4.45

.000*** .004** .577 .049* .001** .012*

*p < .05 **P < .01 ***p < .001. Operational transparency and interaction density are measured on a 3-points scale, the other four properties on a 4-points scale.

non-transparent and require more cognitive effort in understanding their operation. Looking at the type of products, they complain more about products related to their car such as GPS and car computer system than about products saving household labour such as vacuum cleaner and washing machine.
Interaction density: US participants complain more about products with which they have a closer physical interaction than do Dutch participants. South-Korean participants are in between.
Frequency of use: participants from all three countries show that this factor plays an important role in complaining. South-Korean participants e and to a lesser extent US participants e have a significantly higher score than Dutch participants. They complain more often about household products such as the vacuum cleaner and the washing machine as well as kitchen appliances such as the refrigerator and the rice cooker, all products that are frequently used. They hardly complain or do not complain at all about the remote control and the DVD player, which is one of the frequently complained products among US and Dutch participants. Dutch participants complain more often about kitchen appliances but about different products than the other countries, such as the microwave and the coffee machine.

3.2.2

Design properties and age

The results show a significant relationship between participants’ age and the properties ‘perceived performance’ and ‘importance of usability’ (see Table 4). People between 20 and 60 years old more often state that the product they complain about performed worse than expected. People of 70 years and older score lower on ‘importance of usability’ than people below 70, although they also think it is important.

The interaction between usability problems, users and design properties

279

Table 4 Age in relation to design properties: means, (standard deviations) and ANCOVA F-values

Age

Operational transparency Interaction density Product importance Perceived performance Frequency of use Importance of usability

20e29

30e39

40e49

50e59

60e69

70e79

>79

N ¼ 179

N ¼ 120

N ¼ 58

N ¼ 70

N ¼ 93

N ¼ 41

N ¼ 15

x (sd)

x (sd)

x (sd)

x (sd)

x (sd)

x (sd)

x (sd)

2.43 (.79) 2.44 (.75) 2.28 (.77) 2.06 (.88) 2.28 (.80) 2.10 (.89) 2.20 (.94)

F

Sig.

.65 .693

2.60 (.64) 2.58 (.66) 2.52 (.57) 2.63 (.49) 2.62 (.57) 2.49 (.60) 2.53 (.52) 1.26 .277 3.08 (1.01) 3.00 (.97) 2.93 (.99) 3.23 (.87) 3.30 (.96) 2.98 (1.11) 3.13 (.92) .86 .527 2.15 (.90) 2.03 (.86) 1.95 (.76) 2.21 (.93) 2.43 (.97) 2.73 (.97) 2.53 (1.19) 2.88 .009** 3.45 (.88) 3.45 (.84) 3.33 (1.03) 3.46 (.86) 3.34 (1.02) 3.24 (.97) 3.33 (1.05) 1.00 .426 3.31 (.86) 3.42 (.84) 3.33 (.78) 3.54 (.75) 3.53 (.90) 3.05 (1.18) 2.87 (1.19) 5.04 .000***

*p < .05 **P < .01 ***p < .001.

Regarding the type of products, in the group of young participants (between 20 and 30 years old) products such as, MP3 player, printer, earphones, hair dryer are shown, which do not appear in the group with the highest age (79þ). On the contrary, refrigerator, electric blanket, rice cooker, and voice recorder are only seen in the group with the age of 40 years old and higher. Only 60þ participants complain about an e-book reader.

3.2.3

Design properties and gender

As Table 5 shows female and male participants differ significantly on two of the six properties Operational transparency and Interaction density. Men complain more about products that are less transparent. Women complain more about products with closer interaction density. Mobile phones are commonly the most complained products in both groups, which are followed by computers. More women then men complain about vacuum cleaners (6% resp. 2%). Cable TV box and alarm clock are hardly found Table 5 Gender in relation to design properties: means, (standard deviations) and ANCOVA F-values

Gender

Operational transparency Interaction density Product importance Perceived performance Frequency of use Importance of usability

Male

Female

N ¼ 287

N ¼ 289

x (sd)

x (sd)

2.37 2.52 3.03 2.19 3.44 3.37

(.76) (.65) (.99) (.90) (.88) (.85)

2.27 2.65 3.16 2.23 3.37 3.37

(.86) (.55) (.98) (.97) (.96) (.92)

F

Sig.

6.70 19.11 .14 1.45 .70 .98

.010** .000*** .707 .230 .404 .323

*p < .05 **P < .01 ***p < .001.

280

Design Studies Vol 45 No. PB July 2016

among female complaints. Men hardly complain about microwaves and refrigerators.

3.2.4

Design properties and education

Education background plays a significant role with four design properties. The results in Table 6 show that the group of lower educated people deviate on three properties: on operational transparency they have a lower score meaning that they complain more about products with a higher operational transparency. On perceived performance they score higher than the higher educated meaning that they became more positive about the product after buying and using it. And finally they seem to attach less importance to usability than the higher educated, although their score is relatively high. There is a difference in type of products that make participants dissatisfied depending on the level of educational background. Major consumer electronic products such as mobile phone, refrigerator, computer, microwave, washing machine, television, MP3 player, coffee machine, and remote control are commonly complained about regardless of educational level. However, participants with lower educational level additionally complain about simple products such as electric blanket, hair dryer, earphones, and fan, while those who have the highest educational level (i.e. more than university level) additionally complain about more diverse products such as bread maker, camcorder, digital camera, GPS, e-book reader, food processor, printer, scanner, and wireless router, which requires relatively high cognitive load to use. Interestingly, they also have complaints with simple-to-use products such as electric juicer, shaver, toaster, blood pressure monitor and mouse.

3.3

Follow-up (re)action and user characteristics

When consumers experience dissatisfaction and want to complain, there are several ways of how to react: boycott the brand or the store, negative word-

Table 6 Education background in relation to design properties: means, (standard deviations) and ANCOVA F-values

Education

Operational transparency Interaction density Product importance Perceived performance Frequency of use Importance of usability


High school

BSc/BA

MSc/MA

PhD

N ¼ 41

N ¼ 104

N ¼ 169

N ¼ 176

N ¼ 86

x (sd)

x (sd)

x (sd)

x (sd)

x (sd)

1.56 2.59 3.27 2.63 3.51 3.10

(.84) (.55) (.98) (1.13) (.81) (1.16)

2.22 2.61 3.19 2.14 3.43 3.38

(.87) (.61) (.95) (.95) (.93) (.86)

2.40 2.68 3.13 2.16 3.47 3.44

(.80) (.50) (1.00) (.84) (.82) (.84)

2.44 2.49 3.01 2.23 3.32 3.35

(.71) (.66) (1.01) (.84) (1.02) (.90)

2.38 2.56 3.01 2.15 3.35 3.38

(.77) (.60) (.98) (.93) (.93) (.89)

F

Sig.

3.60 6.95 .06 3.57 2.81 4.22

.007** .000*** .994 .007** .025* .002**

*p < .05 **P < .01 ***p < .001.

The interaction between usability problems, users and design properties

281

of-mouth (WOM), seek direct redress or complain to a consumer organisation. But often consumers don’t complain to any party at all. 15 questions were used which are related to these ways of follow-up (re)actions. In order to discover simple patterns in the relationships among the questions, factor analysis was conducted. The results indicate that the questions can be grouped into four types of follow-up (re)actions resulting from soft problems: brand disloyalty, direct redress, helpdesk contact, and passive reaction. In the following tables the questions are clustered accordingly. In the survey participants were asked to tell what they had done after experiencing the problem. Overall, experiencing soft problems does not necessarily lead to strong follow-up reactions. The results indicate that the majority of the people who experience any kind of soft problems will not be disloyal to the brand (81% of the participants, although quite a number of people spread negative comments about the brand (73% of the participants). A decrease in brand loyalty resulting from soft problems would end up with negative purchase intentions (84% of the participants). One of interesting findings is that 30% of the participants are not intended to return the product although they are annoyed by soft problems. This can be explained by the aforementioned assimilation theory regarding user’s expectation (Peyton, Pitts, & Kamer, 2003). According to the theory, people seek to minimize the discrepancy between expectation and actual experience. Therefore, they distort expectations or minimize the relative importance of the disconfirmation. Another finding is that 43% of the participants are willing to actively communicate their dissatisfaction to the helpdesk but not to any consumer organization. To investigate how user characteristics influence particular follow-up actions ANCOVA was performed for all 4 user variables: culture, age, gender and education, meaning that if one user characteristic has been analysed as the independent variable the other three user characteristics are controlled for as covariates.

3.3.1

Cultural influence on follow-up (re)actions

Table 7 shows significant differences between countries in the way people will react after a bad experience with a product. People from USA are more likely to be loyal to the brand and South-Koreans are more eager to spread the (negative) word among friends and relatives; and it is more likely that they complain to a consumer organization. The Dutch will rather ask for refund. But they more likely contact the helpdesk or go to the shopkeeper. This last action also holds for the two other countries.

282

Design Studies Vol 45 No. PB July 2016

Table 7 Culture in relation to follow-up (re)actions: means, (standard deviations) and ANCOVA F-values

Culture

Brand disloyalty Buy products from another brand next time Never buy any products of the brand again Loyal to the brand Negative comments about the brand Tell my friends not to buy that brand products Direct redress Exchange it for one from another brand Speak to the shop manager Demand a refund Helpdesk contact Call the helpdesk to argue with the employee Complain to the company through mail Communicate the dissatisfaction to the helpdesk Complain to consumer organisation Active (re)action Upset but not respond Wait and hope that things improved Remain calm until things are sorted out

F

Sig.

(1.41) (1.50) (1.45) (1.30) (1.55)

21.84 14.17 5.00 8.59 2.31

.000*** .000*** .007** .000*** .100

USA

S-Korean

Dutch

N ¼ 181

N ¼ 210

N ¼ 185

x (sd)

x (sd)

x (sd)

4.65 3.17 2.27 3.64 4.12

(1.23) (1.40) (1.06) (1.65) (1.41)

4.41 3.40 2.20 4.74 4.42

(1.79) (1.79) (1.44) (1.56) (1.62)

4.44 3.15 2.76 4.69 3.62

4.04 (1.39) 3.49 (1.58) 3.37 (1.51)

3.90 (1.68) 4.28 (1.67) 3.49 (1.82)

3.87 (1.45) 4.21 (1.51) 3.18 (1.59)

7.79 2.52 10.98

.000*** .081 .000***

2.56 3.33 4.35 2.12

3.72 2.72 4.28 2.37

3.40 2.55 4.37 2.72

(1.64) (1.57) (1.43) (1.81)

10.36 2.99 5.42 7.35

.000*** .051 .005** .001**

2.85 (1.46) 2.55 (1.40) 3.76 (1.40)

4.24 8.80 3.39

.015* .000*** .034*

(1.44) (1.76) (1.43) (1.41)

2.64 (1.49) 2.51 (1.40) 3.41 (1.25)

(1.73) (1.65) (1.64) (1.47)

2.91 (1.66) 3.55 (1.66) 3.22 (1.58)

*p < .05 **P < .01 ***p < .001.

People from all three countries state that they stay calm until things are sorted out. American participants are:





Less likely to be loyal to the brand Unlikely to call the helpdesk to argue dissatisfaction with the employee Unlikely to complain to governmental agencies or consumer organization Unlikely to wait and hope that things improved

South Korean participants are:
Less likely to be loyal to the brand
Significantly likely to spread negative comments about the brand, especially to their friends telling not to buy the brand products
Very likely to call the helpdesk to argue problems with the employee
Very likely to speak to the shop manager
Less likely to stay calm without taking any actions Dutch participants are:
Significantly likely to spread negative comments about the brand but not to their friends.
Very likely to call the helpdesk to argue problems with the employee
Unlikely to complain to the company through mail

The interaction between usability problems, users and design properties

283

284 Table 8 Age in relation to follow-up (re)actions: means, (standard deviations) and ANCOVA F-values

Age

Design Studies Vol 45 No. PB July 2016

Upset but not responding Speak to the shop manager Exchange it for one from another brand Negative comments about the brand Wait: hoping things improved Call the helpdesk to argue with the employee Buy products from another brand next time Complain to consumer organization Complain to the company through mail Demand a refund Loyal to the brand Never buy any products from the brand again Remain calm until things are sorted out Tell my friends not to buy that brand products Communicate the dissatisfaction to the helpdesk *p < .05 **P < .01 ***p < .001.

20e29

30e39

40e49

50e59

60e69

70e79

>79

N ¼ 179

N ¼ 120

N ¼ 58

N ¼ 70

N ¼ 93

N ¼ 41

N ¼ 15

x (sd)

x (sd)

x (sd)

x (sd)

x (sd)

x (sd)

x (sd)

2.94 3.64 3.89 4.69 3.14 3.08 4.58 2.25 2.83 3.27 2.26 3.22 3.13 4.42 4.21

(1.46) (1.65) (1.44) (1.43) (1.54) (1.65) (1.41) (1.51) (1.71) (1.64) (1.16) (1.53) (1.33) (1.38) (1.45)

2.86 3.64 3.88 4.64 2.90 3.16 4.91 2.08 3.03 3.58 2.08 3.53 3.58 4.37 4.21

(1.48) (1.69) (1.48) (1.45) (1.47) (1.62) (1.24) (1.33) (1.68) (1.64) (1.04) (1.51) (1.38) (1.45) (1.48)

2.78 4.09 3.95 4.48 2.79 3.50 4.33 2.28 2.62 3.22 2.22 3.53 3.26 4.05 4.26

(1.40) (1.53) (1.53) (1.29) (1.44) (1.65) (1.47) (1.35) (1.48) (1.67) (1.21) (1.49) (1.36) (1.42) (1.46)

2.73 4.49 3.99 4.20 3.10 3.43 4.49 2.46 2.96 3.57 2.46 3.20 3.33 3.59 4.44

(1.44) (1.27) (1.44) (1.66) (1.73) (1.57) (1.59) (1.39) (1.57) (1.60) (1.48) (1.66) (1.41) (1.68) (1.49)

2.52 4.51 4.08 3.88 2.33 3.31 4.15 2.69 2.96 3.05 2.69 2.99 3.86 3.66 4.79

(1.72) (1.50) (1.64) (1.74) (1.48) (1.78) (1.71) (1.77) (1.84) (1.51) (1.59) (1.59) (1.53) (1.6$) (1.47)

2.71 4.32 3.90 3.93 3.05 3.42 4.07 3.02 2.83 3.15 3.24 2.85 3.76 3.81 4.49

(1.75) (1.60) (1.74) (1.88) (1.84) (1.88) (1.72) (2.08) (1.75) (1.89) (1.74) (1.74) (1.48) (1.76) (1.58)

3.33 4.73 3.73 3.27 2.67 3.40 4.13 3.53 1.87 4.40 3.00 2.93 4.40 3.07 3.40

(2.23) (1.94) (1.83) (2.01) (1.72) (2.13) (2.00) (2.26) (1.68) (1.92) (1.73) (1.94) (1.88) (2.02) (1.99)

F

Sig.

3.45 1.95 1.93 4.13 1.36 .89 5.73 5.98 1.78 1.94 5.27 .97 1.12 4.43 1.20

.002** .071 .075 .000*** .227 .500 .000*** .000*** .100 .072 .000*** .444 .350 .000*** .303


Very likely to speak to the shop manager
Unlikely to wait and hope that things improved

3.3.2

Age influence on follow-up (re)actions

People of 60þ are less likely to spread negative comments about the brand, contrary to younger people. Although all age groups express that they will go to the shop manager, this is significantly less for the same 60þ group. The 70þ group will more likely ask for refund and communicate their dissatisfaction to the helpdesk. See Table 8.

3.3.3

Gender influence on follow-up (re)actions

Men and women only differ significantly on two items, as Table 9 shows. Even though both men and women are likely to spread negative comments about the brand, men score higher on this action. And second, men seem to be less action-prone: they more likely wait, hoping that things will be improved.

3.3.4

Education background and follow-up (re)actions

It is quite striking that the significant relations between educational background and follow-up (re)actions are mainly found in the deviating scores of the group with the lowest education (see Table 10. The group is less loyal to the brand than higher educated people and is more likely to spread negative messages to friends. Going to the shop manager with their complaint is less likely for the lower educated, but they like to get a refund. This group have

Table 9 Gender in relation to follow-up (re)actions: means, (standard deviations) and ANCOVA F-values

Gender

Upset but not responding Speak to the shop manager Exchange it for one from another brand Negative comments about the brand Wait: hoping things improved Call the helpdesk to argue with the employee Buy products from another brand next time Complain to consumer organization Complain to the company through mail Demand a refund Loyal to the brand Never buy any products from the brand again Remain calm until things are sorted out Tell my friends not to buy that brand products Communicate the dissatisfaction to the helpdesk

Male

Female

N ¼ 287

N ¼ 289

x (sd)

x (sd)

2.75 4.05 3.99 4.57 3.05 3.37 4.51 2.43 2.95 3.41 2.30 3.27 3.28 4.30 4.24

(1.44) (1.52) (1.44) (1.42) (1.49) (1.64) (1.48) (1.47) (1.62) (1.62) (1.24) (1.55) (1.38) (1.41) (1.45)

2.87 3.97 3.88 4.19 2.75 3.25 4.47 2.37 2.77 3.29 2.51 3.22 3.64 3.84 4.42

(1.64) (1.73) (1.59) (1.72) (1.64) (1.68) (1.55) (1.70) (1.76) (1.69) (1.46) (1.63) (1.48) (1.68) (1.56)

F

Sig.

4.94 6.53 3.28 21.27 5.80 .74 .37 4.15 4.85 .90 3.35 .41 .42 17.44 1.39

.027* .011* .071 .000*** .016* .390 .545 .042* .028* .343 .068 .523 .517 000*** .240

*p < .05 **P < .01 ***p < .001.

The interaction between usability problems, users and design properties

285

Table 10 Education background in relation to follow-up (re)actions: means, (standard deviations) and ANCOVA F-values

Education

Upset but not responding Speak to the shop manager Exchange it for one from another brand Negative comments about the brand Wait: hoping things improved Call the helpdesk to argue with the employee Buy products from another brand next time Complain to consumer organization Complain to the company through mail Demand a refund Loyal to the brand Never buy any products from the brand again Remain calm until things are sorted out Tell my friends not to buy that brand products Communicate the dissatisfaction to the helpdesk


High school

BSc/BA

MSc/MA

PhD

N ¼ 41

N ¼ 104

N ¼ 169

N ¼ 176

N ¼ 86

x (sd)

x (sd)

x (sd)

x (sd)

x (sd)

3.12 (2.00) 4.54 (1.76)

3.03 (1.70) 4.29 (1.69)

2.69 (1.52) 3.85 (1.63)

2.64 (1.37) 4.03 (1.52)

3.68 (2.02)

4.04 (1.61)

4.08 (1.50)

3.98 (2.15)

4.44 (1.62)

3.39 (1.95)

F

Sig.

2.97 (1.46) 3.67 (1.60)

6.34 1.36

.000*** .248

3.92 (1.42)

3.66 (1.33)

6.87

.000***

4.33 (1.66)

4.27 (1.49)

4.83 (1.19)

11.82

.000***

2.85 (1.74)

2.88 (1.54)

2.76 (1.47)

3.07 (1.44)

5.86

.000***

3.54 (2.12)

3.40 (1.74)

3.02 (1.60)

3.34 (1.63)

3.20 (1.61)

5.83

.000***

3.76 (2.12)

4.49 (1.61)

4.60 (1.42)

4.59 (1.36)

4.44 (1.48)

9.01

.000***

2.78 (2.16)

2.62 (1.73)

2.21 (1.41)

2.43 (1.51)

2.30 (1.54)

6.42

.000***

1.95 (1.58)

2.77 (1.74)

3.01 (1.72)

3.07 (1.64)

2.67 (1.61)

.82

3.05 (2.09) 3.32 (1.90) 2.78 (1.89)

3.29 (1.77) 2.34 (1.47) 3.12 (1.76)

3.52 (1.63) 2.24 (1.12) 3.37 (1.53)

3.42 (1.53) 2.35 (1.31) 3.33 (1.39)

3.12 (1.57) 2.47 (1.29) 3.19 (1.66)

5.15 7.11 4.61

.000*** .000*** .001***

4.15 (1.80)

3.55 (1.56)

3.20 (1.28)

3.49 (1.33)

3.44 (1.51)

5.26

.000***

3.49 (1.96)

3.81 (1.76)

4.36 (1.50)

4.06 (1.41)

4.09 (1.44)

7.19

.000***

3.78 (1.98)

4.45 (1.50)

4.37 (1.53)

4.42 (1.38)

4.20 (1.43)

5.88

.000**

.514

*p < .05 **P < .01 ***p < .001.

a higher score on ‘being upset but not responding’. Complaining to a consumer organization or calling a helpdesk is less likely.

4

Conclusions and discussion

When household products fail because of a technical problem, it is obvious that the manufacturer will replace the product or parts of it and, if necessary, change the design or production of that particular product. But when in 50% of product failures nontechnical problems are the cause, manufacturers are minded to consider the complaint not as a product failure and put the onus of proof on the user. However, this turns out to be the wrong policy both in terms of denying responsibility for a high level of usability and of damaging brand loyalty; by that we are in danger of doing consumers and businesses a bad turn here.

286

Design Studies Vol 45 No. PB July 2016

The overall results of this study demonstrate the existence of many usability problems in the use of electronic products, which have nothing to do with technical failure. People express a huge amount of complaints about a large variety of products, from computers to e-book readers, and from washing machines to vacuum cleaners. Unbelievable that in a period in which ‘design’ gets high priority and status usability is still a big issue. In our survey functional problems are most mentioned, followed by operational problems and sensorial problems. Going into more detail about the causes of soft problems the study focused on the influence of ‘soft’ design properties as reasons for complaining about soft problems and on how user characteristics interact with these properties. One of the outstanding findings in the study is that soft problems are actually the outcome of the interaction between user characteristics and design properties. The main results were:
Operational transparency: a majority of soft problems in using electronic products occur with low operational transparency; i.e. the less operationally transparent an electronic product is the more likely to have soft problems. Operational transparency does not only refer to cognitive load but also to high-tech dependency (black box) and compatibility issues. Culture, gender and educational background have a significant impact on this property, meaning that the importance of operational property partly depends on these user characteristics.
Physical interaction density: a strong factor to anticipate whether an electronic product gets soft problems. An electronic product with a higher physical interaction density, such as a mobile phone, is more likely to be seen by the user as problematic. The same user characteristics culture, gender and educational background have a significant impact on this property.
Product importance: problems in using a product appear to become critical when it is an important product in user’s life. User characteristics don’t have any significant influence here.
Perceived performance: it is obvious that people complain when the product performs below their expectations. However, the perceived performance of a product is not as critical a factor in the occurrence of soft problems as the other properties. Age and educational background have a significant impact on this property.
Frequency of use: more soft problems are reported with more frequently used electronic products. The importance of this property partly depends on culture.
Importance of usability: whether people consider usability as important or not, it doesn’t show a direct link with the type of products. Age and educational background have a significant impact on this property.

The interaction between usability problems, users and design properties

287

4.1

Soft problems and follow-up (re)actions

Experiencing soft problems leads to different follow-up (re)actions, which partly depends on user characteristics. Without going into detail of how each age group or country behaves in taking or not taking actions, it is more important to show that user characteristics have a significant influence on people’s behaviour. Looking at the following clusters of (re)actions we can see which characteristics have an impact:
Loyalty to the brand. Culture and age influence whether the reaction to the brand will be positive or negative.
Word of mouth. Whether people spread the news about their complaint among friends and relatives depends on culture, gender and age.
Direct redress. Culture, age and educational background have influence on whether this action will be taken or not.
Action towards helpdesk or shop manager. The same characteristics as with the previous action play a significant role: culture, age and educational background.
Passive awaiting. Not taking action at all depends on age and educational background. All results show that it is important for designers and manufacturers to understand people’s interaction with products and their behaviour in case of being frustrated with a product. But are the results as presented in this study directly applicable to the industry?

4.2

Applicability

A company’s project team can already at the beginning of a product development project identify probable soft problems in terms of design properties and target group characteristics. For instance, there is reason for a company to believe that they have to take Culture more serious by studying their foreign target groups. Especially, the findings in the project are useful in case of developing a new product since for a product that never existed before there is lack of information. Anticipated soft problems can be identified in advance through defining the product in terms of design properties and analysing what the target group will be. When these aspects are taken into consideration in the product development process the seriousness of potential problems can be identified. The outcome by way of the newly developed product will increase consumer satisfaction in using products. However, the findings of this study need to be translated into a design language in order to make them useful for design practice. Among many ways to make these findings accessible, two methods could be proposed: (1) an

288

Design Studies Vol 45 No. PB July 2016

interactive tool and (2) a workshop approach. A first framework for a tool has been proposed. Such a tool should provide information in a fast and easy way on the interaction between user, product and use problems. This kind of information is useful especially during desktop studies at the very beginning of a product development process. The second method, a workshop approach, was developed and tested. Such a workshop is a useful way to share a deep understanding and a hands-on experience on the interaction between user, product and context. It will make stakeholders in the product development process aware of the importance of avoiding soft problems and also to provide an in-depth understanding of their target users and products that are being developed.

Acknowledgments The authors gratefully acknowledge the support of the Innovation-Oriented Research Programme ‘Integrated Product Creation and Realization (IOP IPCR)’ of the Netherlands Ministry of Economic Affairs.

References Brombacher, A. C., Sander, P. C., Sonnemans, P. J. M., & Rouvroye, J. L. (2005). Managing product reliability in business processes ‘under pressure’. Reliability Engineering and System Safety, 88(2), 137e146. Dantas, D. (2011). The role of design to transform product attributes on perceived quality. In Paper Presented at the 6th International Congress on Design Research, Lisbon. De Jong, T. (2010). Cognitive load theory, educational research, and instructional design: some food for thought. Instructional Science, 38(2), 105e134. Den Ouden, E. (2006). Development of a Design Analysis Model for Consumer Complaints: Revealing a New Class of Quality Failures. PhD. Eindhoven: Eindhoven University of Technology. Den Ouden, E., Yuan, L., Sonnemans, P. J. M., & Brombacher, A. C. (2006). Quality and reliability problems from a consumer’s perspective: an increasing problem overlooked by businesses? Quality and Reliability Engineering International, 22(7), 821e838. Douthit, D., Flach, M., & Agarwal, V. (2011). A “Returning Problem”: Reducing the Quantity and Cost of Product Returns in Consumer Electronics. Hassenzahl, M. (2004). The interplay of beauty, goodness, and usability in interactive products. Human-Computer Interaction, 19(4), 319e349. Hernandez, S. A., Strahle, W., Garcia, H. L., & Sorensen, R. C. (1991). A crosscultural study of consumer complaining behavior: VCR owners in the U.S. and Puerto Rico. Journal of Consumer Policy, 14(1), 35e62. Khan, S., Phillips, P., Jennions, I., & Hockley, C. (2014). No Fault Found events in maintenance engineering Part 1: current trends, implications and organizational practices. Reliability Engineering & System Safety, 123(0), 183e195. Kim, C. (2012). Anticipating Soft Problems with Consumer Electronic Products: How do Soft Problems Interact with User Characteristics and Product Properties?. PhD thesis Delft: Delft University of Technlogy. Kim, C. (2014). User characteristics and behaviour in operating annoying electronic products. International Journal of Design, 8(1), 93e108.

The interaction between usability problems, users and design properties

289

Kim, C., & Christiaans, H. (2012). ‘Soft’ usability problems with consumer electronics: the interaction between user characteristics and usability. Journal of Design Research, 10(3), 223e238. Kim, C., Christiaans, H., & van Eijk, D. (2007). Soft problems in using consumer electronic products. In Proceedings of the IASDR Conference. Hong Kong PolyU, November 2007. Koca, A., Karapanos, E., & Brombacher, A. (2009). ‘Broken expectations’ from a global business perspective. In Paper Presented at the Proceedings of the 27th International Conference Extended Abstracts on Human Factors in Computing Systems, Boston, MA, USA. Liu, R. R., & McClure, P. (2001). Recognizing cross-cultural differences in consumer complaint behavior and intentions: an empirical examination. Journal of Consumer Marketing, 18(1), 54e75. Madureira, O. M. (1991). Product standardization and industrial competitiveness. In Paper Presented at the International Conference on Standardization and Qulaity, Sao Paulo. Maute, M. F., & Forrester, W. R. (1993). The structure and determinants of consumer complaint intentions and behavior. Journal of Economic Psychology, 14(2), 219e247. Mugge, R., Schoormans, J. P. L., & Schifferstein, H. N. J. (2008). Product attachment: design strategies to stimulate the emotional bonding to products. Product Experience 425e440. Overton, D. (2006). ‘No Fault Found’ Returns Cost the Mobile Industry $4.5 Billion per Year. WDS Global. Patrick, A., & Mu, S. (2009). Usability and Acceptability of Biometric Security Devices. Vol. Updated October 19, 2009. National Research Council of Canada. Peyton, R. M., Pitts, S., & Kamer, R. H. (2003). Consumer satisfaction/dissatisfaction (CS/D): a review of the literature prior to the 1990s. In Paper Presented at the the Academy of Organizational Culture, Communications and Conflict, Las Vegas. Richins, M. L., & Verhage, B. J. (1985). Cross-cultural differences in consumer attitudes and their implications for complaint management. International Journal of Research in Marketing, 2(3), 197e206. Rust, R. T., Thompson, D. V., & Hamilton, R. W. (2006). Defeating feature fatigue. Harvard Business Review, 84(2), 98e107.

290

Design Studies Vol 45 No. PB July 2016