Don’t take away my status! – Evidence from the restructuring of a virtual reward system

Computer Networks 75 (2014) 477–490

Contents lists available at ScienceDirect

Computer Networks journal homepage: www.elsevier.com/locate/comnet

Don’t take away my status! – Evidence from the restructuring of a virtual reward system Tobias Mutter ⇑, Dennis Kundisch 1 University of Paderborn, Warburger Str. 100, 33100 Paderborn, Germany

a r t i c l e

i n f o

Article history: Received 2 December 2013 Received in revised form 14 July 2014 Accepted 11 August 2014 Available online 28 September 2014 Keywords: Online community Virtual reward system Status demotion Natural experiment Field observation

a b s t r a c t In a natural experiment on a popular German Question & Answer community we investigate how user status demotion affects user activity levels in online communities. The virtual reward system on this platform is designed to activate the status seeking behavior of its members. The members’ status within the community is represented by the member’s rank. In the experiment the platform operator restructured the virtual reward system, in the process reducing or abolishing the incentives for selected activities on the platform. After restructuring nearly all users saw their status demoted, with almost three quarters having lost one rank, and more than a quarter having lost two or more ranks. We identify the impact of status demotion by comparing how the members in these two groups respond to the restructuring. We find that users who lost two or more ranks reduce their post-event activities for which the incentives were reduced or abolished by 18% more than those who lost only one rank, and by 9% more than the latter for activities which were unaffected by the event. Ó 2014 Elsevier B.V. All rights reserved.

1. Introduction Since the inception of the Internet the number of online communities is constantly expanding and at the same time information technology is transforming the way people communicate and interact [9]. Online communities are used by people to share information, develop relationships, conduct business, and play games [28]. Despite the success of some online communities, many communities severely struggle with overcoming nonparticipation and low levels of contribution by their users (e.g., [9], [27], [34]). Therefore, a key challenge for online community providers is how users can be activated or incentivized so that they are not merely passive users but active contributors, and at the same time ⇑ Corresponding author. Tel.: +49 5251 60 5531. E-mail addresses: [email protected] (T. Mutter), [email protected] (D. Kundisch). 1 Tel.: +49 5251 60 5533. http://dx.doi.org/10.1016/j.comnet.2014.08.022 1389-1286/Ó 2014 Elsevier B.V. All rights reserved.

how to foster and sustain the activity levels of existing contributors (e.g., [33]). Compared to traditional offline communities, online communities give operators an augmented set of options to stimulate user contribution behavior. This includes technological features like virtual reward systems, which are implemented in many popular online communities (e.g., Stack Overflow, Kahn Academy) [2]. Common features of such virtual reward system are points, levels and ranks, which are designed to activate the status seeking behavior of users (e.g., [6,20]). Typically, users can earn these rewards by performing selected activities, and in this way increase their status within a community. Status or the relative rank in a certain group acts as a potent motivator for human behavior (e.g., [10,16,25,35]). Some research suggests that virtual reward systems can positively affect user activity levels (e.g., [2,15,20]) and, hence, activate the status seeking behavior of users.

478

T. Mutter, D. Kundisch / Computer Networks 75 (2014) 477–490

Virtual reward systems are rarely static, however, since community providers tend to amend them from time to time, with the intention of refreshing or improving their effect on user contribution. Common adjustments typically include modifications made to the structure of the point or award scheme. For example, the number of points needed to achieve ranks may be increased if it is felt that too many users have reached the highest level or rank on the platform, or that the virtual reward system is deemed to be ineffective in activating status seeking behavior. Depending on how the restructuration is designed, users may lose points and ranks in the process, i.e. they may experience status demotion. The effect of the restructuration of a virtual reward system on user activity levels in online communities has, to our knowledge, not yet been investigated. This could however lead to promising and useful results, especially as, in the field of marketing, research on customer loyalty programs suggests that consumer spending levels are negatively affected by customer demotion [41]. Whether these results also apply to user activity levels in online communities is a question that requires further needs to be explored. The difficulty of transferring results from the one environment to the other arises because online communities with virtual reward systems – unlike typical loyalty programs – provide environments without monetary or quasi-monetary benefits (e.g., upgrades, lounge access or priority booking at frequent flyer programs). We exploit a natural experiment to analyze how status demotion affects user activity levels in online communities by using a unique and rich dataset provided by a German Question & Answer (Q&A) community. This exclusive dataset includes detailed information about all user activity on the platform between February 2006 and April 2008. The natural experiment took place in February 2007, in the middle of our observation period, when the operator of the platform undertook a fundamental restructuring of the virtual reward system. All registered users on the platform under study automatically participate in a virtual reward system, which assigns them a status within the community. On performing certain selected activities, users are rewarded with so-called status points, and by accumulating status points, users automatically move up in an ascending (hierarchical) ranking system. As part of the restructuration, the community provider changed the status point scheme for selected activities on the platform and retrospectively recalculated the total number of status points of each user based on the new incentive scheme. In addition, the provider modified the ranking system. These changes had two major consequences: (1) the incentives in form of status points were substantially reduced and, for certain activities, abolished, (2) almost three quarters of users were demoted by one rank and more than one quarter lost two or more ranks. To analyze the impact of the restructuring, we compare the contribution behavior of 1,647 users in the four weeks before and after the event. We observe that both groups of users – those who lose one rank and those who lose two or more ranks – reduce their post-event activity levels, and the drop is even more pronounced for users who receive the stronger treatment. We identify the impact of status demotion on subsequent

user contribution levels by comparing how each of these two user groups (the first losing one rank, and the second, losing two or more ranks) respond to the restructuring. After taking into account the reduction in the post-event activity levels of users who lose one rank, we find that the users who are demoted by two or more ranks reduce their activity levels by 18% for activities for which the incentives were reduced or abolished, and by 9% for those that were unaffected by the event. Hence, user status demotion has a statistically and economically significant negative impact on user contribution behavior. This paper makes novel and significant contributions to research. To the best of our knowledge, we are the first to provide empirical evidence for the impact of status demotion on user activity levels in online communities. Our findings extend the work undertaken by Wagner et al. [41], but with the important difference that their research investigated hierarchical reward systems which offer monetary and quasi-monetary incentives, whereas our reward system only offers non-monetary benefits. Moreover, we contribution to the literature on online communities by offering new theoretical insights on the impact and operation of the key drivers that stimulate or inhibit user contribution behavior. 2. Theory and hypothesis 2.1. Literature review Three streams of literature are relevant to our study. The first is concerned with the general design of online communities. The second investigates how virtual reward systems can be used to incentivize user engagement, while the third is related to the impact of status and status demotion on human behavior. 2.1.1. Design of online communities The literature on the general design of online communities falls into two categories. In the first, the design is typically assumed to be immutable, and researchers purely focus on analyzing motivational factors. Existing research suggests that citizenship behavior and the desire to benefit an organization also act as motivational factors [10]. Users contribute more if they think that their actions enhance their professional reputation, or when they have the opportunity to share information with others, or if they are structurally embedded in a network [40]. Other factors that have been shown to have a positive effect on user activity levels include an awareness of one’s own efficacy, and the enjoyment of helping others [22]; recognition from the community, user experience and individual attributes (such as being a hobbyist) [21]; and perceived identity verification [28]. In the second strand of literature researchers selectively manipulate design features or personalized user information, and subsequently analyze changes in user contribution behavior. Researchers find that users increase their contribution of knowledge and experience an enhanced community attachment if they are reminded of their uniqueness and by setting them specific and challenging

T. Mutter, D. Kundisch / Computer Networks 75 (2014) 477–490

goals [27]; when users receive information about how they perform in comparison to the median user of the community [9]; and by strengthening either group identity or interpersonal bonds due to the implementation of a set of new community features [34]. We contribute to this research by analyzing the potentially negative, inhibiting effects of the status demotion of users on their contribution behavior. 2.1.2. Virtual reward systems Research on virtual reward systems constitutes a subgroup of the literature on gamification. Gamification refers to ‘‘using game design elements in non-gaming contexts’’ [13]. The goal of gamification is to incentivize user engagement to increase the quantity and quality of user contributions, and to support social exchange between users [19]. The set of game-design elements (e.g., leaderboards or ranks) can be related to different user motives (e.g., gaining a sense of achievement) [6]. Hamari et al. provide a comprehensive literature review of empirical studies on gamification [20]. Research so far has analyzed the impact of existing virtual reward systems and on how the introduction of such a system affects user contribution behavior. The majority of the existing studies find a positive effect of gamification on user activity levels. We contribute to this literature by analyzing how the restructuring of a virtual reward system affects user contribution behavior. This could be of practical value to operators of online communities by raising their awareness of actions which are likely to adversely affect user contribution behavior. 2.1.3. Status and status demotion Status is defined as ‘‘an actor’s relative standing in a group based on prestige, honor, and deference’’ ([37], p. 411). Status seeking comprises all activities intended to enhance an individual’s relative position in a group [25]. ‘‘Status matters because the need to compare oneself with others is pervasive and often occurs whether or not individuals intend to do so [. . .] and without them being aware of what they are doing’’ ([11], p. 890). Individuals with higher status tend to benefit from greater social assistance, are healthier on average, have greater social capital, and are offered better opportunities compared to individuals with lower status ([1,32,39]). Several empirical studies indicate the importance of status as a potent motivator for human behavior (e.g., [25,35]). Surprisingly little research, however, has been conducted on how human behavior is affected by status loss, and, hence, by status demotion [29]. Gephart ([18], p. 559) defines status demotion as ‘‘[. . .] relocating an actor in a lower formal social position with lessened status, formal authority, and power to control certain resources’’. Marr and Thau [29] analyze how the loss of status affects the quality of an individual’s task performance. They find that ‘‘High-status individuals experience more self-threat and subsequent difficulty performing well after losing status than do low-status individuals’’ ([29], p. 243). In the context of hierarchical loyalty programs, Wagner et al. [41] investigate the effect of status promotion and demotion on customers’ loyalty intention towards a firm, and thus customer spending levels. They find that the negative effect of customer

479

demotion is stronger than the positive effect of customer promotion. We contribute to the empirical literature on status demotion by analyzing how user status demotion affects user activity levels. This allows us to extend the findings by Wagner et al. on hierarchical reward systems which offer only non-monetary benefits. In Wagner et al.’s analysis customers receive monetary and quasimonetary benefits (e.g., upgrades, lounge access or priority booking at frequent flyer programs) while the virtual reward system on the platform under study offers, at best, non-monetary benefits to its users (e.g., higher reputation or trustworthiness within the community). 2.2. Hypothesis development Online communities commonly use virtual reward systems based on points, levels or ranks in order to activate user status-seeking behavior (e.g., [6,20]). Typically, users can earn these rewards by performing selected activities, and in this way increase their status within a community. The user’s status might be important for several reasons. It provides useful information about their past engagement, which helps other users to assess their reputation (e.g., [24,40]). Thus, status functions as a valuable tool to enable users to assess each other’s trustworthiness and the reliability of content produced by them [3]. This might imply that high status users receive on average more recognition or better ratings for their contributions with the same quality level compared to low level status users [30]. The role of status in communicating previous user engagement might also imply that other users will look more favorably upon those who hold a higher status within a community [5]. Thus, status can provide personal affirmation by reminding users of their, and others’ past levels of engagement [3]. However, community providers modify their virtual reward systems from time to time particularly with a view to refreshing or improving their effect on user contribution. Typical adaptations include adjustments made to the point scheme or an increase in the achievement level for ranks. Depending on the design of the restructuration, users may lose points and ranks in the process, in which case they experience status demotion. Status demotion is generally expected to induce negative emotions within the affected person (e.g., [36,38]) which may turn into anger and disappointment vis-à-vis the online community [17,26], and thus negatively affect users’ subsequent attitudes and behavior (e.g., [7]). In line with status demotion research, we equate status demotion with negative emotions, reduced user motivation and, consequently, expect to see a reduction in user activity levels in online communities. Hence, we formulate our research hypothesis: Hypothesis. Online community users who experience status demotion reduce their activity levels.

3. Research environment The website at the center of our analysis was launched in January 2006 and has requested to remain anonymous.

480

T. Mutter, D. Kundisch / Computer Networks 75 (2014) 477–490

Table 1 Status point scheme (before the event). Main activities

Status points per activity

Ratio of total status points (%)

Answering Questions Asking Questions Adding Friends Adding & Copying Links

0–25 0–4 5–20 1–2

48 14 12 25

pieces of information are also publicly visible to other platform users or guests whenever a user poses or answers a question. Due to the ascending order of ranks, users can easily compare their relative position to other users. Thus, the rank represents a user’s status within the community. 4. Natural experiment2 4.1. Description

The platform offers registered and non-registered users the opportunity to ask questions to members of the community. All registered users automatically participate in the community’s virtual reward system. User participation is incentivized by rewarding nearly all types of active contributions with status points. In Table 1, we present a list of the main activities and the corresponding status point scheme before the restructuration event. Almost all (99%) status points are earned by users taking part in one of the main types of activities. There are further activities which play only a very minor role and account for less than 1% of the total of accumulated status points (e.g., inviting new members to the platform). The ratio of total status points illustrates the ratio of the accumulated number of status points for each activity up until the event as a ratio of the total sum of status points for all activities and users. For example 48% of the overall accumulated status points are earned with the core activity answering questions. Depending on how the questioner or other users rate the quality of their answer, users can earn between 0 and 25 status points for contributing an answer. Apart from the core activity answering questions, registered users can also get status points by asking questions to the community. If a question receives at least one answer or is evaluated as top question by at least one other user, the questioner receives between 1 and 4 status points. No status points are earned if the question remains unanswered. Registered users also have the opportunity to add friends to their network of friends. If a friend request is accepted by another user, both users earn a certain amount of status points. Furthermore, each user has a personal link catalog. Each time a user adds a new link to the catalog or copies a link from another user, she gets status points. By accumulating status points users automatically move up in an ascending ranking system. In Table 2 we provide a list of the available ranks and the status points required for each rank. Before the event the ranking system consisted of 18 ranks in total. The labels of the first eight ranks are noticeably hierarchical, such as ‘‘Student’’, ‘‘Bachelor’’, and ‘‘Master’’. The rank ‘‘Bachelor’’, for example, requires an accumulation of at least 120 status points, while that of ‘‘Master’’ is earned with 720 points. By earning on average 4 status points per answer users would have to answer more than 30 questions to reach the former, and 180 for the latter rank. As the system is cumulative, users have to reach ‘‘Bachelor’’ status before they can start earning points towards the level of ‘‘Master’’. The list of ranks and the required status points for each rank is publicly available on the platform. The rank and the total number of earned status points are displayed in the personal profile of each user. Both

In February 2007, the operator of this Q&A community fundamentally restructured its virtual reward system. According to the operator, the objective of the restructuring was to simplify and enhance the reward system. The provider changed the status point scheme for the activities on the platform, retrospectively recalculated the total number of status points of each user and modified the ranking system. As a result of this restructuration, the number of status points that could be earned for certain activities listed in Table 3 were reduced or abolished. These activities included adding and copying links and adding friends. The activities asking and answering questions – which constitute the main purpose of the community – were not affected by the restructuring. The new status point scheme is illustrated in Table 3. In addition, the community provider recalculated the total number of status points that each user had earned since the day of their registration, based on the new point scheme. For example, by adding a new friend to their network users were previously rewarded with up to 20 status points but received none at all after the event – the reward for this activity had been abolished. Not only this, the provider retrospectively applied the new rules to users’ existing score, so if a user had earned 100 status points by adding new friends before the event, she lost these 100 status points after the restructuring. The new ranking system is illustrated in Table 4. The provider added two new ranks, called ‘‘Beginner’’ and ‘‘Albert Schweitzer’’, changed the labels of the ranks between ‘‘James Watt’’ and ‘‘Leonardo da Vinci’’ (see Table 2), and inflated the number of required status points for each rank. The labels and the order of the ranks from ‘‘Student’’ to ‘‘Nobel Laureates’’ and for the rank ‘‘Albert Einstein’’ stayed the same. Users who held a rank between ‘‘Student’’ and ‘‘Nobel Laureates’’ before the event could compare their new position in the ranking system based on the label of the new rank. This means that they were able to assess the loss of their ranking quite accurately. For example, a user with 3,100 status points held the rank ‘‘Professor’’ before the event. After the event, even without losing any points, the same user only qualifies for the rank ‘‘Doctor’’, due to the inflation of the required status points, 2 Natural experiments are empirical studies which are characterized by a transparent exogenous source of variation in the explanatory variable that determines treatment assignment. A natural experiment can be caused by policy changes or other events [31]. The exogenous source of variation strengthens the claim of a causal interpretation of the results. Natural experiments are most helpful in contexts where controlled experiments would be difficult to implement or unethical (e.g., estimating the economic return on schooling) [14]. We demonstrate the exogenous variation in our natural experiment with a regression model in Section 5.2.5.

T. Mutter, D. Kundisch / Computer Networks 75 (2014) 477–490

481

Table 2 Ranking system (before the event). Label of rank before event

Required status points before event

Label of rank before event

Required status points before event

Student Bachelor Master Research Assistant Doctor Assistant Professor Professor Nobel Laureates James Watt

0 120 720 1,130 1,640 2,250 3,050 3,780 4,690

Archimedes Ts’ai Lun Johannes Gutenberg Alexander G. Bell Gottfried W. Leibniz Max Planck Johannes Kepler Leonardo da Vinci Albert Einstein

4,790 4,890 4,990 5,090 5,190 5,290 5,390 5,490 >6,490

Table 3 Status point scheme (before and after the event). Main activities

Answering Questions Asking Questions Adding Friends Adding & Copying Links

Status points per activity Before event

After event

0–25 0–4 5–20 1–2

0–25 0–4 0 0–1

Status points reduced or abolished?

(Unchanged) (Unchanged) U U

activities adding and copying links and adding friends for which the number of status points are reduced or abolished. We use all four activities to measure user activity levels in order to account for the potential displacement of effort. For example, after the restructuring, users might perform fewer activities of the second group, and more of the first group, because of the change in the number of points that can be earned with these activities. 5. Dataset, sample selection and descriptive statistics 5.1. Dataset

and thus, she lost two ranks. Users who held a rank between ‘‘James Watt’’ and ‘‘Leonardo da Vinci’’ before the event faced greater difficulty comparing their new rank given the changes made to the labeling of these ranks. These users would have had to purposely identify and compare their relative position within the ranking system before and after the event if they wanted to assess whether and how many ranks they had lost. The planned restructuring event was repeatedly announced to the community prior to its implementation from about 5 months before the event. However, it is important for the following analysis to appreciate that its particulars, i.e. neither the recalculation, the deduction of status points nor the specific modification of the ranking system were known to users in advance. Thus, the status demotion had taken them by surprise.

4.2. Contextualization of hypothesis In this natural experiment users are said to experience status demotion when they lose at least one rank. We assume that users refer back to the rank they held before the event as reference point for assessing how many ranks they have lost. Our hypothesis states that users who experience status demotion will subsequently reduce their post-event activity levels. Furthermore, we expect this negative effect of status demotion to be more pronounced for users who receive a stronger treatment, i.e. who have lost more ranks. We use the four main activities on the platform, illustrated in Table 3, to measure user activity levels. The activities can be divided into two activity groups. The first group covers the activities asking and answering questions which are not affected by the restructuring, while the second activity group contains the

We are fortunate in having a unique dataset at our disposal which allows us to analyze this natural experiment provided by the operator of the community. The whole dataset covers all user activities on the platform between the beginning of February 2006 and the end of April 2008. The number of newly registered users was 12,901 in 2006, 54,404 in 2007, and 25,909 up until the end of April 2008. During the observation period, we observe how these users collect 14,132,466 status points, and thereby move up in the ranking system on the platform. To earn status points, users replied to 1,000,542 posted questions with 2,996,446 answers, built 32,696 friendships with other users and added 87,872 links to the link catalog. Our data is at the level of each individual user. Thus we know exactly when a user registers on the platform, when and how often she performs a certain activity, when and how many status points she earns for her activities, and when she moves up in the ranking system. Therefore, we are able to establish a profile for each user based on her user history on the platform. 5.2. Sample selection and descriptive statistics In the next subsections, we present the descriptive statistics and explain how we select the sample for our empirical analysis on user status demotion. How the sample selection decisions affect our results is described in the robustness check section at the end of the paper. 5.2.1. Event window To analyze the impact of the restructuration we compare the user contribution behavior in the four weeks before and in the four weeks after the event. We use only users who were already registered on the online community before the natural experiment took place, and who

482

T. Mutter, D. Kundisch / Computer Networks 75 (2014) 477–490 Table 4 Ranking system (after the event). Label of rank after event

Required status points after event

Rank as before event?

Label of rank after event

Required status points after event

Label of rank as before event?

Beginner Student Bachelor Master Research Assistant Doctor Assistant Professor Professor Nobel Laureates Albert Schweitzer

0 210 530 1,030 1,630 2,430 3,330 4,240 5,240 7,740

New Rank U U U U U U U U New Rank

Robert Koch Immanuel Kant Archimedes Max Planck Isaac Newton T. A. Edison Pythagoras Galileo Galilei Leonardo da Vinci Albert Einstein

8,240 8,740 9,240 9,740 10,240 10,740 11,240 11,740 12,240 >12,740

– – – – – – – – – U

were still active on the platform in the last week before the event. We regard users as inactive when they permanently stopped to perform on any of the platform’s activities. We aggregate the activity data on a weekly level, in line with other studies analyzing user contribution behavior in online communities (e.g., [9], [43]). There are two main reasons for this. First, activity levels for the various activities on the platform tend to fluctuate depending on the day of the week (e.g., activity levels might be higher at the weekend than during the week) and we want to rule out potential biases caused by the selection of a specific day or days. Second, the majority of users are not active on every day of the week. Therefore, aggregating the activity data on a weekly level is a suitable measure for the average activity levels of users. We excluded the week in which the restructuring took place because users discussed the restructuring on the platform – by posing and answering questions on the issue – whereas our aim was to investigate the general impact of the event on user activity levels rather than users debating the event itself. 5.2.2. Distribution of users across ranks In Table 5 we present the distribution of active users across the ranking system of the platform in the week before the restructuring took place. The vast majority of users (98%) are situated between the ranks ‘‘Student’’ and ‘‘Nobel Laureates’’. In general, the more demanding the rank the fewer users hold such a rank. As described in Section 4.1, users holding a rank between ‘‘James Watt’’ and ‘‘Leonardo da Vinci’’ before the event could not easily – if at all – compare their new position in the revised ranking system. Thus it is not clear whether these users experienced status demotion. In addition, these altogether high-ranking users might be more focused on decreasing their distance to the end goal (the highest rank ‘‘Albert Einstein’’) and thus more acutely aware of that distance, and less aware of the next rank in the system and, by implication, of the precise rank they are holding at a specific moment in time. We applied the same reasoning to users who hold the rank ‘‘Albert Einstein’’ before the event and who lose this rank through restructuration. Therefore, we exclude from our sample the 15 users who held a rank between ‘‘James Watt’’ and ‘‘Leonardo da Vinci’’, and the 27 users who held the rank

‘‘Albert Einstein’’, before the event, or a total of 42 users from the upper ranks. 5.2.3. User activity history In Table 6 we present a short summary of users’ activity history from the foundation of the platform to the day of the event (omitting the 42 users in the upper ranks, as described in the previous section). At the time of the restructuration, the average length of registration was 25.7 weeks (Length of Membership), while 50% of users had been registered for 21 weeks or more. Users were registered for a minimum of one week but for no longer than 60 weeks. During the entire period of their membership users contributed on average 73.2 answers (Sum of Answers), asked 18.6 questions (Sum of Questions), had 5.8 friends (Sum of Friends), and added 27.3 links (Sum of Links). As can be seen from the quantiles of the distributions, there is a strong heterogeneity in the history of user contribution. The median values differ substantially from the mean values for each of the four accumulated activity measures. This reveals that a substantial share of the activities is performed by a small number of top contributors. In Table 7 we provide the correlations for the variables of the activity history of users. In general, the correlations for the accumulated number of the four main activities lie in the interval between 0.246 and 0.404. This indicates that when users increase their overall activity levels they perform more of each activity type. Interestingly, the correlations between the Length of Membership and the accumulated number of the four main activities lie in the interval between 0.095 and 0.162 and thus are rather small. This indicates that the most active users are not necessarily those who were registered the longest on the platform. 5.2.4. Number of lost ranks With the restructuring, the provider retrospectively recalculated the total number of status points of each user by applying the new point scheme, and then simultaneously adjusted the ranking system (see Section 4). This resulted in a loss of ranks for nearly all users. We illustrate how users are affected by these changes in Table 8. We present the rank before the event in column (1). In columns (2)–(6) we provide mean, standard deviation, minimum, median and maximum values of the number of deducted status points conditional on user rank, and, in columns

483

T. Mutter, D. Kundisch / Computer Networks 75 (2014) 477–490 Table 5 Distribution of users (before the event). Label of rank before event (1)

Required status points before event (2)

Number of users before event (3)

Label of rank before event (1)

Required status points before event (2)

Number of users before event (3)

Student Bachelor Master Research Assistant Doctor Assistant Professor Professor Nobel Laureates James Watt

0 120 720 1,130 1,640 2,250 3,050 3,780 4,690

776 603 102 76 49 31 26 15 1

Archimedes Ts’ai Lun Johannes Gutenberg Alexander G. Bell Gottfried W. Leibniz Max Planck Johannes Kepler Leonardo da Vinci Albert Einstein

4,790 4,890 4,990 5,090 5,190 5,290 5,390 5,490 >6,490

0 1 2 1 0 0 0 10 27

Table 6 Users’ activity history. Variables

Mean

Std

Min

Q25

Median

Q75

Max

Length of Membership Sum of Answers Sum of Questions Sum of Friends Sum of Links

25.7 73.2 18.6 5.8 27.3

18.4 156.6 36.0 21.8 96.1

1.0 0.0 0.0 0.0 0.0

10.0 4.0 1.0 0.0 0.0

21.0 16.0 6.0 1.0 0.0

43.0 62.0 20.0 5.0 11.0

60.0 1,395.0 617.0 688.0 1,587.0

Table 7 Correlations of Users’ Activity History Variables. Variables

Sum of Answers

Sum of Questions

Sum of Friends

Sum of Links

Sum of Answers Sum of Questions Sum of Friends Sum of Links Length of Membership

1 0.347 0.366 0.339 0.151

1 0.374 0.246 0.110

1 0.404 0.095

1 0.162

(7)–(11), for the number of lost ranks. The average loss is 125 status points. In general, the higher a users’ standing in the ranking, the more status points she lost on average. For example, users who held the rank ‘‘Doctor’’ before the event lost on average 520 points, whereas users who held the rank ‘‘Nobel Laureates’’ lost 903 status points. This is due to the fact that the higher the user rank, the more active the users are on average. And the more active users were before the event, the more status points they lost during the event. As can be seen from the minimum and maximum values of the distribution, some users hardly lost any status points, whereas others lost almost all of their points. This corresponded to an average loss of 1.3 ranks overall. Generally, users with higher ranks lost on average more ranks. For example, users who held the rank ‘‘Master’’ before the event lost on average 1.2 ranks whereas users who held the rank ‘‘Professor’’ lost 2.2 ranks. The minimum and maximum values reveal that there are users in the dataset who did not lose any rank, and users who lost 5 ranks. Important to note is that users who held the lowest rank ‘‘Student’’ before the event are demoted by one rank because the provider introduced a new entry level rank called ‘‘Beginner’’ (see Table 4). This also explains why more than 50% of the users who held the rank ‘‘Bachelor’’ before the event lost 2 ranks as a consequence of the restructuring.

5.2.5. Exogeneity of number of lost ranks The number of ranks lost was partly determined by the user’s ranking before the event, the user activity history on the platform, and by a degree of randomness. Generally, the higher a user’s rank, the more ranks she is likely to have lost. In addition, the more activities a user had performed for which the number of status points had been reduced or abolished, the more status points a user was set to lose. Consequently, the probability of losing more ranks was higher for those in the higher echelons of the ranking system. Finally, the last part is determined by randomness. The main source of randomness is how close a user was to reaching the next highest rank before the event. For example, if a user held the rank ‘‘Professor’’ with 3,100 status points before the event she lost two ranks but if the user held the same rank with 3,500 status points she lost only one rank. To illustrate that the number of ranks which a user lost was substantially caused by randomness, we regress the Number of Lost Ranks on a set of dummies for each available rank on the platform and on the sums of each main activity up until the last week before the event (Sum of Answers, Sum of Questions, Sum of Friends, Sum of Links). The results of the regression are presented in Table 9. Almost all of the coefficients are highly significant. As expected, the coefficients for Sum of Friends and Sum of Links are positive. Users who had performed these two activities extensively up until the event lost on average more ranks than those who had not added links or friends. The coefficients for the variables Sum of Answers and Sum of Questions are negative. This indicates that users who mainly conducted these activities lost on average fewer ranks. The R-squared is 0.5122, indicating that approximately half of the variation in the variable Number of Lost Ranks can be explained by this model. More importantly, a large part of the variation in the dependent variable

484

T. Mutter, D. Kundisch / Computer Networks 75 (2014) 477–490

Table 8 Number of deducted status points and lost ranks. Rank before Event

Number of deducted status points

(1)

Mean (2)

(Beginner) Student Bachelor Master Research assistant Doctor Assistant professor Professor Nobel laureates P

(Rank introduced after event) 12 18 0 108 91 0 263 171 43 415 310 20 520 400 73 572 364 63 764 655 78 903 855 90

Std (3)

125

Min (4)

244

0

Number of lost ranks Median (5)

Max (6)

Mean (7)

0 80 224 347 312 539 632 658

101 565 1,013 1,420 1,372 1,317 2,656 3,209

40

3,209

Table 9 Exogeneity number of lost ranks. Variables (1)

Number of Lost Ranks (2)

Sum of Answers

0.0026⁄⁄⁄ (0.0003) 0.0026⁄⁄⁄ (0.0006) 0.0025⁄ (0.0014) 0.0010⁄⁄⁄ (0.0003) U 1.0247⁄⁄⁄ (0.0038)

Sum of Questions Sum of Friends Sum of Links Rank Dummies Constant Observations R-Squared Robust standard errors in parentheses

1,678 0.5122 ⁄⁄⁄

p < 0.01,

⁄⁄

p < 0.05, ⁄ p < 0.1.

cannot be explained by the model. We use this exogenous variation caused by the natural experiment to analyze the impact of user status demotion on user contribution behavior.

5.2.6. Activity measures We compare user contribution behavior in the four weeks before and after the event, respectively, to investigate the impact of the restructuring. We therefore create two variables to measure user activity on a weekly level based on the four main activities on the platform illustrated in Table 3. The first group covers the activities asking questions and answering questions which are not directly affected by the restructuring. The second group contains the activities adding and copying links, and adding friends for which the number of status points are reduced or abolished. By combining the activities in these two groups we create the two activity measures for our empirical analysis: Friends & Links and Answers & Questions. In Table 10 we provide mean, median and standard deviation for both variables over the four weeks before and after the event. The average number of Answers & Questions increases slightly from 4.13 before to 4.48 per week after the event and the average for Friends & Links drops from 1.28 to 0.31.

Std (8)

Median (10)

Max (11)

(Rank introduced after event) 1 0 1 1.5 0.6 0 1.2 0.6 0 1.6 0.8 0 1.9 0.9 1 2 0.7 1 2.2 0.8 1 2.5 1 1

1 2 1 1.5 2 2 2 2

1 2 3 4 4 3 4 5

1.3

1

5

0.6

Min (9)

0

We also create a log-transformed version of both variables: Ln(Answers & Questions) and Ln(Friends & Links). Since the log of zero is not defined, we add 1 to each of these variables [42]. We log-transform the variables to analyze relative changes in user activity levels and to make the regression outputs for both activity measures comparable. In addition, we give observations in lower quantiles a higher weighting in the subsequent empirical analysis. This allows us to rule out the eventuality that our results could be driven purely by a few top contributors (or outliers) but without actually applying to a substantial part of the users on the platform. Interestingly, the average of both log-transformed variables declines after the event. This indicates that the slight increase in the average for the variable Answers & Questions in absolute terms after the event is caused by a few top contributors but does not reflect how the majority of users respond to the restructuring. 5.2.7. Treatment intensity In Table 11 we present the distribution of the treatment intensity across the users in our sample. Our sample comprises a very small group of 31 users (1.8%) who did not lose any rank. The vast majority of users (70.8%) lost one rank, while approximately one quarter of users lost two or more ranks (27.4%). We aggregate all users who lost two or more ranks in one group (P2), because only 43 users (2.5%) lost between 3 and 5 ranks. Although the sample comprised a small group of users who maintained their position within the ranking system, this group was also negatively affected by the event, and is therefore not suitable to serve as control group for our subsequent empirical analysis. Intuitively, the probability of not losing a rank was highest for users who were very close to moving up to the next highest rank before the event. With restructuration, however, these users saw the goal that had seemed within close reach suddenly receding. According to the goal-gradient hypothesis, user motivation – and thus user activity levels – increases with proximity to the next goal [23]. This phenomenon has been the object of several empirical studies in the field of marketing research which provided empirical evidence for the goal-gradient hypothesis (e.g., [12,23]). Thus, these users might be disappointed because the efforts which they had deployed towards these

485

T. Mutter, D. Kundisch / Computer Networks 75 (2014) 477–490 Table 10 User activity levels. Variables (1)

Mean (2)

Std (3)

Median (4)

Max (5)

Sum (6)

Mean (7)

Std (8)

Median (9)

Max (10)

Sum (11)

Answers & Questions Ln(Answers & Questions) Friends & Links Ln(Friends & Links)

4.31 0.76 1.28 0.27

12.17 1.11 9.44 0.67

0.0 0.0 0.0 0.0

200 5.3 438 6.08

27,721 4,863.63 8,214 1,706.45

4.48 0.64 0.31 0.12

16.7 1.1 2.06 0.38

0.0 0.0 0.0 0.0

369 5.91 78 4.37

30,045 4,282.32 2,066 806.35

Table 13 Proximity to Next Rank II.

Table 11 Frequency distribution treatment. Values (1)

Frequency (2)

0 1 P2 P

31 1,188 459

1.8 70.8 27.4

1,678

100.00

Variables

Percent (3)

activities before the event had come to nothing. In Table 12 we illustrate the proximity to the next rank for, respectively, users who did not lose any rank, users who lost one rank, and all the other users who lost two or more ranks (P2). In columns (2)–(4) we present mean, median and maximum value for the relative number of required status points to reach the next highest rank before the event, and in columns (5)–(7) after the event. On average, users who did not lose any rank just needed another 12% of the required status points to reach the next rank before the event while the two other groups still needed 57% and 76%. After the event, the distance to the next rank increases substantially for users who kept their rank. These users now need on average 84% of the required status points to move up to the next rank. To illustrate that the proximity to the next rank matters, we look at user activity levels over the four weeks before and after the event, and regress both activity measures, Ln(Friends & Links) and Ln(Answers & Questions), on an event dummy variable, the relative number of required status points before the event, an interaction term between these two variables, and a constant. These regression results are presented in Table 13. All estimators are significant and the results are qualitatively the same for both dependent variables. The negative signs of the estimators for the variable Relative Number of Required Status Points reveal that the closer a user gets to the next rank before the event the higher their average activity levels in the four weeks before the event. However, the positive signs of the interaction terms with the event

(1) Event Dummy Relative Number of Required Status Points Event Dummy ⁄ Relative Number of Required Status Points Constant Observations R-Squared

Ln(Friends & Links) (2)

Ln(Answers & Questions) (3)

0.2761⁄⁄⁄ (0.0366) 0.3106⁄⁄⁄ (0.0548) 0.2128⁄⁄⁄ (0.0494) 0.4557⁄⁄⁄ (0.0405)

0.2266⁄⁄⁄ (0.0466) 0.5326⁄⁄⁄ (0.0886) 0.1833⁄⁄⁄ (0.0666) 1.0652⁄⁄⁄ (0.0648)

12,892 0.0320

12,892 0.0165

Cluster robust standard errors in parentheses p < 0.1.

⁄⁄⁄

p < 0.01,

p < 0.05,

dummies indicate that the average activity levels of these users decreases in the four weeks after restructuration. For example, for the dependent variable Ln(Friends & Links) the estimator for the variable Relative Number of Required Status Points is 0.3106 or 26.69% and for the interaction term with the Event Dummy it is 0.2128 or 23.71%. Users who had been very close to the next highest rank before the event (e.g., Relative Number of Required Status Points = 0.01) performed on average 26.69% more of these activities in the weeks before the event compared to users who had only just reached the next rank (e.g., Relative Number of Required Status Points = 0.99) but were on average 23.71% less productive afterwards. Hence the regression results indicate that users who did not lose any rank were also negatively affected by the event. But because of the very small size of this user group there is a risk of it being unrepresentative for a user group who do not lose any rank. Thus, we drop the 31 users who did not lose any rank from our sample as we want to analyze the impact of status demotion and not of how the proximity to the next rank affects user activity levels.

Table 12 Proximity to next rank I. Number of lost ranks

⁄⁄

⁄

Relative number of required status points to get to the next rank Before event After event

(1)

Mean (2)

Median (3)

Max (4)

Mean (5)

Median (6)

Max (7)

0 1 P2

0.12 0.57 0.76

0.09 0.59 0.84

0.37 1 1

0.84 0.75 0.36

0.88 0.8 0.35

0.99 1 0.99

486

T. Mutter, D. Kundisch / Computer Networks 75 (2014) 477–490

Moreover, the remaining two groups are sufficiently large to obtain reliable results. Thus, we focus our empirical analysis on the group who lost one rank and the group who lost two or more ranks to investigate how they respond to the restructuring.

control variables in our model to account for the deterministic part in the natural experiment (see Section 5.2.5) [4]. Thus, we get the following regression model:

Y it ¼ a þ cDG þ hDE þ qðDG  DE Þ þ bX it þ eit

ð1Þ

5.2.8. Summary of the main sample We analyze user contribution behavior in, respectively, the four weeks before and in the four weeks after the event – excluding the event week itself (see Section 5.2.1). We include only users in our sample who had registered on the online community before the restructuring took place, and who were still active on the platform in the last week before the event (see Section 5.2.1). We drop 42 users from our sample who held a rank between ‘‘James Watt’’ and ‘‘Albert Einstein’’ before the event because for these users it is not obvious how many ranks they had lost as a result of restructuration (see Section 5.2.2). Finally, we drop 31 users who did not lose any rank because these users were also negatively affected by the event while having received a different treatment from restructuration (see Section 5.2.7). For our empirical analysis this leaves us with an unbalanced panel of 1,647 users and 12,892 observations on a weekly level (6,304 before and 6,588 after the event) from a total observation period of 8 weeks.

Yit represents the dependent variable. We run the regression for both dependent variables separately. The index i stands for each user in our sample, and t is a time index. The variable DG is the Group Dummy. The estimator for the coefficient c reveals potential differences between both groups in average activity levels before the event. DE is the Event Dummy and the estimator for h represents the difference in average activity levels of the group of users who lost one rank between the four weeks before and after the event. The coefficient q of the interaction term between the Group Dummy and the Event Dummy reveals the difference between the differences in average activity levels for the two groups in the four weeks before and after the event. Hence, the estimator reveals the difference in how each group is affected differently by the restructuring. In addition, we add a vector of control variables Xit to our model accounting for the deterministic part of the treatment, and a random error term e. For statistical inference, we use cluster robust standard errors on the user level to account for potential heteroscedasticity and autocorrelation in the data [42].

6. Empirical analysis

6.3. Identification

6.1. Main variables

A simple comparison of user activity levels before and after the event gives a first indication on how users respond to the restructuring [8]. However, to rule out potential competing explanations (e.g., a potentially superimposed seasonal effect) and to identify the impact of status demotion on user activity levels properly, we compare how the users who lost one rank and those who lost two or more ranks respond to the restructuring. As both user groups receive a treatment, we assume that the treatment effect is the same for both groups and increases with treatment intensity, i.e. both groups are either positively or negatively affected and the treatment effect is stronger for users who lost two or more ranks than for users who lost only one rank. Consequently, if the users who lost two or more ranks reduce their activity levels in post-event activities more than the users who lost one rank this difference could then be solely attributed to the negative impact of status demotion.

We use two variables to measures how user activity levels are affected by the restructuration, Ln(Friends & Links) and Ln(Answers & Questions) (see Section 5.2.6). To account for the non-random variation of the natural experiment, we include user rank as well as the accumulated number of activities up until the last week before the event as control variables in our model (see Sections 5.2.3 and 5.2.5). Therefore, we create dummy variables for each of the available ranks (Rank Dummies). Furthermore, we use the accumulated number of answers (Sum of Answers), questions (Sum of Questions), friends (Sum of Friends), and added or copied links (Sum of Links) as controls. We create a new dummy variable separating the users who lost one rank and two or more ranks (Group Dummy). The dummy variable takes the value zero for the first group and one for the latter (see Section 5.2.7). In addition, we create another dummy variable separating the activity levels in the weeks before and after the event (Event Dummy). We do not include the variable Relative Number of Required Status Points because we want to keep the main model as simple as possible. 6.2. Model We use a differences-in-differences (DD) approach to analyze the impact of status demotion on user activity levels. With the DD framework, we explicitly estimate how each group responds to the restructuring and how the groups’ responses differ from each other. We include several

6.4. Results In Table 14 we illustrate the results for the dependent variable Ln(Friends & Links) and in Table 15 for Ln(Answers & Questions). Both output tables follow a similar structure. In column (1) we illustrate the independent variables that have been introduced in Sections 5.2 and 6.1. We start with a pure DD-design where we regress each dependent variable on a dummy variable separating both user groups (Group Dummy), an event dummy (Event Dummy), and an interaction term between the group and the event dummy (Group Dummy ⁄ Event Dummy). The results are illustrated in column (2). In a second step we add all the variables to

T. Mutter, D. Kundisch / Computer Networks 75 (2014) 477–490 Table 14 Dependent variable Ln(Friends & Links). Variables (1) Group Dummy

Ln(Friends & Links) (2)

Ln(Friends & Links) (3)

0.2652⁄⁄⁄ (0.0381) 0.0875⁄⁄⁄ (0.0107) 0.1985⁄⁄⁄ (0.0341)

Constant

0.1871⁄⁄⁄ (0.0118)

0.0547 (0.0395) 0.0829⁄⁄⁄ (0.0105) 0.2008⁄⁄⁄ (0.0339) U 0.0009⁄⁄⁄ (0.0002) 0.0002 (0.0003) 0.0032⁄⁄⁄ (0.0007) 0.0011⁄⁄⁄ (0.0002) 0.0768⁄⁄⁄ (0.0067)

Observations R-Squared

12,892 0.0431

12,892 0.1738

Event Dummy Group Dummy ⁄ Event Dummy Rank Dummies Sum of Answers Sum of Questions Sum of Friends Sum of Links

Cluster robust standard errors in parentheses p < 0.1.

⁄⁄⁄

p < 0.01,

⁄⁄

p < 0.05,

⁄

Table 15 Dependent variable Ln(Answers & Questions). Variables (1)

Ln(Answers & Questions) (2)

Ln(Answers & Questions) (3)

0.2289⁄⁄⁄ (0.0556) 0.0909⁄⁄⁄ (0.0208) 0.0814⁄ (0.0431)

Constant

0.6692⁄⁄⁄ (0.0274)

0.3354⁄⁄⁄ (0.0692) 0.0752⁄⁄⁄ (0.0208) 0.0921⁄⁄ (0.0429) U 0.0026⁄⁄⁄ (0.0005) 0.0024⁄⁄ (0.0009) 0.0009 (0.0010) 0.0002 (0.0004) 0.2964⁄⁄⁄ (0.0169)

Observations R-Squared

12,892 0.0091

12,892 0.2524

Group Dummy Event Dummy Group Dummy ⁄ Event Dummy Rank Dummies Sum of Answers Sum of Questions Sum of Friends Sum of Links

Cluster robust standard errors in parentheses p < 0.1.

⁄⁄⁄

p < 0.01,

487

number of questions (Sum of Questions), highly significant. Generally speaking, the more active a user had been in the past, the more she contributes in the weeks before and after the event.3 The estimators for the event dummy variable (Event Dummy) and the interaction term between the group dummy (Group Dummy) and the event dummy (Event Dummy) are both negative and highly significant on a one percent level. Both estimators are qualitatively unchanged by including additional control variables. The estimator of the event dummy (Event Dummy) in the main model in column (3) is 0.0829 or 8%.4 This indicates that the users who lose one rank reduce their activity levels for activities for which the incentives are reduced or abolished after the event by 8%. This drop might be caused by status demotion or by a negative seasonal effect in the four weeks after the event. For this user group, we are not able to disentangle these two overlapping effects. However, the estimator for the event dummy (Event Dummy) represents a reference point allowing to identify whether the activity levels of users who lost two or more ranks are negatively affected by status demotion. The estimator of the interaction term between the group dummy and the event dummy (Group Dummy ⁄ Event Dummy) is 0.2008 or 18.19% and reveals how the users who lose two or more ranks respond to the restructuring after taking into account the reduction in post-event activities of the users who lose only one rank. This difference can be solely attributed to the negative impact of user status demotion and reveals that the users who receive the stronger treatment respond much stronger to the restructuring of the incentive system. Hence, we derive our first result. Result I. The online community users who lose two or more ranks reduce their activity levels in post-event activities for which the virtual rewards are reduced or abolished by 18% compared to users who are downgraded by only one rank.

⁄⁄

p < 0.05,

⁄

our model which are related to the treatment intensity. By including these variables to our model, we control for the deterministic part in the treatment. The results for the main model are presented in column (3). As illustrated in Table 14, the estimators for the accumulated activities (Sum of Answers, Sum of Questions, Sum of Friends and Sum of Links) are positive for the dependent variable Ln(Friends & Links) and, except for the accumulated

In Table 15 we present the regression results for the dependent variable Ln(Answers & Questions). We regress the dependent variable on the same set of exogenous variables as before. The estimators for the accumulated activities (Sum of Answers, Sum of Questions, Sum of Friends) are positive and except for the accumulated number of friends (Sum of Friends) significant on a one or five percent level. Generally, the more active a user had been in the past, the more she contributes in the weeks before and following the event. The estimator for the event dummy (Event Dummy) is negative and significant in both models. The estimator of the event dummy (Event Dummy) in the main model in column (3) is 0.0752 or 7%. This indicates that users who lose one rank perform also 7% fewer activities for which the incentives are unaffected by the event. The drop is close to the event dummy for the dependent variable Ln(Friends & Links) presented in Table 14. 3 We do not present the estimators for the Rank Dummies for reasons of simplification. They are available from the authors on request. 4 The dependent variable is log-transformed. Thus, the estimated ^j are interpreted as an approximated percentage change. To coefficients b get the exact percentage change we have to transform the estimated ^j Þ  1Þ  100 coefficients according to the following formula: D% ¼ ðexpðb [42].

488

T. Mutter, D. Kundisch / Computer Networks 75 (2014) 477–490

The estimator of the interaction term between the group dummy and the event dummy (Group Dummy ⁄ Event Dummy) is 0.0921 or 9% and reveals how the users who lose two or more ranks respond to the restructuring after taking into account the reduction in post-event activities of the users who lose only one rank. Again, this difference can be solely attributed to the negative impact of user status demotion and reveals that the users who receive the stronger treatment respond much stronger to the restructuring of the incentive system. Hence, we derive our second result. Result II. The online community users who lose two or more ranks reduce their activity levels in post-event activities for which the virtual rewards remain the same by 9% compared to the users who are downgraded by only one rank. Results I and II confirm our research hypothesis and reveal that status demotion has a negative impact on user activity levels. Users who lose one rank reduce their activity levels in both activity groups. More importantly, users who lose two or more ranks and thus receive the stronger treatment reduce their activity levels in both activity groups more than users who lose only one rank. This finding is significant both in statistical and economic terms. 6.5. Robustness checks Although we find support for our research hypotheses, we examine a number of robustness checks in order to demonstrate the robustness of our results. (1) We estimate the model in Eq. (1) again for both dependent variables in absolute values. (2) To explicitly account for potential differences in the average Length of Membership we add the variable Length of Membership to our main model. (3) To check if our results are driven by the creation of the two activity measures we decompose the two dependent variables into four single variables: Ln(Friends), Ln(Links), Ln(Answers), Ln(Questions) and run our main model for each variable, separately. (4) We include the 42 users who held a rank between ‘‘James Watt’’ and ‘‘Albert Einstein’’ before the event and run the main model again. (5) We estimate the main model for different time windows covering two, three, five and six weeks before and after the event. (6) We add the variable Relative Number of Required Status Points. (7) We include the 31 users who did not lose any rank (see Section 5.2.7) to our sample and create a factorial treatment variable with the following three groups: a small group of users who do not lose any rank, a user group who loses one rank and a user group who loses two or more ranks (see Table 11). (8) Finally, we refine the factorial treatment variable from robustness check (7) and split the user group who loses two or more ranks into two user groups, i.e., users who lose two ranks and users who lose three or more ranks. Our main results remain qualitatively unchanged for each robustness check.5 5 The results of these robustness checks are available from the authors on request.

6.6. Limitations Although our findings are overall consistent with the theory, we recognize that our work is not without its limitations. One limitation is that we do not observe a user group who does not receive any treatment in our natural experiment (see Section 5.2.7). Therefore, we need to rely on the assumption that the treatment effect is either negative or positive for both user groups and becomes more pronounced as the treatment is stronger. Without this assumption we are not able to properly identify the impact of status demotion. In line with theory, we are confident that our assumption is a reasonable one to make. Moreover, we are not aware of any alternative explanation grounded in theory which would predict opposing responses for both user groups (i.e., one group increases while the other group decreases their activity levels). Another limitation might be the low R-squared values in the simple DD-design models without control variables (see column (2) in Tables 14 and 15). We recognize that there are plenty of (unobserved) factors (e.g., age, occupation) which influence user activity levels in everyday life (and which are not included in the model). To be more specific, we have 12,892 observations spanning a period of eight weeks, which we explain with only four mean values (i.e., average activity levels of both user groups before and after the event). Intuitively in such a scenario a high R-squared value would be rather surprising. As we do not observe all the factors which drive user behavior we need random variation in our variable of interest to get reliable estimates [4]. In our discussion of the natural experiment (Section 4) we describe where the random variation in our variable of interest comes from and demonstrate the random variation explicitly in a regression in Section 5.2.5. As the natural experiment provides a solid source of exogenous variation, we are confident that our estimates are still useful despite the low R-squared values. Furthermore, the R-squared values in our main models (see column (3) in Tables 14 and 15) which we use for our empirical analysis are substantially higher than for the simple DD-design models.

7. Conclusion A key challenge for online community providers is the development and implementation of features and policies that can foster and sustain user activity levels. Apart from knowing which features and policies positively affect user activity levels, operators should also be aware of the actions which have been shown to adversely affect user contribution behavior. To the best of our knowledge, our paper is the first to investigate how status demotion affects user activity levels in online communities. We find that user status demotion has a statistically and economically significant negative impact on user contribution behavior. These findings are robust and have survived a range of robustness checks. With these results we contribute to the literature on online communities by augmenting the theoretically based knowledge elucidating the key drivers on how or why specific policies and features in online communities are effective or detrimental in stimulating

T. Mutter, D. Kundisch / Computer Networks 75 (2014) 477–490

and sustaining user contribution behavior. In addition, we contribute to the empirical literature on status demotion by analyzing how user status demotion affects user activity levels. We thereby extend the findings by Wagner et al. [41] for hierarchical reward systems which offer non-monetary benefits. While the quantitative estimates from the Q&A community under study may not be directly applicable to other domains, our results are suggestive nevertheless. Previous research in the field of knowledge contribution in online communities has emphasized that user contribution behavior is influenced by idealistic and altruistic factors (e.g., [21,22,40]). It is reasonable to expect to find the negative effect of status demotion to be more pronounced in an environment where individuals are more extrinsically motivated. Thus, our results indicate that status demotion may also have a negative effect on user activity levels in other online communities that offer some type of system to earn and display status, such as Stack Overflow, OpenStudy or Wikipedia. The impact of status demotion is, in our opinion, an exciting and challenging avenue for future research. For instance, future work could analyze the effects of differential user treatments in a community, and whether and how the information about the treatment distribution moderates the effect of status demotion. For example, if all users on the platform had been downgraded by two ranks and been explicitly informed about the change being applied to all users equally, the impact of status demotion might have been weaker or negligible. Moreover, future research could analyze whether status promotion has a positive effect on user activity levels. This could build on a significant body of literature in the field of marketing where such effects are analyzed in the context of loyalty programs (e.g., [41]). Our results have important managerial implications for operators of online communities. If they plan to implement or have already adopted a virtual reward system that reflects user status they should be aware of the negative impact of status demotion on user activity levels. In the future, operators of such communities might face the challenge of restructuring their ranking system (e.g., adding new ranks) and might wish to consider a review of the extrinsic incentive system for their users. When operators restructure the virtual reward system of their online community, they should avoid status demotion. Otherwise the restructuring might negatively affect the motivation of their users and, hence, defeat the purpose of the reward system by significantly reducing rather than motivating user activity levels.

References [1] N.E. Adler, E.S. Epel, G. Castellazzo, J.R. Ickovics, Relationship of subjective and objective social status with psychological and physiological functioning: preliminary data in healthy, white women, Health Psychol. 19 (6) (2000) 586–592. [2] A. Anderson, D. Huttenlocher, J. Kleinberg, J. Leskovec, Steering user behavior with badges, in: Proceedings of the 22nd International Conference on World Wide Web, Rio de Janeiro, 2013.

489

[3] J. Antin, E.F. Churchill, Badges in social media: a social psychological perspective, in: CHI 2011 Gamification Workshop Proceedings, Vancouver, BC, Canada, 2011. [4] J.D. Angrist, J.S. Pischke, Mostly Harmless Econometrics: An Empiricist’s Companion, University Press, Princeton, 2008. [5] J. Berger, B.P. Cohen, M. Zelditch Jr., Status characteristics and social interaction, Am. Sociol. Rev. 37 (3) (1972) 241–255. [6] I. Blohm, J.M. Leimeister, Gamification. Design of IT-based enhancing services for motivational support and behavioral change, Bus. Inform. Syst. Eng. 5 (4) (2013) 275–278. [7] R. Bougie, R. Pieters, M. Zeelenberg, Angry customers don’t come back, they get back: the experience and behavioral implications of anger and dissatisfaction in services, J. Acad. Market. Sci. 31 (4) (2003) 377–393. [8] M. Caliendo, R. Hujer, The microeconometric estimation of treatment effects – an overview, Allgem. Stat. Arch. 90 (1) (2006) 199–215. [9] Y. Chen, F.M. Harper, J. Konstan, S. Xin Li, Social comparisons and contributions to online communities: a field experiment on MovieLens, Am. Econ. Rev. 100 (4) (2010) 1358–1398. [10] D. Constant, L. Sproull, S. Kiesler, The kindness of strangers: the usefulness of electronic weak ties for technical advice, Organ. Sci. 7 (2) (1996) 119–135. [11] X. Drèze, J.C. Nunes, Feeling superior: the impact of loyalty program structure on consumers’ perceptions of status, J. Consum. Res. 35 (6) (2009) 890–905. [12] X. Drèze, J.C. Nunes, Recurring goals and learning: the impact of successful reward attainment on purchase behavior, J. Market. Res. 48 (2) (2011) 268–281. [13] S. Deterding, R. Khaled, L.E. Nacke, D. Dixon, Gamification: toward a definition, in: CHI 2011 Gamification Workshop Proceedings, Vancouver, BC, Canada, 2011. [14] T. Dunning, Natural Experiments in the Social Sciences: A DesignBased Approach, Cambridge University Press, 2012. [15] R. Farzan, J.M. DiMicco, D.R. Millen, B. Brownholtz, W. Geyer, C. Dugan, When the experiment is over: deploying an incentive system to all the users, in: Proceedings of the Symposium on Persuasive Technology, 2008. [16] L. Festinger, A theory of social comparison processes, Hum. Relat. 7 (2) (1954) 117–140. [17] S. Fournier, S. Dobscha, D.G. Mick, Preventing the premature death of relationship marketing, Harvard Bus. Rev. 76 (1) (1998) 42–51. [18] R.P. Gephart, Status degradation and organizational succession: an ethnomethodological approach, Admin. Sci. Quart. 23 (4) (1978) 553–581. [19] J. Hamari, Transforming homo economicus into homo ludens: a field experiment on gamification in a utilitarian peer-to-peer trading service, Elect. Comm. Res. Appl. 12 (4) (2013) 236–245. [20] J. Hamari, J. Koivisto, H. Sarsa, Does gamification work? – a literature review of empirical studies on gamification, in: Proceedings of the 47th Hawaii International Conference on System Sciences, 2014. [21] L.B. Jeppesen, L. Frederiksen, Why do users contribute to firm-hosted user communities? The case of computer controlled music instruments, Organ. Sci. 17 (1) (2006) 45–63. [22] A. Kankanhalli, B. Tan, K.K. Wei, Contributing knowledge to electronic knowledge repositories: an empirical investigation, MIS Quart. 29 (1) (2005) 113–143. [23] R. Kivetz, O. Urminsky, Y. Zheng, The goal-gradient hypothesis resurrected: purchase acceleration, illusionary goal progress, and customer retention, J. Market. Res. 43 (1) (2006) 39–58. [24] P. Kollock, The economies of online cooperation, in: M. Smith, P. Kollock (Eds.), Communities in Cyberspace, Routledge, New York, 1999. [25] J. Lampel, A. Bhalla, The role of status seeking in online communities: giving the gift of experience, J. Comput.-Med. Commun. 12 (2) (2007) 434–455. [26] R.S. Lazarus, Cognition and motivation in emotion, Am. Psychol. 46 (4) (1991) 352–367. [27] K. Ling, G. Beenen, P. Ludford, X. Wang, K. Chang, X. Li, D. Cosley, D. Frankowski, L. Terveen, A.M. Rashid, P. Resnick, R. Kraut, Using social psychology to motivate contributions to online communities, J. Comput.-Med. Commun. 10 (4) (2005). 00–00. [28] M. Ma, R. Agarwal, Through a glass darkly: information technology design, identity verification, and knowledge contribution in online communities, Inform. Syst. Res. 18 (1) (2007) 42–67.

490

T. Mutter, D. Kundisch / Computer Networks 75 (2014) 477–490

[29] J. Marr, S. Thau, Falling from great (and not so great) heights: how initial status position influences performance after status loss, Acad. Manage. J. 57 (1) (2014) 223–248. [30] R.K. Merton, The Matthew effect in science, Science 159 (3810) (1968) 56–63. [31] B.D. Meyer, Natural and quasi-experiments in economics, J. Bus. Econ. Stat. 13 (2) (1995) 151–161. [32] J.M. Podolny, D.J. Phillips, The dynamics of organizational status, Ind. Corp. Change 5 (2) (1996) 453–471. [33] S. Ray, SS. Kim, JG. Morris, The central role of engagement in online communities, Inform. Syst. Res. (2014) 1–19. [34] Y. Ren, F.M. Harper, S. Drenner, L. Terveen, S. Kiesler, J. Riedl, R.E. Kraut, Building member attachment in online communities: applying theories of group identity and interpersonal bonds, MIS Quart. 36 (3) (2012) 841–864. [35] J.A. Roberts, H. Il-Horn, S.A. Slaughter, Understanding the motivations, participation, and performance of open source software developers: a longitudinal study of the apache projects, Manage. Sci. 52 (7) (2006) 984–999. [36] D. Smith, The humiliating organization: the functions and dysfunctions of degradation, in: AV. Iterson, W. Mastenbroek, T. Newton, D. Smith (Eds.), The Civilized Organization: Norbert Elias and the Future of Organization Studies, John Benjamins Publishing, Philadelphia, 2002, pp. 41–57. [37] S. Thye, A status value theory of power in exchange relations, Am. Sociol. Rev. 65 (3) (2000) 407–432. [38] H.M. Trice, J.M. Beyer, Studying organizational cultures through rites and ceremonials, Acad. Manage. Rev. 9 (4) (1984) 653–669. [39] G.S. Van Der Vegt, J.S. Bunderson, A. Oosterhof, Expertness diversity and interpersonal helping in teams: why those who need the most help end up getting the least, Acad. Manage. J. 49 (5) (2006) 877– 893. [40] M. Wasko, S. Faraj, Why should I share? Examining social capital and knowledge contribution in electronic networks of practice, MIS Quart. 29 (1) (2005) 35–57. [41] T. Wagner, T. Hennig-Thurau, T. Rudolph, Does customer demotion jeopardize loyalty?, J Market. 73 (3) (2009) 69–85. [42] J.M. Wooldridge, Econometric Analysis of Cross Section and Panel Data, The MIT Press, Cambridge, 2010. [43] X. Zhang, F. Zhu, Group size and incentives to contribute: a natural experiment at Chinese Wikipedia, Am. Econ. Rev. 101 (4) (2011) 1601–1615.

Tobias Mutter holds a diploma in Economics from the University of Freiburg, Germany. Since 2011 he is a research assistant at the chair of Business Information Systems, Information Management & E-Finance at the University of Paderborn, Germany. His research interests include Economics of IS and Social Networks.

Dennis Kundisch has been professor of Information Management & E-Finance at the University of Paderborn since July 2009. He holds a Master’s degree in Business Administration from the University of Dayton, USA, and a diploma in Business Administration from the University of Augsburg, Germany. He received his PhD and his habilitation from the University of Augsburg in 2002 and 2006, respectively, for publications in the area of e-finance. From 2002 to 2006 he served as Assistant Professor at the Department of Information Systems and Financial Engineering at the University of Augsburg. From 2006 to 2008 he was acting director of the Chair of Information Systems at the University of Freiburg. He was subsequently Visiting Professor at the University of Calgary, Canada and at the University of New South Wales, Australia. After a guest professorship in Information Systems, Value Management & E-Business at the Technical University of Brandenburg, Germany, he joined the University of Paderborn in the 2009 summer term. Between October 2009 and April 2013 he was Vice-Dean of IT and Public Relations at the Faculty of Business Administration and Economics. In May 2013 Dennis Kundisch was appointed chair of the Scientific Committee on Business Information Systems of the German Academic Association for Business Research (VHB). His research interests include business modeling, the economics of IS, e-finance, IT business value, und e-learning.