A computational cognitive modeling approach to understand and design mobile crowdsourcing for campus safety reporting

Author’s Accepted Manuscript A Computational Cognitive Modeling Approach to Understand and Design Mobile Crowdsourcing for Campus Safety Reporting Yun Huang, Corey White, Huichuan Xia, Yang Wang www.elsevier.com/locate/ijhcs

PII: DOI: Reference:

S1071-5819(16)30154-9 http://dx.doi.org/10.1016/j.ijhcs.2016.11.003 YIJHC2087

To appear in: Journal of Human Computer Studies Received date: 26 February 2016 Revised date: 13 November 2016 Accepted date: 22 November 2016 Cite this article as: Yun Huang, Corey White, Huichuan Xia and Yang Wang, A Computational Cognitive Modeling Approach to Understand and Design Mobile Crowdsourcing for Campus Safety Reporting, Journal of Human Computer Studies, http://dx.doi.org/10.1016/j.ijhcs.2016.11.003 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting galley proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

A Computational Cognitive Modeling Approach to Understand and Design Mobile Crowdsourcing for Campus Safety Reporting Yun Huanga , Corey Whiteb , Huichuan Xiaa , Yang Wanga a School

of Information Studies, Syracuse University of Psychology, Syracuse University

b Department

Abstract The under-reporting of public safety incidents is a long-standing issue. In this paper, we propose a computational cognitive modeling approach to understand and design a mobile crowdsourcing system for improving campus safety reporting. In particular, we adopt drift-diffusion models (DDMs) from cognitive psychology to investigate the effect of various factors on users’ reporting tendency for public safety. Our lab experiment and online study show consistent results on how location context impacts people’s reporting decisions. This finding informs the design of a novel location-based nudge mechanism, which is tested in another lab experiment with 84 participants and proved to be effective in changing users’ reporting decisions. Our follow-up interview study further suggests that the influence of people’s mobility patterns (e.g., expected walking distance) could explain why the nudge design is effective. Our work not only informs the design of mobile crowdsourcing for public safety reporting but also demonstrates the value of applying a computational cognitive modeling approach to address HCI research questions more broadly. Keywords: Mobile Crowdsourcing, Cognitive Computational Method, Public Safety, User Contribution, Drift-Diffusion Decision Model, Nudge Mechanism

Email addresses: [email protected] (Yun Huang), [email protected] (Corey White), [email protected] (Huichuan Xia), [email protected] (Yang Wang)

Preprint submitted to Elsevier

December 9, 2016

1. Introduction In the United States, it is required that universities and colleges that belong to the Institution of Higher Education (IHE) collect and publish their full safety report logs to the public in a timely manner, as specified by the Jeanne Clery Disclosure of Campus Security Policy and Campus Crime Statistics Act [1]. Given that university campuses are close-knit communities dominated by temporary residents who are new to the surrounding area, it is important for students and their families to have easy access to campus safety reports. Students also have a strong desire to be more aware of campus safety conditions, as well as to share safety related information with others [2]. Inspired by the rising popularity of mobile crowdsourcing systems [3, 4, 5, 6, 7] and after witnessing the use of a mobile crowdsourcing app to encourage people to report more transportation issues [8], we proposed a mobile crowdsourcing app for campus safety. People can use the app to send safety reports to the department of public safety (DPS) and have the option to share the reports with others (e.g., friends or family members). Meanwhile, the DPS can also send notifications of safety alerts or request information from the public for crime investigations through the application [2]. However, the design of such an application faces several challenges. First, for the proposed system to function successfully, users’ willingness to share is imperative. According to the Bureau of Justice, about half of all victimizations in the general public are not reported to the police [9]. The under-reporting issue amongst college students is even more severe [10, 11]. Prior studies of public safety reporting contributed good knowledge about various factors that impacted victim (e.g., [12, 13]) and witness reporting (e.g., [14, 15]). Since mobile crowdsourcing for safety reporting is a newly emerging technology, we need to examine which factors would impact people’s reporting decisions in the newly proposed environment. In this regard, mobile sensing technologies could enable new design opportunities [16, 17]. For example, diverse information (e.g., user location, mobility patterns, etc.) can be collected along with the safety report, and push notifications can be sent to selected crowds any time [18, 19]. However, little is currently known about how to leverage the mobile platform to address public safety under-reporting issues.

2

To address the above challenges, we took a computational cognitive modeling approach of using the drift-diffusion model (DDM) from cognitive psychology. The DDM of simple choice has been widely employed to analyze choice behavior in decision tasks (e.g., [20]). This model is well suited for studying two-choice decisions, a decision-making scheme that corresponds to the current application design in which users will decide whether or not to report a safety incident. Specifically, the DDM takes choice probabilities (percentage of share responses) and response time as input, and provides measures of response bias and decision evidence as an output given various stimulus contexts. Thus, this model helps us decompose observed decision behavior into relevant psychological constructs as a function of various contextual factors. In particular, our work has made the following contributions. First, we conducted a series of lab experiments, online studies and interviews to identify important factors and to understand their impact on people’s safety incidents. Our lab experiments and online studies showed consistent results on how location context impacts people’s reporting decisions. Second, with a better understanding of the impacting factors, we designed a novel nudge mechanism to tackle the under-reporting issue and proved its effectiveness by running a second experiment with 84 participants. Third, we presented the implementation of the proposed mobile crowdsourcing system for campus public safety. Our follow-up interviews suggested that different mobility patterns influenced people’s reporting decisions and participants’ feedback helped explain why our nudge mechanism was effective. We reflected on our findings and provided insights for designing mobile crowdsourcing for public safety in general. We also discussed how the cognitive modeling approach could be used to address other HCI research questions. In the remainder of this paper, we will cover the following. We will review related work in Section 2, and introduce the DDM model in Section 3. We will present our experiments and online survey study in Section 4. In Section 5, we will show the nudge design and the second experimental results, as well as the follow up interview study. Finally, we reflect on our findings in Section 6.

3

2. Related Work In this section, we review relevant literature on crime reporting issues, a theoretical foundation to explain crime reporting behaviors, existing crime reporting systems, and factors that can motivate users to make contributions to crowdsourcing or online community systems. 2.1. Crime Reporting & the Under-Reporting Issue About half of all victimizations were not reported to the police [9]. Because both victims and witnesses could use our proposed mobile social systems to send safety reports and the proposed community is a college campus, we review the literature from the following three perspectives: victim reporting, witness reporting and campus reporting. Note that related work reviewed on campus reporting did not distinguish victim reporting from witness reporting. 2.1.1. Victim Reporting Several factors were found to impact victims’ reporting behaviors, such as the seriousness of the crime, demographics, household victimization, and the perceived costs and benefits of reporting. For instance, a meta-analysis was conducted on the crime reporting stats and behaviors around the world [12]. They found that the seriousness of the crime, regardless of if it was attempted or completed, caused injury, or resulted in financial loss, was the most salient positive predictor of reporting. Demographics were also considered a factor in influencing people’s reporting behaviors. For instance, women were found to be consistently more likely to report crime than men, regardless of ethnicity [21]. In developed countries, age was found to be a strong predictor; victims under the age of 35 were less likely to report crime compared to those older than 35 [13]. More specifically, senior citizens above the age of 60 are most likely to report a crime [12]. On the other hand, in developing countries, household victimization was found to be the strongest factor that positively correlated with people’s under-reporting behavior, with the seriousness of the crime being the second strongest factor [22]. The race bias between offender and victim also played a role. It was found that black vic-

4

tims offended by whites were less likely to report than white victims offended by either whites or blacks. [23]. Additionally, the perceived costs of the individual (e.g., time effort, fear, or personal judgment); the benefits (e.g., altruistic motives, incapacitation effect), and belief in retribution to the offenders were also considered as part of a cost-and-benefit decision [24]. Relationships with local law enforcement agencies affected the reporting rate as well. For instance, it was indicated that the community-oriented policing (COP) could influence the community member’s crime reporting behaviors. If a large portion of officers had already been trained in COP, victims were more willing to report to police. If officers were less trained in COP, the victims demonstrated a preference for non-police notification or non-reporting [25]. 2.1.2. Witness Reporting It is noteworthy that the majority of prior research on public safety reporting focused on the “victim’s crime reporting,” and the “witness’s crime reporting” was relatively less investigated [14]. Mawby proposed a model that summarized witness involvement in crimes from four aspects, including offender characteristics, witness characteristics, victim/offense characteristics, and situational variables. In terms of the effects of offender characteristics, it was found that black offenders were more likely to be reported than whites [26] and multiple offenders were more frequently reported than single offenders [27]. Regarding witness characteristics, gender bias existed in different types of crimes. For instance, it was found that female witnesses were more prone to report shoplifting and fraud [26, 28] while male witnesses were more likely to report assault [29]. In Mawby’s model, the victim/offense characteristics referred to the relationship between the witness and the victim. It was found that witnesses would intervene and more readily report if they knew the victim [27]. Finally, the situational variables referred to the environment in which the witnesses situate. For instance, with the presence of other witnesses, the witness would feel reluctant to report, unless the other witnesses were perceived as less helpful or the witness was already familiar with the other witnesses as well as the surrounding environment[15, 14]. In addition, witnesses preferred anonymity. This was possibly due to the fear of re-

5

taliation and embarrassment [30, 15] as well as a reluctance to get involved with police investigations or court testimonies [31]. Prior research has proposed that anonymity might change people’s sharing decisions. Anonymity could be regarded as an aspect of de-individuation [32], which is preferred in an individual’s crime reporting, yet not significantly correlated with an individual’s actual reporting behavior [15]. However, it should be noted that an individual’s perception of anonymity in reporting could be subjective and incomplete [15]. 2.1.3. Campus Reporting To monitor or reduce on-campus crimes, it was suggested to report not only violent crimes, but also all other types of incidents, such as theft, that may be more prevalent among college students [33, 11]. Some scholars attributed the under-reporting issue to not clearly differentiating various types of crimes on campus [34]. Additionally, college-age people seemed to feel social obligation to report crimes to the police, which may have also contributed to the under-reporting on campus [35]. In our prior work, we conducted a log analysis using a full year’s log of public safety reports. For each report, the log data contained when and where the incident happened, the description of the incident, and when the report was sent to the DPS [2]. It allowed us to examine relevant patterns of students’ reporting behaviors, specifically pertaining to where, when, and how soon the community would report safety incidents. We found more reports were sent from residential areas, i.e., where people live, than commercial areas (including work, school and shopping areas). Additionally, most of the reports were sent around midnight (11pm - 3am), or late afternoon (2pm - 5:45pm) [2]. 2.2. Theoretical Foundation for Crime Reporting Past research has proposed several theories to explain the reasons behind crime reporting behavior. Goudriaan et al. proposed a theoretical framework that consisted of two dimensions: Situation versus Context, and Normative versus Rational [36]. The situation referred exclusively to the immediate crime scene or the face-to-face interaction between the victim and the offender, e.g., a college student being confronted

6

by a robber. The context included any social aspects of the location in which a crime occurs outside the immediate face-to-face interaction, e.g., the relationship between community members and police officials, and the level of social cohesion in the community [36]. The second dimension of this theoretical framework included rationality and norms. Rationality referred to the cost-benefit calculations that victims engage in on the basis of the expected expenditures (costs) and returns (benefits) of reporting a crime. For example, the cost of reporting to the police could be further intimidation and injury by the offender, whereas the possible benefit would be the chance to deter offenders and reduce the chance of additional offenses [37]. On the other hand, normative reporting behavior is affected by the victim’s belief regarding the appropriateness of reporting in a particular situation. For instance, some people would report to police even the smallest incident, while others are more concerned with bigger threats [36]. Empirical research is needed to investigate the ways in which these theoretically identified factors affect individuals’ decision-making processes for reporting public safety incidents. Inspired by the results of our previous finding that indicated people sent more reports at certain locations and times [2], in this paper, we use a computational cognitive modeling approach to analyze choice data and response times to better understand how factors like location and timing influence users’ willingness to share. Our study on how any impacting factors could drive people to make sharing decisions faster can potentially contribute to the theoretical foundation with another orthogonal dimension, e.g. the response speed of reporting. The use of computational models of decision making to analyze the data provides a rigorous, comprehensive framework for understanding choice behavior in this crowd-sourcing environment. 2.3. Systems for Reporting Public Safety Incidents There are different ways that people can report safety incidents. The methods of reporting (and their affordances) may also affect people’s reporting behavior. Interactivity seems to be an important factor in enhancing reporting efficiency. Researchers found that crime report frequency increased significantly when reporting was enabled by a computer interactive system as compared to a traditional telephonic mode [38]. Additionally, an Internet-based crime reporting system named “i-recall,” which emu7

lates a police officer conducting a cognitive interview (CI), was designed to improve the completeness and accuracy of a crime report as compared with the non-interactive textbox recording system [39]. This was an initial step in developing an Internet-based on-campus crime reporting system [40]. By comparing it to traditional voicemail reporting, researchers found that the Internet-based crime reporting system was more useful and user-friendly for college students [40]. WikiCrimes was a web tool that also allowed people to report public safety issues, and used a reputation model to address the tension between user participation and the quality of contributed information [41]. Meanwhile, the scholarship in crisis informatics have shown how people have been increasingly using social media to report information during crises such as natural disasters and mass shootings [42]. CrowdMonitor was designed and prototyped [43] for collecting and analyzing participatory sensing data from a mobile crisis application and enhancing the collaboration between emergency services and citizens via a web platform and social media. Very few mobile systems were designed and released. For example, CrowdSafe allowed people to send reports of crime incidents and provided safety alerts to community members to circumvent scenes of public safety violations on mobile devices [44]. Some systems were designed to address specific needs. For instance, a recently proposed mobile crowdsourcing system, called CoSMiC, leveraged observations from nearby people to help find missing children [45]. Using the system, parents didn’t need to ask for help in person and could make inquiries of people found in nearby crowds who were using the system. The parents could also locate their missing child more quickly and effectively with the landmarks found on the missing child’s trajectories, which were provided by the crowds [45]. A framework was proposed for mobile applications of natural disaster management, e.g., floods or wildfires. Four major components were suggested to be built in these systems including: an application for disaster reporting; a server for information gathering and warning; a database for information storage; and a web server for disaster information management [46]. A smartphone application called “BlaCom” was proposed to manage communication between the emergency response organization and the general public in the event of an energy blackout and/or if the mobile network was overloaded [47]. However, the mo8

bile systems described above are all currently in the prototype phase and the number of mobile systems that have been actually deployed for real-world use is limited. Liu proposed a framework to help design crowdsourcing tasks for emergency management [48]. The framework consists of six dimensions: (1) why, which is to identify the problem(s) and determine a crowd task to solve the problem(s); (2) who, which is to identify the types of crowd workers and their expertise; (3) what, which is to identify the interaction flow among the crowd; (4) when, which is to identify the temporal aspects of crowdsourcing tasks, such as prediction, warning, crisis detection, response and recovery, etc. in relation to the lifecycle of the crisis; (5) where, which is to identify the location of the crisis, crowds, and tasks; and (6) how, which is to identify social, technological, organizational, and policy (STOP) interfaces. Our work particularly contributes to a better understanding of the when and where dimensions. 2.4. Motivators for User Contribution Prior scholarship has shown how intrinsic motivators such as social contact, optimal challenge, mastery, and competition, as well as extrinsic motivators such as awards can be used to encourage people to contribute to collective intelligence systems [49], online communities, and/or crowdsourcing environments such as Wikipedia [50]. Similar motivational factors can be found from many mobile crowdsourcing and participatory sensing systems, e.g. [3, 51, 4, 52]. However, it is unclear whether these existing mechanisms can be effective in crowdsourcing public safety related data given its sensitive nature and the potential risks of reporting. Furthermore, while some intrinsic feelings such as fun (desire for peer companionship) and efficacy (desire to influence or obtain power) have been extensively studied to motivate people to make contributions, other intrinsic feelings such as compassion (desire to improve society), love (desire to raise own children), comfort (desire to avoid anxiety and fear) and wonder (desire for knowledge) are under-studied for that purpose [50]. Research in behavioral economics and decision making shows that people often have cognitive and behavioral biases when making decisions [53]. One such bias is status quo (or default) bias [54]. In the context of crowdsourcing safety reports, peo9

ple have a tendency towards not reporting/sharing partly because that is the status quo (default). As such, the status quo bias might contribute to the issue of under-sharing in this context. Nudges have been proposed as soft-paternalistic interventions to help people mitigate behavioral biases that may affect their decision making. These interventions are designed to “nudge” (rather than force) people towards certain behaviors that people desire or that are publicly desired, but are difficult to follow [53]. Our work seeks to understand the motivational factors that could impact users’ safety reporting decisions, and explores an effective nudge design that could help increase user contributions to safety reports.

3. Modeling Sharing Decisions of Campus Safety Reports In our prior work, we proposed a mobile crowdsourcing application that allows campus residents, especially students, to share safety reports [2]. The system allows people to have easy access to the safety related logs, respond to others’ safety requests in a more timely manner, and have a better general awareness of campus safety. The most common design of similar systems usually includes quick dial or message buttons, with time and location automatically recorded. Users can also manually specify the location of the incident in cases where they may be at a different location when reporting. There are many important issues to consider in the crowdsourced data; in this paper, we focus on the very first questions, namely, what factors impact people’s decisions of using the mobile system to report? How will the issue of anonymity affect their reporting/sharing behavior? Better understanding of these questions will provide us with insights on how to design the user interaction of the system that would increase users’ willingness to share safety reports, holding all other conditions constant. At its core, using the application involves a simple, binary choice (to share or not to share) that needs to be made after an incident happens. Computational modeling techniques from cognitive psychology can be employed to explore how factors like time and location influence this choice process. This type of two-choice decision has been successfully accounted for by a class of decision models that describe how the decision unfolds over time. Drift-diffusion model (DDM)

10

Figure 1: DDM representation of decision behavior. A) Noisy evidence is accumulated over time until a boundary is reached, indicating which choice is made. B) Changing the starting point leads to bias for one choice over the other, e.g. increasing preference to share information even before incident has been witnessed. C) Changing the drift rate leads to bias in how the incident is evaluated with respect to the two choices, e.g. increasing preference to evaluate incident as worth sharing.

of simple choice has been ubiquitously employed to understand decision processes across a range of disciplines, including psychology [55], economics [56], and neuroscience [20]. The models posit that when individuals make decisions, noisy (imperfect) evidence is sampled and accumulated over time. Once the accumulated evidence sufficiently favors one response over the other, the decision is made. The models are mathematically specified to make specific predictions about how different decision components affect the duration of the decision and which choice is made. Importantly, this process can be inverted, whereby the decision behavior observed in experimental settings can be modeled with the DDM to identify the contributions of the different decision components. The use of DDM modeling will help disentangle which decision mechanisms are affected by time and location information when people choose whether or not to share safety related information. The DDM framework has two distinct mechanisms through which decision preference can be influenced or biased: (1) changing how the information under consideration is evaluated, and (2) changing which choice is favored overall. The former affects which evidence is provided by the stimulus to drive the decision (e.g., information about a crime), whereas the latter affects how much of that evidence is required for one response relative to the other. Recent work [20] has shown that these two mechanisms can be independently influenced and have distinct effects on decision behavior; changing the evaluation of the stimulus affects preferences in both fast and slow decisions, whereas changing which choice is favored affects only fast decisions. This allows the 11

mechanisms to be dissociated in experimental data by examining the distribution of response times in different experimental conditions. In the experiments, individuals will evaluate information about a witnessed incident and accumulate evidence to decide whether or not they wish to share the information. This process is shown in the DDM framework in Figure 1, where evidence is accumulated over time until one of the two boundaries is reached, signaling a commitment to that choice. Note that to illustrate the model, we simplify the decisions to be binary, either share or ignore. Increasing an individual’s willingness to share can be accomplished through the two mechanisms described above that correspond to the starting point and the drift rate in the DDM (Figure 1). The starting point of the decision process relative to the two boundaries indexes an overall bias where one of the two choices is favored. If the starting point is closer to the “share” boundary (Figure 1. B), the individual is biased for that choice and requires less evidence to reach it compared to the “ignore” boundary. The drift rate, in contrast, indexes the information or evidence about the incident that drives the accumulation process. If the drift rate strongly favors the “share” response (Figure 1. C), the individual is more likely to evaluate the incident as worth sharing. The response time, however, is not used to calculate or compare how soon the participants will report in a real situation. In fact, the quicker the participant selects a ”share” decision (with less response time), the stronger it indicates that the participant is more willing to report the crime under the given condition (e.g., in a specific time, location or crime level). It shows users’ instinctive reaction to the situation.

4. Understanding Impacting Factors of Reporting Behaviors We leveraged the DDM framework and behavioral experiments to understand how different factors influence the likelihood of sharing behavior in the proposed application. Experiments were designed to mimic the user experience in the application. 4.1. Controlled Lab Design and Experimental Results According to our prior log analysis of public safety reports provided by the Department of Public Safety (DPS), three essential factors were included in a report, i.e., time 12

of day, location of the incident, and the incident type [2]. Thus, we started examining the impact of these three factors in our study. Our experiment in this study followed a within-subject design. Each participant was first given instructions about the crowdsourcing application and its purpose. They were then given different scenarios of safety incidents and asked to choose whether they would share the incident through the application. Each scenario included a time of day, a location, and the type of incident. To conduct the experiments, we set the levels of each factor by consulting our log analysis results. For example, for the location factor, more reports were sent from residential areas than commercial areas so the residential areas were defined as the home level [2]. Considering commercial areas could include work/school, which is more professional, and parks or shopping areas, which are more entertaining or social, we defined two levels for the commercial areas. Eventually, a total of three locations were defined to initiate the study. Put simply, participants will be asked to consider whether to report incidents that occur near home, near work/school, or in a public park. For the time of day, because late night and late afternoon revealed some interesting patterns from our log analysis [2], we set up three levels for each time. Namely, in the lab experiment, we presented to participants that the incident could occur during work/school hours, after work/school, or late night (after 9pm). Finally, for the type of the incident, because there were a wide variety of incidents reported, ranging from car thefts to public urination, or drug deals to shootings [34], we grouped different incidents to three main levels. Namely, we presented incidents that could be either drug use or vandalism (low severity), robbery or theft (medium severity), or assault or gunshots (high severity). These incidents were chosen to maximize the range of severity. As discussed above, each factor has three levels. Participants were given 27 trials per list (1 trial for each of the 3x3x3 above), and each scenario was presented in random order 10 times for a total of 270 trials with a short break in between each list. For each trial, the time of day was presented on the screen for 3 seconds, then the location was added for 3 seconds, and then the crime was shown. This presentation scheme was based on the idea that information related to the time of day and location typically 13

precedes the occurrence of an incident, thus was presented first on each trial. As soon as the crime information was presented on the screen, the participants were also instructed to indicate their decision to “share” or “ignore” quickly and accurately by using the ”/” and ”z” keys on the keyboard (button mapping was randomized across participants). Participants were carefully instructed not to make their actual decision until the last piece of information (type of crime) was presented on the screen. The choice and response time were recorded for each decision. We ran this experiment in a computer lab that was equipped with Matlab and the Psychtoolbox package that enabled accurate collection of response times from the tasks. Participants were recruited via email from a participant pool for psychological experiments. They gave consent before participating in the study and were compensated $10 for their participation in the 50-minute experiment. There were a total of 30 participants, including 17 females, and the age range was 20-28 years old. Most of the participants were undergrad and graduate students. The study in its entirety was reviewed and approved by the IRB in advance. The DDM was fit to each participant’s data (see [20], for details of the fitting procedure) with standard parameters for (1) starting point (response bias), and (2) drift rate (decision evidence) for each incident type. Parameter search was conducted using a SIMPLEX [57] routine to make the model best fit the data [58]. The data entered into the routine were the accuracy values, number of observations, and RT distribution shape for share and ignore responses. RT distribution shape was represented by taking the .1, .3, .5 (median), .7, and .9 quantiles of the RT distribution. The starting point (response bias) was calculated relative to the caution parameter, so it gave a percentage of the overall distance. Thus all the behavioral data were used in estimating the DDM parameters. To investigate the effects of the time of day factor, conditions were collapsed across the location factor and fitted with the model. The converse was done to investigate effects of location. The main focus of the analyses was on whether and how the factors of time of day, location, and crime severity affected the willingness to share. The data of interest were the proportion of “share” responses for a factor, and the response times (RT) for those decisions. Bias scores (in milliseconds, from - 200 to 200 ms, or .2 seconds) 14

were calculated for each condition as the RTs for ignore responses minus the RTs for share responses. Thus a positive bias score indicates that participants make a share decision quicker than ignore responses for that condition. The general reasoning behind these analyses is that a factor that encourages sharing would lead to quicker and more probable share responses. 4.1.1. Prediction of Sharing Decisions One-way within-subject ANOVAs were performed on each of the measures separately for each experimental condition (crime type, location, time of day). Post hoc simple comparisons (paired t-tests) were then performed to compare different levels of each factor. The probability of sharing for each factor is shown in Figure 2(A). For the factor of crime, the ANOVA showed the main effect of the crime type (p < .001), and post hoc comparisons showed that participants were much more likely to share the more severe crimes, with robbery shared more than vandalism (p < .001, Bonferroni corrected) and assault shared more than robbery (p < .001, Bonferroni corrected). Time of day (collapsed across the other factors) had no effect on the probability of sharing (all p’s > .4), but location did. The ANOVA for location did not show a main effect (p = .132), but post hoc comparisons revealed that participants were significantly more willing to share if the incident occurred near home (75%) than near work (70%, p = .008, Bonferroni corrected) or a public park (70%, p = .004, Bonferroni corrected). In addition to the % share rates analyzed above, bias can be measured by noting when share responses were faster than ignore (i.e., don’t share) responses (see [20]). Bias RT scores were calculated by taking the difference in RT between share and ignore responses for each participant and condition. These scores are shown in Figure2B, with values above 0 indicating faster share responses than ignore responses (and vice versa). For the bias scores, Figure 2(B) shows that participants made sharing decisions more quickly and were more willing to share at different times and locations. In particular, there was a significant main effect of time of day (p = .021) and post hoc comparisons showed late night incidents leading to faster share responses relative to midday incidents (p = .006, Bonferroni corrected), and a nonsignificant trend for faster share responses relative to evening incidents (p = .06, Bonferroni corrected). Location

15

(A)

(B)

Figure 2: (A) A comparison of the probability of sharing for each factor, (B) RTs are the response time for making one decision. Bias scores were calculated for each condition as the RTs for ignore responses minus the RTs for share responses.

also showed a significant bias main effect, with incidents near home showing faster share responses relative to incidents near work (p = .043, Bonferroni corrected), but not relative to a public park (p = .14, Bonferroni corrected). Overall these results show that the factors of time and location can significantly influence the willingness and speed with which people decide to share information through a reporting app. Participants showed a relative bias to share for late night incidents, though this only affected the speed of their responses and not the probability of sharing. They also showed a robust bias to share for incidents near the home, both in terms of response speed and probability of sharing the incident. 4.1.2. DDM Parameters The above results suggest that certain locations and times of day can significantly affect a user’s willingness to report, but they do not show whether there is an a priori bias for one response over the other. We further fit experimental data (response times and choice probabilities) in the DDM to estimate different decision components for each participant/condition. This allows the data to be decomposed into two distinct processes that could affect the willingness to share: one which affects where the decision starts and thus how much evi16

dence is needed for each response (Fig 1, panel B), and another which affects the drift rate or decision evidence (Fig 1, panel C). Thus the modeling permits determination of how factors like time of day, location, and crime severity affect these two decision components. Of interest are the two components that can affect the overall preference for share or ignore: response bias and decision evidence. Figure 3 (A) shows response bias for the factors of time and location. The response bias can range from 0 (always choose to ignore) to 1 (always choose to share), with .5 halfway between the two decision boundaries (no bias). For example, a value of .65 means that they are closer to the the top response (share) when they begin to gather evidence for the response. This reflects an a priori bias for that response; in essence, it would suggest that the participant is leaning towards the share response after presented with the time or location information, but before they identify what type of crime was committed. The response bias ratios in Figure 3 (A) show that responses for “work” conditions were unbiased (around .5), but for “home” conditions they were biased toward the share response (around .6), meaning participants needed about 20% less evidence for the share response than for the ignore response. For the location factor, a repeated measures ANOVA showed a main effect of location (p = .018). Post hoc comparisons showed the bias to share for “home” was significantly greater than for “work” (p = .042, Bonferroni corrected) and “park” (p = .031, Bonferroni corrected). For the time of day factor, the ANOVA showed a significant main effect (p = .046), and post hoc comparisons showed that only the late night scenario led to a significant bias in favor of sharing the incident (p< .05, paired t-test). Figure 3 (B) shows the decision evidence, separated by incident and time and location. A drift rate (decision evidence) of 0 means that the person is indifferent about sharing or ignoring that incident, which would correspond to sharing 50% of the time for those incidents. Positive value means they were more likely to share than ignore, and negative value means the opposite. And finally, larger absolute values mean stronger evidence for that choice. Separate two-way repeated measures ANOVAs were performed on drift rates; the 17

(A) Figure 3:

(B)

(A) Response bias from the DDM analysis (higher values indicate more bias to share), (B)

Decision evidence from the DDM model analysis (positive values indicate that the incident is worth sharing).

first had incident type (vandalism, robbery, assault) and time of day (midday, evening, late night) as within-subject factors and the second had incident type and location (home, work, park) as within-subject factors. First, there were strong main effects of incident type (p < .001), with the decision evidence tracking the crime severity in the expected manner: assault provided the strongest evidence for sharing, followed by robbery and finally vandalism (all p’s < .01, Bonferroni corrected). Second and more importantly, there were no main effects or interactions with time of day or location (p0 s > .4). Neither time of day nor location had a significant effect on decision evidence, as the evidence values did not significantly differ across these factors (all p’s > .3, Bonferroni corrected). More specifically, assault at a park has a drift rate of +.25, which means that this type of incident is evaluated as strongly (importantly) worth sharing, corresponding to about 95% share, 5% ignore. Robbery incidents at different times and locations all have drift rates of at least +.1, which means that robbery is evaluated as at least weakly worth sharing, corresponding to about 70% share, 30% ignore. Vandalism (a low-severity crime) incidents at different times and locations all have drift rates that are less than -.1, suggesting vandalism is evaluated as at least weakly worth ignoring, corresponding to 30% share, and 70% ignore. But in contrast with the response bias measure in Figure3a, none of the decision evidence measures differed as a function of

18

location or time of day. 4.2. MTurk Study to Verify the Results of the Controlled Lab Study To study how anonymity influences people’s willingness to share, we replicated the general design of the lab study on the online experimental system, MTurk. We decided that the idea of anonymously sharing would be more salient for participants who perform the task at their own computer, away from the researchers, as compared to those in the lab that have had face-to-face contact with the person collecting the data. We conducted a similar study on Amazon Mechanical Turk (MTurk) with a much larger sample size, 230 participants, each only taking two iterations of the 27 scenarios. The MTurk study was modeled after the lab experiment with a few differences. The main difference was the addition of a third answer/decision option (”share anonymously”) to each scenario in order to understand whether and how anonymity might affect people’s sharing decision-making. In addition, since logging how long it took each participant to make a decision turned out to be tricky in MTurk, we did not implement this feature and thus did not have the response time information for each decision. The MTurk study proceeded as follows. First, we started by asking the MTurk users an open-ended question: “In a sentence or two, please tell us under what circumstances you would report public safety incidents.” Then we used exactly the same wording from the lab experiment in presenting two iterations of the 27 different scenarios. All three types of information (i.e., time of day, location, and type of incident) and the decision choices of a scenario were presented at the same time. Finally, we asked participants a number of questions about their demographics such as age and gender. We recruited MTurk users who were from the US and had an acceptance rate of at least 95%. We also checked the answers of the open-ended questions as a way to filter out participants who did not pay attention to the study (e.g., by entering random text). We did not find any participants who answered the survey carelessly. It took participants about 10 minutes to finish this MTurk study and we offered 50 cents to each participant who finished the study. 57% of the MTurk participants were male, and 43% were female. Their ages ranged from 18 to 73 (mean=35.5, sd=12.5). Participants came from a wide variety of backgrounds such as IT, education, engineering, art, business, 19

(A) Willingness to share.

(B) Willingness to share with identities. Figure 4: MTurk Results.

service, self-employed, and unemployed. 4.2.1. MTurk Results In terms of overall willingness to share, regardless of anonymity, the type of crime and location of crime had significant influence. For location, participants were more willing to share when the crime happened near home or at a public park compared to at work (both p’s < .001, Bonferroni corrected, see Figure 4 (A)), but there was no significant difference in sharing between home and public park (p = .09, Bonferroni corrected). For crime type, participants were more willing to share the more severe crimes (robbery and assault) compared to the minor crime of vandalism or drug use (both p’s < .001, Bonferroni corrected). The results also showed that robbery was associated with greater sharing than assault (p = .003, Bonferroni corrected). The factor of time of day did not have a significant effect on willingness to share.

20

We further broke down the responses in the MTurk study to assess whether the factors of time, location, and crime type had an influence on the willingness to report with or without identification. Figure 4 (B) shows the percentage of “share” responses that were given with identification (with anonymous sharing responses given as 1 minus the percentages in the figure). The results were broadly consistent with the overall sharing data in the lab study. Time of day did not have a significant influence, but location and crime type did. A one-way ANOVA showed a main effect of location (p < .01). Participants were more willing to share safety reports with their identifiable information for home and public park locations compared to work location (both p’s < .001, bonferroni corrected), but there was no significant difference between home and public park (p = .105, Bonferroni corrected). The ANOVA also showed a main effect of crime type (p < .001). We interestingly found that participants were more likely to share with identification for robbery/burglary crimes compared to vandalism/drug use and assault (both p’s < .003, Bonferroni corrected). 4.2.2. Participant Feedback All participants provided meaningful responses to our open-ended questions, which indicated that questions regarding public safety reporting are important to them or of interest. We coded the open-ended question “under what circumstances would you report public safety incidents” in the MTurk study. We conducted a thematic analysis commonly used in qualitative studies [59] and followed an iterative coding process. First, two co-authors read through all the responses to the open-ended questions and tried to identify recurring patterns of the responses. They used Atlas.ti [60] to manually code the responses independently and each generated an initial codebook. Next they discussed the codes to reach an agreement on the codes. Afterward, they would finish coding the rest of the responses using the agreed-upon codebook. When new themes emerged, the two co-authors discussed and updated the codebook, until there were no more new themes found. The inter-coder agreement was above 90%. We summarized the responses as follows. First, 131 responses can be classified to altruism, which is a major reason for people to report public safety incidents. Example responses included, “I would report any incidents that cause harm or the threat of

21

harm to individuals in a community,” while another wrote, “I would report something if someone was in danger or if something was happening that went against my beliefs.” Second, 119 responses outlined conditions in which they would report or share general public safety incidents. For example, participants wrote, “I don’t know. Probably if I thought something was life-threatening and nobody else had reported it,” and “I would report public safety incidents when it looked like there was real danger to people’s life and limb, but then only if it didn’t look like anyone else was reporting the same thing.” However, there were also participants who stated that they would unconditionally report under all circumstances, such as “I would try to report public safety incidents 100% of the time when there is a chance of bodily harm or injury to anyone.” Third, anonymity did not seem to be a significant issue for the participants when they first came to this survey; only one of them mentioned it as a premise to report. The participant wrote, “I would report if I remained anonymous and was sure that what I was reporting was beneficial.” 4.3. Reflection on the Impacting Factors Results from our lab experiment showed that both the crime type and incident location significantly affected participants’ propensity to report a public safety incident. This is perhaps not surprising because people tend to pay more attention to more severe crime incidents. Overall findings from the two studies were largely consistent, showing that location and crime type, but not time of day, influenced users’ tendency to share. However, in contrast to the lab study, participants in the MTurk study were slightly (but significantly) more likely to share with identification for robbery/burglary crimes compared to vandalism/drug use and assault. This was an unexpected result. One putative explanation is that there is an interaction between crime severity and the potential for personal harm, as touched on above. That is, the more severe crimes of robbery and assault might be associated with a greater willingness to share with identification. However, because assault is the only crime associated with physical harm we listed in the study, participants could have been more reluctant to share their identification based on fear of retaliation. Another reason may be potentially due to the specific focus on anonymity 22

in the MTurk study that was not present in the lab study. That is, because anonymity was highlighted in the MTurk study, participants might have been more sensitive to the potential for retribution after reporting a crime, which would be more salient for crimes involving physical harm.

5. Design and Evaluation of Nudging Messages Both the controlled lab experiments and the MTurk study showed location matters. This is intuitive because people may pay more attention to incidents that would affect themselves and their family members, so home is more important than other locations. The importance of this finding is that we can exploit this self-interest to our advantage, i.e., encouraging people to report. For instance, the application can make the location factor salient in the reporting interface, which can be enabled by location-based mobile applications. 5.1. Nudging Messages Showing Relative Distances to Home Drawing from the literature from behavioral economics, we designed a novel mechanism to nudge people in sharing safety reports. To help address the issue of under-sharing (and the status quo bias as a potential cause of under-sharing), we designed a nudging message which shows the relative distance between the safety incident and the user’s home, based on our earlier results on the importance of location in people’s sharing decisions. The message does not force people to share, but it might nudge people to report a safety incident if it is close to their home. Figure 5 shows two designs where Figure 5 (S1) does not provide any additional information about user’s current location, and Figure 5 (S2) pops up a message showing the relative location of the user (incident) to his/her home (assuming he/she is willing to tag an area as their living area in the app). 5.2. Controlled Lab Experiment Using the Nudging Messages To evaluate the effectiveness of the nudge design, we replaced the location screen in the first controlled lab study with the nudging messages. Since the goal of our 23

(S1) No Message

(S2) Location Message

Figure 5: Design of Nudging Messages.

study was to assess whether location is an impacting factor rather than determining the threshold values, in the experiment, we selected four values that would be less ambiguous for people to perceive the closeness. In addition to the two mock-ups shown in Figure 5, we added two mock-ups by replacing (S2)’s pop-up message with 1 mile and 5 mile distances, respectively. In the new experiment, we replaced the location screen in the first controlled lab study with these four mock-up screens. Instead of showing different location options, we randomly showed one of these location screens. We replicated the rest of the lab experiment design exactly as we had set up in the first lab study. All the other con-

Figure 6: Results of the lab experiment on the nudging messages. (1) crime type: less serious crimes (e.g., vandalism) are associated with smaller percentages of share decisions; (2) people are more likely to report safety incidents that are closer to home; and (3) people use less time in deciding whether to report safety incidents that are closer to home.

24

figurations, e.g., bias scores in milliseconds, RT distribution, etc., remained the same as in the first lab study. This study was conducted throughout one academic semester. Eventually, 84 participants, including 48 females and 36 males, finished the study. This study yields promising results regarding the nudging message. More specifically, sharing tendency and response time are presented in Figure 6, where the 2nd and 3rd plot show the percentage of share and RT differences compared to the “no distance” condition. First, people are more willing to share the more severe crimes, which is consistent with the results of the first lab experiment. Second, compared to the condition where no distance information is shown, people are more likely to report safety incidents that are closer to home (1-mile distance had the highest sharing/reporting percentage). People also make sharing decisions faster when considering safety incidents that are closer to home. A significant difference was found between “1 mile from home” and no location information (p < .05).

(A) Response Bias

(B) Decision Evidence

Figure 7: The impact of location-based nudge messages on response bias and decision evidence.

According to Figure 7(A), the result of showing a distance of 1 mile away from home is above the sharing point, meaning, knowing the distance was 1 mile from home led to significant bias to share (compared to all other conditions). Compared to having no location information, knowing that they were 10 miles from home actually made 25

them have more of a response bias against sharing. Also, knowing when the distance was 5 miles from home had a similar response bias as having no information (i.e., more biased to share compared to 10 miles). One interpretation could be that with no information you might be close to home, but knowing you are 10 miles away, you know you are not close to home, so you are biased not to share. These results suggest that being far from home induces response bias against sharing, but the closer you get to home the more response bias you have towards sharing. According to Figure 7(B), evidence for sharing was greater for assault than burglary, and greater for burglary than vandalism (this is consistent with the results from our other study). However, interestingly, the location information from the screen shot actually affected the decision evidence (in contrast to our first study with just words on the screen). More specifically, there was no difference in decision evidence between no location and a location of 10 miles from home, but a distance of 5 miles from home led to an increase in decision evidence to share, and a distance of 1 mile from home led to an even bigger increase in decision evidence to share. Another significant finding is that for the weakest crime (i.e., vandalism/drug use), the decision evidence favored ignoring over sharing unless they were very close to home (i.e., 1 mile), in which case the evidence switches from ignoring to sharing, as shown in Figure 7(B). In summary, the results of DDM parameters indicate that the nudging message affects both response bias and decision evidence, with the general pattern being that the closer a user is to home, the more biased the user is to share and the greater the evidence that the user does share the crime. 5.3. Comparing and Reflecting on Two Lab Study Results Comparing the results of the two lab experiments suggests that: a) the nudging message has a strong effect on people’s willingness to share; b) compared with the “words on a screen” from the first experiment (e.g., home, work, park), the nudging message affects both response bias and decision evidence (in other words, when the mobile system shows the users that they are closer to home, it influences both their 26

(A) sample drawing #1

(B) sample drawing #2

(C) App Screenshot

Figure 8: Sample drawings from the participants regarding the areas or routes that they are interested in receiving safety alerts, and a screenshot of the Android app that has two circled areas for notifications.

overall willingness or bias to share and how serious or important they think the crime is, i.e., the decision evidence); c) adding the nudging message and being closer to home can turn crimes with low severity (e.g., vandalism) from “not serious enough to share” to “serious enough to share.” 5.4. Understanding User Defined Notification Areas Since the nudging messages were effective in the lab experiments, we implemented this design in a mobile crowdsourcing app for our campus safety [61]. Figure 8 (C) shows a screen shot of the application, where users can draw circles on a map to define the notification areas for which they are interested in receiving safety alerts. In this example, one circle can be for home, and the other is for work. Both of them are of similar sizes. One way to allow users to specify their own notification area can be as simple as taking their home or work address as the center location, then allowing them to adjust the radius of the circles to define the notification area. Based on the results of the second lab experiment on nudge messages, we expected that users would be more likely to report safety incidents if they are within their self-defined notification areas. To explore this question, we conducted a preliminary interview study in which we asked participants to draw their notification areas on a map and to explain why they 27

defined the areas as such. Seven college students were recruited via school’s email lists and participated in this study (four female and three male students). More specifically, we started by asking them where they lived and how they commuted between home and work (or school) daily. Then we provided each participant a print-out of our local map, asking them to draw the notification areas on the map. Once they finished drawing, we asked them to explain why they drew the areas that way. After collecting all the userdefined areas, we showed them the Android application, and asked for their feedback to improve the app design. We audio recorded the interviews with participants’ permission. The interviews were then transcribed and analyzed to identify major patterns in the interviews. As the two sample drawings showed in Figure 8(A) and (B), the participants chose circles, drew lines along the streets and connected different areas with irregular shapes. The participants’ self-defined notification areas were mostly centered around where they would be, and the sizes of the areas indicated the likelihood of them moving around those areas. For instance, one participant said, “[notification areas] just give me awareness and now I won’t go. I mean I won’t go to this place by walking, maybe I could drive there or passing there on bus, but I won’t walk there.” Participants felt one or two miles would be the acceptable walking distance and would be considered as their active areas. For instance, another participant explained, “One miles, or two miles maybe. It’s in the walking distance...People like me, I mean, most of the time, I would like to stay at home. So maybe one mile around my house would be my active area. But five miles really, for most of the part, I won’t be there, so maybe that [area] is not so important to me.” Since participants thought five miles was beyond walking distance, they were less concerned about those areas they usually would reach by car. These preliminary results suggested that given a location-based nudging message, participants considered their expected mobility in that area, and walking was associated with more safety concerns than driving. Our interview results helped explain why one-mile nudging messages were associated with more sharing decisions than other messages in the second lab experiment and suggested people might define their own interested areas using different 28

shapes.

6. Discussion We are among the first of those who try to systematically understand how different factors impact users’ reporting behaviors in the context of mobile crowdsourcing. Our lab experiment, MTurk study, nudging message design and evaluation, interviews, as well as the modeling endeavor, revealed several results that enhanced our understanding of different factors that might influence users’ willingness to share safety related reports through the mobile crowdsourcing application. In this section, we discuss the application of cognitive modeling to HCI research in general, the design implications to mobile crowdsourcing in the public safety reporting domain, the limitation of this work, and future research. 6.1. Applying Computational Cognitive Modeling to HCI Research Cognitive science and HCI share their historical roots in human factors [62]. Significant efforts have been made to apply cognitive science (theories and methods) to address HCI research questions [63, 64, 65, 66]. For example, the GOMS (Goals, Operators, Methods, and Selection rules) model described a set of knowledge that people need to know to perform a computer based task [64]. Cognitive science could be used to detect or predict certain interaction effects between humans and computers, where sensory input such as touching and hearing could combine with cognitive input, i.e., thinking, and expression motor such as smiling to identify facial expression, like happiness [65]. Computational cognitive modeling is a means of using computer algorithms to emulate or present cognitive descriptions, also referred to as computational psychology [67]. The computational models may be able to bring to light the finer details that math may pass over, details that are crucial in the study of the human mind [67]. In our work, we adopt and benefit from the computational cognitive model (particularly, the DDM) from cognitive psychology. Given the cognitive nature of human decision making, the computational models provide a systematic way to quantify the

29

effects of different factors on our decision making. Compared with other methods traditionally used to study sharing decisions in HCI, such as regression analysis, DDM allows us to examine how willing people are to make these decisions as well as the two aspects of the cognitive mechanism of decision-making, starting point bias and drift rate, both having important practical implications. User (or crowd worker) contributions are a key aspect of crowdsourcing. We believe that the computational cognitive models can contribute to a deep understanding of how users decide whether to contribute or not. In the context of mobile crowdsourcing for safety, the DDM allows us to pinpoint which factors are associated with higher rates of reporting safety incidents. Answers to this question informed new designs (e.g., nudges) that would focus on these important factors in encouraging users to contribute. The DDM can be also used to verify the effectiveness of our new nudging design. 6.2. Design Implications of Mobile Crowdsourcing for Public Safety Reporting Using the DDM, we were able to look into two finer-grained aspects of the cognitive process/mechanism of decision-making: the starting point bias and the drift rate (decision evidence). In brief, our results showed that both location (particularly home) and time of day (particularly late night) generated biases towards sharing. Participants showed a robust bias to share for incidents near the home, both in terms of response speed and probability of sharing the incident. Participants also showed a relative bias to share late night incidents, though this only affected the speed of their responses and not the probability of sharing. This starting point bias is a priori to seeing the actual type of incident that ultimately drives the reporting decision. Therefore, it makes sense to estimate this component independent of the crime type, because the participants saw the time and location before they knew what type of crime occurred. In other words, the influence of time and location occurs before (and independent of) the influence of the crime type. The lab study also found that time of day affected response speed (with faster decisions to share), but did not affect the proportion of times that the participants were willing to share. We also found that more severe crime incidents had more significant decision ev30

idence (or higher drift rate). Starting point bias usually affects fast decisions but not slow decisions because in the later case strong evidence can overturn the decision from one way to the other. Precisely for that reason, drift rate can affect both fast and slow decisions because strong evidence can affect how people evaluate evidence and overturn the decision. This particular result suggests that we should highlight the severity of incidents. For the less severe crime types, we should explore ways to potentially increase their evidence strength, e.g., by suggesting their potential consequences if not getting reported and addressed, or the potential size of population that may be affected. Our nudge design and the results of the second experiment imply that reminding people of their current location and their closeness to home could potentially increase users’ contributions to safety reporting. One challenge is that the current nudges assume that reporting safety incidents are always socially desirable because they can help the police departments improve the public safety. However, the nudge design did not consider the risks for individuals to report safety incidents. The mobile crowdsourcing platform makes it easy to share or report “on the spot.” Under certain circumstances, it might be risky or dangerous for people to report safety incidents or crimes on the spot because of the potential of being retaliated. This calls for context-aware nudge designs that can take into account both the benefits and risks which are dependent on the prevailing context. There has been a global effort on opening crime data, potentially creating an environment where crowdsourcing systems can be built for collecting people’s safety needs [68, 69, 70]. In the U.S., many municipal police departments are trying to implement community policing [71], which is to develop collaborations between the people in a community and local police departments, resulting in solving issues and improving public safety together [72]. As part of the community policing effort, mobile applications for community members to report crimes or help find criminals have been available on the market [73, 74]. It is a timely and important topic to understand how people send or share information on this platform, especially because crime related information can be sensitive to people’s identity and safety. However, because municipal police departments are very cautious on what they release in public, and many times there is a long time delay after the incidents happen [68], the current platforms do not provide 31

us an ideal to study people’s safety reporting behaviors in mobile crowdsourcing for public safety at this scale. On the other hand, our research findings of the campus platform can help inform the design of mobile crowdsourcing for public safety reporting in a larger scope. 6.3. Limitations and Future Work Our study was conducted on a university campus with mostly college students. In our future work, we plan to recruit a more diverse set of participants for our follow-up studies. We acknowledge that there are pros and cons associated with both the lab studies and the MTurk study in the current work. In the lab experiments, 270 trials could be tiresome, affecting participants’ interest and attention. While the Mturk study was shorter, it was very difficult to log response time in this setting, so we did not have that measure. Psychological studies have good performance in terms of producing consistent results in the lab versus in the field [75, 76]. In our study, our experimental results about location and how the severity of the crime impact people’s decisions are aligned with what we observed from real-world safety reports, which suggests we could leverage this DDM method to investigate other factors, e.g., gender. In addition, both the lab studies and the MTurk study used hypothetical scenarios. The responses were self-reported and thus may be biased (e.g., social desirability bias: they might say they will report just because that’s socially desirable and in practice, they might not report). However, for emerging technologies, scenario-based studies and analysis of self-reporting data are widely applied to address research questions [77, 78, 79, 80, 81]. Our lab experiments and online studies showed people were more willing to send safety reports when they were at home. This might be able to explain our log analysis results using the real safety reports, i.e., more reports were sent from residential areas. This study examines the basic factors that could impact people’s instinctive reactions, and the results inform design strategies that can encourage people making contributions to the crowdsourcing system. There could be other factors that affect people’s decisions, for example, female students were found more likely to take self-protective 32

actions [82] and to regard reporting crimes to police as more appropriate than male students [83]. In the future, we plan to study how crime severity can inform the design of the system to address under-reporting issues. To evaluate the effectiveness of the system design, we also plan to run field experiments, however a few challenges need to be addressed. First, the feasibility of running any experiment with a safety reporting system relies on close collaboration with the department of public safety (DPS). In general, the DPS is cautious about introducing any research ideas into their existing system. It took us a few years to gain the trust of our local DPS and their willingness for collaboration on research activities. Because of the critical mission of law enforcement agencies, they have to make sure all the implications and potential impact of using a new technology can be handled well. Second, in order to evaluate the nudge effect in the “wild,” we need to compare users’ reports with the “ground truth” data which contains what should be reported but is not reported. This is particularly challenging, because there are different levels of safety issues. It might be easier to collect the ground truth of some, e.g., traffic incidents, but many of them could be very difficult to detect, e.g., stealing. It may be possible to start with a particular type of report (e.g., gun shots) that could be easily detected using sensors, though this then requires the installation of sensors.

7. Conclusion In this paper, we leverage the drift-decision model (DDM) from cognitive psychology to investigate the impact of factors (e.g., crime type, incident location, and time of day) on people’s willingness to report public safety incidents in a mobile crowdsourcing environment. We present the results of a mixture of lab experiments, an online MTurk study, and interviews. The first lab study and the MTurk study revealed consistently that location and crime severity impact people’s reporting decisions. In particular, when people are close to home or witness/experience severe crimes, they tend to report safety incidents. These findings informed our design of a novel location-based nudge mechanism, where distance to home was displayed on the nudge message, designed to address under-reporting issues. We conducted another lab study to test this

33

nudge design and proved its effectiveness. Our follow-up interviews study explained that mobility patterns influenced people’s safety incident reporting. We also noticed that participants use different ways to identify their interested areas for receiving safety notifications. Overall, our work presents a good use case of how our approach consisting of computational cognitive modeling (DDM) and nudge mechanisms can be applied to advance our understanding and design of mobile crowdsourcing for public safety reporting. We suggest system designers and researchers to consider applying the proposed approach to address other HCI research needs.

8. Acknowledgements We thank our participants for sharing their insights. This material is based upon work supported by the National Science Foundation under Grant No.1464312. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. [1] U.S. of

Department Campus

of

Security

Education, Policy

and

The

Jeanne

Campus

Clery

Crime

Disclosure

Statistics

Act,

http://www2.ed.gov/admins/lead/safety/handbook.pdf (1992). [2] E. Tan, H. Xia, C. Ji, R. V. Joshi, Y. Huang, Designing a mobile crowdsourcing system for campus safety, iConference (March 2015). [3] G. Chatzimilioudis, A. Konstantinidis, C. Laoudias, D. Zeinalipour-Yazti, Crowdsourcing with smartphones, Internet Computing, IEEE 16 (5) (2012) 36–44. doi:10.1109/MIC.2012.70. [4] B. Guo, Z. Wang, Z. Yu, Y. Wang, N. Y. Yen, R. Huang, X. Zhou, Mobile crowd sensing and computing: The review of an emerging human-powered sensing paradigm, ACM Computing Surveys (CSUR) 48 (1) (2015) 7. [5] N. D. Lane, E. Miluzzo, H. Lu, D. Peebles, T. Choudhury, A. T. Campbell, A survey of mobile phone sensing, Comm. Mag. 48 (9) (2010) 140–150. doi: 34

10.1109/MCOM.2010.5560598. URL http://dx.doi.org/10.1109/MCOM.2010.5560598 [6] S. Tilak, Real-world deployments of participatory sensing applications: Current trends and future directions, ISRN Sensor Networks 2013. [7] F. Fuchs-Kittowski, D. Faust, Architecture of mobile crowdsourcing systems, in: Collaboration and Technology, Springer, 2014, pp. 121–136. [8] J. Zimmerman, A. Tomasic, C. Garrod, D. Yoo, C. Hiruncharoenvate, R. Aziz, N. R. Thiruvengadam, Y. Huang, A. Steinfeld, Field trial of tiramisu: crowdsourcing bus arrival times to spur co-design, in: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, 2011, pp. 1677–1686. [9] T. C. Hart, C. M. Rennison, Reporting crime to the police, 1992-2000, in: US Department of Justice, Office of Justice Programs., 2003. [10] D. Thompson, M.and Sitterle, J. Clay, G.and Kingree, Reasons for not reporting victimizations to the police: do they vary for physical and sexual incidents?, in: Journal of American College Health, 2007. [11] P. Wilcox, C. E. Jordan, J. Pritchard, A, A multidimensional examination of campus safety victimization, perceptions of danger, worry about crime, and precautionary behavior among college women in the post-clery era, in: Crime & Delinquency, 2007. [12] W. G. Skogan, Reporting crimes to the police: The status of world research, Journal of research in crime and delinquency 21 (2) (1984) 113–137. [13] M. J. Hindelang, M. Gottfredson, et al., The victim’s decision not to invoke the criminal justice process, Criminal justice and the victim (1976) 57–78. [14] R. I. Mawby, Witnessing crime: Toward a model of public intervention, in: Criminal Justice and Behavior, 1980. [15] L. Bickman, H. Helwig, Bystander reporting of a crime, Criminology 17 (3) (1979) 283–300. 35

[16] J. A. Burke, D. Estrin, M. Hansen, A. Parker, N. Ramanathan, S. Reddy, M. B. Srivastava, Participatory sensing, Center for Embedded Network Sensing. [17] N. D. Lane, E. Miluzzo, H. Lu, D. Peebles, T. Choudhury, A. T. Campbell, A survey of mobile phone sensing, IEEE Communications magazine 48 (9) (2010) 140–150. [18] A. Sahami Shirazi, N. Henze, T. Dingler, M. Pielot, D. Weber, A. Schmidt, Largescale assessment of mobile notifications, in: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, 2014, pp. 3055–3064. [19] D. S. McCrickard, M. Czerwinski, L. Bartram, Introduction: design and evaluation of notification user interfaces, International Journal of Human-Computer Studies 58 (5) (2003) 509–514. [20] C. N. White, R. A. Poldrack, Decomposing bias in different types of simple decisions, in: Journal of Experimental Psychology: Learning, Memory, and Cognition, 40, 385-398, 2014. [21] G. S. Green, Citizen reporting of crime to the police: An analysis of common theft and assault, Ph.D. thesis, University of Pennsylvania (1981). [22] R. R. Bennett, R. B. Wiegand, Observations on crime reporting in a developing nation, Criminology 32 (1) (1994) 135–148. [23] M. Xie, J. L. Lauritsen, Racial context and crime reporting: A test of blacks stratification hypothesis, Journal of quantitative criminology 28 (2) (2012) 265– 293. [24] R. Bowles, M. G. Reyes, N. Garoupa, Crime reporting decisions and the costs of crime, European journal on criminal policy and research 15 (4) (2009) 365–377. [25] S. M. Schnebly, The influence of community-oriented policing on crimereporting behavior, Justice Quarterly 25 (2) (2008) 223–251.

36

[26] M. C. Dertke, L. A. Penner, K. Ulrich, Observer’s reporting of shoplifting as a function of thief’s race and sex, The Journal of Social Psychology 94 (2) (1974) 213–221. [27] B. Latan´e, J. M. Darley, The unresponsive bystander: Why doesn’t he help?, Prentice Hall, 1970. [28] P. R. Bleda, S. E. Bleda, D. Byrne, L. A. White, When a bysder becomes an accomplice: Situational determinants of reactions to dishonesty, Journal of Experimental Social Psychology 12 (1) (1976) 9–25. [29] G. L. Borofsky, G. E. Stollak, L. A. Mess´e, Sex differences in bystander reactions to physical assault, Journal of Experimental Social Psychology 7 (3) (1971) 313– 318. [30] D. M. Gelfand, D. P. Hartmann, P. Walder, B. Page, Who reports shoplifters? a field-experimental study., Journal of Personality and Social Psychology 25 (2) (1973) 276. [31] P. Rothe, L. Elgert, R. Deedo, Dynamic influences on bystander actions: Program recommendations from the field, Alberta Centre for Injury Control and Research, Department of Public Health Sciences, University of Alberta. [32] L. Festinger, A. Pepitone, T. Newcomb, Some consequences of de-individuation in a group., The Journal of Abnormal and Social Psychology 47 (2S) (1952) 382. [33] B. S. Fisher, J. L. Hartman, M. G. Cullen, F. T.and Turner, S. L. Review, Making campuses safer for students: The clery act as a symbolic legal reform, in: Elected officials at all levels of government pass laws in re- sponse to issues that rise to the top of their policy agenda ., 2002. [34] D. E. Gregory, Crime on campus. compliance, liability, and safety, in: Campus Law Enforc, 2011. [35] E. W. Morris, snitches end up in ditches and other cautionary tales, Journal of contemporary criminal justice 26 (3) (2010) 254–272. 37

[36] H. Goudriaan, J. P. Lynch, P. Nieuwbeerta, Reporting to the police in western nations: A theoretical analysis of the effects of social context, in: Justice Quarterly, 2004. [37] S. Singer, The fear of reprisal and the failure of victims to report a personal crime, in: Journal of Quantitative Criminology, 1998. [38] J. Lasley, B. Palombo, When crime reporting goes high-tech: an experimental test of computerized citizen response to crime, in: Journal of Criminal Justice, 1995. [39] A. Iriberri, C. J. Navarrete, Internet crime reporting: Evaluation of a crime reporting and investigative interview system by comparison with a non-interactive reporting alternative, 2014 47th Hawaii International Conference on System Sciences 0 (2010) 1–9. doi:http://doi.ieeecomputersociety.org/ 10.1109/HICSS.2010.460. [40] G. Leroy, N. Garrett, Reporting on-campus crime online: User intention to use, in: Scholarship @ Claremont, 2006. [41] V. Furtado, L. Ayres, M. de Oliveiro, E. Vasconcelos, C. Caminha, J. DOrleans, M. Belchior, Collective intelligence in law enforcement the wikicrimes system, in: Information Sciences, 2010. [42] A. L. Hughes, L. Palen, The evolving role of the public information officer: An examination of social media in emergency management, Journal of Homeland Security and Emergency Management 1976 (2012) 1. [43] T. Ludwig, T. Siebigteroth, V. Pipek, Crowdmonitor: Monitoring physical and digital activities of citizens during emergencies, in: International Conference on Social Informatics, Springer, 2014, pp. 421–428. [44] S. Shah, F. Bao, C.-T. Lu, I.-R. Chen, Crowdsafe: Crowd sourcing of crime incidents and safe routing on mobile devices, in: Proceedings of the 19th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, 2011.

38

[45] H. Shin, T. Park, S. Kang, B. Lee, J. Song, Y. Chon, H. Cha, Cosmic: designing a mobile crowd-sourced collaborative application to find a missing child in situ, in: Proceedings of the 16th international conference on Human-computer interaction with mobile devices & services, ACM, 2014, pp. 389–398. [46] S. H. Seop, M. G. Young, J. D. Hoon, A study on the development of disaster information reporting and status transmission system based on smart phone, in: ICTC 2011, IEEE, 2011, pp. 722–726. [47] C. Reuter, Communication between power blackout and mobile network overload, International Journal of Information Systems for Crisis Response and Management (IJISCRAM) 6 (2) (2014) 38–53. [48] S. B. Liu, Crisis crowdsourcing framework: Designing strategic configurations of crowdsourcing for the emergency management domain, Computer Supported Cooperative Work (CSCW) 23 (4-6) (2014) 389–443. [49] T. W. Malone, R. Laubacher, C. Dellarocas, The collective intelligence genome, IEEE Engineering Management Review 38 (3) (2010) 38. [50] R. Kraut, P. Resnick, S. Kiesler, M. Burke, Y. Chen, N. Kittur, J. Konstan, Y. Ren, J. Riedl, Building Successful Online Communities: Evidence-Based Social Design, MIT Press, 2012. URL http://books.google.com/books?id=lIvBMYVxWJYC [51] J. Goldman, K. Shilton, J. Burke, D. Estrin, M. Hansen, N. Ramanathan, S. Reddy, V. Samanta, M. Srivastava, R. West, Participatory sensing: A citizenpowered approach to illuminating the patterns that shape our world, Foresight & Governance Project, White Paper (2009) 1–15. [52] D. Geiger, M. Rosemann, E. Fielt, Crowdsourcing information systems: a systems theory perspective, in: Proceedings of the 22nd Australasian Conference on Information Systems (ACIS 2011), 2011. [53] R. H. Thaler, C. R. Sunstein, Nudge: Improving Decisions About Health, Wealth, and Happiness, 1st Edition, Yale University Press, 2008. 39

[54] I. Ritov, J. Baron, Status-quo and omission biases, Journal of Risk and Uncertainty 5 (1). doi:10.1007/BF00208786. URL

http://www.springerlink.com/content/

qx07u05234p64l1u/ [55] R. Ratcliff, G. McKoon, The diffusion decision model: Theory and data for twochoice decision tasks, Neural Computation 20 (3) (2008) 873–922. doi:10. 1152/jn.01049.2002. [56] I. Krajbich, A. Rangel, Multialternative drift-diffusion model predicts the relationship between visual fixations and choice in value-based decisions, Proceedings of the National Academy of Science 108 (33) (2011) 13852–13857. doi:10.1073/pnas.1101328108. [57] J. A. Nelder, R. Mead, A simplex method for function minimization, in: The Comput Journal, 1965. [58] R. Ratcliff, F. Tuerlinckx, Estimation of the parameters of the diffusion model: Approaches to dealing with contaminant reaction times and parameter variability, in: Psychonomic Bulletin and Review, 9:438-481, 2002. [59] R. E. Boyatzis, Transforming qualitative information: Thematic analysis and code development, Sage, 1998. [60] S. Friese, Qualitative data analysis with ATLAS. ti, Sage, 2014. [61] SALT lab, Syracuse University, https://play.google.com/store/apps/details?id=syr.dpshl=en (2015). [62] R. L. Boring, Human-computer interaction as cognitive science, in: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 46, SAGE Publications, 2002, pp. 1767–1771. [63] M. M. Gardiner, B. Christie, Applying cognitive psychology to user-interface design, Wiley Chichester, 1987.

40

[64] S. K. Card, A. Newell, T. P. Moran, The psychology of human-computer interaction, L. Erlbaum Associates Inc., 1983. [65] C. L. Lisetti, D. J. Schiano, Automatic facial expression interpretation: Where human-computer interaction, artificial intelligence and cognitive science intersect, Pragmatics & cognition 8 (1) (2000) 185–235. [66] J. Hollan, E. Hutchins, D. Kirsh, Distributed cognition: toward a new foundation for human-computer interaction research, ACM Transactions on ComputerHuman Interaction (TOCHI) 7 (2) (2000) 174–196. [67] R. Sun, Introduction to computational cognitive modeling, Cambridge handbook of computational psychology (2008) 3–19. [68] M. Byrne Evans, K. O’Hara, T. Tiropanis, C. Webber, Crime applications and social machines: Crowdsourcing sensitive data, in: Proceedings of the 22Nd International Conference on World Wide Web, WWW ’13 Companion, ACM, New York, NY, USA, 2013, pp. 891–896. doi:10.1145/2487788.2488075. URL http://doi.acm.org/10.1145/2487788.2488075 [69] L. Tompson, S. Johnson, M. Ashby, C. Perkins, P. Edwards, Uk open source crime data: accuracy and possibilities for research, Cartography and Geographic Information Science 42 (2) (2015) 97–111. doi:10.1080/15230406.2014. 972456. [70] City of Chicago, https://data.cityofchicago.org/Public-Safety/Crimes-2001-topresent/ijzp-q8t2 (2015). [71] T. Hoffman, Nypd turns to social media to strengthen community relations, [Online; accessed 22-June-2015] (2015). URL http://www.1to1media.com/view.aspx?docid=35385 [72] C. P. Consortium, P. Manager, U. S. of America, Understanding community policing: A framework for action.

41

[73] Apprise

Inc,

Mobilepatrol

public

safety

app,

https://play.google.com/store/apps/details?id=com.appriss.mobilepatrol (2014). [74] Alyacom

Emergency,

Mobile

application,

alyacom

emergency,

https://play.google.com/store/apps/details?id=org.alyacom. emergency (2014). [75] G. Mitchell, Revisiting truth or triviality: The external validity of research in the psychological laboratory, Perspectives on Psychological Science 7 (2012) 109– 118. [76] J. J. L. Craig A. Anderson, B. J. Bushman, Research in the psychological laboratory: Truth or triviality?, Current Directions in Psychological Science 8 (1999) 3–9. [77] R. Angeles, An empirical study of the anticipated consumer response to rfid product item tagging, Industrial Management & Data Systems 107 (4) (2007) 461– 483. [78] Y. Wang, H. Xia, Y. Yao, Y. Huang, Flying eyes and hidden controllers: A qualitative study of peoples privacy perceptions of civilian drones in the us, Proceedings on Privacy Enhancing Technologies 2016 (3) (2016) 172–190. [79] M. S. Ackerman, L. F. Cranor, J. Reagle, Privacy in e-commerce: Examining user scenarios and privacy preferences, in: Proceedings of the 1st ACM Conference on Electronic Commerce, EC ’99, ACM, New York, NY, USA, 1999, pp. 1–8. doi:10.1145/336992.336995. URL http://doi.acm.org/10.1145/336992.336995 [80] P. Briggs, L. Thomas, An inclusive, value sensitive design perspective on future identity technologies, ACM Transactions on Computer-Human Interaction (TOCHI) 22 (5) (2015) 23. [81] I. Bilogrevic, M. Ortlieb, If you put all the pieces together...: Attitudes towards data combination and sharing across services and companies, in: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, ACM, 2016, pp. 5215–5227. 42

[82] A. D. Woolnough, Fear of crime on campus: gender differences in use of selfprotective behaviours at an urban university, Security Journal 22 (1) (2009) 40– 55. [83] R. B. Ruback, K. S. Menard, M. C. Outlaw, J. N. Shaffer, Normative advice to campus crime victims: Effects of gender, age, and alcohol, Violence and victims 14 (4) (1999) 381–396.

43