Let’s search together, but not too close! An analysis of communication and performance in collaborative information seeking

Information Processing and Management 49 (2013) 1165–1179

Contents lists available at SciVerse ScienceDirect

Information Processing and Management journal homepage: www.elsevier.com/locate/infoproman

Let’s search together, but not too close! An analysis of communication and performance in collaborative information seeking Roberto González-Ibáñez a,b,⇑, Muge Haseki a, Chirag Shah a a b

School of Communication & Information (SC&I) Rutgers, The State University of New Jersey, United States Departamento de Ingeniería Informática, Universidad de Santiago de Chile, Chile

a r t i c l e

i n f o

Article history: Available online 19 March 2013 Keywords: Collaboration Information seeking Communication Experimentation Human factors Measurement

a b s t r a c t Communication is considered to be one of the most essential components of collaboration, but our understanding as to which form of communication provides the most optimal costbenefit balance lacks severely. To help investigate effects of various communication channels on a collaborative project, we conducted a user study with 30 pairs (60 participants) in three different conditions – co-located, remotely located with text chat, and remotely located with text as well as audio chat, in an exploratory search task. Using both quantitative and qualitative data analysis, we found that teams with remotely located participants were more effective in terms of being able to explore more diverse information. Adding audio support for remote collaboration helped participants to lower their cognitive load as well as negative emotions compared to those working in the same space. We also show how these findings could help design more effective systems for collaborative information seeking tasks using adequate and appropriate communication. We argue that collaboration is an important aspect of human-centered IR, and that our work provides interesting insights into people doing information seeking/retrieval in collaboration. Ó 2013 Elsevier Ltd. All rights reserved.

1. Introduction Most of the studies in information seeking/retrieval have focused primarily on people as individual entities (Reddy & Jansen, 2008; Shah, 2010b). Many scholars have recognized, however, that information seeking is often a social process involving groups of people (Morris, 2008; Reddy & Dourish, 2002; Sonnenwald, 1996; Twidale, Nichols, & Paice, 1997) who engage in several activities to reach goals that are difficult to achieve individually. In today’s organizations and groups, the speed and complexity of information and communication flows often exceed the processing capacities of individuals (Yuan, Fulk, & Monge, 2007), which require individuals to work together to collect, analyze, and synthesize information. Some studies have already examined social practices that affect collaboration during the information seeking process. For instance, an extensive literature review on collaborative information seeking (CIS) has identified information sharing, coordination, and awareness as core processes of collaborative work (Shah, 2010b). In addition, other studies have revealed awareness, division of labor, persistence (Morris & Horvitz, 2007), and brainstorming (Morris, 2008) as supporting components of collaborative Web search. However, these studies have not considered potential costs and benefits of communication processes in CIS. In contrast, the first goal of this work is to explore the communication processes of teams in CIS.

⇑ Corresponding author at: School of Communication & Information (SC&I) Rutgers, The State University of New Jersey, United States. E-mail addresses: [email protected], [email protected] (R. González-Ibáñez). 0306-4573/$ - see front matter Ó 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.ipm.2012.12.008

1166

R. González-Ibáñez et al. / Information Processing and Management 49 (2013) 1165–1179

A common finding in communication research is that physical proximity facilitates communication by increasing spontaneous interactions. Although the literature has shown that communication can facilitate greater exchange of high-quality information, research has also demonstrated negative effects of social interaction on problem solving, generation of ideas, and affective as well as cognitive load. For instance, Kratzer, Leenders, and Van Engelen (2006) have found that interaction frequency negatively influences team creativity. Along with these lines, the second goal of this work is to compare the CIS process and products of teams across three different communication contexts (co-located, text chat, audio + text chat) within the theoretical framework of computer-mediated communication (CMC). Specifically, we investigate how communication contexts constrain or improve communication processes and the outcomes in CIS. Social and collaborative aspects of information retrieval/seeking are essential for designing better information retrieval (IR) and human–computer interaction (HCI) systems. Early studies showed that communication is necessary for system designers of collaborative solutions to understand the costs and benefits related to communication choices under various circumstances (Grudin, 1994). Besides communication needs, introducing support for collaborative search as well as sense-making could help to improve exploratory search experience (Morris, 2007). Given that proximity has different effects on communication processes and outcomes, the final goal of this work is to identify social features needed in different communication contexts that could help to design better systems. To meet our goals, we conducted an experiment with 60 participants in 30 pairs. Each pair was randomly assigned to three different experimental conditions (collocated, text chat, and text plus audio chat) defined based on spatial context and communication support. The following section provides background for this work, along with specific gaps in our understanding of this subject and their corresponding research questions. Next, we present our study design, results, and analyses. Finally, the paper concludes with reflections on the results, as well as implications for CIS system design. 2. Background and research questions Several studies have investigated the effect of communication context on the performance of collaborative teams. Within the framework of computer supported cooperative work (CSCW), studies have examined the effect of CMC (Fidas, Komis, Tzanavaris, & Avouris, 2005). Other studies have compared face-to-face (F2F) and chat communication (Newlands, Anderson, & Mullin, 2003), and F2F, co-located, and video-mediated communication (VMC) (Doherty-Sneddon et al., 1997). However, none of these studies have focused on the communication processes of teams in information seeking tasks across different contexts, which may have an impact on the performance of collaborative teams. This section presents a review of relevant literature constituting the research framework of this work. In addition, as the literature is discussed, the research questions are introduced. 2.1. Collaborative information seeking CIS is commonly characterized by the ‘‘systems and practices that enable individuals to collaborate during the seeking, searching, and retrieval of information’’ (Foster, 2006, p. 330). While the literature on CSCW covers a wide spectrum of problems related to collaboration, CIS is a unique research topic due to particular social practices that emerge around information. In this sense, understanding particular aspects of CIS, such as communication, behaviors, and performance, are essential for designing better systems that can support collaboration in searching, assessing, and making sense of information. As a key concept in CIS, collaboration has been studied from different perspectives in information studies. As a result, multiple definitions have been developed for this term. In this paper, we define collaboration as a social process in which two or more individuals intentionally and explicitly work together with the aim of cooperating to accomplish common goals, either synchronously or asynchronously, co-located or remotely located, using communication to interact with as well as to coordinate actions among group members. 2.2. Social processes in collaborative information seeking During recent years, many studies have explored different aspects of CIS in both naturalistic and experimental settings. The focus has typically been on describing users’ behaviors as well as information seeking processes of teams. For example, Hyldegard (2006, 2009) studied the applicability of Kuhlthau’s (1991) information search process in the context of groups. Similarly, Shah and González-Ibáñez (2011) attempted to map the stages in Kuhlthau’s model to collaborative information seeking. Both studies revealed that even though some stages of this model may apply to CIS, they do not cover the social dimension of CIS. Studies of search strategies in educational (Large, Beheshti, & Rahman, 2002; Twidale et al., 1997) and organizational settings (Morris, 2008) revealed that users often prefer to collaborate on search tasks. For example, studies in school settings have identified joint information retrieval as students often work together on team projects and assist each other with strategies for finding relevant information online or in library databases (Large et al., 2002; Twidale et al., 1997). Such collaborative work on searching information has the potential to yield better results than individual efforts due to shared activities. In an organizational context, Morris and Horvitz’s study (2007) revealed three key features for supporting collaborative Web search: awareness, division of labor, and persistence. Awareness refers to the ‘‘understanding of the activities of others,

R. González-Ibáñez et al. / Information Processing and Management 49 (2013) 1165–1179

1167

which provides a context for your own activity’’ (Dourish & Belloti, 1992, p. 1). Awareness allows team members to evaluate each other’s activities and progress, and has potential to improve the collaboration process. Participants in this study also reported employing various methods for managing collaborative search tasks in order to increase efficiency. Such methods included explicitly dividing up the space of potential keywords, search engines, or subtasks, and assigning a part to each group member. In terms of communication media used, instant messaging was mostly used for discussion about the task and division of labor and facilitating awareness. Associating audio feedback while sharing information was suggested as an alternative way for notifying users about each other’s activities. In a follow-up study, Morris (2008) investigated the effects of distance on the collaborative search process. Participants working on a collaborative Web search indicated both obstacles and advantages of working remotely. On one hand, working remotely enabled them to follow different strategies to search for information, increasing their productivity and facilitating creativity in gathering information. On the other hand, while working remotely they faced the problem of gathering redundant information and difficulty in navigating with the collaborator to the same content. That is, distributed context restricted awareness between collaborators and made the exchange of detailed information difficult. These findings demonstrate the need for varying degrees of communication and coordination during search process for awareness, creativity, communication and thereby positive outcomes. As such, the following section reviews the theoretical frameworks that help us better understand the relationship between the level of interaction a task needs and the type of communication medium used to fulfill these needs. 2.3. Communication needs and media use Several theories of CMC suggest a fit between the communication needs of a task and the capacity of a communication medium for information conveyance and social presence. According to media richness theory (Daft & Lengel, 1984), F2F communication is considered as the richest medium in that people can rapidly convey content and meaning both verbally and nonverbally. Audio communication allows conveyance of certain nonverbal information, and text chat communication only allows conveyance of textual information. Another early theory of CMC is social presence theory (Short, Williams, & Christie, 1976), which suggests that social presence will affect the way individuals perceive their discussions and their relationships with others (Sonnenwald & Pierce, 2000). According to this theory, communication media vary in their degree of social presence and people tend to choose media compatible with their needs for social presence. Research indicates that F2F communication is the highest in social presence followed by video, multichannel audio, speakerphone, and written text (Shaw, 1981). Research within these theoretical frameworks showed that tasks that require highest levels of interaction and collaboration call for richer medium and higher levels of social presence. As the literature above suggests collaborators need varying degrees of interaction throughout a CIS task, and therefore should be matched with a particular communication media. For instance, while division of labor is a decision making process that may require high levels of interaction, information sharing activities do not necessitate higher levels of social presence. Along with the premises of media richness and social presence theories, each task needs to be supported by a different communication medium. As such, our first research question aims to explore the varying interaction needs throughout a CIS task and to identify the optimum communication medium to be used to fulfill these interaction needs in order to develop systems that support those interactions (Table 1). Prior research also demonstrated that a lack of fit between the medium and a task’s social need can negatively influence the performance of communication partners (Mennecke, Valacich, & Wheeler, 2000). For instance, communicating through audio is found to be important when collaborating at a distance and to improve both task performance and perceived affordances in comparison to text chat (Matsuura, Fujino, Okada, & Matsushita, 1993). However, adding video communication did not add significant advantages to task performance compared to audio (Anderson et al., 1996; Olson, Olson, & Meader, 1995; Whittaker & O-Conaill, 1997). Given the insignificant impact of video communication revealed by early studies of collaboration, we excluded video chat from our current study and only focused on F2F, text-based, and audio communication. Continuing our research exploration along the direction of evaluating the effects of communication contexts in CIS, our second research question addresses the relation between communication contexts and their implications in the performance of teams (Table 1). 2.4. Cognitive and affective factors in communication Communication context may also influence the cognitive and affective loads of team members. Cognitive load refers to the limitations on working memory capacity (Howarth & Anderson, 2007). As pointed out by Howarth and Anderson, in a Table 1 Research questions. #

Research question

RQ1

How do communication dynamics in different communication contexts (face-to-face, text chat, and audio + text chat) affect teams’ interaction within a CIS task? How does communication context (face-to-face, text chat, audio + text chat) affect the performance of teams within a CIS task? How does communication context (face-to-face, text chat, audio + text chat) affect the cognitive and affective load of teams within a CIS task?

RQ2 RQ3

1168

R. González-Ibáñez et al. / Information Processing and Management 49 (2013) 1165–1179

collaborative problem-solving task, cognitive load, and communication context influence the way team members collaborate with one another. This study found that team members in a video-mediated context used fewer question-form introductions followed by an informative response than those in a F2F context. In addition to cognitive load, research has also shown that CMC contexts result in various degrees of affective load for individuals. For example, using pre-task and post-task questionnaires, Satar and Özdener (2008) found in an experimental study that participants using text-based communication experience a decrease of anxiety levels when compared to those that used voice communication. Considering these two aspects, we also wish to address the relationship between communication and cognitive and affective load through our third research question (Table 1). 3. Method 3.1. Participants We conducted a laboratory study involving a total of 60 participants in 30 collaborative pairs. We chose the minimum group size in this study to avoid the incorporation of new and potential intervening variables. Early studies showed that as the number of collaborators working together increases, the complexity of possible interactions increases exponentially, thus increasing the likelihood of misinterpretation and misunderstanding (Tang et al., 2010). Minimizing group size allowed us to have better control of each experimental condition with respect to the variables of interest, which in this case relate to communication practices and performance. Participants were recruited from Rutgers University by sending experiment advertisements to email lists and posting announcements around campus. Participants were asked to sign up in pairs with someone with whom they had previous experience collaborating. This was to make sure that the participants were comfortable working with their teammates and to ensure common ground (Clark & Brennan, 1991). Participants were informed of compensation for participating in the study, which consisted of $10 per person and the possibility to obtain additional prizes if they were among the three best-performing teams ($50, $25, and $15 per person additionally) at the end of the study. 3.2. System In order to provide appropriate tools and support for information seeking processes of the participants working in various collaborative conditions, we developed a modified version of Coagmento (González- Ibáñez & Shah, 2011; Shah, 2010a) – a plugin for the Firefox web browser. As shown in Fig. 1, the add-on included a toolbar and a sidebar. The toolbar had the following buttons: (1) Home – for taking the participant to appropriate questionnaires, (2) Bookmark – for bookmarking a webpage, (3) Snip – for collecting a snippet using highlighted text from a webpage, and (4) Editor – for accessing a shared editor for writing the report. The sidebar had two major components: a chat-box and a resources panel. The chat-box allowed the collaborators in a given team to communicate with each other and also to receive instructions from the researcher conducting the study. The resources panel included tabs for bookmarks, saved snippets, and executed queries. See Fig. 1 for details.

Fig. 1. Snapshot of the experimental system with parts of it shown in details.

R. González-Ibáñez et al. / Information Processing and Management 49 (2013) 1165–1179

1169

3.3. Session workflow Each experimental session lasted less than an hour and was structured in six parts, which are described below: 1. 2. 3. 4. 5.

Participants were introduced to the study and asked to sign a consent form. Participants watched a brief tutorial in order to learn the basic functionalities required during the task. Participants individually filled out pre-task questionnaires (demographics and PANAS). Participants read the task description (given later). Each team worked for 25 min on the given task, which included searching and collecting relevant information, and using it to compose a report. 6. Participants filled out post-task questionnaires. The researcher conducting the study communicated with the participants through the chat-box at different times during the study instructing them to start/stop the task or fill in questionnaires. Although setting an artificial time limit for completing the task is one of the limitations of this study, it was necessary to keep the experiment manageable for both the researchers and the participants. Setting such a limit is a common practice for such studies (c.f., Morris & Horvitz, 2007; Shah & Marchionini, 2010). It is possible that allowing the participants to take as much time as they like would yield different results, but that is a subject of investigation for a future study. 3.4. Conditions To investigate how communication relates to various aspects of group work while working in collaboration for an information seeking task, we conducted experiments with three different conditions: two participants at the same room with different computers (C1f2f); two participants at different rooms with individual computers and the ability to communicate only via text-based chat (C2text); and two participants at different rooms with individual computers and the ability to communicate via audio and text-based chat (C3audio+text). Note that participants in conditions C2text and C3audio+text were located at different rooms separated by walls, and not just a partition. They could not see or talk to each other directly, and the only communication channel they had were the ones provided in each condition. 3.5. Task The participants were asked to collect relevant information for an exploratory search task that was designed to be a realistic work-task (Borlund & Ingwersen, 1999). We chose ‘‘gulf oil spill’’ as the topic for this study since it was quite relevant at the time this study was being conducted (early fall 2010). Our preliminary investigations, including several pilot runs, indicated that there was a substantial amount of material on this topic, and that the participants would find it interesting and challenging enough as an exploratory search task. Each participant was given the following task description: ‘‘A leading newspaper has hired your team to create a comprehensive report on the causes, effects, and consequences of the recent gulf oil spill. As a part of your contract, you are required to collect all the relevant information from any available online sources that you can find. To prepare this report, search and visit any website that you want and look for specific aspects as given in the guideline below. As you find useful information, highlight and save relevant snippets. Make sure you also rate a snippet to help you in ranking them based on their quality and usefulness. Later, you can use these snippets to compile your report, no longer than 200 lines, as instructed. Your report on this topic should address the following issues: description of how the oil spill took place, reactions by BP as well as various government and other agencies, impact on economy and life (people and animals) in the gulf, attempts to fix the leaking well and to clean the waters, long-term implications and lessons learned.’’ The participants saw this description on the screen and were also given a printed copy to refer to during their session. 3.6. Protocol for data capturing During the lab session, we captured various forms of data from the users’ physical actions, such as visited pages, bookmarks, queries, and chat messages. We also used Camtasia Studio 7 to record desktop activity and participants’ faces. All of these multimedia sources were synchronized with an error margin of a few seconds. In the case of C1f2f and C3audio+text, we also recorded audio of their conversations during the session. 4. Data analysis In order to study the communication within teams, we devised a specific method that consisted of four stages: (1) preprocessing, (2) coding, (3) post-processing, and (4) evaluation.

1170

R. González-Ibáñez et al. / Information Processing and Management 49 (2013) 1165–1179

4.1. Data pre-processing Prior to conducting our analyses, we prepared the data obtained from each experimental session. From the 30 teams that participated in our study, we ended up with more than 25 h of audiovisual data and thousands of records of participants’ actions in our system. From this data, we conducted a preprocessing procedure in order to segment and synchronize our data sources. The segmentation was performed in order to identify the moment in which each participant completed the last pretask questionnaire and the moment right before they started filling in the first post-task questionnaire. We then synchronized the sources of data for each pair of participants within a team in order to identify properly when both participants reached the task description stage. After performing segmentation, we extracted the audio and text messages containing the communication of each team in conditions C1f2f and C3audio+text. For C2text, on the other hand, we retrieved all the messages from our chat logs. At the end of this stage, we possessed approximately 9.5 h of audio for C1f2f and C3audio+text, and 263 text messages for C2text. 4.2. Content analysis Content analysis of communication was used to investigate communication patterns during intragroup collaboration. We used both a qualitative and a quantitative approach, such that communication between team members were coded, and then frequencies used for statistical comparisons. We used a sentence or part of a compound sentence as the unit of analysis (Strijbos, Martens, Jochems, & Broers, 2004). A specific procedure to segment transcripts in these units was developed. Content analysis was performed on audio and text messages generated by 60 subjects equally distributed across three conditions (C1f2f, C2text, and C3audio+text). The categories for the structuring content analysis were adapted from a framework previously used for a CSCL study (Strijbos et al., 2004), which includes five main categories – task coordination (TC), task content (TN), task social (TS), non-task (NT), and non-codable (NC). These categories refer to all types of statements regarding coordination such as decision making about how the task should be performed (TC); statements that are related to the content of the task such as information assessment, layout, structure, and revision of report (TC); statements that concern group functioning, effort, or attitude, as well as opinions in regards to information obtained or information sources (TS); and all statements with a social orientation that are not related to the assignment or regarding technical issues of the system being used (NT). A summary of these categories and examples are depicted in Table 2. Following this coding scheme, two human judges coded a total of 3826 messages across all experimental conditions. For the coding procedure of audio conversations, we used a subtitle editor program called Aegisub. Although this tool was developed as a subtitle editor, the main functionalities are particularly useful for coding as well as for transcription. 4.3. Data post-processing After completing the coding stage for each team, we conducted a post processing procedure in order to ensure the proper formatting of the coded data and evaluate the reliability of the two judges. The inter-rater reliability for the judges was found to be Kappa = 0.799 (p < 0.01).

Table 2 Coding scheme. Code

Description

Examples of Statements

Task coordination (TC)

All types of statements regarding coordination, which involve decision making about how the task should be performed

1. We should start writing now 2. Can you search the second one? 3. How do you want to do this? 4. I will work on the reactions and you work on the consequences

Task content (TN)

All types of statements that are related to the content of the task, which include information assessment, layout, structure, and revision of report

1. I found something about consequences 2. Ok, I found stuff on the impact on economy life, people and animals 3. Well, I have how they finally capped off the leak, but I will look up some failed attempts

Task social (TS)

All types of statements that concern group functioning, effort, or attitude as well as opinions in regards to information obtained or information sources

1. This task is really hard 2. We did good 3. Wow, so many animals were killed during the spill 4. I think my answer is the best

Non-task related (NT)

All statements with a social orientation that are not related to the assignment or regarding technical issues of system being used

1. 2. 3. 4.

Non-codable (NC)

All types of statements that do not belong any category specified

1. What happened?

I am hungry How do we save the report? I saw a great movie yesterday What are you going to do tomorrow?

R. González-Ibáñez et al. / Information Processing and Management 49 (2013) 1165–1179

1171

5. Evaluation In order to perform comparisons across conditions, we devised an evaluation method, which consisted of four sets of measures: communication, productivity, information synthesis, and cognitive/affective load measures. 5.1. Communication measures For each team, we computed frequencies of codes under each coding category (i.e. TC, TN, TS, NT, and NC). These numbers were used in the definition of three implicit communication measures, namely: volume, effort, and balance. First of all, volume corresponds to the amount of messages exchanged by team members during the task; communication effort represents the overall time spent by participants in communication; and the last measure indicates the communication balance of the team in terms of the number of messages issued by each team member. Description of each measure and their formulations are presented in Table 3. 5.2. Productivity measures In order to measure the productivity of teams, we used the set of measures described in Shah and González- Ibáñez (2011). These measures included the union of all webpages visited by all teams (U); the union of all webpages bookmarked or from where one or more snippets were collected (Ur); the total number of distinct webpages visited by a given team within U (C); the total number of pages visited only by given team within U (uC); the total number of distinct relevant webpages visited by a given team within Ur (Cr); the total number of relevant pages visited only by a given team within Ur (uCr); an implicit measure of usefulness based on the dwell time on a webpage (UsW) (White & Huang, 2010); and Lavenshtein distance between pairs of queries for a given subject/team (QD) (Table 4). The various dimensions offered by this framework of productivity evaluation are particularly useful for comparing teams across different scenarios of collaboration. 5.3. Information synthesis measures As indicated above, in addition to bookmark pages and collect snippets from them, teams in all conditions had to produce a report with answers to specific questions in the task description, using information collected. To get a sense of the writing quality of these reports, we evaluated them using standard tests of readability as well as assessments given by human coders. Note that these tests and the associated evaluation do not truly evaluate the quality of the content or the retrieval performance.

Table 3 Implicit measures of communication. Communication measure

Description

Volume

The overall number of messages exchanged by team members during the task (Eq. (1)). This was computed for each team

V ¼ # messages Effort

Overall time spent in communication. This implicit measure of effort was operationalized through calculating the duration in seconds of each message. This was computed for audio data (C1f2f and C3audio+text) by subtracting the ending time and the starting time of each message (Eq. (2)). For written messages (C2text), we estimated the effort considering the average number of words per minute (wpm). Based on the literature and considering that this was not a transcription task, we established this rate at 50 wpm (Ostrach, 1997). Then we computed the number of words for each message to calculate the estimate time spent in writing it (Eq. (3))

EAudio ¼

EText ¼ Balance

ð1Þ

X ðendðmsgÞ  startðmsgÞÞ

X #wordsðmsgÞ  60 50

ð2Þ

ð3Þ

This implicit measure considers the balance of communication within a team (if participants exchanged similar number of messages). This was done by subtracting the volume of communication of each participant (PA and PB) within a team (Eq. (4)). The closer this value to zero, the more balanced the communication

BV ¼ jVðPA Þ  VðPB Þj

ð4Þ

1172

R. González-Ibáñez et al. / Information Processing and Management 49 (2013) 1165–1179

Table 4 Productivity measures for CIS. Productivity measure

Description

Universe (U) Universe of relevant webpages (Ur) Coverage (C) Unique Coverage (uC) Relevant Coverage (Cr) Unique Relevant Coverage (uCr) Precision (Prec), Recall (Rec), and F-measure (F-m) Useful webpages (UsW) Query diversity (QD)

The union of all webpages visited by all teams (Total pages = 1396; Distinct pages = 814) The union of all webpages bookmarked or from where one or more snippets were collected (Total pages = 571; Distinct pages = 348) The total number of distinct webpages visited by a given team within U The total number of pages visited only by given team within U (Total pages = 673) The total number of distinct relevant webpages visited by a given team within Ur The total number of relevant pages visited only by given team within Ur (Total pages = 212) Three of the most common evaluation measures in information retrieval. We compute this measures using the results from Ur, C, and Cr An implicit measure based on the dwell time on a webpage (at least 30 s) (Watson et al., 1988) Lavenshtein distance between pairs of queries for a given subject/team

As standard readability tests we used: Flesch Kincaid Grade Level (F-K) (Kincaid, Fishburne, Rogers, & Chissom, 1975), Automated Readability Index (ARI) (Senter & Smith, 1967), and Flesch Reading Ease (Kincaid et al., 1975). Generally speaking these tests indicate the difficulty of understanding a given text through the evaluation of different units of analysis such as characters, syllables, words, and sentences. Some of these tests have been used to assess magazines, handbooks, academic reports, and health messages, among other types of content. Although these measures share similar principles, we computed all three with the aim of ensuring consistency. In addition to automated readability analyses, documents were assessed by two external coders. They evaluated the reports based on four categories, namely content, resources, organization, and style, and for each category they gave a score ranging from 1 to 5, 1 being poor and 5 being excellent. As a result, a team can get up to 20 points as the final score. Content of reports was evaluated in terms of the amount and quality of information provided for the questions being asked in the task. Resources were evaluated based on the number as well as the variety of resources used in the report. Organization of reports was assessed with regards to the extent to which the information is synthesized and grouped in an organized manner. Finally, style was evaluated in regards to spelling/grammar, punctuation and capitalization. The inter-rater reliability for all four categories was found to be Kappa > 0.7 (p < 0.01). 5.4. Cognitive and affective measures In order to measure the cognitive load and affective experience of team members, we applied two instruments as pre-task and post-task questionnaires. First, we used a simplified version of NASA-TLX (Hart & Staveland, 1988) for measuring cognitive load of participants. Second, we used PANAS (Positive Affect Negative Affect Schedule) for measuring affective states of participants (Watson, Clark, & Tellegen, 1988). We adapted Nahl’s (2007) formulation to estimate the affective load of each participant (Eq. (5)).

Affective Load ¼ Uncertainty  TimePressure

ð5Þ

In the expression above (Eq. (5)), uncertainty can be expressed through specific negative affects such as anxiety, frustration, and irritation. Time pressure, on the other hand, was estimated through a specific question in the NASA-TLX questionnaire: ‘‘How hurried or rushed was the pace of the task?’’ 6. Results 6.1. Communication Our first research question addressed the communication dynamics of teams in different communication contexts in a CIS task. We used an existing coding scheme based on a CSCL task to identify the communication dynamics of teams in a CIS task (Table 2). During the coding process, our analysis revealed three subcategories of communication messages, namely: strategy (S), information seeking (IS), and awareness (A). These specific kinds of messages resulted from the nature of task and context (Table 5). More specifically, strategy subcategory included all types of statements regarding how to perform the task. Teams used different strategies for dividing up the task and writing the report. They negotiated when to stop searching and start writing the report. For example, in the following quote ‘‘ill do the impact on the economy and life (people and animals) in the gulf and attempts at fixing it and then we can both do the last one (long term implications and lessons learned) together’’ a team member proposed an initial plan to tackle the task without room for negotiation. In other examples (such as ‘‘wanna start at the top of the list and work your way towards the middle and ill start at the bottom and work my way towards to middle?’’ or ‘‘do you want to split it up or how do you want to do the project?’’) team members both suggested a strategy and left room for negotiation by asking to their partners if they agreed with the proposed plan. We also found the exchange of strategy messages in the middle of the session, suggesting dynamic adaptation. For example in the following

R. González-Ibáñez et al. / Information Processing and Management 49 (2013) 1165–1179

1173

Table 5 New categories identified through textual analysis. Subcategory

Description

Example

Strategy (S)

All types of statements regarding strategy

Information seeking (IS) Awareness (A)

All types of statements regarding coordination in information seeking All types of statements regarding the awareness of task or time

I guess what we can do to speed up everything is like skim the articles very relevant to the point we are trying to make and then snip it in Do you wanna use two different browsers, I mean search engines? I am gonna find one more I am on Forbes now; We have 8 min left

messages ‘‘last 2 min well reorganize our paper’’ and ‘‘at 3:45 lets try and start putting together the paper’’, participants adapted their initial strategy based on their progress during the session and the remaining time they had to complete the task. The information seeking subcategory included all types of statements regarding coordination with respect to the information seeking process such as which and how many information sources to use. For example, in the message ‘‘search ‘gulf oil spill overview’ and click on the 2nd result ‘times topics: gulf of mexico deepwater horizon oil spill’’’, the participant provided detailed directions to her partner to find a specific article. Interestingly, although Coagmento provides a feature to share bookmarks and snippets, the participant in this example preferred communication over the tool for sharing information. A similar case is observed in the following message ‘‘with google scholar i found 2 articles about the effect of the spill on human health and ecosystem health’’, in which the participant reported to his partner what seemed to be useful findings. Finally, the last subcategory included all types of statements regarding the awareness of task or time. For instance, team members made each other aware of which question they were addressing at the given moment, how much time left to finish up the task, or the guidelines of writing task. Examples of this kind of messages are: ‘‘im on nytimes.com’’, ‘‘we have less than 10 min lol’’, and ‘‘Oh, I was looking at the snippets. I see them now’’. These new themes align with the literature in that coordination and awareness are central in CIS (Shah, 2010b). Additionally, we investigated how communication context affects interactions of teams within a CIS task. Fig. 2 depicts the proportions of communication volume in each condition, in terms of task coordination, task content, task social, and nontask related. The results and pairwise comparison among the three conditions are reported in Table 6. As shown, we found that those with audio support (C1f2f and C3audio+text) produced a higher volume of communication (C1f2f = 153.33; C2text = 28.9; C3audio+text = 215.7), F = 8.698, p < 0.01. There were also significant differences in terms of communication measures among the conditions (Table 7). Our results showed that those in C1f2f and C3audio+text exchanged more task and nontask messages compared to those in C2text. However, those in C1f2f also exchanged more messages than those in C2text for task coordination, while no differences were found between C2text and C3audio+text. This shows that the participants in C3audio+text talked more about the task (good for problem solving) and non-task related topics (good for promoting social ties and engagement) than those in C2text, without having wasted any additional effort in coordination as in C1f2f did. Thus, C3audio+text got the best of both in C1f2f and C2text conditions. Second, an examination within teams revealed variations in terms of communication between partners across conditions. In C1f2f, teams produced similar amounts of task content and task social messages. In C2text condition, teams exchanged the majority of messages on task coordination and content with limited social messages. As well, teams in C2text had better balance of communication messages than those in C1f2f or C3audio+text. In other words, unlike those in audio supported

Fig. 2. Proportion of volume for each communication factor within condition.

1174

R. González-Ibáñez et al. / Information Processing and Management 49 (2013) 1165–1179

Table 6 Means for communication measures. Mean (standard deviation)

Volume Effort Balance Task coordination Task content Task social Non-task related

F(p)

C1f2f

C2text

C3audio+text

153.33 (40.1) 390.556 (125.981) 40.00 (9.042) 22.11 (11.952) 29.89 (18.169) 6.56 (4.362) 4.22 (3.701)

28.9 (18.248) 185.16 (106.066) 13.00 (8.138) 8.00 (4.372) 8.20 (5.138) 3.00 (3.162) 3.90 (3.348)

215.7 (165.235) 636.0 (424.412) 38.80 (27.1) 34.20 (30.622) 48.80 (35.627) 23.30 (28.546) 7.60 (6.204)

8.698 7.161 7.546 4.583 7.502 3.999 5.399

(0.001) (0.003) (0.003) (0.020) (0.003) (0.031) (0.11)

Table 7 Pairwaise comparison between conditions for each communication measure. Bold values indicate significant difference at p < 0.05. Bold and italic values indicate significant difference at p < 0.01. Bold and underlined values indicate borderline significant difference at p < 0.07.

conditions (co-present or remotely located), which had disparity in communication initiation/production, those in C2text exchanged messages only when a team member initiated a conversation either for a question or a comment. In such cases, other team members either answered to a question or confirmed to a feedback. In C3audio+text condition, teams had balance of task content, task coordination, and task social messages. Finally, although teams in C1f2f and C3audio+text were also allowed to exchange text messages, audio communication was predominant. Indeed, text messages were mainly used for interactions with the researcher to get answers for technical or task-related issues. Third, we also wanted to map out how teams working in a CIS task spend their communication efforts throughout the task. By doing a minute-by-minute analysis for each communication factor, we found certain patterns in terms of cumulative effort and cumulative task-content communication. From minutes 2 to 16, the teams in C3audio+text spent significantly more time communicating than C2text, but no significant difference between C1f2f and C2text was found. Likewise, with regard to task-content communication, we found that from minute 1 to 21, teams in C3audio+text dedicated more time discussing task content than teams in C2text. 6.2. Productivity Our second research question addresses the effect of co-located, text chat, and audio + text chat communication on performance of teams within a CIS task. We found no significant differences in productivity among conditions as measured by various forms of coverage and relevance (precision, recall, F-measure). To gain a better understanding of differences in

1175

R. González-Ibáñez et al. / Information Processing and Management 49 (2013) 1165–1179

working styles that may impact productivity, we looked at the amount of useful information (webpages) found (see Table 3). We discovered that while C1f2f performed at par with C2text and C3audio+text, there was a borderline significant difference between C2text and C3audio+text, giving those in C3audio+text an edge over C2text (p < 0.07). More importantly, we looked at the searches performed by every team and measured query diversity between the members of a given team to understand how interdependently they worked. We found that those in C1f2f achieved less diversity in searching than those in C2text or C3audio+text (Tables 8 and 9). 6.3. Information synthesis As one of the products of the CIS task that participants performed, we also explored the effects of different forms of communication on information synthesis within a CIS task. Evaluations based on readability tests and human assessment did not reveal significant differences among the final reports generated by teams in all three conditions. According to these tests, the average report would be better understood by people with at least 13 years of formal education, in other words, university students. In terms of human assessment, the average score of reports in a scale of 20 points was 13.56 (s.d. = 3.82) points. 6.4. Cognitive and affective factors Our third research question asked how communication context influences the cognitive load of teams within a CIS task. To better understand the implications of providing a specific communication channel, we looked at self-reported cognitive load on the participants. No differences were found between C1f2f and C2text, or C2text and C3audio+text, however, those in C3audio+text reported experiencing less cognitive load than those in C1f2f (see Tables 10 and 11). In terms of affective load, an analysis of the responses to the PANAS questionnaire revealed that the ratings to the negative emotional factors raised more for C1f2f than for C3audio+text between pre-task and post-task, indicating a heightened level of negative affects for those sitting in the same room. In addition, using the adapted formulation of Nahl’s (2007), we found significant difference (p<.05) between C1f2f and C3audio+text, which was lower for teams in the latter condition (Tables 10 and 11). 7. Discussion and implications The findings present the cost-benefit picture of each communication context in interaction and performance of teams. For example, we found that F2F setup allows team members to interact effortlessly, generating a larger volume of communication compared to those that collaborate remotely. While such effortless communication involves discussion about the task, more chatting on non-task related topics presents challenges for team performance. In fact, F2F participants were less diversified in their information exploration compared to remotely located teammates. Along with the literature presented above, computer-mediated context seems to limit the social aspects of communication in a CIS task, due to limited capacity of information exchange and extra effort needed for communication. For example, in the case of C2text, team members needed to stop doing their search or writing activities in order to chat with their collaborators. Although, such divided efforts could be time-consuming our results showed that chat messages were used more purposefully and effectively with a focus mostly on task-related conversations. The condition with remotely located participants with audio support (C3audio+text) seemed to be the condition with the best cost-benefit ratio of communication effort and task productivity. More specifically, the level of social presence capacity in this condition was sufficient to support interactions for decision making processes, but not too high to distract teams from doing work. As reported in the previous section, although text-chat channel was available for participants in C3audio+text condition, audio communication was predominant. An interesting finding was revealed in our minute-by-minute analysis. Teams in C3audio+text spent significantly more time communicating than C2text between minutes 2 and 16. We suggest the following explanation for this: As soon as teams in C2text decided on a strategy during the first minute, they may have preferred to avoid communication as much as possible, in order to conserve their efforts for the task. Likewise, the analysis in terms of task-content communication revealed that teams in C3audio+text spent more time discussing task content than those in C2text. Based on these findings, audio communication seems to be highly useful, especially at certain times during a CIS task to facilitate communication and coordination. These instances show that teams communicated strategically to compensate the limitations of the text channel, which allowed them to spend more time on the task.

Table 8 Means for productivity measures. Mean (standard deviation)

Useful webpages Query Diversity

F(p)

C1f2f

C2text

C3audio+text

18.10 (5.13) 19.69 (8.863)

16.90 (4.557) 23.13 (8.465)

22.10 (5.259) 22.29 (8.870)

2.975 (0.068) 21.056 (0.000)

1176

R. González-Ibáñez et al. / Information Processing and Management 49 (2013) 1165–1179 Table 9 Pairwaise comparison between conditions for each communication measure. Bold values indicate significant difference at p < 0.05. Bold and italic values indicate significant difference at p < 0.01. Bold and underlined values indicate borderline significant difference at p < 0.07.

Table 10 Means for cognitive and affective measures.

Table 11 Pairwaise comparison between conditions for each communication measure. Bold values indicate significant difference at p < 0.05. Bold and italic values indicate significant difference at p < 0.01.

With regards to our second research question which explored the implications of communication contexts in the productivity of teams working in a CIS task, we found no significant differences among conditions in terms of a set of measures of coverage and relevance. This could be attributed to the time-bound and recall-oriented nature of the task. That is, there was a sufficient amount of relevant material on the topic, and teams had 25 min to do searching and collecting information, as well

R. González-Ibáñez et al. / Information Processing and Management 49 (2013) 1165–1179

1177

as preparing the report. In other words, 25 min could be sufficient for teams in any condition to perform well in the CIS task as defined in this study. The analysis of diversity, however, showed that those working co-located (C1f2f) achieved less diversity while searching for information than those working in remotely-located conditions (C2text and C3audio+text). As the level of social presence increased, the level of information and opinion exchange increased, leading to similarity in search strategies and information access. On the other hand, remotely located participants were able to have more independence in their exploration than those co-located (C1f2f) due to diminished interaction in less socially present context. Overall, the findings show that C3audio+text gets the better of the other two conditions – it allows participants to achieve independence like C2text – something deemed to be important for a dividable task – while helping teams discover more useful information through their interactions. Through our third research question, we also explored the implications of communication contexts in the cognitive and affective load of teams within a CIS task. Our results indicated that those working face to face (C1f2f) experienced higher cognitive load than those working remotely located with an audio channel enabled (C3audio+text). Higher levels of social presence increased the levels of content related information exchange, and thereby increased the cognitive load of team members. Along these lines, being remotely located diminished the levels of interaction and task related information exchange and hence allowed participants to be more relaxed and distraction-free. These findings were also supported in our end-session interviews. Those in C1f2f reported often being distracted by having their teammate located in the same space. In terms of affective load, results indicated a significant increase of negative affects in participants working co-located (C1f2f) compared to those working in C3audio+text. Similar to cognitive load, this finding can be explained by the higher levels of information sharing in this condition. In particular, participants shared higher levels of task related social information through which they express their personal experiences while working on a task. For instance, participants expressed negative opinions when they could not find relevant information or when they stressed out due to time constraints while trying to finish writing the report. Although they also expressed their positive feelings when they identified relevant information, the amount of both positive and negative expressions was higher in this condition, leading to higher levels of affective load. Lower levels of social presence in audio context diminished the amount of task related social information exchange, leading to decreases in affective load. Although sharing negative opinions and experiences about the task may promote social support and positive ones may be motivating for participants, too much sharing of positive and negative experiences seem to diminish possible positive effects for the task. In that sense, we found that C3audio+text provides better conditions than C1f2f for managing both affective and cognitive loads. This study has several implications. For instance, it has been argued that F2F communication is the richest way of communicating (Daft & Lengel, 1984; Foster, 2006) and for years many have attempted to enrich mediated communication by providing resources that make mediated interactions close to face-to-face ones. Our findings suggest that F2F is not always better or more productive; the choice for a spatial setup and/or communication preference may depend heavily on the nature of the task. This is similar to the findings reported in the background section by McGrath and Hollingshead (1993) as the task-media fit hypothesis. It also supports hypotheses previously proposed by Shah (2010b), i.e., non-dividable tasks (e.g., brainstorming) are better performed in F2F, whereas easily dividable tasks (e.g., the task used in our experiment) benefit more from the collaborators working remotely (independently). Of course, this implication should be understood in proper context of the task requirements – we are not advocating a lack of social interactions or disconnected work by ‘‘independence,’’ since even independent work done in collaboration may require assessment of partial results, strategy definition, and awareness. Therefore, our findings indicate that there has to be adequate and appropriate amount of communication for the given CIS project. This notion of adequate and appropriate communication bears more discussion. We found that C2TEXT provided a good amount of independence among the participants while giving them minimal choices for communication. To promote more social interactions and interactions in general, teams could work in the same space (C1F2F), but as we found, this leads to less diversity and more distractions during their work. In other words, simply providing more freedom to communicate was not enough; the assigned task required at least some level of independence, which C1F2F was missing. C3AUDIO+TEXT provided more communication choices compared to C2TEXT, at the same time allowed the participants to be more independent than those in C1F2F. This also created more balanced cost-benefit factors for cognitive and affective load. C3AUDIO+TEXT provided both adequate and appropriate communication channels for the given CIS task. As a system designer, one could put these lessons in practice by understanding (1) the space and time contexts in which a CIS system will be used, (2) the nature of the task(s) performed, and (3) various expectations related to the task and social interactions. For instance, if the collaborators are working on strategy forming phase of a project, they may require F2F (if possible), audio and/or video (if remotely located) forms of communication, whereas for information gathering parts, the communication options could be taken away or put in periphery so that the participants could focus on the task. If, on the other hand, promoting social interactions among the participants is highly important (e.g., an educational group project), one may make audio or video channels prominent and/or mandatory. There are limitations associated to the experiment design. First, the experimental nature of the study lacks the effect of contextual factors in the performance of participants. Our work here was limited by the nature of the task (time-bound, divisible, and exploratory search), time context (synchronous), and group size (two members per team). For instance, organizational and educational context may offer intrinsic or extrinsic rewards that motivate or discourage the performance of teams. In addition, extrinsic factors such as time limit might also influence the type and amount of information shared

1178

R. González-Ibáñez et al. / Information Processing and Management 49 (2013) 1165–1179

through different communication modes. For instance, the assigned task was limited to 30-min, which might have affected the coordination of search activities. All together, these factors limit the generalizability of our findings and should be addressed in future studies. 8. Conclusion Through a controlled laboratory study, we assessed the pros and cons of three different setups of people working in synchronous collaborative information seeking (CIS) task. The contributions of the work reported here based on this study are several. First, we provided a methodology that is not often seen in studying collaboration; it included a well-informed study design, a unique combination of interesting and challenging task, and a system for providing CIS-related tools and logging user activities. Second, we proposed a set of evaluation measures that incorporated assessment of communicated messages (text or audio), productivity, and various cognitive and affective factors. Third, we showed costs and benefits of using different spatial settings while performing a CIS task. Our analysis demonstrated that for a divisible and exploratory search task with a recall-oriented goal, those with more independence achieve more diversity in information resources, which may help them explore a greater variety of relevant information, and thus, those with remotely-located participants were found to be more diverse in their exploration than colocated teams. Having more social presence increased interactions among the participants as it was natural for them to talk to each other while working, but these interactions were often found to be distracting for a time-bound task. One may argue that more interactions could be beneficial characteristics to have for improving participants’ engagement in collaboration even if it means sacrificing some productivity. While that may be the case, we showed that remotely located participants with audio support could achieve the best of both the setups; they exercised the same amount of independence as those remotely located with only text interactions, while carrying out just as many interactions with each other as those in the co-located condition with fewer distractions. We have shown that the implications of these contributions go beyond addressing the research questions investigated here, by providing guidelines to balance costs and benefits of various communication channels while supporting group work. These contributions are valuable not simply for CIS, but the larger domain of human–computer interaction and information retrieval (HCIR). For instance, the reported findings demonstrate how the issues of control, communication, and awareness could be balanced in an interactive information retrieval task by understanding various trade-offs with different information seeking conditions. These issues are considered to be highly important for designing and evaluating user-centered IR systems (Rodden, 1991; Rodden & Blair, 1991). References Anderson, A., Newlands, A., Mullen, J., Flemming, A. M., Doherty-Sneddon, G., & Van der Velden, J. (1996). Impact of video-mediated communication on simulated service encounters. Interacting with Computers, 8, 193–206. Borlund, P., & Ingwersen, P. (1999). The application of work tasks in connection with the evaluation of interactive information retrieval systems: Empirical results. In MIRA: Electronic Workshops in Computing (pp. 29–46). Clark, H. H., & Brennan, S. E. (1991). Grounding in communication. In L. B. Resnick, R. M. Levine, & S. D. Teasley (Eds.), Perspectives on socially shared cognition (pp. 127–149). The American Psychological Association. Daft, R., & Lengel, R. (1984). Information richness: A new approach to managerial behavior and organizational design. In B. Straw & L. Cummings (Eds.), Research in organizational behavior. Greenwich, CT: JAI Press. Doherty-Sneddon, G., Anderson, A., O’Malley, C., Langton, S., Garrod, S., & Bruce, V. (1997). Face-to-Face and video-mediated communication: A comparison of dialogue structure and task performance. Journal of Experimental Psychology: Applied, 3, 105–125. Dourish, P., & Bellotti, V. (1992). Awareness and coordination in shared workspaces. In Proceedings of the 1992 ACM conference on Computer-supported cooperative work (pp. 107–114). Fidas, C., Komis, V., Tzanavaris, S., & Avouris, N. (2005). Heterogeneity of learning material in synchronous computer-supported collaborative modeling. Computers & Education, 44, 135–154. Foster, J. (2006). Collaborative information seeking and retrieval. Annual Review of Information Science and Technology (ARIST), 40, 329–356. González- Ibáñez, R., & Shah, C. (2011). Coagmento: A System for supporting collaborative information seeking. Integrated demo at ASIST’2011, New Orleans, LA. Grudin, J. (1994). Groupware and social dynamics: Eight challenges for developers. Communications of the ACM, 37(1), 92–105. Hart, S. G., & Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In P. A. Hancock & N. Meshkati (Eds.), Human mental workload (pp. 239–250). North Holland Press. Howarth, B., & Anderson, A. H. (2007). Introducing objects in spoken dialogue: The influence of conversational setting and cognitive load on the articulation and use of referring expressions. Language and Cognitive Processes, 22, 272–296. Hyldegard, J. (2006). Collaborative information behaviour – Exploring Kuhlthau’s Information Search Process model in a group-based educational setting. Information Processing and Management, 42, 276–298. Hyldegard, J. (2009). Beyond the search process – exploring group members’ information behavior in context. Information Processing and Management, 45, 142–158. Kincaid, J. P., Fishburne, R. P., Rogers, R. L., & Chissom, B. S. (1975). Derivation of new readability formulas (automated readability index, fog count, and Flesch reading ease formula) for Navy enlisted personnel. Research branch report 8–75. Chief of Naval Technical Training: Naval Air Station Memphis. Kratzer, J., Leenders, R. T. A., & Van Engelen, J. M. (2006). Managing creative team performance in virtual environments: an empirical study in 44 R&D teams. Technovation, 26(1), 42–49. Kuhlthau, C. C. (1991). Inside the search process: Information seeking from the user’s perspective. Journal of the American Society for Information Science and Technology, 42(5), 361–371. Large, A., Beheshti, J., & Rahman, T. (2002). Gender differences in collaborative web searching behavior: An elementary school study. Information Processing and Management, 38, 427–433. Matsuura, N., Fujino, G., Okada, K., & Matsushita, Y. (1993). An approach to encounters and interaction in a virtual environment. In Proceedings of ACM computer science conference. Association for computing, machinery (pp. 298–303).

R. González-Ibáñez et al. / Information Processing and Management 49 (2013) 1165–1179

1179

McGrath, J. E., & Hollingshead (1993). Groups, tasks, and technology: The effects of experience and change. Small Group Research, 24, 406–420. Mennecke, B. E., Valacich, J. S., & Wheeler, B. C. (2000). The effects of media and task on user performance: A test of the task-media fit hypothesis. Group Decision and Negotiation, 9, 507–529. Morris, M. R. (2007). Interfaces for collaborative exploratory web search: Motivations and directions for multi-user designs. CHI 2007 Workshop on Exploratory Search and HCI. Morris, M. R. (2008). A survey of collaborative web search practices. Proceeding of the twenty-sixth annual CHI conference on human factors in computing systems – CHI’08.2008:1657. Morris, M. R., & Horvitz, E. (2007). Search together: An interface for collaborative web search. Proceedings of UIST, 2007, 3–12. Nahl, D. (2007). The centrality of the affective in information behavior. In D. Nahl, & D. Bilal (Eds.), Information and emotion (pp. 3–37). Medford, NJ: Information Today. Newlands, A., Anderson, A. H., & Mullin, J. (2003). Adapting communicative strategies to computer-mediated communication: An analysis of task performance and dialogue structure. Applied Cognitive Psychology, 17, 325–348. Olson, J. S., Olson, G. M., & Meader, D. K. (1995). What mix of video and audio is useful for small groups doing remote real-time design work? In Proceedings of the ACM CHI’95 conference on human factors in computing systems (pp. 362–368). New York: ACM Press. Ostrach, T. R. (1997). Typing speed: How fast is average. Orlando, FL: Five Star Staffing, Inc. Reddy, M. & Dourish, P. (2002). A finger on the pulse: Temporal rhythms and information seeking in medical work. In Proceedings of the ACM conference on computer-supported cooperative work CSCW 2002, Chicago, IL. Reddy, M. C., & Jansen, B. J. (2008). A model for understanding collaborative information behavior in context: A study of two healthcare teams. Information Processing and Management, 44(1), 256–273. Rodden, T. (1991). A survey of CSCW systems. Interacting with Computers, 3(3), 319–353. Rodden, T., & Blair, G. (1991). CSCW and distributed systems: the problem of control. In Proceedings of ECSCW (pp. 49–64). Amsterdam, The Netherlands. Satar, H. M., & Özdener, N. (2008). The effects of synchronous CMC on speaking proficiency and anxiety: Text versus voice chat. The Modern Language Journal, 92, 595–613. Senter, R. J., & Smith, E. A. (1967). Automated readability index. Aerospace Medical Research Laboratories. Shah, C. (2010a). Coagmento – A collaborative information seeking, synthesis and sense-making framework. Integrated demo at CSCW 2010. Savannah, GA: February 6–11, 2010. Shah, C. (2010b). Collaborative information seeking: A literature review. Exploring the digital frontier advances in librarianship, 32. Shah, C., & González- Ibáñez, R. (2011). Evaluating the synergic effect of collaboration in information seeking. In Proceedings of the 34th annual international ACM SIGIR conference on research and development in information retrieval. Beijing, China, July 24–28. Shah, C., & Marchionini, G. (2010). Awareness in collaborative information seeking. Journal of American Society of Information Science and Technology (JASIST), 61(10), 1970–1986. Shaw, M. E. (1981). Group dynamics: The psychology of small group behavior. New York: McGraw Hill. Short, J., Williams, E., & Christie, B. (1976). The social psychology of telecommunications. London: Wiley Press. Sonnenwald, D. H. (1996). Communication roles that support collaboration during the design process. Design Studies, 17, 277–301. Sonnenwald, D. H., & Pierce, L. G. (2000). Information behavior in dynamic group work contexts: Interwoven situational awareness, dense social networks and contested collaboration in command and control. Information Processing and Management, 36, 461–479. Strijbos, J. W., Martens, R. L., Jochems, W., & Broers, N. J. (2004). The effect of functional roles on group efficiency: Using multilevel modelling and content analysis to investigate computer-supported collaboration in small groups. Small Group Research, 35, 195–229. Tang, A., Pahud, M., Inkpen, K., Benko, H., Tang, J. C., Buxton, V. (2010). Three’s company: Understanding communication channels in three-way distributed collaboration. In Proceedings of CSCW 2010. Twidale, M., Nichols, D., & Paice, C. (1997). Browsing is a collaborative process. Information Processing and Management, 33, 761–783. Watson, D., Clark, L. A., & Tellegen, A. (1988). Development and validation of brief measures of positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology, 54(6), 1063–1070. White, R. W., & Huang, J. 2010. Assessing the scenic route: Measuring the value of search trails in web logs. In Proceedings of the annual ACM conference on research and development in information retrieval (SIGIR). Geneva, Switzerland. Whittaker, S., & O-Conaill, B. (1997). The role of vision in face-to-face and mediated communication. In E. E. Finn, A. J. Sellen, & S. B. Wilbur (Eds.), Videomediated communication (pp. 23–50). Mahwah, NJ: Lawrence Erlbaum. Yuan, Y., Fulk, J., & Monge, P. R. (2007). Access to information in connective and communal transactive memory systems. Communication Research, 34, 131–155.