Keeping the family together: Sustainability and modularity in community source development

Information and Organization 30 (2020) 100274

Contents lists available at ScienceDirect

Information and Organization journal homepage: www.elsevier.com/locate/infoandorg

Keeping the family together: Sustainability and modularity in community source development

T

⁎

Manlu Liu , Sean Hansen, Qiang Tu Rochester Institute of Technology, Max Lowenthal Hall, 107 Lomb Memorial Drive, Rochester, NY 14623, United States of America

A R T IC LE I N F O

ABS TRA CT

Keywords: Community source Modular design Kuali Open source Sustaining Collaboration Software development Design Structure Matrix

Community-based open source software development (“community source”) is an approach that leverages selected strengths of both the commercial and open source software development. The community source approach centers on the formation of formal communities among institutions based on principles of shared investment and collaborative action to achieve mutually-desired goals. However, as with ordinary open source projects, sustainability can be a significant challenge as development communities grow and evolve. In the present study, we explore the opportunities and challenges to the community source approach through a longitudinal study of a real world community source project, Kuali. Specifically, we apply the principles of modular systems design and design structure matrix modeling to achieve an in-depth understanding of the mechanisms for sustainability in collaborative systems development. In so doing, we introduce a quantitative measure of architectural modularity within an inter-organizational structure. A number of implications are presented for the conduct of research on community source development as well as the design of collaborative communities more broadly defined.

1. Introduction Large-scale information technology (IT) investment decisions are far from trivial undertakings, with enterprise systems implementation representing one of the single largest investments many organizations ever make. Organizations may struggle to find enterprise application software in the commercial marketplace that satisfies their specific needs or processes. At the same time, the costs of in-house development or the customization of commercial software are often prohibitively high. As a result, organizations seek a way to be more effective in developing their desired software. An emerging approach to the development of enterprise application software focuses on the formation of formal communities around the development mechanism of open source software (Agerfalk & Fitzgerald, 2008). This approach is referred to as community-based open source, or simply “community source” (Liu, Wang, & Zhao, 2012; Wheeler, 2004). These communities are formed based on principles of shared investment and collaborative action to achieve mutually-desired goals. Collaborative communities require members to limit their autonomy in the interest of directed action to pursue shared objectives (Wheeler & Hilton, 2012). The community source approach can be understood as a hybrid form of development that lies between commercial and “traditional” open source development (Liu, Hansen, & Tu, 2014). In Raymond's (2004) widely-cited metaphor, commercial software development is analogized to builders constructing cathedrals, with extensive planning beforehand and a long, linear build process. By contrast, open source software development is equated with a bazaar, a loosely organized marketplace that harnesses the energies and interests of diverse actors, remaining open to new ideas and new participants. Extending this metaphor, a community source ⁎

Corresponding author. E-mail address: [email protected] (M. Liu).

https://doi.org/10.1016/j.infoandorg.2019.100274 Received 12 July 2018; Received in revised form 20 July 2019; Accepted 7 November 2019 1471-7727/ © 2019 Elsevier Ltd. All rights reserved.

Information and Organization 30 (2020) 100274

M. Liu, et al.

project might be described as a shopping mall, where diverse participating organizations maintain significant autonomy at the same time that they coordinate efforts to pursue common objectives (Wheeler & Hilton, 2012). Community source creates a formal virtual organization where multiple partner institutions collaborate with each other to develop custom software solutions (Liu, Hull, & Hung, 2017). Three features that distinguish the community source approach from more common open source phenomena are worth noting. First, unlike “traditional” open source development, community source reflects a formal collaboration among multiple institutions via a virtual organization. Second, all of the application developers in a community source effort are employees assigned to the project by the partner institutions. Third, the community source project requires significant initial investment by the partner institutions. While some of these features appear in other types of open source projects, a community source project reflects all of them. In resting between the commercial and open source sides of the development spectrum, the community source approach draws certain benefits from each. One of the most prominent advantages of the community source approach is a significant reduction in costs while developing targeted systems. The pooling of resources by participating organizations enables them to focus on the functionality that they desire at a much lower cost to each organization than would be incurred with commercial software purchases and customization. The community also benefits from the open innovation of the open source platform. By enabling individual participant organizations to experiment with modifications that meet their unique needs (adaptations which in turn are shared with the rest of the community), the community source effort fosters a much broader solution set than a standardized platform would allow. At the same time, community source projects maintain an institutional imperative, which enables the community to direct development efforts, rather than simply relying upon the interests of volunteer developers. In this way, the community source approach leverages certain strengths of both commercial and open-source development. As one might expect, the community source approach's unique combination of benefits are mirrored by a number of challenges, especially with regard to coordination and community sustainability (Liu et al., 2014). Coordinating the efforts of distinct partner organizations becomes increasingly difficult as the community grows. Furthermore, a change in the goals or vision among the members of a collaborative community might shift behaviors and commitments even to the point of community dissolution. These observations raise a number of critical questions for the undertaking of community source software development: How can a growing development community be sustained in the long term? Is there an architecture that will enable the creation of strong and vibrant communities, providing an organic roof for the growing community? There is a clear need for a theoretically rigorous and practically relevant framework for understanding organizational mechanisms in community source projects. In this study, through a longitudinal case study of a real world community source project called Kuali, we try to better understand the mechanisms of community collaboration and the processes involved in developing and sustaining a community as it grows. In analyzing the Kuali case, we draw upon the principles of modular design and theories of modularity.1 As Langlois (2002) notes, “Modularity is a very general set of principles for managing complexity” (p. 19). Contemporary theories of modularity have their origins in the early development of the cognitive science discipline (Seok, 2006) and have been widely employed in the field of computer science (Sutcliffe, 2002). More recently, the concepts of modularity have been applied to the question of organizational design (Langlois, 2002). In this study, we apply modularity theory in the context of community source development and examine how communities achieve sustainability through the modular design of organizational structures. Furthermore, we examine how modularity can be applied among collaborating parties in community-level and project-level activities. In developing this analysis, we employ the representational tool of the design structure matrix (DSM; Eppinger & Browning, 2012) to model the degree of modularity in the Kuali community at different phases in its evolution. A quantitative measure of modularity is also developed to measure the level of modularity accordingly. The research questions that guide our study are as follows: RQ1: What are the challenges to sustaining community source development in a growing community? RQ2: How can collaborative communities achieve sustainability in the long term? This research offers two primary contributions. First, to the best of our knowledge, this is the first study to use design structure matrix modeling and a quantitative measure of modularity to evaluate the level of modularity in an inter-organizational structure. We integrate research on both modular organizational design and software development in an effort to understand community source development in a growing community. Secondly, building upon our analytical findings, we provide a critical lens for identifying threats to the sustainability of communities that arise with growth and complexity. We believe this methodology can be readily applied to other inter-organizational collaborations and organizational contexts. In the next section of the paper, we introduce the Kuali initiative, the real world community source project at the center of our analysis. We then review the literature on modular design, with particular emphasis on the general modular systems theory (Schilling & Steensma, 2001). In the subsequent section, we outline the research methodology employed. The fifth section of the paper presents the findings from our analysis, including the challenges faced in the management of the focal community source project, the development of design structure matrices to model organizational structures of the community in four distinct phases, and the quantitative measurement of modularity in each of these phases. We then turn to a discussion of the insights that flow from our analysis, centered on the evolution of the design structure matrix at each phase which highlights ways to enhance modular design in community source. Finally, we discuss the implications for research and practice before offering some concluding points for

1 Rather than being applied a priori, the applicability of modular design theory emerged from the initial analysis of the case study data. As we discuss in the Research Methods section, the emerging focus on modularity did inform our subsequent analyses.

2

Information and Organization 30 (2020) 100274

M. Liu, et al.

consideration.

2. The Kuali case “But I don't think the real story [of community source] is about open source or particular products. What is really momentous is the pace at which the higher education community is coming together to meet its own needs.” – Chairman of Kuali Kuali is a comprehensive suite of administrative software that meets the needs of the higher education sector. The original motivation for the Kuali project was that existing financial systems used in the initiating universities were outdated and too difficult to maintain. Commercial products are often too expensive and hard to customize, with some colleges and universities needing to operate expensive “shadow systems” to provide desired features absent from commercially available ERP packages. On the other hand, building a financial system in house is equally daunting and can only be considered by the largest universities. The Kuali project provides an attractive alternative to this “buy or build” dilemma (Liu, Wu, Zhao, & Zhu, 2010). It pools institutional resources to develop an open source financial system, thus dramatically reducing the cost of managing fiscal data and processes in higher education. Kuali was initiated in 2004, with the migration of Indiana University's financial information system to an open source web-based platform. Indiana University and four other development partners collaborated to build the system with an eye to the unique financial management needs of the higher education environment. The partner institutions are generally colleges and universities that share a common vision of open, modular, and distributed systems for their software requirements. In addition to these educational institutions, the community includes a number of commercial affiliates – private, for-profit organizations that provide guidance, support, implementation, and integration services related to the Kuali software in exchange for a fee. From its inception in 2004, Kuali has grown rapidly. By 2011, more than seventy universities and institutions had joined Kuali. In 2016, the number of development partners reached over one hundred. While the initial mission of the Kuali project focused on the development of a baseline system for financial services, the community's efforts have expanded to include modules for research administration (Kuali Coeus), student information management (Kuali Student), library management (Kuali Open Library Environment; OLE), business continuity (Kuali Ready), human capital management (Kuali People Management for the Enterprise; KPME), mobile device integration (Kuali Mobility), and middleware

PROJECT BOARD FUNCTIONAL COUNCIL

Financial System

Coeus for Research Administraon

Student System

Business Connuity Planning (SaaS)

Open Library Environment

People Management for the Enterprise

Mobility

PROJECT MANAGEMENT DEVELOPMENT TEAMS SUBJECT MATTER EXPERTS

APPLICATION ROADMAP COMMITTEE

PROJECT BOARD ENTERPRISE WORKFLOW

RAPID APP DEV

PROJECT MANAGEMENT

ENTERPRISE NOTIFICATION

IDENTITY MANAGEMENT

DEVELOPMENT TEAMS

TECHNOLOGY ROADMAP COMMITTEE

SERVICE BUS

MEMBER INSTITUTIONS

COMMERCIAL AFFILIATES

BOARD OF DIRECTORS LEGAL SERVICES INTELLECTUAL PROP.

FINANCIAL MANAGEMENT HOSTING SERVICES

LICENSING

COLLABORATION INFRASTRUCTURE.

COMMUNITY COORDINATION

VENT MANAGEMENT

Fig. 1. Kuali systems (from the presenatation by the Chair of Kuali Foundation at Kuali Days 2011). 3

Information and Organization 30 (2020) 100274

M. Liu, et al.

integration (Kuali Rice) (Fig. 1). As the most mature project, Kuali Financial System (KFS) is now the institutional financial system for a wide range of colleges, large and small. The research administration (Kuali Coeus) and middleware modules (Kuali Rice) are also mature and installed fairly widely (www.kuali.org). In an effort to sustain the rapidly growing community, the Kuali governance structure has evolved significantly since the community's inception. The initial governance structure for Kuali was informed by an earlier community source project called Sakai. The Sakai project (www.sakaiproject.org) was focused on the development of a new collaboration and learning environment for higher education and included participation of several of Kuali's founding institutions. For these Kuali founders, the perceived shortcomings of the Sakai effort provided valuable lessons about community governance. Sakai was mainly led by technical staff. The lack of communication between technical staff and managerial staff was a significant source of concern and frustration on the project. To guard against this challenge, the Kuali community created both a functional council and a technical council in the organizational structure, with the two councils taking ownership of the functional and technical issues of Kuali, respectively. The early success of KFS brought more projects to the Kuali community. As Kuali expanded to include eight projects, there was an increasing demand for resource sharing among projects. The governance structure needed to be refined to support these collaborations. Multiple modules are developed under each project, and some modules can be reused across projects, fostering connections among projects. This expansion and exchange has engendered a variety of changes to the governance structure over the years, including greater specialization through the articulation of new roles and responsibilities, the development of new units to oversee different facets of the design process, and new mechanisms for inter-organizational collaboration. Interestingly, the evolution of the Kuali community reflects a greater emphasis on modularity in the design of community structures. To gain theoretical insights into this evolution and its implications for the long-term viability of the Kuali project, we believe it is fruitful to review the existing theory on modularity as applied to organizational design. 3. Modularity theory Techniques for dividing effort and knowledge are fundamental to the creation of highly complex manmade things. – Baldwin & Clark, Design Rules 3.1. Modularity in design Complexity is an inherent characteristic of systems in which multiple parts interact in dynamic ways to achieve certain ends. Specifically, complexity emerges from both the diversity of components2 that make up a system and the interactions and interdependencies between those components (Langlois, 2002). As a result of the unpredictability that it engenders, complexity is generally considered a challenge to be overcome in the design, instantiation, and management of artificial and socio-technical systems. One commonly-employed strategy for addressing complexity in designed systems is an emphasis on modularization (Ethiraj & Levinthal, 2004; Pil & Cohen, 2006). Indeed, modularity has been defined as “a very general set of principles for managing complexity” (Langlois, 2002, p. 19). While the concept of modularity has a long history, recent research reflects a renewed interest in the value of modularity for the design of diverse social and technical systems (Baldwin & Clark, 2000; Ethiraj & Levinthal, 2004; Pil & Cohen, 2006). Precise definitions of modularity vary widely (Seok, 2006), yet a general description is that modularity in the context of system design reflects an emphasis on decomposing a system into multiple discrete sets of components (i.e., modules), with clear and limited mechanisms for communication or interaction between those different sets. This general description highlights several critical features. First, to achieve modular design, we must be capable of taking a whole system and breaking it into discrete modules, or subsets of components. This process of segmenting a system into distinct modules is the fundamental concept of decomposition (Simon, 1962). Secondly, the relative independence of the resulting modules must be enforced. One of the defining characteristics of modular designs is that the degree of interaction between components within a module greatly exceeds the interactions between separate modules (Seok, 2006), reflecting the principle of self-containment (Gunnar & Maratsos, 1992). With respect to within-module interactions, a module should display high cohesion, which reflects the degree to which the components of a module “belong together” or are functionally consistent (Stevens, Myers, & Constantine, 1974; Yourdon & Constantine, 1979). Indeed, in an ideally cohesive module, all of the components would be directed toward the execution of a single function (Perepletchikov, Ryan, & Frampton, 2007). Third, while interactions between modules are less intense than those within modules, a mechanism for communication and relationships between modules must exist. In modular design, this is achieved through the creation of interfaces (Baldwin & Clark, 2000). Interfaces represent the rules for the flow of information and action from one module to another. Drawing upon the notion of self-containment, effective modularization means that such communication is limited to what is required by an interface. Other information or visibility of action occurring within a module should be hidden from other modules within the system. This principle of information hiding (Parnas, 1979) reinforces the relative independence of modules in a 2 In this analysis, the terms component and part are often used interchangeably. In modular systems, a component represents “a physically distinct portion of the product that embodies a core design concept and performs a well-defined function” (p. 11; Henderson & Clark, 1990). As such, a component is a key discernible part of a system. Furthermore, components may be nested – i.e., components may be made up of other, lower-level components. For the sake of clarity, we use components to refer to lower-level parts of a system and modules for the discrete sets of components that serve a specific purpose within the system.

4

Information and Organization 30 (2020) 100274

M. Liu, et al.

system. Finally, as a by-product of self-containment, cohesion, and information hiding, modular systems are generally characterized by loose coupling. Coupling refers to the degree to which modules (i.e., discrete sets of components) of a system are tied together and dependent upon one another. Tight coupling reflects situations in which the modules of a system are highly-interdependent and therefore relatively difficult to separate. By contrast, loose coupling reflects conditions in which the modules of a system interact or respond to one another in meaningful ways, but they also maintain a degree of independence (Sanchez & Mahoney, 1996; Weick, 1976). Relative to tightly coupled systems, loosely coupled systems allow for adaptation and variation, because modules can be added, removed, or rearranged in novel ways with minimal impact on the other modules. Thus, loose coupling enables continuous change in designs (Sanchez & Mahoney, 1996). To illustrate these various principles of modularity, consider the example of an organization. In an organization with a significant level of modularity, several characteristics would be discernible. First, the organization would be made up of many individuals (components), with those individuals being organized into multiple, relatively independent sub groups or business units (modules). The business units would be relatively easy to identify within the broader organization (decomposition), because each business unit would serve a clear purpose or function which was distinct from that of other business units (coherence). Furthermore, the degree of interaction between individuals within a business unit would be greater than the interaction of individuals from different business units (self-containment). The need for individuals in one business unit to know the detailed processes of another business unit would be low (information hiding) as long as they knew how to exchange relevant information with those other business units (interfaces). As a result of all these qualities, the business units would be relatively independent (loose coupling). This example then may engender the question of why or under what conditions an organization would pursue a modular design.

3.2. Organizational modularity In a series of studies, Schilling (Schilling, 2000; Schilling & Steensma, 2001) developed the general modular systems theory, which addresses the conditions under which modularity tends to emerge or increase in the design of organizational forms. This theory states that pressures toward or away from modularity are a reflection of multiple factors, including the heterogeneity of both inputs and demands, the benefits of asset or skill specificity, the presence of market standards, the pace of technological change, and the competitive intensity within a marketplace (Schilling, 2000). While the latter three factors are general qualities of a given market, the other factors may warrant some explanation. Returning to the example of an organization, heterogeneity of inputs would reflect the diversity of skills, knowledge, and technology available to, or possessed by, the organization. Heterogeneity of demands relates to the range of different customers and their specific expectations of the organization's products or services. Specificity in assets or skills (which Schilling dubs “synergistic specificity”) represents the degree to which assets within an organization or the skills of its members are tailored to a particular business process or configuration. To illustrate the application of the theory, Schilling and Steensma (2001) highlight multiple examples of more modular organizational forms that have emerged in recent years, including contract manufacturing, alternative work arrangements (e.g., contractbased or temporary employment), and inter-organizational strategic alliances. Fig. 2 provides an overview of the general modular systems theory. Within this model, the heterogeneity of inputs and demands are primary drivers for increased modularity. In environments marked by heterogeneous inputs, organizations will tend to move toward more modular forms, because this heterogeneity increases the value of the reconfiguration and adaptation that modularity affords. Similarly, heterogeneity of demands in the marketplace, will

Fig. 2. General modular systems theory (Adapted from Schilling & Steensma, 2001). 5

Information and Organization 30 (2020) 100274

M. Liu, et al.

promote a movement toward modularity, because diverse demands can be better addressed with the potential for dynamic reconfiguration. The tendency for increased modularity is countered by gains that can be achieved through synergistic specificity (Schilling, 2000): “Through the optimization of [modules] for a particular configuration, an integrated system might achieve functionality unobtainable through the combination of more independent [modules]” (p. 1154). If organizations can obtain significant scale efficiencies through the creation of highly specific assets and processes (e.g., a unique piece of equipment or production process), these gains may outweigh the benefits of flexibility and reconfiguration offered by modular design. Finally, the model posits a number of additional forces which moderate the influence of heterogeneity of inputs and outputs on the emergence of modularity. Schilling and Steensma (2001) refer to these collectively as catalysts, and they include the availability of standards, the pace of technological change, and the competitive intensity within the marketplace. The availability of market standards enhances a trend toward modularity in that such standards reduce specificity and the potential gains associated with it. A rapid pace of technological change within an industry will similarly enhance the tendency toward modular organizational forms. Industries marked by high levels of competitive intensity will tend to foster modular forms. 3.3. Application of the general modular systems theory In applying the general modular systems theory (Schilling & Steensma, 2001) to the Kuali case, we see significant impetus for the adoption of modular organizational structures in the community source context. We briefly review the evidence in the Kuali case for each of the core constructs in the Schilling and Steensma (2001) model. 3.3.1. Heterogeneity of inputs The heterogeneity of inputs within an environment includes the diversity of technological options available and the differential capabilities of institutions (e.g., with respect to in-house software development or maintenance). In the Kuali setting, both of these factors suggest a high level of heterogeneity. The diversity of technological options is high because of a wide range of legacy systems and vendor platforms, multiple development options, and diverse functionality with platforms. First, the technology environments confronted by the Kuali member institutions vary widely. The Kuali community reflects participation from a broad gamut of educational institutions – from large public universities to small liberal arts colleges. Similarly, the technological environments with which the Kuali technologies must conform in development and implementation are markedly diverse, ranging from large enterprise platforms offered by multinational software firms such as Oracle and SAP to homegrown applications created by localized developers. Secondly, the Kuali technologies themselves reflect significant heterogeneity. The Kuali community employs a vast array of up-todate approaches and technologies, including open source software components, workflow automation, and service-oriented architecture (SOA). At the center of the Kuali system is the Kuali enterprise service bus (KSB), the core concept of service-oriented architecture that coordinates the interactions between an application module and the various lower-level system services. Kuali also has a modularized middleware infrastructure called Rice, sitting between the server and applications. The service centric approach of Kuali makes it possible to “develop once and use anywhere and on any platform.” Kuali started as a financial system and has evolved to an umbrella suite of administrative higher education systems. Finally, the Kuali community prides itself on the diversity of approaches to development and implementation that it enables. The technical capabilities of the Kuali member institutions vary widely, with the larger universities having more resources and diverselyskilled personnel than smaller institutions. In addition, access to personnel differs across the member institutions based on geographic or regional idiosyncrasies. Thus, the differentiation in the capabilities of member institutions is high in the Kuali community. 3.3.2. Heterogeneity of demands Significant customer heterogeneity in desired functions and scale is observed in the Kuali community. As noted above, the Kuali membership reflects very different types of schools – from small liberal arts colleges to sprawling multiple-campus universities. As such, the information needs of these members vary widely. Indeed, this heterogeneity of demands is evidenced by the emergence of multiple Kuali platforms with varying functionality and focus. The Kuali Foundation is now home to eight projects (Financial, Research Administration, Student, HR/Payroll, Middleware/Workflow/Identity, Library, Business Continuity Planning, and Mobile) that are in varying stages of maturity. With the exception of the KFS platform around which the community was initially formed, each of these projects was initiated based the specific IS needs of a subset of the member institutions. The diversity of needs and motivations is frequently noted by Kuali participants. 3.3.3. Catalysts In the general modular systems theory, catalysts moderating the impact of heterogeneity of inputs and demands include the availability of standards, the pace of technological change, and the competitive intensity within the marketplace. In the Kuali case, these catalysts are again fairly prominent. Information system capabilities are in a state of constant flux due to the speed of technological change. As a community focused on the development and implementation of software, the Kuali participants consistently recognize this market dynamic. Accordingly, system flexibility plays a critical role in enhancing the robustness of system development in Kuali. In Kuali, the degree of competitive intensity can be assessed at multiple levels. In higher education, competitive intensity is generally high, because institutions are trying to attract and matriculate the same students. However, in the Kuali community, this level of competition tends to be downplayed as universities collaborate to address shared information management needs, but a second 6

Information and Organization 30 (2020) 100274

M. Liu, et al.

level of competitive intensity is noteworthy. In the IT platform space, competitive intensity is high because a few large players are vying for enterprise platform business. This is the space that Kuali has entered by offering institutions an alternative to the large commercial software vendors. The juxtaposition of Kuali with these dominant market players is a consistent topic in discussions of the Kuali community. Since Kuali systems are designed particularly for particular institutional niche (i.e., higher education), the availability of standards for data management and retrieval are quite high. While the need for specific software functionality varies by member institution, the data storage and reporting needs of the member institutions are largely consistent. In addition, standards in the domains of data management (e.g., relation database structures) and software development (e.g., CMMI, SOA) are inherited from the broader domains of software engineering and IT management. 3.3.4. Synergistic specificity The final facet of the general modular system theory is the countervailing force of synergistic specificity – i.e., performance advantages (or cost savings) achieved through the tight linkage of particular resources or activities (e.g., business units or business processes). In Kuali, the forces favoring synergistic specificity are low. There is limited scale efficiency gained through creating a single, tightly-coupled configuration of member universities or development teams, because the individual member institutions have varying information needs. Considering the Kuali experiences holistically, this analysis suggests that the conditions encountered by the community source initiative heavily favor the adoption of modular design. In the analysis that follows, we explore the degree to which such modular design can be observed in the Kuali community and consider the relationship between the community's organizational structure and the challenges that it experiences. Before turning to the findings from our longitudinal case study of the Kuali community, we briefly review the research methods employed in this analysis. 4. Research method In this research, we conduct a multi-methods analysis of data from a longitudinal case study of the Kuali community (Yin, 2009). Our case inquiries were conducted in accordance with prevailing case study field procedures, including the development of a case study protocol prior to data collection, triangulation using multiple sources of evidence, and the maintenance of a chain of evidence (Paré, 2004; Yin, 2009). The data collection effort included semi-structured interviews with Kuali participants, direct observation of project events, milestone review presentations and minutes of meetings, and formal project documentation. Our examination of the Kuali initiative began in 2006. We conducted formal interviews in four different periods of time. The first round of interviews was conducted in the beginning of the Kuali project. The main respondents were the founding partners and the development partners. The second round of interviews was conducted between 2008 and 2009 when the first Kuali project, Kuali financial systems (KFS), was successfully launched. The main respondents were the development partners of Kuali. The third round of interviews was conducted between 2011 and 2012 when Kuali expanded to eight projects and started to attract more interest from the industry. The main respondents in this stage were deployment partners (i.e., partner institutions who deploy and use Kuali systems, but do not participate actively in the development effort) and commercial affiliates. The fourth round of interviews was conducted between 2014 and 2016, when the main respondents were commercial affiliates and observers. During all four interview periods, respondents were encouraged to discuss their Kuali participation in an open-ended fashion. We asked respondents to share their specific experiences in the project, reflecting upon their activities, problems encountered, lessons learned, how they or others interacted, steps taken to resolve problems, and general perceptions of the benefits and challenges of the community source development experience. We further asked for respondents' perceptions of project success and project control. Most of the questions were open-ended and non-directive. In each of the interview sessions, the researchers allowed for flexibility, enabling the participants to focus on particular questions, factual information and evaluative comments, and to skip others, based on a respondent's personal experience and expertise. Data analysis in this study included the application of multiple distinct techniques, including the application of thematic analysis, the creation of design structure matrices, and the assessment of system architecture modularity. The separate analytic treatments focus on distinct aspects of the Kuali case, but they are fundamentally interconnected in the community phenomena that they highlight. 4.1. Thematic analysis First, interview transcripts, observational field notes, and documentary evidence were coded using NVivo, a qualitative data analysis tool. This round of data analysis was conducted in line with thematic analysis techniques (Boyatzis, 1998; Braun, Clarke, Hayfield, & Terry, 2019). While the research team has reviewed and analyzed elements of the Kuali data corpus multiple times over the course of this longitudinal case, the present thematic analysis centered on the social interactions and governance of the Kuali community, as well as perceived challenges to the community's functioning. Accordingly, the interview transcripts were the primary form of data, although observational field notes, meeting minutes, and project documentation were incorporated as well. The thematic analysis of the data proceeded through several iterative rounds of coding. The first round of coding focused on the identification of a wide range of initial codes related to community challenges and governance issues (Braun et al., 2019). This initial coding was conducted independently by two members of the research team. Following these initial rounds, the research team met to review the initial coding results. Points of similarity and difference between the two coding sets were discussed to merge corresponding codes and establish a joint code set. From the joint set, the research team sought to collaboratively identify key underlying 7

Information and Organization 30 (2020) 100274

M. Liu, et al.

themes,3 using memoing (Corbin & Strauss, 1990) and visual modeling techniques (Braun et al., 2019). This led to a second round of independent coding using the preliminary set of themes. Following this second round, the research team reconvened to discuss the results. Based on the insights of the individual coders, the team considered the merger of selected themes (based on difficulty differentiating them in the coding process). This led to the creation of a unified set of themes. The data was subjected to a final round of coding using the unified theme set. This final coding round both contributed to the identification of key themes around community sustainability (summarized in Section 5.1. of the findings) and supported the development of design structure matrices in the second of our analytical treatments. 4.2. Design Structure Matrix A Design Structure Matrix (DSM) is a modeling technique used to capture the network of relationships and interdependencies that exist within a complex system (Eppinger & Browning, 2012; Yassine & Braha, 2003). Commonly employed in engineering domains, DSM techniques have been applied in the design of a wide range of electrical and mechanical systems (Browning, 2001). Given its emphasis on interdependencies between system components, a DSM is a powerful tool for assessing the degree of modularization within a system (Baldwin & Clark, 2000). In a DSM, the rows and columns of the matrix reflect the core components of any given system being analyzed (e.g., parts in a product, teams/business units in an organizational system, or steps in a business process). The matrix is then completed by indicating the occurrence of dependency or interaction between the components, by placing an indicator (e.g., an “x”) in any cell for which the columnar component influences the corresponding row component (Baldwin & Clark, 2006; Yassine & Braha, 2003). In this study, we use the DSM technique to model the activity structure – and especially the degree of modularity – for Kuali at different points in time. For the components of the matrices (i.e., the columns and rows labels), we use all activities associated with a traditional systems development life cycle (SDLC) as well as components of community coordination/governance (Boehm, 1988; Davis, Bersoff, & Comer, 1988; Somers & Nelson, 2004). The matrices are then completed by using the x-indicators to identify interdependencies between task components. For example, because domain analysis influences the execution of requirements determination, an x appears in the cell corresponding to the “domain analysis” column and the “requirements determination” row. In our DSM models, we have augmented the standard matrix with an indication of the functional units of the community responsible for various sets of tasks (i.e., the far-right hand column(s) in each matrix). These sets are established based on a visual assessment of clustering in the interdependencies between tasks. Finally, we have color shaded a different set of tasks based on the key functional categories (governance, project coordination, systems analysis, systems development, and systems implementation) of the SDLC (Boehm, 1988; Davis et al., 1988; Somers & Nelson, 2004). These functional categories represent an “idealized” set of modules within each of the DSM models. As noted above, we modeled the Kuali community at different points in time. The four primary analytical DSMs reflect the community structure at the points of community inception (Fig. 3), expansion (Fig. 4), modularizing (Fig. 5), and standardizing (Fig. 6). All models are generated based on the data collected on a longitudinal basis. Specifically, we analyzed the responsibilities and tasks for each organizational unit at the distinct time points and marked the perceived interdependencies in the matrices. Similar to the qualitative coding activities described above, preliminary DSM modeling for each stage was conducted independently by two members of the research team. The entire research team then met to review the proposed models and discuss any areas of nonagreement (i.e., lack of consensus between the initial models regarding perceived process interdependencies). These discussions were used to arrive at a mutually agreed-upon DSM model for each stage of the community’s evolution. At that point, we sought confirmation and other feedback on the proposed DSMs from several members of the Kuali Foundation. The DSMs were revised based on this feedback. 4.3. Modularity level measurement The third core analytical treatment of the data built upon the DSM modeling activity and focused on the measurement of organizational modularity levels within the Kuali community. This specific technique for measuring the modularity of the community was borrowed from the software engineering literature (Emanuel & Surjawan, 2012). The measurement of modularity has been a recurring challenge and area of focus for scholars of software engineering. A variety of methods for measuring the modularization of a piece of software have been proposed, including the use of object oriented metrics (Chidamber & Kemerer, 1994), effort estimation (Asundi, 2005), and coupling-based structural metrics (Aruna, Devi, & Deepa, 2008). Recently, Emanuel and Surjawan (2012) have proposed measures for modularity at the class, package, and system levels in reference to open source software systems. Of particular interest for the present analysis, Emanuel and Surjawan's (2012) metric for System Architecture (SA) combines an evaluation of the coupling and cohesion of a package/module to assess the modular quality of a broader system. Module coupling (cij) is defined as the count of dependencies from classes (i.e., basic programming components) in one package/module to the classes in other modules. Module cohesion (cii) is defined as the count of dependencies between classes in one module to other classes in that same module, including dependencies to the class itself. Specifically, the SA is calculated using the following formula (Emanuel & Surjawan, 2012): 3 In thematic analysis, a theme is a key concept or principle that emerges in the data analysis (Braun et al., 2019). Through recursive analysis, thematic analysis zeroes in on those themes that recur in the data and represent central organizing concepts.

8

Information and Organization 30 (2020) 100274

X

.

X

X

X

X

X

.

X

X

X

X

X

.

X

X

X

X

X

X

X

X

.

X

X

X

X

X

X

X

.

X

X

X

X

X

X

X

X

X

.

X

X

X

X

X

X

X

X

X

.

X

X

X

X

X

X

X

X

.

X

X

X

X

X

X

X

X

X

X

.

X

X

X

X X

X

X

Domain analysis

User engagement

Project evaluation X

X

X

X

X

.

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X X

X

X

X

X

Governance

X

Project Coordinaon

X

X

X

X X

X

X

X • Kuali Foundation • Func. Council • Project mgmt.

X X

X

X

X

X

.

X

X

X

X

X

.

X

X

X

X

X

X

.

X

X

X

X

X

X

X

.

X

X

X

X

X

X

X

X

Systems Analysis

X

X

X

X

X X

X

X X X

X

X

X

X

X

X

.

X

X

X

X

X

.

X

X

X

X

.

X

X

X

X

.

X

Systems Development

• Func. Council • Project mgmt. • Tech. Council • Dev. teams

X X

X

X

X

X X X

X

X X

X

X X

X

X

X

X X

X

Organizational Unit(s) Involved

X

X X

X

X

X X

X

X

X

X X

X

X

X

X

X

Software development

X

X X

Technical design

X

Project planning

X

Project management

X

Licensing

X

X

Systems support

X

X

X

Hosting

X

X

.

Modularity (SA) = 0.664

User training

X

X

Software implementation

X

. X

Quality assurance

X

Configuration management

Conceptual design

X

Project integration

X

Marketing

X

X

Policy setting

X

Community coordination Community coordination Policy setting Licensing Marketing Project integration Project planning Project management Project evaluation User engagement Domain analysis Requirements determination Conceptual design Technical design Software development Configuration management Quality assurance Software implementation User training Hosting Systems support

Requirements determination

M. Liu, et al.

• Tech. Council • Project mgmt. • Dev. Teams • Commer. Affiliates

Systems Implementaon

X

.

X

X

X

X

X

.

X

X

X

X

X

.

X

X

X

X

X

.

X

X

X

X

X

X

X

X

.

X

X

X

X

X

.

X

X

X

X

X

X

X

.

X

X

X

X

X

X

X

X

X

.

X

X

X

X

X

X

X

X

.

X

X

X

X

X

X

X

.

X

X X

X

X X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

Project Coordinaon

X

X

X

X

X

X

X

X

X • Kuali Foundation • Commer. affiliates • Project mgmt.

X

X

X

X

X

X

X

X

.

X

X

X

X

X

.

X

X

X

X

X

X

.

X

• Project mgmt. • Tech. Council • Dev. teams X • Comm. X Affiliates • Comm. affiliates • Kuali X Foundation X

X

X

X

.

X

X

X

X

X

.

X

X

X

.

X

X

X

X

X

X

X

X

.

X

X

X

X

X

X

X

.

X

X

X

X

X

X

.

X

X X

X

X X

X

X

Systems Analysis

X

Systems Development

X

X X

Systems Implementaon

Fig. 4. Design Structure Matrix – Kuali expansion.

9

X

• Func. Council • Project mgmt. • Comm. Affiliates

X

X

X

Systems support

Hosting

User training

Software implementation

Quality assurance

Configuration management

X

X

X

X X

X

X X

Organizational Unit(s) Involved

X

X

X

X

X X

Modularity (SA) = 0.722

X

X

X

Governance

Software development

X X

X

X

Technical design

Conceptual design

Requirements determination

X

Domain analysis

X

User engagement

X

Project evaluation

Project management

X

Project planning

Project integration

X

Marketing

.

Licensing

Policy setting

Community coordination Policy setting Licensing Marketing Project integration Project planning Project management Project evaluation User engagement Domain analysis Requirements determination Conceptual design Technical design Software development Configuration management Quality assurance Software implementation User training Hosting Systems support

Community coordination

Fig. 3. Design Structure Matrix – Kuali inception.

• Func.Council • Project mgmt. • Dev. Teams

X

X

X

.

X

X

X

X

.

X

X

X

X

X

X

.

X

X

X

Project Coordinaon

X

X

Systems Analysi

X

X

X

X

.

X

X

X

X

X

X

.

X

X

X

X

X

.

X

X

X

X

.

X

X

X

.

X

X

X

X

X

.

X

X

X

X

X

.

X

X

X

X X

X

X

Domain analysis

X

X

X

X

X

X

Organizaonal Unit(s) Involved

• Kuali Foundation • Commer. affiliates • Project Board X • Roadmap committees X

• Project Board • Roadmap Committees • Project mgmt. • SM Es X • Func. Council • Project mgmt. • SM Es

X

X

X X X

X

X

X

X

X

X

X

X

.

X

X

X

X

X

.

X

X

X

X

X

X

.

X

X

X

.

X

• Commer. affiliates X • Dev. teams X • Testing teams

.

X

X

X

.

X

X X

X

X

X

X

Project Coordinaon

Modularity (SA) = 0.844

X

X

Governance

Hosting

.

X

User training

X

X

Software implementation

X

X

Configuration management

X

Quality assurance

X

Software development

X

Technical design

.

Conceptual design

Requirements determination

X

User engagement

X

X

Project evaluation

Project integration

X

Project management

Marketing

X

X

Project planning

Policy setting

X

.

Licensing

Community coordination

X

Governance

. X

Systems Systems Development Development

Community coordination Policy setting Licensing Marketing Project integration Project planning Project management Project evaluation User engagement Domain analysis Requirements determination Conceptual design Technical design Software development Configuration management Quality assurance Software implementation User training Hosting Systems support

Systems support

Information and Organization 30 (2020) 100274

M. Liu, et al.

Systems Analysis

X X

Systems Development

X

• Func. Council • Tech. Council • Project M gmt. • Commer. affiliates • Dev. teams • Testing teams

Systems Implementaon

Project planning

Project management

Project evaluation

User engagement

Domain analysis

Requirements determination

Conceptual design

Technical design

Software development

Configuration management

Quality assurance

Software implementation

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

.

X

X

X

X

X

.

X

X

X

X

X

.

X

X

X

X

X

X

.

X

X

X

X

X

.

X

X

X

X

X

.

X

X

X

X

X

.

X

X

X

X

X

X

X

.

X

X

X

X

X

X

X

.

X

X

X

X

X

X

.

X

X

X

X

.

X

X

X

X

X

X

.

X

X

X

X

X X X

X

X X

X

X

X

X

X

X

X

Governance

X

X

X • "Portfolioness" Platform

X

X

X

X X

Systems support

Project integration

X

Hosting

Marketing

Community coordination Policy setting Licensing Marketing Project integration Project planning Project management Project evaluation User engagement Domain analysis Requirements determination Conceptual design Technical design Software development Configuration management Quality assurance Software implementation User training Hosting Systems support

Organizaonal Unit(s) Involved

• Kuali Foundation • Commer. affiliates X • Project Board • Project Board • Roadmap Committees • Project mgmt. • SM Es

X X

X

X

X

X

X

X

.

X

X

X

X

.

X

X

X

X

X

.

X

X

X

X

x

X X X

X

X

.

X

X

X

X

.

X .

X • Commer. affiliates X • Dev. teams X • Testing teams

X

X

.

X X

Systems Analysis

X

Systems Development

X

Systems Implementaon

Fig. 6. Design Structure Matrix – Kuali standardizing.

10

• Func. Council • Project mgmt. • SM Es

X

X

Project Coordinaon

Modularity (SA) = 0.827

User training

Licensing

. X

Community coordination

Policy setting

Fig. 5. Design Structure Matrix – Kuali modularizing.

• Func. Council • Tech. Council • Project M gmt. • Commer. affiliates • Dev. teams • Testing teams

Information and Organization 30 (2020) 100274

M. Liu, et al.

d

SA =

∑i = 1 cii2 d ∑i = 1

d

∑ j = 1 cij2

This SA calculation results in a single, normalized value for the level of modularity in a given system. In this study we adapt the SA metric (Emanuel & Surjawan, 2012) to measure the degree of modularity in the Kuali community at multiple points in its evolution. This adaptation builds upon the DSM analysis described above. We calculate module cohesion (cii) as the count of dependencies (as identified in the DSM) between organizational activities in one functional category (e.g., community governance, systems development; graphically represented with colored shading) to other activities in that same category. Conversely, module coupling (cij) is calculated as the count of dependencies from activities in one functional category to the activities in other functional categories in the Kuali community. As with the DSM modeling analysis, the SA of the Kuali system is calculated at multiple points in time. 5. Findings In line with the multi-method nature of our analysis, the findings from the Kuali case study are revealed at two levels. First, we delineate the findings from our thematic analysis, focusing on the challenges confronting the community as it seeks to balance growth, complexity, and long-term sustainability (RQ1). Second, we present the findings of the DSM modeling analysis, which explores the extent of modularity in the evolution of Kuali and the community's efforts to achieve sustainability in the long term (RQ2). 5.1. Challenges confronting the community source approach In the years since its initiation, the Kuali effort has grown dramatically. The participating institutions are held together by a system of shared values that no single institution could provide. However, Kuali faces a number of challenges with the community growing. On one hand, it is optimal for the Kuali Foundation to attract as many participating institutions as possible to share resources, reduce costs, and minimize risks. On the other hand, the management of the community has become increasingly complicated as Kuali has grown. Our thematic analysis of the case data reveals a series of central themes which highlight key challenges in this balancing act of growth and complexity. These challenges include issues around role clarification (specifically, for the foundation and the commercial affiliates), the maintenance of a family environment, new project development, and project management. Table 1 provides a summary of the themes identified, including descriptions and illustrative statements from the data. As this summary suggests, the thematic challenges identified are closely tied to the dramatic growth that the community has experienced and the associated rise in complexity in coordinating the activities of the diverse member institutions. In keeping with the general modular systems theory, this rising complexity has been largely addressed through a movement toward more modular organizational structures. 5.2. Modularity and the pursuit of community sustainability Our analysis of the forces within and around Kuali reveals that the community faced significant impetus for modular design in its pursuit of long-term sustainability. Indeed, several of the challenges identified in the Kuali community are indicative of issues arising from points of low cohesion and tight coupling within the functioning of the community. Questions regarding community governance and project coordination reflect the desire for a single unit (i.e., the Kuali Foundation) to oversee governance and coordination between the diverse efforts of various collaborators. Concerns regarding the role of commercial affiliates turn on questions of both cohesion (i.e., the degree to which teams or functional units have a clear and relatively limited purpose) and coupling (i.e., the degree of interdependence among the functional units). With respect to cohesion, the commercial affiliates' role within the community was ill-defined, often extending to nearly every aspect of project planning and execution. On the coupling front, commercial affiliates actually create the potential for loose coupling within the system by being a conduit for lessons learned across implementation efforts. However, concerns with the variable quality of the affiliates may lead to breakdowns in cross-project knowledge sharing. In light of these observations, the second phase of our analysis focused on capturing the evolution of the Kuali community and the organizational structures that it adopted to achieve sustainability in the face of growth and increasing complexity (RQ2). In this regard, we believe it important to consider both the evidence of modularity in the Kuali community and areas where modularity may be undermined by tight coupling between functional units or the absence of cohesion in the roles served by those units. To support this assessment, we have created DSM models of the interactions within the community at four distinct points in its development. We label these developmental phases inception, expansion, modularizing, and standardizing. In comparing the community structures at these distinct points in time, we seek to juxtapose the initial structure of the Kuali community to its structure at the time of our latest round of data collection. We augment this structural analysis with the calculation of the modularity of the system architecture (SA) of the community at each point in time, based on our adaptation of the Emanuel and Surjawan (2012) model. Figs. 3 through 6 reveal the DSMs for the four time points respectively. Within each matrix, the color-shaded areas are used to designate functional categories in the systems development life cycle as well as community coordination activities (Boehm, 1988; Davis et al., 1988; Somers & Nelson, 2004). While the structure of the Kuali community is not “purely” modular at any stage in our analysis, these functional categories represent the idealized sets of modules for such a development community. Fig. 3 reflects the DSM for the inception phase of the Kuali community. In this phase, there is only one project (i.e., KFS) under 11

Information and Organization 30 (2020) 100274

M. Liu, et al.

Table 1 Challenges in community source development. Challenge

Description

Illustrative statements

Community governance

The role of the Kuali Foundation in directing the activities of the community is a recurring debate as the community grows and projects proliferate. As more projects have emerged, institutions are expecting the Kuali Foundation to take the lead in helping with implementation and resource allocation. With multiple systems reaching maturity, there is a need to market these systems to prospective clients. There is a concern among the institutions that the Kuali Foundation does not demonstrate sufficient leadership in this regard. A major question for many Kuali participants concerns ambiguity with regard to the role for certain community participants, most notably the commercial affiliates. For-profit commercial affiliates provide an ecosystem of services for delivering Kuali software systems. However, the growth of commercial affiliates (and competition between them) has raised concern for some members. For institutions which decide to implement Kuali systems, hiring a commercial affiliate as a consultant has both advantages and possible drawbacks. On one hand, institutions receive help from the commercial affiliates. On the other hand, institutions lose the opportunity of building a knowledge base by themselves. There is a concern among the institutions that the commercial affiliates have taken too much control over the community. Throughout most of Kuali's history, the presence of a family atmosphere – i.e., the sense of connectedness through close interpersonal ties – has been central to the community's collaborative efforts. As the community has grown dramatically in recent years, the ability to maintain a “familial” connection between participants has been strained. Several respondents expressed concern that growth has hindered the family atmosphere that has been such a strength of the Kuali community. As projects proliferate, the ability to share insights or lessonslearned across projects has not always been effective. Knowledge management becomes an important component to enforce the learning process among various projects. Some of the new projects did not move along quickly because they failed to learn from existing successful projects. For example, Kuali Student (i.e., the student information system module) managers failed to recognize a mistake they made in the development stage, which they should have learned from KFS in the beginning. Each participating institution is required to contribute a certain number of developers to the building of Kuali systems. As the number of development partners increases, the number of developers increases dramatically. The increasing number of development partners has put pressure on the Kuali organization for project management across partner institutions.

“The foundation has to remain strong or even get stronger. I think they have to recognize and promote to the rest of the colleges the better installations, the better successes, the more cost effective and sustainable results that they are seeing… They [the Foundation] have to promote themselves because right now they don't have a budget to do it.” – CEO of a commercial affiliate, a middle size IT company

The role of commercial affiliates

Maintaining the family atmosphere

Cross-project knowledge sharing

Project coordination

“In some ways, having an implementation partner [commercial affiliate], they've already learned a lot of the lessons that we learned along the way. In that manner, that would have been great. On the other hand, doing it ourselves built a knowledge base within our institution which you don't necessarily get when you have an implementation partner because the knowledge base is there.” – Team leader of KFS of a development partner, a middle size private research university

“In the early stage in Kuali, I felt like we were very close, like a family. When I attended Kuali Days, I basically knew 90% of the people there. Right now, when I walk around, I don't know most of the people. It's a good thing for the community to get bigger. However, sometimes I regret that we lose the family sense in the community.” – Leader of KFS deployment, a large size public university

“As we have grown to be in eight projects now, we're having to think, not only about, ‘What do the research people want, or the student people?’ but, ‘How do those things go together when a campus installs multiple Kuali projects?’” – Top management, Kuali Foundation

“The number of development partners is starting to get too large. The number of developers is too large to manage effectively. Sustainability is a big issue…Managing a virtual team is not easy. Our developers work in a different location, different times.” – Project manager of KFS, a large size public research university

development and the Kuali community is very small. A review of the DSM reveals significant interdependence across community and project coordination functions. Furthermore, the majority of organizational units (e.g., functional council, project management) are engaged in a wide range of distinct activities. This structure reveals that cohesion within the community was relatively low, with significant indication of tight coupling. The SA value for the community at that point in time was 0.664. Fig. 4 reflects the DSM for the second phase of the Kuali community, which we have labeled the expansion phase. In this phase, Kuali expanded from one project to multiple projects. As the community took on additional partner institutions and began to replicate its development approach across multiple projects, we see a move toward modularization. Specifically, while several functional units (esp., commercial affiliates, Functional Council) continued to play a wide range of roles within the community, other units (e.g., Technical Council, development teams) became more narrowly focused in their activities and the interdependence between activities decline. This suggests a shift toward greater cohesion for at least certain roles within the community and a “loosening” of the coupling between the functional units. The SA value for the community at that point in time was 0.722. The Kuali community continued to change and evolve its organizational structures and business processes in response to challenges or tensions within the community. In the third discernible phase of its evolution, the Kuali community undertook a deliberate effort at modularization of its technical systems. Fig. 5 reflects the DSM for the community during this “modularizing” phase. In this phase, Kuali further expanded to include eight projects. As the number of projects grew, the community (directed by the Kuali 12

Information and Organization 30 (2020) 100274

M. Liu, et al.

Foundation) focused on the creation of software modules that could be reused across projects. While the community was explicitly focused on technical rather than organizational modularity, our analysis reveals that this phase reflects a more modular structure of the community itself. Specifically, in moving from the expansion phase to the modularizing phase, interdependencies across development and coordination functions were greatly reduced, enhancing cohesion and further reducing the prevalence of tight coupling within the community. These changes are reflected in the substantive rise in the SA to 0.844. To a significant extent, this move toward greater modularization was driven by the introduction of new organizational units and the clarification of roles for all of the units. The Application Roadmap Committee and Technology Roadmap Committee were created to guide Kuali functionality determination and technical resource sets at the level of the broader community, suggesting the desire for greater cohesion. Similarly, the designation of subject matter expert (SME) roles and testing teams with clear domains of responsibility aided cohesion. In the final observed phase of its evolution, the Kuali community made a push for standardization across all of its technical platforms – an effort which the Kuali Foundation dubbed “portfolioness.” Fig. 6 reflects the DSM for the community during this “standardizing” phase. Through the portfolioness platform, institutions were encouraged to contribute their modifications back to the community, with the coordination of these modifications being managed and maintained by the Kuali Foundation. This portfolioness platform was tasked with managing only the Community Coordination function, i.e., providing guidance, documenting lessonslearned, and coordinating broader community-wide activities. Interestingly, by taking control of the community coordination function, the portfolioness platform became the primary mechanism of coupling within the community. The SA value for the community at that point in time was 0.827. Thus, looking across the evolution of the Kuali community, we see the emergence of an increasingly modular organizational architecture in pursuit of greater sustainability in response to community growth and rising complexity. This architectural transformation included the introduction of new functional units as well as more clearly defined roles within the community, engendering higher unit cohesion and looser coupling between groups. Despite this general move toward greater organizational modularity, the adoption of the portfolioness platform created a reduction in the measurement (i.e., the SA value) of the community's modularity in the final phase of our observations. This raises valuable questions about the role of organizational modularity in confronting complexity and fostering long-term sustainability (e.g., Is more modularity always better? If not, is there a desirable level of modularity at specific phases of an organizational evolution?). We offer more thoughts in the following discussion section. 6. Discussion As we observed earlier, several of the challenges encountered by Kuali as it became larger and more diverse derived from the existence of tight coupling and lack of cohesion within the community. The need for greater role clarity for both the Kuali Foundation and the commercial affiliates provides a clear example of issues that can arise when unit cohesion is compromised. The challenges encountered with new project development (e.g., the failure of learning across distinct projects) call attention to the relative absence of effective mechanisms for knowledge transfer within the broader Kuali community. In the language of modularity, this implies an inadequacy of interfaces between distinct functional units (i.e., modules) that operate at the project level. The perceived need for improved project management to integrate the diverse development teams contributed by distinct member institutions similarly signals a concern over the interfacing of separate groups of developers. Finally, the difficulty of maintaining a family environment suggests that community growth has dramatically increased the complexity of interpersonal relations within the community and undermined the close interactions previously experienced within a given organizational unit. As we have highlighted, the Kuali community has responded to these challenges by shifting toward a more modular organizational architecture, mirroring the modularization of its technical platforms. Yet, this evolution was not predestined or linear in nature. Indeed, our findings reveal some movement away from strict modularity with the introduction of the “portfolioness” platform, a move that enables greater community-wide exchange and communication. Of course, the implications of these changes for the sense of family within the community remain to be seen. Furthermore, it is unclear to what degree the changes enacted by Kuali will engender long-term sustainability for the community source effort. Thus, these findings present a number of both provocations and insights concerning research and practice on the community source development approach and other contexts of collective action. 6.1. Implications for research From a research perspective, the Kuali case presents a number of valuable insights regarding the role of a modular organizational architecture in balancing the objectives of growth and sustainability. Methodologically, we believe the study provides a valuable lens for assessing modularity in organizational structures and its impact on both the complexity faced by organizations and their longterm sustainability. Despite evidence of its value (Browning, 2016; Eppinger & Browning, 2012), DSM modeling has seen limited application to the analysis of organizational designs. The Kuali case study underscores the analytical value of such modeling for capturing the evolution of an organizational architecture. In addition, we leverage the analytical tool of the DSM to develop a novel measure of system architecture modularity for an inter-organizational collaborative community. In so doing, we create the possibility of formal comparison of modularity across organizational designs and governance structures. Furthermore, this focus on modularity in organizational design provides a critical lens for identifying threats to the sustainability of communities that arise with growth and community complexity. We believe that the methodology introduced in this study can be fruitfully applied to other inter-organizational collaborations and a wide variety of organizational contexts. Such application will support further confirmation and 13

Information and Organization 30 (2020) 100274

M. Liu, et al.

definition regarding the role and impact of modularity on project or organizational sustainability. This analysis suggests several avenues for research. Since this is the first time that the SA index has been used to measure the level of organizational modularity, the numerical results are still subject to interpretation. More applications are needed to confirm the utility of this approach. In addition, more research is needed to aid in the interpretation of DSMs and SA values in combination. We believe the combined methods provide a critical lens for identifying both the threats to the sustainability of communities that arise with growth and complexity and potential solutions to address such threats. However, the question of how much modularity is “right” remains open. For example, what SA levels are conducive to organizational sustainability? Under what conditions (drawing on modular system theory or other environmental analysis framework) are various degrees of modularity desirable? In the case of Kuali, even though we see general movement toward greater modularity, some organizational decisions do reduce the measure of modularity (i.e., the SA value) within the community (e.g., in the portfolioness phase). Will the outcome of these shifts ultimately be beneficial or detrimental to the community's long-term sustainability? The answer remains to be seen. Community source as a new approach of collaborative system development is still in its infancy. We are hopeful that this study can attract more researchers' interest in studying this development approach. The research also contributes to our understanding of the evolving phenomenon of open source software (OSS) development. Despite the high profile projects that draw great attention and effort, the success rate of OSS projects is remarkably low (Lee, Kim, & Gupta, 2009). Identifying mechanisms that can support the sustainability of OSS efforts is a critical need for the advocates of this distributed development approach. Our study suggests that attention to the design of OSS communities, especially with respect to the modular characteristics (e.g., unit cohesion, loose coupling) that they display, may be central to this pursuit. Interestingly, modularity in software design has been identified as one of the key success factors of OSS projects (Stamelos, Angelis, Oikonomou, & Bleris, 2002). In terms of engendering community sustainability, the present study indicates that as much attention must be given to the design of the OSS communities themselves. 6.2. Implications for practice In turning to the practical implications of the research, one might ask why we care about the sustainability of community source projects like Kuali. As we have noted, to some degree, the community source approach represents “the best of both worlds” vis-à-vis the commercial and open source approaches. However, as the Kuali case reveals, growth can be a double-edged sword for community source projects – the influx of new participants brings access to new ideas, financial support, and potential solutions, but it can also blur the lines of responsibility, strain the bonds of interpersonal relationship, and threaten stable patterns of knowledge sharing and exchange. Growth increases complexity, which in turn undermines the sustainability of the underlying community. One prominent strategy for mitigating such complexity is through modularization. Indeed, our findings reveal the critical role of organizational modularization in the evolution of the Kuali initiative. From a practical perspective, the insights from this research will be useful for those managing a variety of community-oriented development project, including community source projects similar to Kuali, open source projects, or commercial companies sponsoring open source efforts. Given its promise of enabling organizations to cut costs, increase collaboration, and avoid vendor lock-in, the community source approach can be used by practitioners to address common business challenges shared within the same industry, especially for public sector entities for whom budgets are essential concerns. Not surprisingly, we see evidence of such initiatives emerging in a variety of sectors. For example, the State of Utah Department of Health pooled together technical expertise and financial resources from its 29 counties, 12 local health departments, and the University of Utah to develop a community source public health management and response system called TriSano (Herrick, 2010). Of special note in the TriSano case is the engagement of a commercial development partner called Collaborative Software Initiatives (CSI) that acts as the governance body for this community. Interestingly, CSI is trying to replicate the community source approach to other similar projects. Such a new and potentially more sustainable software development approach has also been adopted by commercial software companies to handle internal complexity and develop new products or features that better satisfy customer needs at lower cost. IBM used the community source approach to bring thousands of developers from different business units and geographic locations to contribute to its community source program, which now has more than 1400 projects (Gongla & Rizzuto, 2001). A web-based governance application was also developed to facilitate knowledge sharing and component reuse. In this initiative, principles of modularity played a big role in encouraging componentization. IBM's community source efforts contributed directly to cutting development cost and time while improving product innovation. Initiatives of this nature could greatly benefit from the lessons from Kuali. Beyond the specific context of community source development, this study identifies some critical factors for sustainability of interorganizational communities like Kuali. While the realities of a community source project in higher education may not generalize to all community-based collaborations, the Kuali experience highlights the benefits of modular organizational design for sustaining commitments and engagement as a community grows. In this regard, the analytical tools of the DSM and system architecture modularity (SA) provide helpful mechanisms for identifying fruitful enhancements to community designs. 7. Conclusion The community source development approach promises the “best of both worlds” – the dynamism and tailored functionality of open source software paired with the control of commercial development. However, to achieve this promise, community source efforts must create community structures that remain stable as memberships grow and evolve. Through a longitudinal case analysis of 14

Information and Organization 30 (2020) 100274

M. Liu, et al.

the Kuali initiative, we explore the milestones and growing pains that mark the flowering of a large-scale community source effort. By rethinking community source development using modular design principles, we develop a deeper understanding of the mechanisms for sustainability in collaborative systems development. Indeed, this work represents the first empirical research applying modularity design to the community source domain. The findings from this research can help managers of community source and other collaborative development initiatives to identify the challenges confronting their effort and the degree to which such challenges reflect concerns over tight coupling and low cohesion within their organizational structure. References Agerfalk, P. J., & Fitzgerald, B. (2008). Outsourcing to an unknown workforce: Exploring opensourcing as a global sourcing strategy. MIS Quarterly, 32(2), 385–409. Aruna, M., Devi, M. S., & Deepa, M. (2008). Measuring the quality of software modularization using coupling-based structural metrics for an Oos system. First International Conference on Emerging Trends in Engineering and Technology, Nagpur, India (pp. 1130–1135). IEEE Computer Society. Asundi, J. (2005). The need for effort estimation models for open source software projects. Proceedings of the fifth workshop on open source software engineering (pp. 1–3). St. Louis, MO: ACM. Baldwin, C. Y., & Clark, K. B. (2000). Design rules, volume 1: The power of modularity. Cambridge, MA: MIT Press. Baldwin, C. Y., & Clark, K. B. (2006). Modularity in the design of complex engineering systems. In D. Braha, A. A. Minai, & Y. Bar-Yam (Eds.). Complex engineered systems: Science meets technology (pp. 175–205). Berlin, Germany: Springer. Boehm, B. W. (1988). A spiral model of software development and enhancement. IEEE Computer, 21, 61–72. Boyatzis, R. E. (1998). Transforming qualitative information: Thematic analysis and code development. Thousand Oaks, CA: Sage Publications, Inc. Braun, V., Clarke, V., Hayfield, N., & Terry, G. (2019). Thematic analysis. In P. Liamputtong (Ed.). Handbook of research methods in health social sciences (pp. 843–860). Singapore: Springer. Browning, T. R. (2001). Applying the design structure matrix to system decomposition and integration problems: A review and new directions. IEEE Transactions on Engineering Management, 48(3), 292–306. Browning, T. R. (2016). Design structure matrix extensions and innovations: A survey and new opportunities. IEEE Transactions on Engineering Management, 63(1), 27–52. Chidamber, S. R., & Kemerer, C. F. (1994). A metrics suite for object oriented design. IEEE Transactions on Software Engineering, 20(6), 476–493. Corbin, J., & Strauss, A. (1990). Grounded theory research: Procedures, canons and evaluative criteria. Qualitative Sociology, 13(1), 3–21. Davis, A. M., Bersoff, E. H., & Comer, E. R. (1988). A strategy for comparing alternative software development life cycle models. IEEE Transactions on Software Engineering, 14(10), 1453–1461. Emanuel, A. W. R., & Surjawan, D. J. (2012). Revised modularity index to measure modularity of Oss projects with case study of Freemind. International Journal of Computer Applications, 59(12), 28–33. Eppinger, S. D., & Browning, T. R. (2012). Design structure matrix methods and applications. Cambridge, MA: MIT Press. Ethiraj, S. K., & Levinthal, D. (2004). Modularity and innovation in complex systems. Management Science, 50(2), 159–173. Gongla, P., & Rizzuto, C. R. (2001). Evolving communities of practice: Ibm global services experience. IBM Systems Journal, 40(4), 842–862. Gunnar, M. R., & Maratsos, M. (1992). Modularity and constraints in language and cognition: The Minnesota Symposia on child psychology. New York, NY: Lawrence Erlbaum. Henderson, R., & Clark, K. B. (1990). Architectural innovation: The reconfiguration of existing product technologies and the failure of established firms. Administrative Science Quarterly, 35(1), 9–30. Herrick, M. (2010). Trisano: Collaborating for the good of global public health. Technology Innovation Management Review. (December) http://timreview.ca/article/405. Langlois, R. N. (2002). Modularity in technology and organization. Journal of Economic Behavior & Organization, 49, 19–37. Lee, S.-Y. T., Kim, H.-W., & Gupta, S. (2009). Measuring open source software success. Omega, 37(2), 426–438. Liu, M., Hansen, S., & Tu, Q. (2014). The community source approach to software development and the Kuali experience. Communications of the ACM, 57(5), 88–96. Liu, M., Hull, C. E., & Hung, Y.-T. C. (2017). Starting open source collaborative innovation: The antecedents of network formation in community source. Information Systems Journal, 27(5), 643–670. Liu, M., Wang, H. J., & Zhao, J. L. (2012). Technology flexibility as enabler of robust application development in community source: The case of Kuali and Sakai. Journal of Systems and Software, 85(12), 2921–2928. Liu, M., Wu, X., Zhao, J. L., & Zhu, L. (2010). Outsourcing of community source: Identifying motivations and benefits. Journal of Global Information Management, 18, 36–52. Paré, G. (2004). Investigating information systems with positivist case study research. Communications of the AIS, 13(18), 233–364. Parnas, D. L. (1979). Designing software for ease of extension and contraction. IEEE Transactions on Software Engineering, SE-5(2), 128–138. Perepletchikov, M., Ryan, C., & Frampton, K. (2007). Cohesion metrics for predicting maintainability of service-oriented software. In A. Mathur, W. E. Wong, & M. F. Lau (Eds.). Seventh international conference on quality software (QSIC’07) (pp. 328–335). Portland, OR, USA: IEEE. Pil, F. K., & Cohen, S. K. (2006). Modularity: Implications for imitation, innovation, and sustained advantage. Academy of Management Review, 31(4), 995–1011. Raymond, E. S. (2004). The cathedral and the bazaar. Sudbury, MA: Jones and Bartlett Publishers. Sanchez, R., & Mahoney, J. T. (1996). Modularity, flexibility, and knowledge Management in Product and Organization Design. Strategic Management Journal, 17(S2), 63–76. Schilling, M. A. (2000). Toward a general modular systems theory and its application to interfirm product modularity. Academy of Management Review, 25(2), 312–334. Schilling, M. A., & Steensma, H. K. (2001). The use of modular organizational forms: An industry-level analysis. Academy of Management Journal, 44(6), 1149–1168. Seok, B. (2006). Diversity and unity of modularity. Cognitive Science, 30(2), 347–380. Simon, H. A. (1962). The architecture of complexity. Proceedings of the American Philosophical Society, 106(6), 467–482. Somers, T. M., & Nelson, K. G. (2004). A taxonomy of players and activities across the erp project life cycle. Information & Management, 41(3), 257–278. Stamelos, I., Angelis, L., Oikonomou, A., & Bleris, G. L. (2002). Code quality analysis in open source software development. Information Systems Journal, 12(1), 43–60. Stevens, W. P., Myers, G. J., & Constantine, L. L. (1974). Structured design. IBM Systems Journal, 13(2), 115–139. Sutcliffe, A. (2002). The domain theory: Patterns for knowledge and software reuse. Mahwah, NJ: Lawrence Erlbaum Associates. Weick, K. E. (1976). Educational organizations as loosely coupled systems. Administrative Science Quarterly, 21(1), 1–19. Wheeler, B. (2004). The open source parade. Educause Review, 39(5), 68–69. Wheeler, B., & Hilton, J. L. (2012). The marketecture of community. Educause Review, 47(6), 66–82. Yassine, A., & Braha, D. (2003). Complex concurrent engineering and the design structure matrix method. Concurrent Engineering, 11(3), 165–176. Yin, R. K. (2009). Case study research: Design and methods. Thousand Oaks, CA: SAGE. Yourdon, E., & Constantine, L. L. (1979). Structured design: Fundamentals of a discipline of computer program and systems design. Upper Saddle River, NJ: Prentice-Hall, Inc.

Manlu Liu ([email protected]) is an Associate Professor of Management Information Systems and Accounting and a Director of MS in Business Analytics in the Saunders College of Business at Rochester Institute of Technology, Rochester, NY. Her main research interests are community-based open source (community source), open innovation, blockchain technology, and data analytics in accounting and finance. Sean Hansen ([email protected]) is an Associate Professor of Management Information Systems and a Department Chair of MIS, Marketing & Digital Business in the Saunders College of Business at Rochester Institute of Technology, Rochester, NY. His main research interests are requirements engineering, IT strategy, Web 2.0 technologies, and the application of contemporary cognitive theory to information systems development. Qiang Tu ([email protected]) is a Full Professor of Management Information Systems and a Senior Associate Dean in the Saunders College of Business at Rochester Institute of Technology, Rochester, NY. His main research interests are information systems and operations management strategy, technology management, and behavioral issues in information systems and operations management.

15