A tale of two cities: case studies of GSS transition in two organizations

   Introduction

A GSS is a suite of collaborative software tools that can be used to focus and structure a team's deliberation, while reducing cognitive costs of communication and information access and minimizing distraction among teams working collaboratively towards a goal ^[10]. There is a huge body of research over the benefits that teams and organizations derive from Group Support Systems (GSS¹). However, the diffusion of this technology has been slow. Therefore, many organizations do not derive the benefits the technology appears to offer. Even more, in some organizations there is a growing community of users while in others the technology is not used any more. This dilemma of usefulness vs. use (lack of) is the main concern of this paper.

Vogel and Vreede ^[29] conducted a survey about the usefulness of GSS with 441 GSS users at IBM and Nationale Nederlanden (NN), one of the largest insurance firms Europe. Respondents in both companies rated the technology as highly useful, and reported large time savings and strong satisfaction with its use (Table 1).

Yet a further look at the business market for GSS reveals an interesting picture. Figure 1 shows number of organizations in the Netherlands from 1994 to 2002 that acquired and used GSS maintenance contracts and the number that chose not to renew maintenance contracts ^[12].

As can be seen, the maximum number of maintenance contracts in one year was 29 for 1999. From 1998 to 2001 the quantity of new contracts diminished by a half, while the quantity of contracts terminated matched the number of new contracts. These and other results suggest that self-sustaining communities of GSS users may be slow to form. A self-sustaining community of users is defined as one that uses a technology as a standard practice; that trains new people to use the technology as a matter of course, and that fixes a technology if it breaks, rather than abandoning it. A growing community of users is one where more people find more uses for the technology over time.

The results reported by Gordebeke ^[12] suggest that at least some of the GSS adopters in the Netherlands may have become rejecters over time. Based on the previously presented evidence, it seems useful to address the question, if people derive great benefit from GSS, why do they not continue to use it? What causes a self-sustaining and growing communities of GSS users to form?

Figure 1.Figure 1.

This paper describes two case studies of technology transition for GSS in organizations where GSS was formally adopted and used with good results. In one of these, a self-sustaining community of users formed; in the other, the GSS is rarely used. The paper proceeds as follows. Section 2 summarizes the Technology Transition Model, the theory used to guide the investigation. Section 3 describes the research methodology, and Section 4 presents critical incidents from the two cases. Section 5 discusses the findings, and Section 6 offers conclusions, describes limitations and recommends future directions.

1   Theory

1.1 Theories and Research

Authors have taken two general approaches to studying technology transition. Some explore processes for successful transition, while others seek the causes of success. ^[5]

Rogers' ^[26] Innovation Decision Process is a seminal example of a process-centric view. Rogers' posits five successive stages, each of which is a prerequisite for the next. They are:

• Knowledge stage - potential users become aware and gain understanding of an innovation.
• Persuasion stage - Potential users form favorable attitudes towards the innovation.
• Decision stage - Activities lead to a choice to adopt or reject an innovation.
• Implementation stage - People use the innovation
• Confirmation stage – individual seeks reinforcement of the innovation-decision.

Another body of process-centric work draws on Structuration Theory ^[21][24], characterizing technology as one part of two interdependent sub-systems: the social and the technical, which comprise a socio-technical system. In an organization, the social system might be jobs, workflows, work groups, while the technical might include hardware, software tools, devices, etc. Both of the social and technical are open systems, thus prone to be effected by external influences, like new products on one side, and new members or practices in the other. Any change in one side influences the other side. While process models of technology transition are useful for reducing the risk of transition failure, it might also be useful to have a causal model to explain why these process produce success, and to guide the design of new processes.

Davis' Technology Acceptance Model (TAM) is a seminal example of a causal model for technology transition. TAM posits that people will develop an intention to use, and therefore will use a new technology to the extent that it is perceived as useful, and to the extent that it is easy to use. TAM also posits that other un-named external factors may influence a person's perception of usefulness and ease of use.

Moore, and Benbasat ^[10] identify seven attributes of an innovation that affect adoption:

• Relative advantage - the degree to which an innovation is perceived as being better than its precursor.
• Compatibility - the degree to which an innovation is perceived as being consistent with the values, needs, and experiences of its adopters.
• Complexity - the degree to which an innovation is perceived as being difficult to use.
• Observability - the degree to which the results of an innovation are observable to others.
• Trialability - the degree to which an innovation may be experimented with before adoption.
• Image - the degree to which the use of an innovation is seen to enhance one's status.
• Voluntariness of use - the degree to which use of the innovation is voluntarily.

TAM is useful to predict system use a few weeks after initial exposure to a new technology. Moore and Benbasat's attributes are a useful topoi for planning an intervention and predicting its success. Parthasarathy and Bhattacherjee ^[25] posit a set of hypothesis based mainly on the work of Rogers on diffusion of innovations. They state that post-adoption behavior is related to a set of factors when applied to a case of online subscription services. Parthasarathy et al had tested usefulness and ease of use - as defined by Rogers ^[26] - and found that perceived usefulness could be a significant indicator while ease of use did not seem to explain self-sustainment.

It might also be useful to have a model to explain why an organization might abandon a technology some years after a successful adoption, and to explain the mechanisms that give rise to the effects of the attributes of innovation.

1.2 Technology Transition Model

The Technology Transition Model (TTM) ^[4] builds on the logic of TAM as it seeks to explain the existence (or non-existence) of self-sustaining and growing communities of users for innovative technologies. Like TAM, TTM posits that system-use (U) is a positive function of behavioral-intentions (BI). ^[5] TTM posits that intention-to-use is a multiplicative function of:

• The Perceived Magnitude of the Net Value (V) that might be obtained after a switch to the new technology, and
• The Perceived Frequency (F) with which said value might be obtained.

  Figure 2.Figure 2.

TTM posits that Perceived Magnitude of Net Value (V) is an overall sense, derived through the parallel processing of the brain, rather than a rational, deliberate summation of each individual cost or benefit. Indeed, people may arrive at some V without being able to articulate all the factors that weighed in the assessment. TTM posits that as people consider using a new system, they perceive value (both cost and benefit, both negative and positive) along a number of dimensions, among them:

• Usefulness: The extent to which the user beliefs the technology will enhance or hinder job performance (incorporated from TAM).
• Affective: The extent to which the technology will invoke positive or negative emotional response in the user.
• Economic: The extent to which the technology will increase or decrease the user's cash, assets, marketability, etc.
• Physical: The extent to which the technology will increase or decrease the user's health or comfort.
• Political: The extent to which the technology will increase or decrease the user's power or influence within and/or across organizations.
• Social: The extent to which the technology will enhance or detract from the user's personal relationships with other people, such as colleagues, friends, and family.
• Cognitive: The extent to which the technology will increase or decrease the user's amount of mental effort expended to complete tasks the technology supports. This dimension has at least three components:
• Perceptual Load. User friendliness - the amount of mental effort required finding and controlling the features and functions of the technology required to accomplish the task at hand.
• Access Load. Availability - the amount of mental effort required to gain permission and access to use the components of the technology needed for the task at hand.
• Conceptual Load. Understanding - the amount of effort required to understand what the technology is supposed to do for the user.

TTM posits that the direct causal relationship between V*F and B is moderated by two factors:

• A multiplicative function of the degree of Certainty (C) about perceptions of magnitude and frequency of value. The model posits that Certainty is an asymptotic function of exposure to the technology through testimony, observation, and experience. Although increasing exposure to a system may cause perceptions about the system to swing dramatically, the more exposure one has to a new system, the more certain one feels about ones perceptions of that system.
• An additive function of the Perceived Net Value of Transition (T). There are both costs and benefits to the transition process that are different from the costs and benefits of using the new system once it is in place.

TTM can be summarized as a mathematical approximation as follows: $${\rm U}\cong f({\rm BI})\cong f({\rm VFC}+{\rm T})$$

The Technology Transition Model served as a framework for guiding the observations and interviews of the case study. The purpose of this was two fold; first, to check whether the causal relationships posited by TTM seemed to manifest in the field; and second, to learn whether other factors appears to come into play which could not be explained by the model.

2   Methodology

We chose a qualitative approach for this study because “Qualitative research methods are designed to help researchers understand people and the social and cultural contexts within which they live . ” ^[21] . Myers ^[21] identifies four approaches to qualitative enquiry: action research, case study, ethnography, and grounded theory. We chose to conduct case studies following Yin ^[31], who argues that this approach is useful for addressing “ ‘how’ or ‘why’ questions about a contemporary set of events over which the investigator has little or no control. ”

We used two different methods for data gathering: interviews, followed up by focus groups, following Morgan ^[20], who argues that “following individual interviews with focus groups allows the researcher to explore issues that came up only during the analysis of the interviews.”

The various dimensions of the TTM were incorporated into a structured set of prompts to guide the interview process. Data was collected through semi-structured interviews, with employees in different roles related to the adoption and use of GSS. When interviewees deviated from structured interview topics to explore other issues, the interviewer probed the new topic until the interviewee was finished with it. Then the interviewer returned to the structured prompts.

Interviews were conducted face-to-face and by phone. The average duration of the conversation was 30 minutes. All interviews were taped in a digital format with the permission of the informants. The recorded interviews were transcribed. These transcripts were then analyzed and coded.

The analysis of the interview transcriptions sought to identify key concepts and events that affected technology transition. This case analysis was distributed to the informants. They were asked to validate factual information and to discuss interpretations of the researchers. This led to further refinements of the analysis.

3   The case studies. a Tale of Two Cities

3.1 The Rotterdam Port Authority

The Port of Rotterdam encompasses a total port area of 10500 ha which stretched along 40 km from the Rotterdam City to the North Sea coast. About 61,000 people work on port-related jobs. About 322 millions metric tons of goods passed through the port in 2002, which makes the RPA one of the biggest ports in the world.

The Bureau Harbor Master sub-division is responsible or the preparation and training for incident response. This task is accomplished by the use of a GSS and described in the following chapter.

3.2 The Ministry of Foreign Affairs

The Ministry of Foreign Affairs coordinates and carries out Dutch foreign policy at its headquarters in The Hague and through its missions abroad. It is the channel through which the Dutch Government communicates with foreign governments and international organizations. ^[36]. As of January, 2002, these objectives were carried out by 3202 persons, of which 1960 were in The Netherlands and 1202 in missions abroad. Furthermore, another 1921 were locally engaged as staff at the missions.

One of the departments from the array of services and departments under the Secretary-General is the Human Resources Department. In the 2000, a redesign of the Ministry's organizational structure took place with new policies for personnel handling. This, in its time, new Human Resources department became the main user and responsible of the Group Decision Room (GDR). This GDR was geared with a GSS license already existing at the Ministry's Headquarters in The Hague.

4   Case comparison and discussion

In the RPA, the system was acquired for a specific purpose, to prepare and provide training for disaster relief. The training sessions follow a set of repeatable steps. These steps are repeated every time training is conducted. Also, the trainings are design by the writing of scenarios, which are created by trainers from each key service. In this manner, the system is frequently used, benefits drawn often and the cognitive effort is low for anyone since everyone knows what has to be done without much difficulty.

On the other hand, in the MFA the system was acquired for a general-purpose use. Thus no clearly repeatable steps were involved. Every time a meeting needed to be run, outcomes depended on the facilitator's skills and knowledge. In order to be able to satisfy the demand for sessions and to tackle different approaches to different problems, a budget for training facilitators was created, and about 25 facilitators were trained.

4.1 The GSS User Community at RPA

The primary GSS users at RPA were the Incident Training unit within the Bureau Harbor Master. This unit started to use GSS in May 1999. The sole motivation for its acquisition was the development of a tool to support disasters relief training practices. The people in charge of the program saw the advantages that the system could provide to them. One of the first contacts with the technology was during a research conducted in the RPA with Delft University of Technology.

The training for disaster relief involves bringing personnel from all the key services together at the same place. The goal is to evaluate and improve the interaction and response between services in a wide variety of possible disasters at the Rotterdam Port. The key services involved are: the Port of Rotterdam Authority (coordination body), the fire brigade, the police department, the environmental service and the health service.

A standard training session is set as follows. First, the trainers from all the key services prepare a scenario. This scenario is designed in a comprehensive way, with a myriad of different occurrences where all the services need to take action, in the form of “images”. These images are descriptions of a given incident, which added up make a disaster as it unfolds. Then, during the session, the participants are prompted with all these different images and asked to react on them. Each respondent generates information that is also shared to the others. Finally, the trainees are evaluated based on their reactions.

Before the adoption of GSS, training was conducted using flip charts, post-it notes and the like. A representative of every service had to be present as a trainer-supervisor in every session to evaluate the trainees on the spot, because there were no records left available once the session was over. Thus, every time a session was run, at least 5 senior trainers had to be present. Consequently, schedules had to be coordinated, all available human resources were used for one session and the quantity of training sessions reached a “ceiling”.

Given a need on improving the efficiency of the trainings, the staff in charge decided to search for a tool to support them. As a result, GSS was adopted and shaped for the task. The process stayed the same, but sessions were conducted with the GSS and a presentation tool. The participants interacted on line, and were prompted through the system. There were significant improvements on several fronts:

• Time savings for all phases of the process, e.g. designing and writing of the training scenarios.
• A single senior trainer could conduct the session because all the trainee interactions were saved in the server. The outcomes are delivered afterwards to the responsible for evaluation from each service.
• More training sessions were possible, and were conducted more efficiently.

4.2 The GSS User Community of GSS at MFA

The newly formed Human Resources Department received a mandate to provide GSS facilitation services to the whole Ministry in a specially equipped group decision room. Others could make reservation for the room, but each department was to provide its own facilitator. It fell to Human Resources to assure that there were sufficient trained facilitators trained in each department, which meant that each department needed a budget to fund these trainings.

The departments used the GSS facility for issues that ranged from development of policies in different regions of the world to career development within the ministry. No specific predetermined procedure was used by any group, although it was reported that some generic patterns of tool use did emerge over time.

4.3 Perceived Frequency of Net Value

At the RPA, increasing the quantity and frequency of training, while using fewer human resources amounted to increased efficiency on a frequent basis. The technology champion reported that the innovation “gave people the opportunity to do more training with less supporting personnel, that was the whole issue…”

In the MFA, no one could really forecast the frequency of use. The aim seemed to be rather to allow a more participatory way of decision-making, than a specific improvement in the efficiency of a given process. The system was simply available in case someone needed it. A whole new way of doing meetings was attempted, without the certainty on how frequently those sessions would run. Thus, benefits were drawn randomly, only when the need for the use of the system rose.

In the RPA the system was perceived as having a potential to be used often, which, consistent with TTM, seemed to strengthen the case for continued system usage. Meanwhile, in the MFA, the need for the use was random, so users could not forecast when a session might be run. Thus the community that had an expectation of high-frequency benefit became self-sustaining. The community that could not determine a likely frequency of benefit did not.

4.4 Perceived Magnitude of Net Value

4.4.1 Economic value

When prompted, informants from the RPA did not discuss the economic impact of the system extensively. They focused more on time savings that cost savings. Answers on this topic tended to be short, and the general impression was that the economic value of the system was slightly positive.

Informants from the RPA had no insights about the economic value of the GSS one way or the other.

4.4.2 Physical value

The users from the RPA reported neutral physical value from GSS use. No changes in level-of-effort and no changes in level-of-comfort came to mind during interviews. The same outcome was reported by the MFA.

4.4.3 Social value

Neither RPA nor MFA reported negative or positive shifts in social value with respect to GSS use. If such shifts occurred, they were not salient to the informants at the time of the interviews.

4.4.4 Cognitive value - Access Load

In the RPA, there are facilities destined for training sessions where the trainees whether a GSS is being used or not. Further, these facilities are nearby the posts where the personnel stand on watch on a 24/7 basis. Thus, it took little additional effort to access the technology. Nor did the technology interfere with or complicate other tasks. Hence, the access load for the users was low.

In contrast, the GDR at the MFA was not in anybody's normal workplace. The room had to be scheduled in advance, and arrangements had to be made for a facilitator to be available before the meeting to plan the process, during the meeting to conduct the process, and after the meeting to close down the facility and produce the final reports. When the sessions took place, all the participants had to move from their working spaces to another place (the room). Thus, access load for this community was high.

Consistent with TTM, the community of users that reported low access load for GSS became self-sustaining, while the community of users that reported high access load did not.

4.4.5 Cognitive conceptual load

In the RPA, the system was acquired for a specific purpose, to prepare and give trainings for disaster relief. The earlier users learned how to use the features of the system that supported their routine, step-by-step process, and trained others how to follow the process. Users reported that it was not difficult to understand what the system was supposed to do for them.

In the MFA, each session addressed a different kind of problem, so each session required a different set of tools, in different configurations, used in different orders, with different guidance to the group for each step in the process. Infrequent users of the system reported that it was difficult to understand which of the tools and configurations might be most useful to them for a given situation. It was also difficult to guess in advance what patterns of group interaction might emerge when a given tool and configuration were placed before a group. Therefore, they could only use the system when a trained facilitator was on hand to conduct the sessions for them. The facilitator required skills in both group dynamics and in the technology. Some of the MFA respondents found it easy to learn both the facilitation and technology skills. Others found it daunting.

The set of repeatable steps made it easy for all RPA users to understand and remember how to use the system. The wide variety of tasks faced by the MFA users raised their conceptual load considerably. Consistent with TTM, the community of users with low conceptual load became self-sustaining, while the community of users with high conceptual load did not.

4.5 Political value

The political dimension of the Perceived Magnitude of Net Value appeared to have a substantial impact on the continuance over time of GSS system use. RPA, informants felt empowered by the system. They felt that they were more likely to be promoted because the system made them more effective, and freed up time for the most senior users.

In the RPA the product champion was still in place, but respondents reported they were likely to continue using the system if he left. They enacted horizontal agreements across services, whereby anyone could take over if someone else was promoted.

On the other hand, in the Ministry of Foreign Affairs, informants reported that the key supporter of GSS moved to a new job, and frequency and duration of system use declined quickly as budgets for training and facilitation dried up. They reported no ongoing political value in continuing to use the system.

4.5.1 Usefulness

In the case of the PRA, respondents reported that the GSS made it easier for them to do their jobs well. During the interviews, there were a number of remarks about the time savings the system provided. They also valued the permanent storage of training interactions that had previously been ephemeral. In the case of the MFA, respondents reported isolated incidents where they had derived job-related benefit from the system. Consistent with TTM, the community of users that reported frequent opportunities to obtain positive value along a number of dimensions became self-sustaining, while the community that expected infrequent, unpredictable value did not.

4.6 Certainty about perceptions

TTM posits that exposure to a technology gives rise to certainty about expectations of value and frequency. Remarks in the interviews confirmed that that the users in these two cases became more convinced of their assessments of the capabilities and advantages of the GSS as they used it more. In the RPA, the exposure before adopting the technology came from a previous project with the Delft University of Technology. They also read white papers about the use of GSS by the U.S. Navy. Some interviewees, those in charge of assessing the future use of the system in the RPA, reported that these papers helped them see new possibilities for using the system.

In the case of the MFA, the original adopters - the IT department - were over taken as system users by the newly designed human resources department. Thus the person that implemented the use of GSS as a tool for personnel handling was presented with the technology instead of choosing it for himself. The person who followed was also presented with the technology and given a mandate to continue its use, which did happen as frequently as was intended.

4.7 Perceived Net Value of Transition

In both cases, the product champions took a training program outside the organization and the vendor provided assistance in the setting of the respective systems. The level of effort for implementation was almost identical in both cases. In the RPA, only the product champion followed a formal training to become a facilitator. All the other users learned the system by using it and guided by the champion and it was centered on the technical aspects. In the MFA some 25 persons were internally trained for both, becoming a facilitator and know how to use the system.

5   Conclusions

5.1 Implications for research

The results of these two cases suggest that the Technology Transition Model may be a useful model for explaining, and perhaps predicting when a self-sustaining community of users is likely to form, and when such a community of users is likely to disband. However, much more research is required to test the model for its usefulness to be confirmed.

If the model continues to receive empirical support, it might be possible at some point in the future to use it as the basis for deriving an instrument to predict when an organization is likely to change from its current practices to other practices. Indeed, it may turn out that TTM can serve as more general model of organizational change, rather than just a more specific model of technological change.

5.2 Implications for practice

The differences in the dimensions of value across the two cases could all be traced back to the fact that one organization used the technology for repeatable processes while the other used it for ad-hoc projects. Those who needed the technology every day could foresee ongoing, frequent benefits. Those who used it only for special projects could not foresee when they might need it again. Those who used it for repeatable processes learned to use the features they needed, and, for the most part, ignored the rest. Those who needed it on an ad hoc basis had to understand many of the ins-and-outs of the systems and the variety of nuances it could produce in group dynamics, which was far more difficult. Those who used it frequently to improve their job performance experienced affective and political value. Those who used it occasionally could not demonstrate that their ongoing performance would be enhanced if they made the effort to learn the system.

It may, therefore, behoove practitioners who seek to implement GSS in an organization to first seek out a group of uses whose need is frequent, ongoing, and routine. As a first step, design a process for this group to follow that they will find useful in their day-to-day work. Once the first team is in place, the role of the product champion should be to find user groups one after another whose routine tasks can be made easier and more productive through GSS use. Because the tasks are routine, users should not require trained facilitators to conduct their sessions. Because the technology supports day-to-day work, when the technology champion moves on, users may continue deriving the value the technology can provide. Thus, the champion would leave behind a lasting legacy.

5.3 Limitations and Future Directions

This research is the first of many studies that will be required to test and refine TTM. This study examined only two organizations. It would be illuminating to understand how GSS transitions have proceeded in many other organizations. Thus, additional case studies would be useful. In order to gain knowledge about the utility of TTM, it would be useful to have a broader perspective than case studies can provide. More-rigorous tests of the model will be required, but technology transition is not a phenomenon that can be readily tested in a laboratory setting. It may be that survey research could provide both a broader perspective and additional rigor to challenge the model. If the model holds, then it should be possible to design interventions that increase the likelihood of successful technology transitions. Therefore, action research might be an additional means to shed light on the usefulness of TTM.