An Experimental Study of Computer-Mediated Collaborative Design

Mary Lou Maher and Anna Cicognani
Key Centre of Design Computing
University of Sydney

[Excerpt]

• Abstract
• Overview
• The experiment
• Preliminary observations
• Discussion
• Instructions
• Briefs
• Web Based paper

1. Overview

The use of computer technology in design practice is moving towards a distributed resource available to a team of designers. In addition to CAD, designers now make use of databases, spreadsheets, and analysis and simulation programs. The focus of this paper is a study of how these various technologies support the collaboration of individuals when combined with computer-based communications. The paper presents a formal approach to collecting and analysing data on computer-mediated collaborative design for the continuing development of design information models.

The development of software to support designers has been based largely on the assumption that there will be a single person using the software at a time. The development of user interfaces and representation schemes has been directed by the limitations of computer systems to deliver resources to a one user per window or input/output device. The development of client/server architectures, desktop video conferencing, groupware, and the availability of the Internet has changed our expectations for design software to allow multiple designers to interact in a computer-mediated environment. The implications of these developments includes the possibility of computer-mediated collaborative design as an alternative to collaboration across a table or at a meeting that is largely paper-based. The problem is that very little is understood about the use of this technology in a design environment and the need for design documentation that enables collaboration among professionals.

This paper presents a review of the research that has led to the need for a formal study of computer-mediated collaboration. The paper then discusses the similar developments in protocol analysis and data collection in two areas: design protocols and collaboration protocols. The development of an experiment in which computer-mediated collaborative design is presented that focuses on the type and content of the information recorded in a collaborative design session is presented. The summary discusses what has been learned from the experiments so far and identifies future directions for such research.

4.2 The experiment

In this experiment, designers are observed while designing alone and then while designing in a team of two. In each session, the designer(s) will record their designs using computer tools. The data collected is the information the designer records on the computer. In the first part of the experiment, a designer will be studied to observe how he externalises or describes his design ideas when designing alone. In the second part, two designers will be observed while they are collaboratively designing. This is not an experiment about the collaborative process, but, rather, an observation of how a designer has different needs of computer support when designing alone or with a remote partner.

Each experiment has two sessions. Session 1 provides a base for each designer and the way he describes his design using design software. The hypothesis will be tested during Session 2, where designer A, using the same design software, works collaboratively with designer B to describe their combined design ideas.

During Session 1, Designers A and B will solve Design Problem 1 (DP1) by themselves, using design software such as a simple drawing program and/or a CAD system and/or a simple database. The designers will not be asked to "think aloud", but to give a description of what they are doing, and, possibly, why. The observer will assist this session as a spectator, eventually putting questions, but with a purpose of "normal understanding" (as anybody would put questions if the explanation is not clear).

During Session 2, Designer A and Designer B will be asked to collaboratively solve Design Problem 2 (DP2), where DP2 is a similar type of problem to DP1 but does not have the same brief. B will be asked to operate the design software, and A will be asked to give instructions, remotely, using video conferencing and the pointer on the shared windows. A will be asked not to make changes to the design description directly, but to ask B to make the changes. The observer will record the session, helping to overcome any technical problems in the use of the selected software.

If in Session 2, as the hypothesis suggests, the designers record more semantic information, researchers will be able to develop more accurate models of collaboratively designed products and determine requirements for computer support for synchronous collaborative design.

Even though Designer A will be asked to instruct Designer B, B will not be considered as a simple executer. B will play the part of an "intelligent" operator, and will influence the process, with both design and computer skills. What should be avoided is that A takes control of the whole design process, as A will be the "speaker" and B the "operator".

The collaboration will have an aspect of remoteness, and of physical absence. The communication will be computer-mediated by video conferencing and shared drawing boards reflecting the current state of the technology, with the technical problems that inevitably will come up during such sessions. These problems are an important aspect of the study for understanding how the role of software support facilitates or frustrates collaboration. These technical problems will be observed although they are not the major focus of the study.

4.4 Preliminary Observations

Two pairs of designers have been observed following the experiment as outlined above. Although this does not provide enough information to prove or disprove our hypothesis, it does provide the basis for some preliminary observations. We describe our observations in terms of the technology the designers used, their response to the technology, and a qualitative observation of the difference between the two sessions.

Each designer used two computers for both sessions. One computer provided video conferencing and the second computer provided a multi-user environment without a personal communication channel. We decided to use two computers rather than one so that the designers had a choice of technology that suited them personally and to alleviate the problem of lack of screen space. The video conferencing software used was InPerson on a Silicon Graphics Workstation that includes video, audio, and shared whiteboard facilities. The multi-user environment was Timbuktu on Macintoshes with access to a broad range of CAD systems, 3D modellers, word processing, spreadsheets, etc.

The designers learned to use the video conference and whiteboard facilities very quickly and to rely on it entirely for communication during the collaborative phase. The designers used only the software they were familiar with on the Macintosh and when they had technical difficulties due to lack of knowledge of the software, they were stalled in their design process, making no progress at all. In other words, the technology problems caused the designers to stop thinking about the design and to focus only on the technology problem. In the first pair of designers, the technology problems were not overcome, while the second pair used familiar software allowing them to progress to a design solution they were pleased with. Where our original intention of using the same software for all designers participating in the experiments would provide a similar data set, we now realise that the software must be familiar to the participants or they are unable to think about the design.

An interesting observation of the experiments so far is that the designers could not follow the instructions in the brief for session 2 that one person operate the software and the other describe his ideas verbally. Both designers operated the software. One pair of designers collaborated by both operating the shared whiteboard using the video conferencing facility. Another pair of designers split the job by having one designers use the shared whiteboard to develop the floor plans and the second designer using a CAD system to develop alternative 3D massing studies of the design. Each of the designers in the second pair relied on feedback from the other, but only made changes to the software he was operating.

Our preliminary observations are that there is some difference in the way the designers documented their designs when working alone and when working together. Much the design semantic was communicated verbally using the video conference facility rather than additional design documentation. During the alone session the designers did not articulate their design semantic at all, occasionally used labels to indicate to themselves what the graphic symbols were for, but overall the design semantic was not documented. When collaborating, the design semantic was communicated but not documented in most cases.

5. Discussion