NASA Position Paper for the CSCW 2002 Workshop on
Public, Community and Situated Displays: MERBoard

Jay Trimble Roxana Wales Rich Gossweiler

NASA Ames Research Center Science Applications International Computer Science Corp

jtrimble@mail.arc.nasa.gov NASA Ames Research Center NASA Ames Research Center

rwales@mail.arc.nasa.gov rcg@email.arc.nasa.gov

ABSTRACT

This position paper describes the ongoing process by which a multidisciplinary group at NASA’s Ames Research Center (social science and computer science) is designing and implementing a large interactive worksurface called the MERBoard Collaborative Workspace. A MERBoard system involves several distributed, large, touch-enabled, plasma display systems with custom MERBoard software. A centralized server and database back the system. We are continually tuning MERBoard to support over two hundred scientists and engineers during the surface operations of the Mars Exploration Rover Missions. These scientists and engineers come from various disciplines and are working both in small and large groups over a span of space and time. We describe the multidisciplinary, human-centered process by which this MERBoard system is being deployed, the usage patterns and social interactions that we have observed, and issues we are currently facing.

Keywords

Pervasive computing, ubiquitous computing, human-centered computing, CSCW, user-interface.

1. INTRODUCTION

In 2003, NASA will send twin robot rovers to explore the surface of Mars. Called Mars Exploration Rovers (MER), they will operate as mobile science platforms and be the most capable systems ever sent to explore the surface of the Red Planet. With a planned mission lifetime of 90 days per rover, every day on the Martian surface represents high value time to gather important science data.

To maximize the productivity of the MER Rovers, the Mission Operations Team at the Jet Propulsion Laboratory will communicate with the rovers every day. The science and engineering teams will receive data from the rovers, analyze the data, determine a strategy for operations for the next day, and develop command sequences to send to the rovers that will execute that strategy.

The MER mission will require collaboration among people and teams at multiple levels in order to develop appropriate science and engineering strategies and integrate them into sequences of commands.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.

Figure 1, Scientists and Engineers on Earth communicate with the rovers on Mars each day.

The MER Human Centered Computing Team at NASA Ames has worked for the last year and a half to develop a collaborative information tool called the MERBoard that will help to facilitate this process. As members of this group, we offer cross disciplinary (computer science, anthropology and cognitive psychology) insights into the process of conceiving and designing the MERBoard as a new collaborative, situated, large screen technology for creating, accessing, displaying, annotating, sharing and saving information within the MER mission environment.

1.1 Collaboration within small Science Theme Groups

The MER Science Team begins their day with the receipt of data from the rovers. The science team is organized into theme groups according to science discipline: geology, geochemistry, astrosoilology, and atmospheric studies. There is also a long term planning theme group whose job is to plan rover operations in a strategic sense, i.e. over periods of days, weeks and months. Every day, the various theme groups analyze their data and determine the science objectives and priorities for their group. Each theme group is composed of many scientists who must work collaboratively to determine a set of objectives and priorities that the group members can support. As each theme group analyzes data and develops their plans, team members circulate among other groups to share plans and strategies. This is currently an informal process.

1.2 Collaboration within a large Science Operations Working Group

After the science theme groups have developed their plans and priorities, the collaborative process continues as all the theme groups move to another room in the Mission Support Area and gather in a Science Operations Working Group (SOWG) meeting to develop an overall integrated science plan. During this meeting all of the theme groups present their requests and priorities and develop together a single integrated science plan of associated rover instrument activities and movements that is acceptable to all theme groups and achieves agreed upon scientific objectives. This meeting is lead by a single person, the SOWG Chair, who is responsible for delivering a time ordered list of requested science activities to the engineering team.

2. UNDERSTANDING THE DOMAIN Fieldwork, Observations and Analysis from the FIDO 2001 Field Test

In the spring of 2001, we observed a series of “Mars Yard” and “Field” tests at the Jet Propulsion Laboratory. During the tests, scientists practiced tele-robotic science using the FIDO (Field Integrated Design and Operations) rover, commanding the remotely located rover and collaborating to make science decisions.

Ethnographic observation and analysis, as well as interaction analysis [1] of video film taken during the above tests, revealed constraints on the collaborative process during these tests. As scientists and engineers received data, analyzed it and made decisions related to the rover, we saw limitations on their ability to view, share, present and save important information.

Figure 2: A scientist using flip charts to present to the SOWG at the 2001 FIDO Field Tests

For example, in Figure 2 note the use of flip charts, laptop computers, print outs and projection screens that are displaying information from the team’s Science Activity Planner tool.

Scientists had important supporting data on their laptops they could not share in a form the rest of the team could easily see. They held up their laptops in an attempt to display the information. The projection screens facilitated group viewing of some information but were not configured to display others. During the Science Operations Working Group meeting, the team used flip charts for creating and presenting a flow chart, called a Sol Tree, that was used in the decision making process. While flip charts support natural, rapid, handwritten representations they do not allow for the embedding of related information such as images. They are difficult to store, retrieve and search over long periods of time, and they are not easily shared beyond team members who are collocated and in close viewing range.

All of the teams in the test were collocated in one room. Scientists and engineers were able to share and discuss their process more easily because of this. However, mission events will take place on three floors of a large building and will require information sharing among distributed as well as collocated groups. Participants will move from floor to floor and room to room for a variety of work practice events and specialized meetings.

3. IDENTIFICATION OF NEEDS

The MERBoard design is intended to assist the mission operations process by addressing the direct user needs that we observed during the tests, and issues that we anticipate from our participation in the mission operations system design process. This includes the need to display, annotate and share information in a natural way, the need for a medium such as an electronic whiteboard that allows for brain storming, sketching ideas, compositing, and lists that can then be captured and shared easily, and the capability to easily save, recall and distribute the data. By inserting the MERBoard into the mix of tools shown in Figure 2, we proposed to augment the teams existing work practice, providing additional functionality that supports work across the large Mission Support Area. Our goal is to provide the following:

· Designated interactive displays in the science assessment and engineering areas for accessing, viewing, annotating, saving, distributing and sharing science and engineering data and information, and for creating and saving new information representations within and across small groups.

· A designated interactive display in the Science Operations Working Group area that provides a pervasive information space for presenting information to a larger group within a formal meeting process.

· Functionality that supports knowledge sharing and the creation of representations that support the long term decision process within and across teams in the mission setting

· Pervasive access to information, so that a team member can access personal and group information at any of the large screen interactive MERBoards in the mission support area.

· A virtual network computer that allows viewing and control between one MERBoard and another and/or viewing and control between personal computers and MERBoards.

4. DESIGN AND IMPLEMENTATION:

The MERBoard: A Large Interactive Work Surface for Collaboration

Based on these needs, we designed, implemented and iterated with a small set of individual users/scientists to develop the initial MERBoard architecture and user interface.

The MERBoard was inspired by the BlueBoard project at IBM [2]. The designs of the BlueBoard and MERBoard have diverged as the development of each system responds to the needs and work practice of users in their intended domains.

4.1 Hardware Architecture

The MERBoard system consists of several 50-inch plasma screens with touchscreen overlays backed by a standard computer. These computers are connected through the ethernet to a centralized server and database.

4.2 Software Architecture

The MER team is a multiplatform group, using Windows, Macintosh, Linux and Unix systems. Given this, the MERBoard architecture has to be platform independent; we chose Java for this reason.

We also recognized that this architecture needed to not only support large screens in multiple places, but also needed to support a diverse set of data and communication channels.

Figure 3: MERBoard Network

4.3 User Interface

We divided the user interface into two areas -- the globally accessible area and the client area. The global area is a small region where primary applications and helper functions are always available. The much larger client area hosts each application's window.

The four core applications that MERBoard supports are: the whiteboard, the web browser, the Virtual Network Connection (VNC) and the MERSpace (an information repository). The two primary helper functions are "email screenshot" and "capture screen."

Figure 4: screenshot of MERBoard showing tabbed interface for Browser, VNC, Whiteboard and MERSpace, and meta-tools for screen capture and e-mail

4.3.1 WhiteBoard

This is the primary workspace for the scientists. In addition to being able to sketch and brainstorm on the whiteboard using a small set of tools, the board also supports image manipulation and simple flowchart capabilities. These capabilities came directly from mission task support requirements.

We made the decision to build one application with a simple set of tools rather than break the application out into three tools (e.g. a flowchart tool, a sketching tool and a separate image annotation tool). We felt this "one-place-to-work" model was simple, and inevitably scientists would want features from one application to be in the other.

Since scientists may be working on several items simultaneously, since there may be different groups of people in front of the board, the whiteboard also supports tabs. This allows team members to quickly “flip” from one board to another, a flip-chart like capability.

The whiteboards file format is SVG, a vector-based object model for drawing entities. This allows whiteboard drawings to have multiple editable objects that may be saved and retrieved in an editable format. It also allows for export and editing in external applications that are SVG compatible.

The whiteboard evolved into the workspace/work area for scientists, either for sketching, image annotation or flow-chart creation and editing.

4.3.2 Browser

A large volume of data is available through the web. Other mission related tools were constructing web-based interfaces to allow access mission data and reports. We needed to support a mechanism for getting at this information, so a core tool is a Java-based Browser called ICE [3].

4.3.3 VNC

We recognized that scientists would have data and applications on their personal computers that they would want to display on the MERBoard. We needed a way to allow them to present that information from their computers on the MERBoard. We also wanted to support Board-To-Board communication and control. Virtual Network Computing (VNC) makes this possible. Scientists can install a client on their personal computer and then run that client to display their screen on the larger displays. Scientists can control the MERBoard from their computer or vice versa, control their computer from the MERBoard.

4.3.4 MERSpace

The MERSpace is a file storage space for individuals and groups. Whiteboard pages can be saved in MERSpace. Users can also post content from their personal computers into their MERSpace. While any file or document can be stored in MERSpace, the MERBoard is currently capable of displaying rtf, PDF, jpg and SVG files. More file types will be supported in future versions. Because the user and group MERSpaces are available at any MERBoard, content can be created at one board, and presented or used at another. This supports the flow of people and teams through the daily command creation process, where information is created in science theme groups, is presented at the science operations working group, and then moves into the engineering process of creating commands.

4.3.5 Meta-Tools and Other Data Communication

In addition to the four core applications, the MERBoard supports globally accessible meta-tools. Current meta-tools are screen capture and e-mail.

Screen capture provides a simple way for users to capture image information from different applications (e.g. from their VNC session or from the browser) and place it in the whiteboard for annotation and comparison.

Email screen allows the user to send information from the MERBoard to another MERBoard user. The reason for choosing e-mail as a means of communication is that our user community is already in the habit of checking their e-mail regularly. This provides a means of getting content from one user to another without requiring new workpractice. We will also provide the capability for one user to send content to another users MERSpace. However, having a user check their MERSpace for new content is new work practice, and it is uncertain how many users will do this and how often.

5. THE SECOND FIDO OBSERVATIONS

In the summer of 2002, we observed the next FIDO field test, placing two MERBoards within the test facility. The second FIDO test was larger than the first and was held in two rooms instead of one, with a larger number of participants. The test area now more closely simulated the design of the Mission Support Area by putting the larger three of the five science theme groups in one room, where they did their analysis and having them move to a larger room next door, where the other two theme groups were located, for the SOWG meeting. The MERBoard collaboration design assumes that each science theme group has their own MERBoard, however, due to the limitations of the facility, we were only allowed to place one MERBoard in each room.

We used a variety of camera setups to capture interactions around the MERBoard and screen capture and logging setups to capture the activity on the board itself. We were able to observe and capture how the board was used, and to compare the actual use to the expected use.

Figure 5: Scientists using the MERBoard to view a terrain image

5.1 Group Interactions Around the Board

As we had anticipated, the board in the smaller room supported the work practice of the small groups and was initially used to access, and view images and mission relevant information. In fact, groups sometimes grew from shoulder to shoulder collaborators to over the shoulder collaborators as several people gathered at the boards to view images and create representations of interest. The large size of the board facilitated group interactions around images in several ways. First, the size and height of the board allowed people to easily view images in groups (Figure 5). The ability of the board to display large scrollable images, combined with the touchscreen which facilitated interactions, allowed groups of scientists to view large terrains such that all of them could scroll through the terrain, point out features of interest and be active participants in the decision process of selecting features to be designated as targets. Contrast this with the more traditional means of a group of people using a personal computer or workstation to do examine large terrain images. In that case several people are crowded around a relatively small screen, usually with one person controlling the computer. The size of the screen, locus of control, and type of interaction is changed significantly in this case. It’s also worth noting that large terrain and target images are displayed on the wall during field testing. While this practice continued even in the presence of the MERBoards, the interaction and use of wall sized images is more constrained and less interactive (Figure 7). Scientists tended to view, rather than interact with, the wall mounted images.

Figure 7: Scientists viewing wall mounted terrain images

5.2 MERBoard as a Pervasive Device and Presentation Tool

With the initial help of some early adopters, scientists used the pervasive set up of the two boards to create and save information in one room and then display it in the second room during the SOWG meeting. Figure 8 shows a scientist presenting to the SOWG. Over the period of the tests, the team members adapted their use of the MERBoard during the presentations, using content created on the MERBoards, content created on individual users lap tops and posted to MERSpace, as well as content created on users lap tops then shown on the MERBoard using VNC. Note that the presenter in figure 8 can show images to the group, and still be able to directly interact with the image on the screen. This is in contrast to the projection screens where remote interaction using a pointing device is required.

Figure 8: A scientist using the MERBoard to present to the Science Operations Working Group

As we had also anticipated, placement of the boards is crucial to their use. The long term planning group that was located closest to the board in the smaller science assessment room rapidly became “owners” of that board. Additionally, we had provided some simple tools on the board (boxes and lines) to help this group create Sol Trees, a science process decision tree representation. Over the period of the test, the consistent day to day representation of this “tree” on the board formalized both the representation and the use of the board. From time to time, group members who were creating the tree would move to a large white board to make quick notes before inserting them into the decision tree. When the tree was running on one of the screens, other team members appeared reluctant to minimize it and use the board for other purposes.

5.3 MERBoard and Traditional Media

Note in figure 2 that flip charts were used as a means of developing content and presenting it to the SOWG at the 2001 FIDO Field Tests. The image in figure 2 shows a scientist presenting Sol Trees. As previously mentioned, the MERBoard design includes some basic features in the whiteboard specifically designed to facilitate the development of Sol Trees. A key question for the MERBoard team going into FIDO was to what extent team members would use the MERBoard in place of a traditional medium, such as flip charts. Going in to the test we believed that an electronic medium, such as the MERBoard, must have several advantages to have any chance of replacing traditional media that team members were already comfortable using. The advantages that the MERBoard offered for Sol Trees were a large interactive workspace, the ability to easily save, recall and revise drawings, electronic drawing tools, and the ability to develop the Sol Trees on the board in the theme group area then easily bring the same drawing up on the board in the SOWG area for presentations to the group.

Figure 9: Scientists using the MERBoard to develop Sol Trees, a flow chart representation of long term objectives for the rover

Figure 9 shows the Long Term Planning Science Theme Group using the MERBoard to develop Sol Trees. Note how the board facilitates group interaction. Figure 10 shows the flip charts half way through the FIDO test period. Note that they are unused at this point. Though the flip charts did get some use towards the end of the FIDO test period, the use pattern clearly showed that the MERBoards advantages were enough to get the team members to change their work practice and use an electronic medium in place of a traditional one.

Figure 10: A flip chart at FIDO 2002, in the same location as the flip chart shown in Figure 2 at FIDO 2001, sits unused half way through the test

6. ISSUES AND FUTURE WORK

Given the current state of the MERBoard system and observations from the latest FIDO, we raise several issues and note some future paths.

6.1 Issues

· Ownership and Individuals– during collaboration, how do we deal with an individual's information? This involves issues of personalization, security and privacy as well.

· Ownership and Groups- during the mission will people be reluctant to use MERBoards that are not being used by others simply because their placement near a group identifies them as owned by that group?

· Logging in and out – what are the optimal requirements? If people don’t save, close out or log out, this affects how long information needs to stay on the board (clutter, security, etc.)

· Enabling user recognition by the board and knowing who is currently interacting with the board.

· Expectations of users of running computer software (treating the large display as a computer)

· Poor input mechanisms (touching and dragging can cause skipping and inaccurate selection)

· Knowledge and Information Management -- where does the information live and how does the user organize this information, especially in light of the user belonging to multiple groups?

· Physical location and height of the Boards themselves

· Group organization management

· Formalization -- when users create something on the board, the document becomes formalized. The digital medium makes it more so than a whiteboard. Typed text makes it more than handwriting.

· Diverse technology -- everything from firewall constraints to different OSs to different display resolutions to different clients

6.2 FUTURE Paths

While mission-oriented, there are several paths for the future of the MERBoard, each with their own set of issues and opportunities. As we develop this mission-specific application, we are keeping mindful of longer term possibilities such as:

· Non-mission use (e.g. workplace collaboration)

· Federated research centers

· More powerful clients (clients are applications that run on other appliances with different constraints and affordances).

· Synchronous distributed collaboration

· Asychronous distributed collaboration

7. REFERENCES

[1] Jordan, B., Henderson, A. Interaction Analysis: Foundations and Practice, Institute for Research on Learning, Report 94-0027, (July, 1994).

[2] Russell D.M. and Gossweiler R., On the Design of Personal & Communal Large Information Scale Appliances, UbiCom'01, (October, 2001).

[3] ICE Browser http://www.icesoft.no/main/main.php