This website uses cookies to store information on your computer. Some of these cookies are used for visitor analysis, others are essential to making our site function properly and improve the user experience. By using this site, you consent to the placement of these cookies. Click Accept to consent and dismiss this message or Deny to leave this website. Read our Privacy Statement for more.
Article by Alan Friedman - 1979
Share |

Article reprinted from The Planetarian, Autumn, 1979.


Alan J. Friedman, Laurie P. Eason, Cary I. Sneider

Lawrence Hall of Science

University of California

Berkeley, California 94720

*This material was prepared with the support of a gift from Mr. William K. Holt, and National Science Foundation Grant #GPP74-14112. Any opinions, findings, conclusions, or recommendations expressed are those of the authors and' do not necessarily reflect the views of the National Science Foundation.

When we set out to develop a participatory astronomy exhibit for the Lawrence Hall of Science, we knew it would be difficult to design an unattended astronomy exhibit that would actually involve visitors in astronomy experiments. To improve our chances of making this complex exhibit an effective educational experience, we committed ourselves to designing it through formative evaluation. In such a process, every major component of the exhibit is subjected to "trial by visitor." The results are then used to redesign the components if necessary. The improved facets may be tested again, and the whole process repeated indefinitely until the designer-developer is satisfied, or has run out of time and/or money. Some of the tools we used to determine visitor responses were observations, questionnaires, and interviews.

As we will show in this paper:

1) Formative evaluation can be very painful to the egos of the exhibit designers;

2) our exhibit would have been an expensive disaster if we had built it exactly as we originally planned; and

3) the improvements made as a result o' formative evaluation are directly responsible for the popularity the exhibit now enjoys.

Star Games image 1

Figure 1 Ten visitor-operated telescopes are central to Star Games.

The objective of Star Games: Discover Astronomy was for the visitors to learn how to manipulate high quality telescopes in an unattended indoor exhibit. We wanted people to feel accomplished and confident in being able to use real scientific tools to learn about science. Star Games consists of ten small telescopes (Fig. 1) which visitors must learn to aim and focus at a backlighted panel (Fig. 2) containing simulated views of stars, planets, nebulae, and galaxies. This 'sky window" is constructed to reproduce roughly what these same telescopes would see in parts of the real nighttime sky. Once visitors have taught themselves how to aim and focus the telescopes, they are rewarded by seeing astronomical objects invisible to the naked eye. After experiencing this basic function of the exhibit, the visitors can try one of the formal activities available to learn more about how astronomers use telescopes: locating radio sources; finding galaxies, nebulae, star clusters, and planets; determining the effect of atmospheric turbulence or of changing telescope aperture or magnification; and discovering the composition of stars. In all of these activities the optics and the physics are real--only the target is simulated.

Star Games image 2

Figure 2 The simulated "Sky Window" is the target of the telescopes.


Star Games image 3

Figure 3 Three telescope designs undergo "trial by visitor" during formative evaluation.

The right-hand telescope in Fig. 3 was the first casualty of our formative evaluation. That telescope was custom-built to our specifications, and had every feature we thought would contribute to a successful visitor-operated exhibit. In addition to being rugged, it had a built-in mirror in the back, allowing the viewer to bend over and Iook down into the instrument. This is more comfortable for the viewer than craning his or her neck backwards to look through the main tube (the position of the person on the left in Fig. 3) as in an old-fashioned "spyglass" telescope. Some mirror arrangement is used on most astronomical telescopes.


The trials revealed that visitors had a hard time aiming this first telescope. They learned to look down into the telescope, but they had difficulty deciding in what direction to point the instrument itself. Very few members of the public could aim and focus in thirty seconds, and most gave up before then.

Next we tried a small "finder" telescope along the barrel of the main tube (visible in Fig. 3). That worked too well. People enjoyed looking through the small finder and didn't bother with the main telescope, even though looking through the main one was necessary to discover the fainter astronomical images. Then we tried "aiming tubes"--empty cylinders with cross-hairs on each end. Now when we asked people to try the telescope, they would begin by trying the main telescope, discover the difficulty of aiming, find the aiming tube and use it, and then go back to the main telescope. We were sure the, problem was solved.

As a final test (we thought), we left the improved configuration unattended for public scrutiny. Would people operate the telescope without our specifically asking them to try it? No, they wouldn't. To our surprise, very few people advanced beyond discovering the difficulty of aiming the main telescope. Apparently, knowing we were watching them had given the previous visitors the necessary motivation to try the aiming tube. Without "official" observers, the three-step process was too much; other exhibits nearby gave rewards with less difficulty.

Star Games image 4

Figure 4 Visitors using the final design.

The present Star Games telescopes are shown in Fig. 4. These instruments work as the public expects a telescope to work. You look through the main tube and, because of a prism arrangement inside, you see an image right-side-up and correct left-to-right; in most astronomical telescopes the image is inverted and moves in an opposite direction to your push. The present telescope design is much less comfortable to use than the first design, but people do use it on their own. While aiming and focusing are still a challenge, the majority of visitors succeed. The most frequent comment visitors are overhead to make is, 'Hey, come look what / found!" That personal discovery fosters the sense of accomplishment and confidence about using a scientific instrument that we sought to develop.

Star Games image 5

Figure 5 Original conception of a supplementary panel display.

Similar improvements in other aspects of the exhibit were' suggested by the visitor trials. Fig. 5 shows the original conception of an illustrated panel developed by astrophysicist Philip Dauber and executed by Star Games architect Dan Quan. Fig. 6 is a mid-development mockup of the panel being examined by visitors. After interviews to determine both cognitive and affective results of viewing each revision of the panel, a final design wasn constructed (Fig. 7).

Star Games image 6

Figure 6 Visitors examine a mock-up of the panel undergoing revision.

Star Games image 7

Figure 7 Final panel design.

Directions for the additional activities were originally printed as step-by-step paragraphs. People generally ignored these instructions, even though we tried to reduce each step to a few words. Fig. 8 shows the final format, conceived and executed by Budd Wentz. In ourour final evaluation we learned that 69% of the teenage students we studied read these directions. Most people. in other words, will follow a cartoon set of directions even if they ignore a printed set having the same words.

Once Star Games was completed, we began a final evaluation of the exhibit's effectiveness (summative evaluation). In addition to observing public visitors of all ages, we studied 138 high school students from four schools who were scheduled to visit the Lawrence Hall of Science on field trips.

Each school group was divided randomly into two parts. The half that was asked to visit Star Games was contrasted with the half that visited other exhibits. Three tests were given to each student: 1) an Astronomy Quiz, which sampled cognitive learning; 2) a Focus Time Procedure, which sampled psycho-motor skills; and 3) a Raffle Choice Instrument, which sampled attitudes toward astronomy materials.

These tests and the results are described in detail in a technical report published elsewhere (Sneider, Friedman, Eason: 1978). The important conclusions of that report are:

1. The Astronomy Quiz results indicate that the Star Games experience does increase visitors' knowledge about how astronomers use several kinds of scientific instruments.

2. The Focus Time Procedure results indicate that the Star Games experience does develop visitors' psychomotor skills which will enable them to use telescopes successfully.

3. The Poster Raffle conducted during the formative evaluation indicates that subjects' preferences for viewing stars and galaxies are positively affected by the Star Games exhibit. However, the Book Raffle conducted during summative evaluation indicates that the effect is not necessarily generalized to include reading about astronomy..

In addition, the study pr ovides the following information about the effect of Star Games on visitors of different ages and sexes:

4. As might be expected, older subjects performed better than younger subjects on the most difficult question of the Astronomy Quiz.

5. Although both boys and girls who experienced Star Games did better on the Quiz than those who did not, the girls' improvement outpaced that of the boys' on one question.

6. Boys are far more likely to have owned or used a telescope than girls.

These last two significant results suggest that although boys have more occasion to use scientific instruments, both boys and girls benefit when given the opportunity to perform activities like the Star Games.

Our data indicate that the high school students in our studies, as well as public visitors, spend an average of almost two minutes at each telescope. That is substantially longer than the 40 seconds visitors spend at typical non participatory visits (Parsons, 1968), and also longer than the 30 seconds it took for visitors to become frustrated and abandon the telescopes we had originally designed. We conclude that formative evaluation turned Star Games from a frustrating into a rewarding exhibit. We highly recommend formative evaluation to all planetarium educators who wish to develop effective exhibits.

Many planetariums and museums have based new exhibits on components of Star Games, and we will be glad to provide assistance to anyone wishing to replicate this exhibit.

Star Games image 8

Figure 8 Cartoon format, rather than printed paragraph format, was a superior method of presenting instructions.


Many of the staff of the Lawrence Hall of Science contributed to the development and evaluation of Star Games. We would like to thank the Star Games advisors panel, consisting of W. M. Laetsch (Principal Investigator), David Cudaback, Phil Dauber, Lawrence Lowery, Dan Quan, Dennis Schatz, and Budd Wentz. Kenneth Bethel and Dennis Galloway made major contributions to the design of the activities. Monty Reed, Peter Rowe, Tim Aaronson, and Lee Ehmke provided computer assistance, and Leonard Marascuilo advised about methods of statistical analysis. The exhibit was constructed by the Lawrence Hall of Science shops led by John Fredericks, Marshal Montgomery, and Vincent Ryan. Photographs were taken by David Best, Laurie Eason, and John Quick. We would also like to thank the hundreds of students and visitors who patiently put up with our questions.


Sneider, Cary, Alan Friedman, Laurie Eason, "Summative Evaluation of a Participatory Science Exhibit," Science Education (in press) 1978.

Parsons, Lee A. "Systematic Testing of Display Techniques for an Anthropological Exhibit," Curator 8, 2, 167-189, 1968.

This article originally appeared in the Autumn 1979 issue of the Planetarian, the quarterly journal of the International Planetarium Society. You may print it for personal use and link to this web page from another web site, but the file may not be printed for distribution or reproduced in another web site without permission of the Executive Editor Sharon Shanks.



about ips | find a planetarium | officers | privacy policy

International Planetarium Society, Inc | c/o Ann Bragg, Treasurer | Marietta College, 215 Fifth Street | Marietta, Ohio 45750 USA