Illuminating Learning in the Dome
Share |

 

Illuminating Learning in the Dome:
Constructing the international STudies of Astronomy education Reseach DataBase

 

Timothy F. Slater, Ph.D.
University of Wyoming
Laramie, Wyoming USA
timslaterwyo@gmail.com

 

From Planetarian, Vol. 46 No. 4, December 2017
Pages 12-16, 32

 

Ever since the very first carefully positioned, star-like pinpoints of light were shown to students under a planetarium dome, educators have wondered about the true educational effectiveness of the planetarium. One might naturally assume that the planetarium represents the ultimate astronomy teaching tool. Rather than braving all manner of windy weather and annoying insects by going outside night after night after night for decades with mentoring elder sky watchers to learn the sky, the planetarium allows one to readily demonstrate all heavenly motions and phenomena that can be seen from anywhere on Earth—even during daytime hours. In this sense, the educational promise of the planetarium is hindered only by the imagination of its operator.


Over the last century, most people who have visited a planetarium of any sort report that their visit is a grand experience. Even in the current digital age where children are bombarded with complex scientific visualizations and more-realistic-than-life special effects using CGI on their mobile smart phones, planetarium visitors generally describe their experiences as positive across the board. As such, most people intuitively believe that a planetarium should inherently be a superior educational learning environment to a traditional classroom. And, at the same time, most people tacitly assume that acquiring a planetarium in their community is probably worth the distributed cost.


Given the high cost of constructing a dedicated brick and mortar planetarium facility— or even the cost of purchase and maintenance of a portable planetarium using an inflatable dome—administrators and education stakeholders throughout time have often demanded some semblance of evidence of a planetarium’s educational effectiveness. Such a request seems reasonable, and one might naturally assume that planetarium education research methods and results would be widely available in the scholarly education research literature. In other words, a quick trip to a nearby university library should readily result in sufficient planetarium education research data to satisfy even the most skepti- cal stakeholders.


The problem is that it doesn’t.


At the same time, when discipline-based astronomy education and planetarium education researchers begin any systematic research study—graduate students, for instance, working on their dissertations—the first step is to conduct a thorough review of the scientific literature (Plummer 2015; Slater, Slater, Heyer, & Bailey, 2015).

 

In days not so far in the past, the three-fold task was: first, to visit the university library to look at databases and published literature reviews to find articles or dissertations about planetarium education; second, to make photocopies from illuminating microfiche readers or by spending hours making photocopies one page at a time of long awaited items ordered through interlibrary loan; and third, to then create a critical summary of the research landscape in order to establish an as yet unsolved education problem that was both useful to the broader community and pragmatically something that could be studied given the limited time and fiscal resources available.


Today this task of surveying the literature seems many times easier given the powerful Internet-based scholarly articles, proceedings, and theses aggregators such as ERIC, ProQuest, GoogleScholar, ResearchGate, and Academia.edu. Yet, simply searching online databases for “planetarium education” or “planetarium learning effectiveness” most often yields sorely disappointing and vapid results. In other words, planetarium education researchers building comprehensive literature reviews have much the same challenge as those aforementioned people trying to readily establish the tacitly assumed educational power of the planetarium as a teaching tool—the readily available scholarly evidence of effectiveness is far more difficult to find than one might wish.

 

Where is all the planetarium effectiveness evidence?

Planetarium educators and enthusiasts are left with the question of where are all of the educational effectiveness studies and hard-earned community knowledge reports about successfully teaching in the planetarium? To our surprise, written accounts for most astronomy teaching strategies and planetarium education research results are far more widely dispersed than we anticipated.

Although some reports of planetarium learning do readily appear in a traditional literature search, the number of "hits" when searching is vastly smaller than if one were doing a survey of numerical problem solving in physics education research, or mapping skills in geosciences education research. Certainly, decades of work have been described and documented in Planetarian, but this represents only a tiny fraction of what is known globally about learning in the planetarium. Perhaps the larger problem is that the most successful and busy planetarium educators are busy educating in the planetarium, rather than writing archival articles, as the veteran Planetarian editors can willingly attest.

Moreover, much of what is known about teaching and learning in the planetarium does not appear in the scholarly literature at all. Instead, this teaching knowledge is sometimes captured in professional association meeting presentations, regional planetarium educator newsletters, the 1970’s Spitz Planetarium Director’s Handbook edited by Michael Bennett, or even books such as LHS’s Planetarium Activities for Successful Shows (PASS) booklets (Gerald Mallon and Alan Gould), among many others. At the same time, much of the systematic planetarium research studies are done in pursuit of graduate degree require-ments—unpublished masters theses and doctoral dissertations.

These diverse avenues distributing research reports that are not formal refereed articles or graduate juried these or dissertations are collectively known as the "grey literature" (Slater, 2015, 2016).

As an example, consider a hypothetical planetarium director who did a first-rate, well-conceived masters thesis in 1975 doing a two-group controlled comparison study of some highly valued aspect of learning or attitude enhancement in the planetarium. Taking years to complete, this thesis study would probably have been guided and juried by three to five scholars, far more than the typical one or two peer-reviewers who referee most scholarly journal articles. One might reasonably assume this has potential to be a strong and valuable study.

But, just how would a scholar today find that study so as to either replicate it or expand upon it? The disappointing fact is that no one today would likely ever find this imaginary unpublished, decades old thesis no matter how valuable, resulting in the same study being done over and over and over again, reinventing the wheel each time by well-intended but largely unaware researchers. This represents a substantial inefficiency in the planetarium education community.

Conceptualization of the iSTAR database

Given the wealth of existing planetarium education research and teaching effectiveness knowledge that seems to be hidden beyond the traditional library walls, a concerted effort to build a system to capture a and collect these resources was initiated. The project is led by Dr. Stephanie J. Slater, director of the International CAPER Center for Astronomy & Physics Education Research in collaboration with Australia’s Michael T. Fitzgerald and graduate student Saeed Salimpour of Edith Cowen University’s Institute for Education Research, Brazil’s Paulo S. Bretones of the International Astronomical Union’s Working Group on Astronomy Education, among many others, including myself and University of Wyoming Ph.D. candidate Coty B. Tatge.

Conceived as an internationally-accessible database of both easyand hard-to-find systematic astronomy education research studies and astronomy teaching insights, this internationally collaborative project is generally known as the international STudies of Astronomy education Research database project—or iSTAR for short.

The first-steps plan to create the iSTAR database was to do the hard work of finding, categorizing, and uploading astronomy education research dissertations. Our naïve thinking was that there were not very many—predicting maybe 50 such documents existing at the most—and that using dissertations would be a good way to test out our initially established database categorization criteria.

When looking at astronomy education research dissertations written in the 1970s, the included literature review might have summarized five or six previous dissertations in the field. You might imagine our great surprise when we uncovered nearly 300 astronomy education research dissertations and theses across a myriad of topics and utilizing diverse research methods on our first cursory search.

Seemingly overnight, the sufficiently difficult task of reading and categorizing a predicted few tens of dissertations became an arduous task of reading and categorizing the actual number of hundreds of dissertations.

Any large database needs to be efficiently searchable in order to be useful. After numerous searchable in order to be useful. After numerous iterations and valuable feedback from the community, we decided to use a categorization scheme that identified studies as those being quantitative or qualitative; done in formal, school-based settings or informal, out-of-school settings; and based on particular participant-demographic groups, among many others. The categorization scheme used for every entry in iSTAR is summarized in Figure 1.

In a seemingly unending stream of surprises, another thing we quickly learned is that dissertations completed more than twenty years ago are not readily available in any digital format via the Internet, and masters theses rarely are available electronically. The implication here is that iSTAR team members—and by “team” I mean mostly hardworking graduate students tirelessly working late into the night—needed not only to obtain bound, hard-copy dissertations through interlibrary loan, but also needed to carefully scan these documents into digital PDF format. You can probably imagine how bleary-eyed this would make even the most dedicated research assistant.

In some cases, university libraries no longer had hard copies they were willing to loan, resulting in us having to contact the original authors—or their families if they had passed away—to beg for copies to be mailed back and forth with carefully organized tracking numbers for these family heirlooms.

These steps all had to be done before the documents could be read, categorized, and then uploaded, which is time consuming and intellectually exhausting in and of itself. Taken together, the time required to do these thankless tasks was unexpectedly Herculean, but incredibly valuable in the end.

Although we certainly have in no way exhaustively collected all the extant dissertations and theses, let alone meeting proceedings and newsletter contributions, we have made a solid start and hope that the broader community will now voluntarily help by creating a free account and uploading their own research dissertations, theses, papers, and research reports if they are not already in the database.

Planetarium education in iSTAR’s first-light survey

In much the same exciting way that a new telescope will make first-light observations to “see what one can see,” we completed a cursory “what can we see?” survey of the initially 300 uploaded dissertations and theses. Reported in detail elsewhere (Slater, Tatge, Bretones, Slater, Schleigh, McKinnon, & Heyer, 2016), we see that astronomy education has been being published since the 1920s and steadily increasing today. We can also see that the earliest researchers were also dealing with many of the same tenaciously thorny teaching problems that exist today: moon phases, seasons, and gravity.

When looking specifically at planetarium education research revealed in iSTAR’s firstlight survey, our initial observation is that there was a dramatic change in the volume and nature of planetarium education research from before 1990 to after 1990. This is shown in the histogram in Figure 2, adapted from Slater and Tatge (2017, p. 10). In particular, there was surprisingly little research production in the 1990s: We suspect this is a result of planetarium educators being consumed by the effort of absorbing the combined widespread introduction of digital planetarium projectors with the simultaneous widespread distribution of portable planetariums with inflatable domes.

Figure 2 clearly shows a bimodal shape of publications of empirical studies in planetarium education research over time, first peaking in the late 1970’s (n=13) and then again in the late 2000’s (n=17). One observation worth emphasizing is that the majority of empirical research studies were conducted as planetarium research dissertations and theses before 2005, and since that time the majority of research studies are published as peer-reviewed works in scholarly journals.

This is important for interpreting the historical context. It has been argued that dissertations and theses should be considered part of non-peer-reviewed gray literature base, and should not be considered or valued as seminal or critical literature in the same serious way that peer-reviewed journal articles are.

For example, when working with National Research Council committees to summarize the existing knowledge base for astronomy education (viz., NRC, 2012), only results from peer-reviewed articles are supposed to be considered. However, work by Slater (2008) as well as Bailey and Lombardi (2015) clearly show that much of the insightful astronomy education research results are found among unpublished dissertation research documents, rather than formally published peer-review articles.

Moreover, prior to 1990, when a graduate student completed their dissertation that work was often considered “finished” and historically there was little motivation or remuneration to republish the same study results a second time in a scholarly journal.

Today, the scholarly world is a quite a bit different and students in all fields are encouraged to publish the results from dissertations again in the form of scholarly journals, such as the peer-reviewed Journal of Astronomy & Earth Sciences Education (JAESE.org) or the peer-reviewed research sections of Planetarian.

The data from Figure 2 lends further weight to the critical importance of including dissertations and graduate theses related to planetarium education as part of any summarizing reviews of the broader planetarium education literature landscape. iSTAR allows us to make some strong distinctions in planetarium education research before and after 1990. Summarized in Figure 3, we see that general changes in each of these planetarium education research attributes are generally aligned with the gradual change from an analog to digital era. Planetarium education research was primarily conducted and distributed through dissertations and theses using quantitative methodologies, whereas today the majority of research uses both qualitative and quantitative methodologies and is more often published through the peer-reviewed process characteristics of journals.

Although fewer peer-reviewed articles were published pre-1990, George Reed at West Chester State College is responsible for the majority of those works, and today Julia Plummer and colleagues at Penn State University have published and continues to publish the bulk of studies regarding the planetarium.

Planetarium education researchers from both the analog and digital eras were mostly focused on K-12 students’ understanding of astronomy content; it wasn’t until digital planetariums that researchers narrowed their scope to focus specifically on the sun-earth-moon system, such as lunar phases and seasons, largely due to the broadening emphasis on the related national education standards.

Pre-1990, the School Science and Mathematics journal was the most central platform for peer-reviewed, empirical planetarium education studies, whereas today Planetarian and the Journal of Astronomy and Earth Sciences Education have largely usurped that role.

Recurring themes emerge

Recurring big themes do emerge when looking at research reports in the iSTAR database. These themes are useful for planetarium educators to keep in mind when designing learning experiences for their diverse audiences. I suggest these themes in no particular order of importance, and kindly remind the reader that these are generalizations, and exceptions do exist.

The first theme is that planetariums in and of themselves are not automatically more educationally effective than classrooms just because they are a planetarium. In fact, it may be that if the prescribed learning targets are memorizing star and constellation names by rote, a planetarium-based classroom might be actually be an unnecessary distraction to students more familiar with a traditional classroom setting. Planetarium educators would be well advised to plan purposeful educational experiences for which the planetarium setting is uniquely situated.

The second is that students are unlikely to remember facts when relayed in the planetarium any more than when facts are relayed in the classroom. What we know from contemporary educational theory is that ideas are best inserted into students’ long-term memory when matched with a memorable, narrative story that encourages students to repeatedly think about and encounter the idea.

Remember how students seemingly remember the smallest details from movies they see, yet struggle with disconnected facts presented in the purposeful learning environment of traditional schools? A corollary to this is that students can only internalize so much, and too many planetarium presentations present an overwhelming deluge of facts and figures because of the limited time allocated to a planetarium visit. Planetarium education program designers need to be well versed in the learning sciences and interactive learning techniques, such as those promoted by members of the Live Interactive Planetarium Symposium (LIPS) community.

The third is related to many planetariums’ goals for improving attendee interest, attitudes, and values about science, and science-related career paths. As with the unrealized hope that planetariums would inherently be better teaching facilities for improving student achievement, the planetarium does not in and of itself automatically enhance students’ attitudes toward STEM. Attendees can have a great time at the planetarium and thoroughly enjoy the experience, but no extant educational theory exists that can attribute an isolated 40-minute experience as overriding decades of students’ negative experiences in learning STEM.

Planetarium educators need to plan extended, repeated, and purposeful learning experiences when they want to enhance students’ affect toward STEM, perhaps using student internship programs, as an example.

The future of iSTAR

iSTAR is only in its infant stages and it is our intention that it becomes a community nurtured, well-fed, and growing resource. Perhaps self-serving to a researcher like myself, we hope that iSTAR will help discipline-based astronomy education and planetarium education researchers and graduate students to put their energy into timely topics, to avoid reinventing the wheel by doing unnecessary projects, and to more efficiently understand the broader landscape of astronomy education. Creating a comprehensive critical review and synthesis of the literature as part of a larger education research project is hard enough without also have great difficulty in finding extant research reports, and iSTAR should make this task a bit easier.

We also hope that iSTAR will help stakeholders better argue for the next generation of constructed and designed facilities. Rather than “yet another cool planetarium,” perhaps more compelling cases can be made to prospective donors by designing the “next generation learning facility,” to which a lanetarium
serves as the keystone.

Finally, it is our desire that planetarium educators and show designers will have ready access to a foundation of research that guides better and more effective show design. For most of the last century, the pedagogical content knowledge of astronomy teaching has been largely based either on the long-earned expertise of planetarians or what tacitly seems like a good idea to well-intentioned educators, and then shared informally and inefficiently from one planetarium educator to the next. iSTAR is well positioned to be a clearing house for systematic research reports related to teaching and learning in the planetarium.

 

What is most needed now is for the planetarium community to help contribute resources to the iSTAR database. As of this writing, there are 989 documents in iSTAR, and this probably only represents a small fraction of what could be available to the community at large. The infrastructure is in place to easily create an account at istardatabase.org and upload one’s own research or to contribute research reports, meeting contributions, public evaluation reports, informal white-papers, articles, theses, and dissertations done by others. iSTAR depends entirely on the gracious contributions of the community, and we enthusiastically encourage your participation.


Acknowledgements


The iSTAR community database is built upon the collective contributions of the broader community, and we greatly appreciate their contributions. We would like to specifically acknowledge the tireless intellectual contributions of iSTAR designers Dr. Stephanie J. Slater, Dr. Paulo S. Bretones, and Dr. Michael T. Fitzgerald. Among many contributors, we greatly appreciate the thankless, heavy, day-to-day lifting by graduate students Coty B. Tatge from the U.S. and Saeed Salimpour from Australia.


We also greatly appreciate the financial support of the International CAPER Center for Astronomy & Physics Education Research, the University of Wyoming Endowment for Excellence in Higher Education in the U.S., and Edith Cowan University’s Institute for Education Research in Australia.

 

Detailed information requests about iSTAR should be addressed to the iSTARdatabase.org Coordinator Dr. Michael T. Fitzgerald from the Institute for Education Research at Australia’s Edith Cowan University at psyfitz@gmail.com.


References

  • Bailey, J. M., & Lombardi, D. (2015). Blazing the trail for astronomy education research. Journal of Astronomy & Earth Sciences Education, 2(2), 77-87.
  • National Research Council. (2012). Discipline-based education research: Understanding and improving learning in undergraduate science and engineering. Washington, DC: National Academies Press.
  • Plummer, J.D., Schmoll, S., Yu, K.C., & Ghent, C. (2015). A guide to conducting educationalresearch in the planetarium. Planetarian, 44(2), 8-24, 30.
  • Slater, S. J., Tatge, C. B., Bretones, P. S., Slater, T. F., Schleigh, S. P., McKinnon, D., & Heyer, I. (2016). iSTAR First light: Characterizing astronomy education research dissertations in the iSTAR database. Journal of Astronomy and Earth Sciences Education, 3(2), 125.
  • Slater, S. J., Slater, T. F., Heyer, I., & Bailey, J. M. (2015). Conducting astronomy education research: an astronomer’s guide, 2nd edition. Hilo, Hawai’i: Pono Publishing. ISBN: 1515025322.
  • Slater, T. F. (2008). The first big wave of astronomy education research dissertations and some directions for future research efforts. Astronomy Education Review, 7(1), 1-12.
  • Slater, T. F. (2015). “Is the best astronomy education research ‘grey’?” Retrieved from AstroLrner.org Blog at astronomyfacultylounge.wordpress.com/2015/09/24/is-the-best-astronomy-education-research-grey.
  • Slater, T. F. (2016). Undiscovered value of grey astronomy education research results. The Grey Journal, 12(3), 153-155.
  • Slater, T. F., & Tatge, C. B. (2017). Research on teaching astronomy in the planetarium. Netherlands: Springer. ISBN: 978-3-319-57200-0.

 

 

about ips | find a planetarium | officers

International Planetarium Society, Inc | c/o Ann Bragg, Treasurer | Marietta College, 215 Fifth Street | Marietta, Ohio 45750 USA