This article reports an investigation of preservice teachers’ interactions with a computer simulation designed to allow them to explore the nature and practices of science. Participants included 188 preservice, secondary-level, science and mathematics teachers who were enrolled in one of seven consecutive semesters in a professional development course as part of the teacher certification program at a large research university. Artifacts, including articles published in an online journal, responses to focus questions, reflections on the activity, as well as audio and video recordings of the activities themselves, were analyzed following a grounded approach. The simulation activities qualified in many respects as authentic science as identified by Chinn and Malhotra (2002). Further, what these activities revealed about student beliefs in regard to the nature and practice of science correlated with their reactions toward the simulation and their views of how it might be used in high school classes.
Science education standards provide a clear mandate for students to understand the nature of science (National Research Council, 1996). There is, however, no consensus on what the nature of science (NOS) is, nor are the standard surveys typically used to assess it necessarily reliable in measuring student understanding of this multifaceted construct (Abd-El-Khalick & Lederman, 2000). Such measures tend to identify declarative knowledge of the nature of science, but fail to assess students’ ability to generate knowledge using the accepted practices of science, i.e., their ability to construct and critique scientific claims through a socially embedded process (Ford, 2008). This work embraces a view of NOS as more than the nature of scientific knowledge; in this framework NOS includes the nature of scientific practice, i.e., what working scientists do. The more students engage critically with how scientific knowledge is generated and disseminated, the more scientifically literate they will become. Benchmarks for Science Literacy (AAAS, 1993) puts it this way:
When people know how scientists go about their work and reach scientific conclusions, and what the limitation of such conclusions are, they are more likely to react thoughtfully to scientific claims and less likely to reject them out of hand or accept them uncritically... The myths and stereotypes that young people have about science are not dispelled when science teaching focuses narrowly on the laws, concepts, and theories of science. Hence the study of science as a way of knowing needs to be made explicit in the curriculum. (p. 3)
Following Ford (2008), “knowing” in the statement above is interpreted here as a verb rather than a noun – coming to know, rather than the nature of the inert knowledge itself. Ford presents an existence proof that it is possible to develop this ‘grasp of practice’ in sixth-grade students through properly designed curriculum activities, inviting further exploration into how instruction interacts with students’ understanding of the nature and practice of science, and how technology might influence that interaction.
Coming to understand NOS is arguably especially important in the case of students who are preservice teachers, and whose views will undoubtedly affect the instruction they will implement in their own classrooms. Teachers have reported that they do not view understanding the nature of science to be as important as other learning objectives, a view that is often reflected in district standards and assessment, and that they do not have the time, resources, or knowledge for the extra effort needed to infuse NOS into their curriculum (Abd-El-Khalick, Bell & Lederman, 1998; Posnanski, 2010). As Edelson (1998) notes:
Traditional training for teachers has not prepared them for new roles in which they must engage students in uncertain science, help them to formulate and refine research questions, identify resources and tools that will allow them to expand their understanding, and foster authentic scientific debate. Providing scientific resources, tools, and techniques for use by students requires the modification of facilities designed for expert scientific practitioners to allow students to ask questions and pursue them in ways that are similar to those of scientists. (p. 320)
Thus teachers need a support system to scaffold the authentic process of science, one that they can experience as valid in their own right. Technology can be a resource in this regard and “can place a greater range of tools and resources at the disposal of teachers and students, and… increase the opportunities for the social interchange that is at the heart of authentic science practice” (Edelson, 1998, p. 317). The NOS simulation offered by Epistemological Engineering (EEPS, 2015) provides such a resource. See Big Time Science (2015) to use the beta version of the simulation. Developed with funding from the National Science Foundation, the simulation lets students take on the role of scientists ‘writ large’ in a limited amount of class time, while still developing some specific science content ideas.
The UTeach secondary STEM teacher preparation program has used the NOS simulation to prompt its students, preservice teachers earning degrees in mathematics, science or engineering, to examine their own thinking about the nature of science and how to convey it to their future students. In particular, the intention was to expose them to some aspects of the scientific enterprise they may not have experienced, including securing funding; peer review; and public dissemination, replication and refutation of results. The online simulation provides students with ‘worlds’ in which they conduct investigations in teams representing research organizations. Students must decide what they will research and how, possibly inventing new data collection and analysis procedures in the process; submit their results for peer review; and determine next steps based on their results and the actions of other scientists. The group can watch a scientific concept evolve in these simulated worlds from its first creative inception, through many challenges and possibly refutation, until it finally is either discarded as incorrect or simply irrelevant, or becomes part of the scientific canon of the class, all in a matter of hours. Throughout the simulation, as well as before and after, students reflect on the nature of scientific practice, both individually and as a group.
This paper reports on a multi-year study of UTeach students’ interaction with the NOS simulation activity, intended to shed light on preservice teachers’ thinking about the nature and process of science, as revealed in their interaction with, and reflection on, the simulation. The research questions that guided the investigation were (1) Does engagement with the NOS simulation in fact constitute authentic science inquiry? (2) What does the NOS simulation activity reveal about preservice teachers’ thinking with regard to the nature and practice of science? and (3) How do preservice teachers view the possible use of such an activity in their own classrooms?
The next section reviews an expanded framework of authentic science that will be used in answering the first question, as well as results of previous research on the effect of instruction in enhancing students’ and teachers’ understanding of NOS, and technology as an affordance in enhancing understanding of NOS.
Authentic Science
Chinn and Malhotra (2002) present a theoretical framework for epistemologically authentic science tasks in school settings. They go well beyond Ford’s generation and critique of scientific claims to identify the cognitive processes involved in doing authentic science, in contrast to school science. Chinn and Malhotra (2002) lists the cognitive processes employed by scientists in authentic science tasks. These include generating research questions, designing studies (including selecting variables) and determining their own procedures, making observations, explaining results, developing theories and studying research reports. Their evaluation framework also describes the epistemological dimensions of authentic science, including the purposes of research, theory-data coordination, theory-ladenness of methods, response to anomalous data, nature of reasoning, and social construction of knowledge. These authors examined 468 textbook inquiry tasks and 26 inquiry tasks developed by education researchers. They found that the textbook activities included, on average, less than one of the elements of authentic science reasoning they had identified. Although the researcher-developed tasks included many more elements, they still, on average, incorporated less than half, leaving much room for improvement.
NOS Curriculum
Although an understanding of NOS is common in state (Kumar & Berlin, 1998) and national standards (e.g., National Research Council, 1996), investigations of student understanding rarely find it to be adequate, regardless of either the assessment measure chosen or whether the students have engaged in inquiry activities (Abd-El-Khalick & Lederman, 2000). It might be the case that the inquiry activities in question constituted school science rather than authentic science, as noted by Chinn and Malhotra (2002). However, even scientific activities that involve students in collaboration with professional researchers do not necessarily increase students’ understanding of the nature of research or the bigger picture of the scientific enterprise.
Moss, Abrams and Kull (1998) studied the effects of project-based instruction using authentic data from ongoing research projects and intended to foster ‘student scientist partnerships’ (SPS). These authors found that even after engaging in such partnerships, through curriculum tied to ongoing research, students retained only a “rudimentary” understanding of the nature of science and its practices. In more recent work, Moss (2003) found that working as part of the JASON project did not “foster a perceptible change in [students’] conceptions about what science is or how it is practiced,” despite increasing student content knowledge in science (p. 28). Although students can learn about experimental design and the methodological development of a professional science program, as novices their active participation can sometimes be limited to carrying out procedures that have been designed by others so that the integrity of the experiment can be insured. In the worst case, students may not even be able to interpret or explain their own results.
On the other hand, self-directed, personalized inquiry projects, which mimic some aspects of authentic science such as selecting a question, designing an experiment, and communicating results, do not generally expose students to the full range of scientific pursuits. Students (other than those engaged in specialized independent research at advanced academic levels) are rarely required, for example, to study research reports from other scientists in deciding what to study. They rarely engage with epistemological dimensions of authentic science identified by Chinn and Malhotra (2002), such as theory-ladenness of methods, nature of reasoning, and social construction of knowledge, e.g., by undergoing peer review, having their work replicated or discredited by other scientists, or having to find funding for their work. As noted by Edelson (1998), pre-college teachers will need an understanding of the nature and practices of authentic science in order to make modified versions of scientific infrastructure accessible to their students.
Enhancing Teachers’ Understanding of NOS
In a review of efforts to improve teachers’ understanding of NOS, Abd-El-Khalick and Lederman (2000) found that teachers were generally found to hold positivist views of science, often believing that scientific knowledge is absolute, not socially constructed, and not subject to re-evaluation, and that these views are difficult to change. The authors found that professional development efforts that were explicit about NOS (N = 9) were more successful at bringing teachers’ views into alignment with those of the researchers than programs that implicitly modeled NOS through inquiry activities alone (N = 8), but that none of the studies reviewed were “successful in fostering among science teachers understandings of NOS that would enable them to effectively teach this valued aspect of science” (p. 693). They argue that explicit instruction on the nature of science, requiring reflection, is essential for effective teacher education on NOS.
However, although it may be necessary, explicit instruction in NOS may not be sufficient to ensure that teachers have a true grasp of scientific practice. Akerson, Abd-.El-Khalick and Lederman (2000) demonstrated that after engaging in ongoing discussions of the nature of science in a science methods class, substantially more preservice teachers held ‘adequate’ views of the nature of science at the end of the course than at the beginning. However, even some of those who seemingly internalized adequate views still retained naïve views alongside them. Further, the measured changes dealt primarily with the traditional epistemology (nature of knowledge) rather than the practices of science.
In more recent work, Hanegan and Bigler (2009) studied five female future secondary biology teachers as they resolved original research problems through protocols of their own design and implementation. In this case study, the authors were primarily interested in the preservice teachers’ acquisition of biology content knowledge; however, they noted an increase in the participants’ understanding of the nature of science. In particular, these preservice teachers gained understanding of the collaborative nature of scientific inquiry, the possibility (and possible utility) of error in science, and the importance of persistence, in addition to knowledge of experimentation techniques. Still, although all participants indicated that they planned to incorporate authentic inquiry in their own teaching, further study would be required to investigate the effect of this experience on their future classrooms.
Posnanski (2010) undertook such a study, investigating the effect of a two-year professional development program explicitly addressing NOS on inservice elementary (first through seventh grade) teachers’ understanding and classroom teaching. Participants were again primarily female and all Caucasian, a volunteer sample selected based on their commitment and support from their school administrators to complete the program. The teachers received graduate credit and a small stipend for supplies and equipment for their participation. The program consisted of a weeklong summer institute and evening sessions during the academic year. University scientists and science educators shared their current research and strategies for teaching NOS. Participants discussed and reflected on NOS explicitly, while engaging in inquiry activities and planning for classroom instruction. They developed action research plans and were observed after returning to their classrooms.
Posnanski and her graduate students found that composite scores on the VNOS-C (Lederman, Abd-El-Khalick, Bell, & Schwartz, 2002), as identified using a rubric, improved for the majority of the participants (18 of 22) from pre- to post-intervention. Overall, the teachers showed a statistically significant increase in scores. No patterns were identified in the open-ended VNOS-C responses in regard to NOS; however, analysis of interview data indicated that the explicit NOS activities and reflections had an impact on teacher understanding and action research plans. However, interview and survey data indicated that teachers identified real and perceived constraints to implementing their plans for explicit NOS activities in the classroom and questioned whether such activities would benefit their students. Although the teachers did incorporate some aspects of NOS when implementing their action research plans, these aspects were not necessarily present in their other classroom activities, and there was no indication of significant gains on the theory/law and observation/inference components of NOS. The authors conclude that activities explicitly involving reflection on NOS and embedded within appropriate science content investigations including contemporary research experiences may enhance the professional development of teachers. Further, professional development programs must address barriers to classroom implementation as perceived by teachers, in particular the relationship between NOS and content to be tested in standardized exams.
The NOS activity used in the UTeach program is designed to address these issues by demonstrating that teachers can provide students access to multiple content-based, student-directed research opportunities (e.g., in genetics, cosmology, physics and engineering) through the use of technology as a scaffold for authentic inquiry within the constraints of the standard curriculum. The next section reviews research on technology as a scaffold for authentic inquiry.
Technology as a Scaffold for Authentic Inquiry
Beyond its established simulated worlds, the NOS simulation infrastructure used in the UTeach program can be applied as a guiding structure for any inquiry activity within the standard curriculum. Eslinger, White, Frederiksen and Brobst (2008) reported on the effectiveness of an interactive environment, Inquiry Island, a similar general-purpose framework for supporting inquiry-based science learning. This software is designed around the Inquiry Cycle, a pre-set series of steps to guide students through the inquiry experience. In addition, Inquiry Island provides prompts for self-assessment and reflection. This prescriptive scaffolding can be individualized and gradually reduced. In the implementation of the software studied by these authors, students investigated genetics by making predictions and then viewing static representations of crosses between simulated plants. However, the simulation was not interactive and the outcomes were limited to static representations of predetermined crosses. In addition, at least initially, students were given specific research questions to investigate, and the inquiry process was highly scaffolded. The Inquiry Island genetics exercise improved students’ scores on measures of some inquiry skills, but did not attempt to challenge students’ understanding of the norms of professional science or their ability to conceive, devise, and implement investigations in a more open-ended environment.
Muwanga-Zake (2007) also evaluated the use of a computer simulation (game) related to biology, including genetics. His participants were 26 South African secondary biology teachers, many of whom had limited previous experience with computer technology. These teachers explored the use of Zadarh, an adventure game that explores biological phenomena. Although this setting and participants differed substantially from US classrooms and teachers, particularly in the present day, the need it highlights for considering the beliefs and needs of end-users (in this case, teachers) continues to be relevant. This study also pointed out the complex role of ‘fun’ in motivation and learning.
In work that is more in alignment with the overarching goals of the NOS simulation, the Co-Vis project (Edelson, 1998) has created tools for both scientific visualization (display techniques used by scientists in analyzing data) and scientific collaboration (the social generation of scientific knowledge). Edelson identifies three categories of key features of authentic inquiry: attitudes, tools and techniques, and social interactions, and argues that the “successful adaptation practice for learning will place tools and techniques of scientists into the hands of students in a context that reflects [these] characteristics of scientific practice” (p. 319), a difficult task given the constraints on average classrooms, particularly those in under-resourced communities. Co-Vis employs a technology infrastructure to meet these challenges. The visualization tools provide students with information in graphical format, using modified versions of scientists’ tools with more accessible interfaces. These are similar to the graphical interfaces of the NOS simulation, described below. The Co-Vis collaborative software allows students to communicate and collaborate both with each other and with other students and experts outside the classroom through the Collaboratory Notebook, as well as more traditional means such as email. The NOS simulation also incorporates this feature, allowing student teams to view (and possibly review) each other’s work through the online journal feature, moderated by the journal editor, a role that might be filled by the classroom instructor, or an expert collaborator outside the classroom.
Using evidence from Co-Vis and related projects, Edelson (1998) argues that it is indeed possible, using technology, to meet each of the three essential requirements (curriculum structures, teacher support, and scientific tools, techniques and resources), for successful adaptation of authentic science practice for classroom learning. This paper examines preservice teachers’ engagement with the NOS simulation activity to determine whether it likewise serves to promote authentic scientific practice, and how it reveals and interacts with these future teachers’ understanding of NOS.
Setting and Participants
Participants included 188 preservice, secondary-level, science and mathematics teachers enrolled during one of seven consecutive semesters in a professional development course as part of the UTeach STEM teacher certification program at the University of Texas at Austin. Approximately 60% of the participants were female, 40% were male, and 40-50% were ethnic minority, depending on the semester. The sample comprised eleven different sections of the course, taught by four different instructors. Approximately half of these future secondary teachers were being certified in science and half in mathematics, the majority at the high school level but some also in middle school. All students were earning (or had already earned) bachelor’s degrees in natural sciences or engineering (or in a small number of cases had accrued the equivalent coursework in a STEM discipline area in addition to a non-STEM bachelor’s degree.)
The artifacts used in this study were the result of regularly scheduled classroom activities, but students were informed of the formal study of the simulation activity after the fact, with IRB approval (so as to minimize the influence of the research on their actions), and given the option of not having artifacts that they had generated included in the study. No students availed themselves of this option.
The Simulation Activity
In two hours of class time, the students worked in teams of two (representing institutions) to explore two of the NOS simulation ‘worlds’: Extraterrestrial Critter Genetics and Four Color Universe.
| Figure 1. An illustration showing the layout of one team’s ‘institute’ (left), alluding to possible scientific activity such as reading journals and collecting data, as well as the results from a previous observation of the universe (center). Students learn the relative distribution of colors but not their positions. The highlighted squares on the grid at the right indicate the position of the next observation to be made |
|---|
|
In Four Color Universe, students explore a 12x12 ‘universe’ in which each cell is red, blue, orange, or green. They make measurements from which they learn the color distribution in either a 3x3 or 2x2 sub-array. One example is shown in Figure 1. Here the Inst. Technológico de Costa Rica is set to make a 2x2 observation near the upper left corner of the universe. Their previous 3x3 observation (under “Latest Result”) yielded the information that none of the cells in the region they sampled are red (see the red zero under the “R”), five cells are blue, two are green, and two are orange—but they do not learn the locations of the colors within the 3x3 block. Students can reason deductively using overlapping observations to try to determine the pattern in detail. Observations of different sizes cost different amounts, and groups have limited funding to make observations, in analogy to actual experiments.
Even if they do not learn the pattern’s details, however, students can infer about population and symmetry. A typical early student paper based on these observations might be, “There is no red in the universe.” Another group could refute that assertion as soon as it observes red light from somewhere in the universe.
Another typical paper would be, “The universe has up-down mirror symmetry.” Given groups' limited resources, it is impossible for any one group to prove that this assertion is true. But other groups can publish new data supporting the claim, promoting the authentic science behavior of studying and building upon the research of others. The simulation keeps track of how often groups read each other’s reports and when. Even working in coordination, the entire class cannot prove this assertion absolutely, and the entire universe is not viewable by simulation participants, providing excellent fodder for the debriefing at the end of the activity and for later student reflection.
In Extraterrestrial Critter Genetics, participants observe and experiment with hypothetical animals captured on the planet Eta Cassiopiae IV. They build cages, capture ‘critters,’ and maintain them over time. Students control environmental conditions in the cage and make measurements (weight, sex) and observations of the appearance of the animals in it. Occasionally, small animals appear within the cage, leading students to hypothesize that the animals have reproduced. In the process of observing the animals, students can discover laws of genetics very similar to those that govern heredity here on Earth, as well as mathematical relationships between growth rate, temperature and other variables.
Participants in the simulated worlds can submit papers for review and publication in online journals. The limited resources allocated to the students typically result in their prompting each other to publish what they have found, although there are also instances of refusing to divulge findings in the hopes of ‘scooping’ other scientists with a more definitive result. The role of the journal editor has been enacted at different times by the instructor, a teaching assistant, classroom visitors, and offsite collaborators.
Prior to the simulation activity, students are typically asked to submit a reflection on the nature of science. This has taken many forms, from ‘one minute papers’ to more extensive reflections on previous experiences with science classrooms and research.
To begin the activity itself, an instructor models the basics of the simulation, including publishing papers. But students are not prompted to investigate particular phenomena. Instructors answer students’ technical questions regarding the simulation, but record all other questions that arise on a white board, and discuss them with the entire group at the end of the activity. As homework, students typically submit a reflection on the activity itself and some students reflect on this activity as part of their professional portfolio.
Data Collection
All artifacts related to the nature of science or the NOS simulation that were produced by students during the seven semesters of the study were collected and catalogued. Specifically, these included (1) student reflections on the NOS simulation activity itself, (2) responses to pre- and post-activity prompts about the nature of science and previously experienced science instruction, (3) artifacts (journal articles, requests for funding, publication counts, etc.) created within the simulation environment itself, and (4) student comments as recorded on whiteboards during the simulation, in field notes, and, with consent, in archival video documenting the activity for course dissemination purposes.
Data Analysis
To address the first research question (Does engagement with the NOS simulation in fact constitute authentic science inquiry?), the artifacts were evaluated against the framework presented in Chinn and Malhotra (2002) for epistemologically authentic inquiry in schools. To investigate what the simulation activity reveals about preservice teachers’ thinking about the nature of science and how they view possible use of the simulation in their own classrooms, artifacts were also analyzed using a grounded approach (Corbin & Strauss, 1990). The grounded approach employs a qualitative methodology to “provide a thorough theoretical explanation of social phenomena under study… [which] should explain as well as describe” although not necessarily predict (p. 5).
This methodology does not follow the canons of quantitative research, but rather its own canons and procedures. In grounded research, data collection and analysis are necessarily interrelated and evolving processes. These processes require systematic collection and archiving of data from multiple sources. Concepts are the basic unit of analysis. Concepts do not merit inclusion in the developing theory on the basis of quantitative evaluation, but rather on the fact that they show up repeatedly under different circumstances, or because they show up under some circumstances but not others. Concepts are grouped into categories, which are then related to make a theory. Sampling proceeds on the basis of the theory being developed, looking specifically for instances that confirm or limit the theory under development. The analysis makes use of continued comparison between the existing theory and new observations. Exceptions to the developing pattern must be accounted for in the theory, and hypotheses must continually be verified against new data (Corbin & Strauss, 1990). In the work presented here, hypotheses evolved over the years of the study to include limiting conditions, for example variations in whether the preservice teachers had engaged in science research themselves, statewide testing expectations, etc.
Following the tenets of the grounded approach, artifacts related to student interaction with the NOS simulation were reviewed as they were collected, examining them for emerging themes. Emergent themes from classroom discussions were also recorded at the end of each implementation of the simulation activity. At three stages over the course of the study reports on emergent concepts and categories (‘theoretical memos’) were drafted and used to inform subsequent implementations of the activity. Prior to the final round of implementation and data gathering, two of the authors independently coded the existent artifacts using codes that had evolved over the course of the study, allowing them to confirm and revise the major categories that had arisen as needed and determine what questions were needed to further refine and clarify the emerging theory. After the results of the independent coding were compared and discrepancies discussed, a working version of the codes was created. The first author used this version of the codes to reexamine the entire data set after the final implementation of the simulation, while still allowing for additions and modifications as needed to satisfy the requirements of grounded theory. The second author performed a spot check on the final coding version.
Table 1 shows the set of codes derived from the initial implementation of the simulation; Table 2 shows the refined set of codes used in the final analysis. As can be seen by comparing the tables, the initial code of ‘fun’ as descriptor that arose regarding engagement in the simulation in the first round of coding was eventually joined by ‘not engaged’ as through repeated observations students were identified who were not enthusiastic about the simulation and the circumstances surrounding those reports (students with heavier course loads and teaching duties at the point in the semester when the simulation was implemented, increased standardized testing requirements). Likewise, some codes identified in the first round (e.g., ‘English as language of publication’) did not show up again in later rounds of coding. These codes were valid in the instantiation where they arose, but were not repeated in every implementation and therefore deleted from the ultimate descriptive theory. Ultimately codes were grouped into views of epistemology and practice of science, views on science teaching, and affective reactions. Following the procedures of grounded analysis these were ultimately related in a final theory in the selective coding process (Corbin & Strauss, 1990).
Table 1. Codes arising from the initial round of open coding
| Codes Arising From Initial Coding |
|---|
| Claim stated w/o evidence |
| Claim stated w/o limitations; recognition of limitations |
| Progression of claims |
| English as language of publication |
| Definitive or exact answers/real answer or lack of closure |
| “Given” info- assumptions w/o proof |
| Citing others |
| Speculation/proposed explanations |
| Funding |
| Realism of simulation |
| Enthusiasm (as learners or as teachers) |
| Fun |
| Value of pursuing their own research question |
| Specific measurement or analysis |
| Narrowing/refining of research question/limitations on what can be researched. |
| Collaboration |
| Testing /revision of hypotheses, ‘spontaneous science’, recognized or not |
| Refutation (challenging another’s work) |
| Knowledge as revisable (Can you publish something if it’s wrong.) |
Table 2. Final code set with categories after axial coding
| Final Set of Codes | Categories |
|---|---|
| Fun (as student, as teacher) | Affective |
| Enthusiasm (for students, teachers) | Affective |
| Not engaged | Affective |
| Unique/constantly changing solutions | Epistemology |
| Repeatability/refutability | Epistemology |
| Logic/mathematical proof | Epistemology |
| Experts/learned in school | Epistemology/Teaching practice |
| Experiments, testing against reality | Epistemology/Teaching practice |
| Limitations/testing/revision | Practice of science |
| Norms of science | Practice of science |
| Speculation, proposed explanation | Practice of science |
| Realism | Practice of science |
| Collaboration/competition | Practice of science |
| Refutation | Practice of science |
| Development of procedures | Practice of science |
| General convention/consensus | Practice of science |
| Research literature | Practice of science |
| Citation of others | Practice of science |
| Facilitation of simulation | Teaching practice |
| Content learning | Teaching practice |
| Ownership of research questions | Teaching practice |
| Find out for your self | Teaching practice |
| Technology | Teaching practice |
Does Engagement With the NOS Simulation Constitute Authentic Science Inquiry?
In this paper, results pertaining to this first research question will be summarized in order to focus more deeply on the second.
To what extent does the simulation promote the cognitive processes of authentic inquiry as identified by Chinn and Malhotra (2002) in their Table 1? In brief, although all of authentic science cannot be simulated in one or two class periods, a considerable proportion of the Chinn and Malhotra’s authentic-science processes were evident in the work of the participants.
Here are examples, however, of processes that were not extensively observed:
All of these make sense, given the nature of the simulation.
But there are numerous examples of students exhibiting authentic processes in the activities. Participants recognized their assumptions; found flaws in reasoning (especially others’); made multiple representations; invented experimental procedures; examined research reports; and developed theories about unobservable mechanisms.
And they created their own questions. Many students noted this in their reflections, some in fact finding it initially disconcerting. As one wrote,
It allowed us to think, create, make important decisions, and experiment on our own terms. … I experienced a slight confusion at the beginning of the lesson when we were “let loose” to experiment and work on our own. Even in the second simulation with the creatures, we were not told what to observe or look at, we were simply told to experiment and “publish papers.” It was fun and educational at the same time, which is something I think students really need (especially in a science class).
Participants reasoned indirectly. The limitations of the simulation seemed to force students to employ chains of argumentation as they strove to understand, as in this example:
Our first cage set had offspring so rapidly - four in the first week - we wondered if the first born Jenny could have been the mother of the last born Bobby. We put Jenny and adult Bob, who had reproduced before, in a cage together for a week. They did not reproduce as Bob had with Cindy, and Jenny is still getting bigger and therefore [more mature]. Jenny and Bob had the same cage conditions as Cindy and Bob. We postulate that Jenny was not yet mature enough in age to reproduce… So, aging to mature adult takes longer than a week.
Likewise, students were quick to realize that papers making broader claims (generalizations) seemed to be regarded as more valuable than claims made just about the data in hand, but also were more easily subject to refutation. Students had to determine whether they would risk eventual refutation in order to scoop the other teams.
The simulation activity was also evaluated against dimensions of the epistemology of authentic inquiry as identified by Chinn and Malhotra (their Table 2). As before, there are dimensions the simulation did not stimulate, e.g.:
However, participants did seek global consistency, coordinating theory and data: if they discovered symmetry in one area of the universe simulation, for example, they searched for it elsewhere. They used heuristic, nonalgorithmic reasoning, multiple acceptable forms of argumentation, and uncertain reasoning. In fact, the participants made claims on a variety of bases. They had to determine whether a result was certain enough to warrant publication (and risk possible refutation). Further, the simulation clearly exposed these students to the uncertainty of science (“Can you publish something that’s wrong?”), which some of them had apparently never considered.
Finally, the area where this simulation activity most exemplified authentic inquiry as characterized by Chinn and Malhotra is the social construction of knowledge. These student scientists worked in groups and routinely built on each other’s work as they realized that was a way to publish papers, both through extension and refutation. The simulation offered a clear example of establishing institutional norms through an expert review process that mimicked the peer review process in a faithful, albeit accelerated, way.
In summary, the simulation did facilitate many, but not all, of the features of both the cognitive processes and epistemology of authentic inquiry as identified by Chinn and Malhotra, at least for the class collectively. As noted by Bereiter (1994) in describing scientific inquiry as discourse, individual inquiry is likely to lead to enhanced understanding only insofar as it is brought into the larger discourse in and beyond the classroom, coordinating with previous results. Thus, any one student does not enact authentic science alone; rather it is as part of the larger community that students enact authentic science.
What Does the NOS Simulation Activity Reveal About Preservice Teachers’ Thinking With Regard to the Nature and Practice of Science?
Students were asked to respond to a prompt, prior to the simulation, about what it means to do science (including mathematics). Some respondents simply indicated that scientists and mathematicians must proceed logically, but other responses specifically indicated that scientists or mathematicians seek solutions that made sense to them personally, e.g., “follow the procedures that make sense in the context. Check to make sure the answer you got makes sense/works.”
Another common response highlighted the complementary notion that scientists test their ideas against reality, i.e., that science is data driven. Typical responses were “the ideas are tested, supported by our best understanding of reality,” “You have to have evidence (experiments, observations, data) to support your ideas,” and “Experimental verification— whatever most accurately reflects observation is the most correct. Nature of Science- the process by which science makes inquiry into nature.”
Other responses coded for the collaborative nature of science. For example, “We decide what is right by the conventions of the math community and valid proofs accepted by the math community,” “[Scientists arrive at answers] [b]y the general consensus of scientists over time. This could be obtained by referencing published literature. This also includes staying up to date because science ideas are constantly changing,” “Mathematics provides the tools and processes necessary to execute the science; to answer the questions, using a universal language transcending cultural regions, allowing people all over the world to show each other their answers to these questions,” and “You work individually and as a group at the same time. You do experiments, collect data, draw conclusion, make prediction individually, and at the same time all the members work together as a group. It is like a system. Each members work affects the whole system, and the system success depends on each member achievement. [sic]”
In contrast, students’ responses that focused on science as proceeding from authority, and transmission of a canon of knowledge, were coded under the category of ‘experts.’ For example, one student responded, “I have no idea [how science is done]. What we learn is just a universal agreement based on observations experts have made.” “[To determine an answer in science] I would look up the ‘correct answer’ to the topic either in the student textbook or in one of my textbooks.”
The simulation and reflections generally revealed a lack of familiarity with details of the practice of science, e.g., the peer review system, particularly on the part of students who had not engaged in it themselves. Issues that frequently arose in discussions and questions included funding and its relationship to what research is undertaken; whether authors pay or are paid for publications; whether it is possible to publish something that has already been published (and if not, how then is the repeatability of results established); and whether authors should be obliged to share their results.
The activity and reflections afterward also revealed students’ understanding of the epistemology of science. Despite the current emphasis on the tentative nature of science in today’s standards, these students were shocked to find that published research could ultimately be falsified: “It was an experience seeing the articles published, then people disproving what had been thought to be correct by the publishers.” “Can you publish something that is wrong?” is a question that arose in every implementation in some form or another. This led to frequent discussions of the limitations of data and the inherent assumptions in interpreting it.
How Do Preservice Teachers View the Possible Use of the Simulation in Classrooms?
For many students, the simulation represented an opportunity to engage their future students; ‘Fun’ was the first code to arise in the analyses and remained a major theme throughout.
I love that the exercise was modeled after the process of writing & publishing research papers because it is so “real-world” and fun at the same time. I think that what is often emphasized in science education is tedious and thus a turn off to students. The excitement of figuring things out is often lost at the expense of doing the experiment correctly and memorizing the terms.
I really enjoyed the program. I would have like to been able [sic] to play around with it more if I were a student. Again, the genetics one really got me going. I just wanted to run so many tests and make connnections. The program is something that I would love to use when I have a class of my own.
However, cases of students who were not engaged by the activity (“I was not engaged through the whole thing and [in] the end we kind of gave up”) were also encountered. These fell into two categories. In the first case, some of the students were concerned that the activity did not provide sufficient guidance; even as college students they felt overwhelmed at some points with the lack of structure (“When I first began the activity, I was very lost and confused”) and felt their future students would need more direction. “When we weren’t told what to find, I felt that this might leave students to be confused or decide to not do the activity because they are lost and not interested.”
The second cause for disengagement was seeing no relevance for the activity or feeling that there were more important or pressing things they should have been doing. These future teachers did not see the need for investigating the nature of science or why they should spend time on it, especially at a busy point in the semester.
In contrast, others considered it important to expose students to the working reality of science, and saw the simulation as a unique way to do that. According to one future teacher with graduate research experience, “This is exciting. This is the first thing that I have ever seen that mimics the real kind of, [sic] what you do in grad school and what professors are doing all the time that nobody has any idea about until they get to grad school.”
For some of these students, however, the ultimate goal of instruction is the factual content to be delivered, and for these students the simulation did not seem a worthwhile expenditure of class time. No matter how valid the process, if their future students would not come up with ‘real science answers,’ the exercise was not beneficial enough to these future teachers. This stance is exemplified by this student reflection:
The process that we went through to gain knowledge about the 4-color universe was not highly related to any other 'real' science experiment. Therefore there was no content knowledge to attach the process to.
In other words, if there is no ‘real’ content at the end of the process it is not justified as part of the curriculum.
The importance of teacher facilitation was another theme that arose as students reflected on how they might someday teach. In the words of one, “The teacher must be well-involved and keep up with each group in order for both activities to flow well.” The students also argued that the amount of time dedicated to the simulation and how it was used as part of the larger curriculum at the high school level would determine its usefulness: “I think that it would be interesting to use this activity throughout the school year as an organizing tool to explore lots of science & math topics. I’m not sure how many topics could be studied using this interface” and “In thinking about implementing Wednesday's activity in a classroom, time is a major factor. Like I said, it was almost frustrating to have to walk away from the activity so incompleted. These should be ongoing projects as their aim is for student to feel invested.”
Finally, students frequently made note of the issue of collaboration versus competition and how it affects the practice of science, as well as the importance of highlighting this issue for students in their future classes. As one student eloquently put it in a reflection:
I think that besides the obvious ways that [the simulation] exemplifies different ways that scientific investigations and experiments can be carried out, the extra special feature is that it shows students how the scientific community works together. Now that is one aspect of the scientific inquiry process that was not stressed so much to me in any of my previous classes. I think the coolest thing is that through this activity, students can learn that science is much more productive and efficient when scientists communicate and collaborate with each other.… Students need to be aware of these kinds of issues because it will improve the future efficiency of science.
Table 3 (after Chinn and Malhotra [2002], Table 1) lists cognitive processes associated with authentic inquiry as practiced by scientists. Likewise, Table 4 (after Chinn and Malhotra [2002], Table 2) lists the six dimensions of the epistemology of inquiry along with characteristics of that dimension associated with authentic scientific practice. The elements highlighted in bold in each table were consistently demonstrated in the simulation as enacted in this study. The elements in italics were occasionally or partially demonstrated and those in plain text were rarely or never evident.
Table 3. Cognitive processes involved in inquiry with corresponding actions associated with authentic science. Bold text indicates characteristics consistently demonstrated in the simulation as enacted in this study. Italics text indicates characteristics occasionally or partially demonstrated, and plain text indicates characteristics inconsistently or never demonstrated. After Chinn and Malhotra (2002), Table 1
| Cognitive Process | Scientists Engaged in Authentic Inquiry: |
|---|---|
| Generating research questions | • Generate their own research questions. |
| Designing studies | • Select and invent variables. • Invent complex procedures. • Devise analog models. • Employ multiple controls, which can be difficult to determine. • Incorporate multiple measures of variable. |
| Making observations | • Devise elaborate techniques to guard against observer bias. |
| Explaining results | • Repeatedly transform observations into other data formats. • Question whether their own and others' results are correct or artifacts of experimental flaws. • Relate observations to research questions by complex chains of inference. • Observed variables not identical to theoretical variables of interest. • Judge when to generalize. • Employ multiple forms of argument. |
| Developing theories | • Postulate unobservable mechanisms. • Coordinate results from multiple, possibly conflicting, studies that may involve unobservable mechanisms as well as observable variables. • Use strategies for resolving differences between studies. |
| Studying research reports | • Study other scientists' research reports for multiple purposes. |
Table 4. Dimensions of the epistemology of science with corresponding characteristics of authentic science inquiry. Bold text indicates characteristics consistently demonstrated in the simulation as enacted in this study. Italics text indicates characteristics occasionally or partially demonstrated, and plain text indicates characteristics inconsistently or never demonstrated. After Chinn and Malhotra (2002), Table 2
| Dimension of Epistemology | Characteristics Associated With Authentic Scientific Inquiry: |
|---|---|
| Purpose of research | • Building and revising theoretical models with unobservable mechanisms |
| Theory-data coordination | • Coordination of theoretical models with multiple sets of complex, partially conflicting data • Seeking global consistency |
| Theory-ladenness of methods | • Awareness of theory-method entanglement |
| Responses to anomalous data | • Rational and regular discounting of anomalous data |
| Nature of reasoning | • Heuristic, nonalgorithmic reasoning • Multiple acceptable forms of argument • Uncertain reasoning |
| Social construction of knowledge | • Construction of knowledge in collaborative groups • Building on previous research by many scientists • Establishment of institutional norms through expert review processes |
None of these features of authentic inquiry appeared in the work of every team in any instantiation of the simulation. However, almost every cognitive process listed in Table 3 was manifested in each implementation by at least one team. Given the process of the online publication system and the directed class discussions, the authors argue that overall the entire class engaged in a collective process that successfully addressed aspects of authentic scientific inquiry otherwise missing from school science. Likewise, the simulation activity exhibited the majority of the epistemological characteristics of authentic inquiry as shown by the elements in bold or italics in Table 4.
The areas where the activity fell short of authentic inquiry had to do primarily with inventing variables and measures of variables, ways to guard against bias in observations, and conflicting or anomalous data. These were constrained by the simulation itself. The ‘fixed and determined’ data presented by the simulation inhibited these aspects of authentic inquiry while enabling others, such as generating viable research questions, coordinating multiple research studies, and studying the research of others. The limited but quickly obtainable data allowed for multiple observations, and let students design investigations to respond to published results. A physical, ’wet-lab‘ experiment might force students to address some of the shortcomings identified in tables 1 and 2 (e.g., students might have to invent their own variables and figure out how to measure them), but it would be difficult to capture other aspects, particularly those associated with long-term interactions with other scientists, in the limited time available in K–12 classrooms.
| Figure 2. Model of student thinking about the nature of science that emerged from this study. The vertices indicate different aspects of student views of the nature of science |
|---|
 |
The model of student thinking about the nature of science that emerged from this study is shown in Figure 2. Along one dimension, student views ranged from individual (belief that an individual must discover information for her/himself) to collective (scientific ideas arrived at by the consensus of a community in which all might participate). Examples of responses coded as individual are “The interaction between an individual and the questioning hypothesis is a very reliable and influential experience in science. The personal discovery is much more than the mechanical reading that majority of peers use as their scientific background,” and “Doing science is actually researching something yourself and not something learned in a book.” An example of a response coded as collective is “Virtually nothing we teach in math or science are right answers, but rather a notion of observations that when experimented will produce the same or similar results. Thus what we teach in class are thousands & millions of observations & experiments proven to decide [what] is the ‘right’ answer.”
In contrast to both of these is the notion of scientific knowledge transmitted from authority, created only by ‘experts’ and transmitted through a canon of knowledge that it is critical for students to know and understand. Responses typical of this category include “What we learn is just a universal agreement based on observations experts have made” and “The ‘right answers’ in science and math depend on what we as teachers have learned in college/school and textbooks/research.”
| Figure 3: Model of student views of what should be taught in science classes. The different components (science content, process of inquiry, norms of professional science) are not necessarily mutually exclusive, and most teachers would fall somewhere between the vertices |
|---|
 |
A parallel model emerged when considering what students thought should be taught, and consequently how the NOS simulation should or should not be used, in science classes (see Figure 3). Here, the process of inquiry is positioned at one end of an axis with science content at the other. For some teachers, students must understand the processes of science, how to discover answers for themselves, either individually or collectively. Students who fell in this camp saw great potential in the NOS simulation. In their words, “[S]tudents learn more in a trial-error type environment and where the students have to figure out what works and what doesn’t work” and “I think the activity on Wednesday was very successful. It used pre-designed games to get students interested by having them actively investigating the natural world. In astronomy and geology classes, students usually learn about the make-ups of the universe, but because they’re spoon-fed with this information, they rarely go thinking about how scientists came up with these ideas.”
For the preservice teachers who fell near the ‘content’ vertex, it was more important to ensure their students develop a firm grasp of science facts and concepts, particularly as identified in standards. This might well indicate awareness of the realities of school science where students will be held accountable for knowing the right answer in the current high-stakes testing environment (“a con [to the activity] is that you’re constrained by time and the standardized tests are better at testing knowledge than process anyway.”) Even some of these future teachers who valued having their students experience the process of science had a foot firmly at this vertex as well, arguing that that there had to be a connection to real science facts in order for the activity to be valuable in the classroom: “I do see some benefit to occasionally invest in the ‘learning curve’ associated with specific technologies (such as…the Extraterrestrial Critters game) …however, these tools are useless if there is no connection to real applications of the information” and “One of the downsides of the activity is that while it gave the students an entertaining and possibly enriching activity, it didn’t promote any building of content knowledge that may be on a standardized assessment.” As noted by Edelson (1998), “In adapting science practice to the classroom, it is seductively easy to focus on scientific knowledge, tools, and techniques at the expense of other elements of scientific practice. However, scientists’ attitudes and their social interactions are also defining features of scientific practice. For students, understanding these attitudes and interactions is essential in order to understand the scientific process and to interpret the products of science” (p. 318).
A third view that emerged (“norms of science”) is that a scientifically literate public needs to understand scientific practice in order to “react thoughtfully to scientific claims and [be] less likely to reject them out of hand or accept them uncritically…” (AAAS, 1993, p. 3). As noted earlier, the question always arose whether a flawed result could be published (“If you [an editor] accept a paper does that mean that it is right? Or does that just mean that it’s a good paper and it could be right?”). Students were also surprised at how funding and other resources interact with the process of discovery: “It was very interesting to experience what it would be like to be a scientist. It was extremely realistic and even dealt with aspects of being a scientist that a student might not think about (Like my group for example ran out of money [before being able to complete an investigation]).”
Ethical considerations with regard to the practice of science also came into play. For example, in the extraterrestrial genetics simulation, breeding individuals had to be isolated in order to determine the offspring’s parentage. Students expressed concerns about the ethics of determining mating partners solely in order to answer the scientist’s questions. Although ethics is a component of high school science standards in the state where the study took place, students had rarely had to make actual ethical decisions in their own experimentation.
Likewise, the constraints of funding, the tentative nature of results (“It was an experience seeing the articles published, then people disproving what had been thought to be correct by the publishers”), and the effects of competition and collaboration were elements of the practice of science students had not generally experienced, and were identified as important for their future students to experience.
Moss (2003) speaks to the importance of not giving precedence to either content or process, but rather blending the two, saying that science content “should be the context in which the nature of science is explored and not the aim of all learning” (p. 26). To satisfy both goals (and possibly both camps of preservice teachers), the instructor facilitating the simulation should make a direct link to ‘real’ science. For example, the genetics of the extraterrestrial ‘critters’ included both dominant and recessive traits. The simulation can help students develop these concepts for themselves—and make connections to real-life biology. As one future teacher noted, “I also liked the critter simulation because I was able to determine genotypes by mating different characteristics together and seeing the outcome. This would be great to incorporate into a genetics lesson without having to resort to the traditional way of using the capital and lower case letters [a reference to Punnet squares].”
Without such connections, experiencing the processes and norms of professional science did not seem worthwhile to some of these students. Even those who valued the process goals indicated this in their reflections, e.g., “It gave the students time to explore and investigate the world they had in front of them. They came up with their own ideas and techniques and tried to understand the world they had in front of them (us). I think that it needed to be more emphasized though on its relation to our world. If students do not pick up on that, then they will miss the main idea of the activity.” An obvious solution is to simulate actual terrestrial genetics, which could promote content learning while affording opportunities for discovery that would be precluded by the time and resource constraints of most classrooms. It is important, to remember that in designing professional development “discourses and paradigms that are not considerate of value systems erode the participants’ voices and so could yield contestable outcomes that are not in concert” (Muwanga-Zake, 2007, p. 476).
Prior research, however, speaks to the difficulty of finding “any widely available genetics curriculum that was based on a satisfactory model of inquiry” (Eslinger et al., 2008, p. 613). The technology-supported activity described by those authors comprises one example, dealing directly with the phenomenon of plant genetics and using the Inquiry Island software, that does meet this criterion. Results indicate that the NOS simulation used in this study constitutes another. The NOS-simulated worlds do not have as much embedded scaffolding, and instead present a very large (albeit bounded) range of possibilities for student investigations. The students valued this interactivity and open-endedness, despite concerns that it might overwhelm their students if not used long enough in the classroom: “I wonder if, in a high school classroom, if [sic] there would be students who would be overwhelmed by the exercise because it is so self-guided. I would definitely not advocate redesigning the activity to make it more obvious, that would ruin it. I would hope that students would be able to experience it enough times to become comfortable.”
To ensure that the simulation would be beneficial, students identified the teacher’s role, rather than scaffolds embedded in the software itself, as critical in supporting learning about inquiry. This was also the case in the Inquiry Island investigation (Eslinger et al., 2008). Other research has shown that the teacher plays a critical role not only in enabling the inquiry process, but also in supporting reflection on both it and the nature of science generally. Akerson et al. (2000) found that the use of explicit prompts to engage students in discussion about the nature of science was critical in changing student thinking on that issue. In this case, focused discussions of the activities and written reflections after the fact, as opposed to the simulation itself, ensured that students analyzed their own beliefs about the nature of science and the best ways to teach it. Even though some new observations about the nature of science occurred spontaneously to individual students or groups, the group discussions and posted reflections made these ideas accessible to the entire class.
Finally, the instructor role was also critical in ensuring that the students engaged in authentic scientific inquiry, for example by requiring that they generate their own questions and procedures, publish accounts of their work, and be cognizant of relevant published research. The role of the editor (often played by an instructor) is also critical in challenging students to develop models to explain their observations, articulate the limitations on their contentions, and justify their results.
Does engagement with the NOS simulation in fact constitutes authentic science inquiry according to Chinn and Malhotra (2002)? These results indicate that it satisfies the majority of requirements of the authors’ frameworks, but not all. Further, some elements of the inquiry framework were satisfied in the aggregate, i.e., by the class as a whole, but not necessarily by every student. And although the affordances of the NOS simulation facilitated authentic science inquiry, they by no means guaranteed it. The design of the activity, and the instructor’s role in it, were key in that regard. Had the instructors directed students to research particular questions, instructed them in procedures to follow, not prompted them to read the online journal, etc., the simulation could have constituted merely school science or simply a game.
Perhaps not surprisingly, students’ views on the possible use of the simulation in their future classrooms seemed to be related to their views on the nature of science itself. Students who viewed science facts and concepts as already clearly accepted in the canon of science, and students who valued personal discovery as a means to reach an understanding of facts and procedures, saw the simulation activity as falling short of meeting classroom requirements. In contrast, those who viewed science primarily as a way of knowing saw great potential in the simulation activity for engaging their students. The students themselves suggested that improving the connection of the simulation to ‘real world’ science might satisfy both camps.
They also highlighted the importance of the classroom teacher as the designer of how the simulation would be incorporated into the curriculum, as well as the facilitator of student engagement with the simulation. Finally, these results indicate that preservice teachers would likely benefit from opportunities to experience the full range of aspects of the scientific endeavor, as well as the norms of professional science, as they may not already be familiar with these.
It is important to note that the study participants were all certification candidates from a single teacher training program and thus these results may not apply to other preservice teachers. The size of the participant sample, the time extent of the study, as well as the variety of instructors who implemented the activity all served to increase the likelihood that the findings are indeed representative. As noted above, constraints on data acquisition opportunities limited the ability to take process in its progressive sense into account, that is, to “break the phenomenon [in this case, student interaction with the simulation] down into stages, phases or steps;” rather the methodology was limited to developing a theory around “purposeful action/interaction that is not necessarily progressive but changes in response to prevailing conditions” (Corbin & Strauss, 1990, p. 10). Details of how the simulation activity might or might not have effected changes in student understanding of the nature of science, or how much exposure would be required to bring about a change were not the target of this investigation; rather, the intent was to identify the conditions under which certain reactions to the simulation were prevalent. Thus, these results speak to the possibilities of the simulation activity rather than its effectiveness at eliciting a particular mindset with regard to the nature of science in these students.
This research was previously published in the International Journal of Gaming and Computer-Mediated Simulations (IJGCMS), 7(2); edited by Brock Dubbels, pages 24-45, copyright year 2015 by IGI Publishing (an imprint of IGI Global).
Abd-El-Khalick, F., Bell, R. L., & Lederman, N. (1998). The nature of science and instructional practice: Making the unnatural natural. Science Education , 82(4), 417–436.
Abd-El-Khalick, F., & Lederman, N. (2000). Improving science teachers’ conceptions of nature of science. International Journal of Science Education , 22(7), 665–701. doi:10.1080/09500690050044044
Akerson, V., Abd-El-Khalick, F., & Lederman, N. (2000). Influence of a reflective explicit activity-based approach on elementary teachers’ conceptions of nature of science. Journal of Research in Science Teaching , 37(4), 295–317.
American Association for the Advancement of Science (AAAS). (1993). Benchmarks for science literacy . New York: Oxford University Press.
Bereiter, C. (1994). Implication of postmodernism for science, or science as progressive discourse. Educational Psychologist , 29(1), 3–12. doi:10.1207/s15326985ep2901_1
Big Time Science. (2015). Science worlds public beta. Retrieved March 1, 2015, from http://www.eeps.com/bigtimescience/
Chinn, C. A., & Malhotra, B. A. (2002). Epistemologically authentic inquiry in schools: A theoretical framework for evaluating inquiry tasks. Science Education , 86(2), 175–218. doi:10.1002/sce.10001
Corbin, J., & Strauss, A. (1990). Grounded theory research: Procedures, canons and evaluative criteria. Qualitative Sociology , 13(1), 3–21. doi:10.1007/BF00988593
Edelson, D. (1998). Realising authentic science learning through the adaptation of scientific practice . In Fraser, B. J., & Tobin, K. G. (Eds.), International handbook of science education (pp. 317–331). Dordrecht, NL: Kluwer. doi:10.1007/978-94-011-4940-2_19
EEPS. (2015). Simulating the nature of science. Retrieved March 1, 2015, from http://www.eeps.com/projects/NoSProject.html
Eslinger, E., White, B., Frederiksen, J., & Brobst, J. (2008). Supporting inquiry processes with an interactive learning environment: Inquiry Island. Journal of Science Education and Technology , 17(6), 610–617. doi:10.1007/s10956-008-9130-6
Ford, M. (2008). ‘Grasp of practice’ as a reasoning resource for inquiry and nature of science understanding. Science and Education , 17(2-3), 147–177. doi:10.1007/s11191-006-9045-7
Hanegan, N. L., & Bigler, A. (2009). Infusing authentic inquiry into biotechnology. Journal of Science Education and Technology , 18(5), 393–401. doi:10.1007/s10956-009-9155-5
Kumar, D., & Berlin, B. (1998). A study of STS themes in state science curriculum frameworks in the United States. Journal of Science Education and Technology , 7(2), 191–197. doi:10.1023/A:1022520725386
Lederman, N., Abd-El-Khalick, F., Bell, R., & Schwartz, R. (2002). Views of nature of science questionnaire: Toward valid and meaningful assessment of learners’ conceptions of nature of science. Journal of Research in Science Teaching , 39(6), 497–521. doi:10.1002/tea.10034
Moss, D. M. (2003). A window on science: Exploring the JASON project and student conceptions of science. Journal of Science Education and Technology , 12(1), 21–30. doi:10.1023/A:1022151410028
Moss, D. M., Abrams, E. D., & Kull, J. A. (1998). Can we be scientists too? Secondary students' perceptions of scientific research from a project-based classroom. Journal of Science Education and Technology , 7(2), 149–161. doi:10.1023/A:1022564507639
Muwanga-Zake, J. W. F. (2007). Evaluation of an educational computer programme as a change agent in science classrooms. Journal of Science Education and Technology , 16(6), 473–490. doi:10.1007/s10956-007-9078-y
National Research Council. (1996). National science education standards . Washington, D.C.: National Academy Press.
Posnanski, T. J. (2010). Developing understanding of the nature of science within a professional development program for inservice elementary teachers: Project nature of elementary science teaching. Journal of Science Teacher Education , 21(5), 589–621. doi:10.1007/s10972-009-9145-8