Project Activities
To begin, the research team will integrate the basic VTG instructional module into the SimScientists platform. Next, researchers will develop three versions of the Voyage to Galapagos learning environment to run in the SimScientists learning management system, each of which uses a different mode of assistance. Finally, the team will complete a randomized, comparative pilot study with middle and high school biology students that will investigate what mode of assistance maximizes learning outcomes for students of differing beginning abilities in science.
Structured Abstract
Setting
This study will take place in three middle or high schools in North Carolina, Wisconsin, and California.
Sample
Participants include 100-150 biology students at each school, for a total of approximately 300-450 students per year. The students represent a diverse population geographically, ethnically, and socioeconomically.
Researchers will create three versions of the Voyage to Galapagos learning environment to run in the SimScientists learning management. Version 1, No Intervention Assistance, will replicate the assistance approach already embedded in the existing VTG design. Feedback is provided only after students complete a given phase of a given level. Version 2, Contingent Tutoring Assistance, will include the same interface and instructional designs that comprise Version 1, but will replace the passive feedback system with a pro-active tutor that intervenes when a student is taking actions that indicate particular misconceptions related to the content or the inquiry skills. Version 3, Model-Driven Assistance, also is also a proactive form of tutoring, and is also known as an Intelligent Tutoring System (ITS). It will note student learning patterns as they proceed through the VTG interface and infer from those patterns whether a student shows a high, medium, or low level of ability of both conducting inquiry and of understanding the key concepts.
Research design and methods
During the iterative development of the two assistance systems, researchers will conduct three design studies that will allow them to test initial designs and make subsequent design revisions based on results. Feasibility Study 1 will test preliminary designs for the model-driven tutor to determine the extent to which the initial assumptions made about predicted student behavior in the context of VTG match what students actually do when they work with the VTG tasks and interface. Feasibility Study 2 will test how well VTG's contingency tutoring model and its Bayesian tutoring model work in terms of their accuracy at assessing student proficiencies in the key assessment target areas. This study will ask if both systems function as expected or if they need additional refinement. A Pilot Study will be conducted once the project has integrated VTG with the SimScientists learning management system and has completed the first two feasibility studies and refined the initial tutoring designs. The Pilot Study will then implement a quasi-experimental design.
Key measures
Researchers will use the Conceptual Inventory of Natural Selection (CINS) as an independent (distal) measure of student knowledge and inquiry skills that will be the target of the VTG modules. The CINS is a 20-item multiple choice test that employs common alternative conceptions as distractors. In addition, researchers will develop a proximal measure of content deliberately aligned to the specific content of VTG. For that instrument, researchers will use a selection of items from WestEd's proprietary bank of items from the Partnership for the Assessment of Standards-based Science (PASS). The instrument will be pilot tested in the first year of the project.
Data analytic strategy
Mean student learning gains across the three treatments will be compared using pre- and post-testing. Comparison of results will include analysis of covariance using demographic covariates and analysis of variance level blocking factors to examine which of the three modes of assistance is most beneficial for students of differing initial abilities in science.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
Products: Products include the three versions of the VTG system integrated into the SimScientist learning platform: one that replicates the assistance approach already embedded in the existing VTG design; the second that responds to students contingent on their responses (termed the Contingency Tutoring Version); and the third that interacts with the student based on the system's "knowledge representation" of that student (termed the Model-driven Version.)
Scholarly reports of the findings will also be produced.
Book chapter
Quellmalz, E.S., Silberglitt, M.D., Buckley, B.C., Loveland, M.T., and Brenner, D. (2016). Simulations for Assessing and Supporting Science Literacy. In Y. Rosen, S. Ferrara, and M. Mosharraf (Eds.), Handbook of Research on Computational Tools for Real-World Skill Development (pp. 191-229). Hershey, PA: IGI Global.
Proceeding
McLaren, B.M., Timms, M., Weihnacht, D., and Brenner, D. (2012). Exploring the Assistance Dilemma in an Inquiry Learning Environment for Evolution Theory. In Proceedings of the Workshop on Intelligent Support for Exploratory Environments 2012: Exploring, Collaborating and Learning Together at the 11th International Conference on Intelligent Tutoring Systems. New York: Springer.
McLaren, B.M., Timms, T., Weihnacht, D., Brenner, D., Luttgen, K., Grillo-Hill, A., and Brown, D.H. (2014). A Web-Based System to Support Inquiry Learning: Towards Determining How Much Assistance Students Need. In Proceedings of the Sixth International Conference on Computer-Supported Education, Volume 1 (pp. 43-52). Setubal, Portugal: Science and Technology Publications.
Journal article
Brenner, D. G., Matlen, B. J., Timms, M. J., Gochyyev, P., Grillo-Hill, A., Luttgen, K., & Varfolomeeva, M. (2017). Modeling Student Learning Behavior Patterns in an Online Science Inquiry Environment. Technology, Knowledge and Learning, 22(3), 405ndash;425.
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
