Skip to main content

Breadcrumb

Home arrow_forward_ios Information on IES-Funded Research arrow_forward_ios Improving the Assessment Capability ...
Home arrow_forward_ios ... arrow_forward_ios Improving the Assessment Capability ...
Information on IES-Funded Research
Grant Closed

Improving the Assessment Capability of Standardized Tests: How High-Stakes Testing Environments Compromise Performance

NCER
Program: Education Research Grants
Program topic(s): Cognition and Student Learning
Award amount: $427,786
Principal investigator: Sian Beilock
Awardee:
University of Chicago
Year: 2005
Award period: 3 years (06/01/2005 - 05/31/2008)
Project type:
Development and Innovation
Award number: R305H050004

Purpose

In this project, the researchers proposed to first examine how stereotype threat affects women's performance in math testing situations and then develop and pilot test new assessment tools that aim to reduce the negative effects of stereotype threat. Stereotype threat arises when members of a marginalized group feel there is a negative stereotype about them and perform worse or demonstrate apprehension in circumstances when the stereotype is salient. For example, there is a stereotype that women are not good at math. The theory is that women may not perform at their full potential in math situations if they perceive this stereotype. By the end of this project, the researchers aimed to provide new guidance on how standardized assessments can be developed that accurately reflect all students' abilities and potential.

The second component of this work investigates techniques to reduce the negative impact of high-stake testing situations in real-world education assessment by modifying the major test used in college admissions (i.e., SAT) based on the above findings and administering it to students in the residential program for women in math and science. Studies 7 and 8 modify the Quantitative portion of the SAT using principles (tested above) designed to diminish stereotype threat effects. It is anticipated that these new tests will produce better overall math performance and reveal greater predictive validity for academic achievement (e.g., college math grades) than those currently used. Study 9 examines SAT performance following training designed to inoculate students against the negative consequences of high-stakes situations.

Structured Abstract

Setting

The experimental research is being conducted in a medium-sized Midwestern town and a large Midwestern city.

Sample

Approximately 900 female undergraduates and women in a residential program for female students with interests in math and science are participating. Because stereotype threat is most likely to occur for those highly invested in performing well, women pursuing math and science careers provide an ideal population to examine in a real-world education assessment setting.

Research design and methods

The first component of this work explores how stereotype threat interferes with women's math performance. Studies 1 and 2 test the idea that stereotype threat induces verbal worries that consume phonological resources needed for solving certain types of math problems. Studies 3 and 4 test the counterintuitive predictions that individuals with greater working memory capacity are more susceptible to stereotype threat. Studies 5 and 6 examine if stereotype threat carries over and impairs subsequent tasks that rely on working memory but are not implicated by stereotype threat.

Students are randomly assigned to experimental or control conditions in every experiment described above.

Control condition

Students in the control conditions are not provided with framing information that precipitates stereotype threat.

Key measures

Student performance on math problems is the primary data being collected.

Data analytic strategy

Analysis of variance techniques are being used to examine student performance on mathematical problems as a function of participation in the experimental or control condition.

People and institutions involved

IES program contact(s)

Elizabeth Albro

Elizabeth Albro

Commissioner of Education Research
NCER

Products and publications

Publications:

ERIC Citations: Find available citations in ERIC for this award here.

Select Publications:

Book chapters

Beilock, S.L. (2007). Choking Under Pressure. In R. Baumeister, and K. Vohs (Eds.), Encyclopedia of Social Psychology (pp. 141-142). Los Angeles: Sage Publications.

Beilock, S.L., and Lyons, I.M. (2009). Expertise and the Mental Simulation of Action. In K.D. Markman, W.P. Klein, and J.A. Suhr (Eds.), Handbook of Imagination and Mental Simulation (pp. 21-34). New York: Psychology Press.

Beilock, S.L., and Ramirez, G. (2011). On the Interplay of Emotion and Cognitive Control: Implications for Enhancing Academic Achievement. In J.P. Mestre, and B.H Ross (Eds.), The Psychology of Learning and Motivation, Volume 55 (pp. 137-169). San Diego: Elsevier, Inc.

DeCaro, M.S., and Beilock, S.L. (2013). The Benefits and Perils of Attentional Control. In M. Csikszentmihalyi, and B. Bruya (Eds.), Effortless Attention: A New Perspective in the Cognitive Science of Attention and Action (pp. 51-73). Cambridge, MA: MIT Press.

Schmader, T., and Beilock, S.L. (2011). An Integration of Processes That Underlie Stereotype Threat. In T. Schmader, and M. Inzlicht (Eds.), Stereotype Threat: Theory, Process, and Application (pp. 34-50). New York: Oxford University Press.

Journal articles

Beilock, S.L. (2008). Math Performance in Stressful Situations. Current Directions in Psychological Science, 17(5): 339-343.

Beilock, S.L., and Decaro, M.S. (2007). From Poor Performance to Success Under Stress: Working Memory, Strategy Selection, and Mathematical Problem Solving Under Pressure. Journal of Experimental Psychology: Learning, Memory, and Cognition, 33(6): 983-998.

Beilock, S.L., and Gonso, S. (2008). Putting in the Mind Versus Putting on the Green: Expertise, Performance Time, and the Linking of Imagery and Action. The Quarterly Journal of Experimental Psychology, 61(6): 920-932.

Beilock, S.L., Jellison, W.A., Rydell, R.J., Mcconnell, A.R., and Carr, T.H. (2006). On the Causal Mechanisms of Stereotype Threat: Can Skills That Don't Rely Heavily on Working Memory Still be Threatened?. Personality and Social Psychology Bulletin, 32(8): 1059-1071.

Beilock, S.L., Lyons, I.M., Mattarella-Micke, A., Nusbaum, H.C., and Small, S.L. (2008). Sports Experience Changes the Neural Processing of Action Language. PNAS Proceedings of the National Academy of Sciences of the United States of America, 105(36): 13269-13273.

Beilock, S.L., Rydell, R.J., and McConnell, A.R. (2007). Stereotype Threat and Working Memory: Mechanisms, Alleviation, and Spillover. Journal of Experimental Psychology: General, 136(2): 256-276.

Decaro, M.S., Thomas, R., and Beilock, S.L. (2008). Individual Differences in Category Learning: Sometimes Less Working Memory Capacity Is Better Than More. Cognition, 107(1): 284-294.

Decaro, M.S., Wieth, M., and Beilock, S.L. (2007). Methodologies for Examining Problem Solving Success and Failure. Methods, 42(1): 58-67.

Ping, R.M., Dhillon, S., and Beilock, S.L. (2009). Reach for What You Like: The Body's Role in Shaping Preferences. Emotion Review, 1(2): 140-150.

Rydell, B.J., McConnell, A.R., and Beilock, S.L. (2009). Multiple Social Identities and Stereotype Threat: Imbalance, Accessibility, and Working Memory. Journal of Personality and Social Psychology, 96(5): 949-966.

Sibley, B.A., and Beilock, S.L. (2007). Exercise and Working Memory: An Individual Differences Investigation. Journal of Sport and Exercise Psychology, 29(6): 783-791.

Questions about this project?

To answer additional questions about this project or provide feedback, please contact the program officer.

 

Tags

CognitionData and AssessmentsK-12 EducationMathematicsPostsecondary Education

Share

Icon to link to Facebook social media siteIcon to link to X social media siteIcon to link to LinkedIn social media siteIcon to copy link value

Questions about this project?

To answer additional questions about this project or provide feedback, please contact the program officer.

 

You may also like

Zoomed in IES logo
Workshop/Training

Data Science Methods for Digital Learning Platform...

August 18, 2025
Read More
Zoomed in IES logo
Workshop/Training

Meta-Analysis Training Institute (MATI)

July 28, 2025
Read More
Zoomed in Yellow IES Logo
Workshop/Training

Bayesian Longitudinal Data Modeling in Education S...

July 21, 2025
Read More
icon-dot-govicon-https icon-quote