Colleges that ditched test scores for admissions still struggle with bias

One college admissions officer at a large public university described how test-optional admissions had spurred more disagreements in his office. A third reader on an application was often called in to break a tie when one staffer said ‘yes’ and another said ‘no.’ Without SAT and ACT scores, he explained, the job of admitting students had become more subjective and more time-consuming. “I feel like everyone who reviews applications has their own perspective or opinion,” he said.

This sobering anecdote comes from a research project led by Kelly Slay, an assistant professor at Vanderbilt University, who has been conducting in-depth interviews with admissions officers in 2022 to understand how the elimination of SAT and ACT testing requirements has been playing out inside colleges and universities. According to Slay, admissions officers often described a “chaotic” and “stressful” process where they lacked clear guidance on how to select students without test scores. Admissions officers at selective colleges were also “overwhelmed” by the volume of applicants that test-optional policies had unleashed.

“One of our key findings were the tensions that were emerging around these test optional policies,” said Slay. “There’s a struggle on how to implement them.”

Slay’s work gives us a rare, unvarnished glimpse inside college admissions offices. It’s especially significant now because a college admissions case is currently before the Supreme Court that could strike down affirmative action, a practice that gives preferences to groups that have been discriminated against. As colleges experiment with alternative solutions, these interviews help shed light on why test-optional policies haven’t been helpful for increasing diversity on college campuses.

Earlier quantitative studies found that the test-optional movement, which has spread to over 1,700 colleges, failed to substantially raise the share of low-income students or students of color. For example, one study published in 2021 found that the share of Black, Latino and Native American students increased by only 1 percentage point at about 100 colleges and universities that adopted the policy between 2005-06 and 2015-16. A separate study of a group of selective liberal arts colleges that adopted test-optional policies before 2011 didn’t find any didn’t find any diversity improvements on those campuses.

Before the pandemic, the move to test-optional admissions was already gathering steam as concerns mounted over the fact that wealthier students could hire tutors, take the tests multiple times and post higher scores. Other critics said that the paperwork to waive testing fees was a barrier for many low-income students. Then, during the pandemic, it became nearly impossible for students to sit for exams and the vast majority of colleges eliminated testing requirements. Some have since restored them, but many haven’t.

Slay’s research is still ongoing, and she presented her preliminary findings at the 2022 annual conference of the Association For Education Finance & Policy. When I interviewed her in October 2022, she and her research team had interviewed 22 admissions officers from 16 colleges and universities. All were four-year institutions, but they ranged from public to private, large to small, and religious to nonreligious. Four of the colleges had dropped testing requirements in the years before the pandemic with the remaining 12 doing so during the pandemic.

It’s not surprising that colleges that went test-optional during the pandemic were suddenly scrambling to decide how to review applications without standardized tests. But the researchers learned that even colleges who had years of experience with test-optional admissions were still working out the details of how to implement it.

Admissions officers worried that their colleges were replacing standardized tests with metrics that were even more biased toward wealthier and white students, such as letters of recommendation and expensive extra-curricular activities. One college purchased a data service that ranked high schools and factored those high school rankings into each application. Students from underserved high schools received a lower ranking, an admissions officer explained. It wasn’t a fair process.

Many admissions officers said that they were struggling with how to select candidates fairly and didn’t know how to weigh an application with test scores against one without. “I think the students that do have the strong test scores still do have that advantage, especially when you have a student that has strong test scores versus a student who doesn’t have test scores and everything else on the academics is more or less the same,” an admissions officer told Slay.

“It’s really hard to ignore test scores if that’s the way you were trained to review applications and think about merit,” said Slay. “If the standardized test is there in the file, it might still bias you in ways that you’re not aware of. It’s an anchoring bias.”

Admissions officers also described how they struggled to answer a frequent, but basic question: are you really test optional? Students wanted to know if they would have an advantage if they did submit a test score. Slay said admissions officers wished they had better guidance on how to answer this question. Since college entrance exam scores could also be used for certain scholarships and determining course placements once admitted, it was difficult for admissions officers to say that the test wasn’t still important.

Larger workloads were a common complaint. College admissions officers said they were spending more time on each application in an effort to be diligent. Plus, the volume of applications had increased “a lot” at selective schools, Slay said. Meanwhile, many offices lost staff during COVID. Some employees resigned amid the strong job market. Budget cuts at some schools led to layoffs and furloughs. Slay said that some admissions offices were operating with a “skeletal” staff.

The stress and pressure of being short-staffed and confused could affect anyone’s decision making. The conditions were ripe for amplifying implicit biases – exactly the opposite of the intent of the test-optional policy.

Slay is hearing from colleges that test-optional policies have increased the diversity of the applicant pool, but it may not translate into a more diverse student body.

“One of the things we concluded is that test optional does not mean an increase in diversity – racial diversity or socio-economic diversity,” said Slay. “If we haven’t figured out how to review students who come from diverse backgrounds who come from schools where they may not have the same access to AP or IB courses, then that could mean that these students still aren’t going to be admitted.”

This story about test-optional admissions was written by Jill Barshay and produced by The Hechinger Report, a nonprofit, independent news organization focused on inequality and innovation in education. Sign up for the Hechinger newsletter.