Stanford AI Model Helps Locate Racist Deeds in Santa Clara County

Even in Santa Clara County, home to many of the companies and centers of innovation that have earned Silicon Valley its name, governments often do things in an old-fashioned manner.

So when California lawmakers handed down a mandate in 2021 that all counties in the state needed to cull their property deed records to find and redact racially restrictive covenants, Santa Clara County put two employees on the daunting task.

They began in 2022 what they expected might be an up to five-year project to manually sift through tens of millions of pages of paper and digitized property deed records. They were looking for racist language that barred people of specific races or ethnicities from owning properties in the county.

“I mean, it was literally eyes on paper turning pages, then it was eyes on the computer going through those same type of pages on the reels. And they did an excellent job,” said Louis Chiaramonte, the county’s assistant clerk-recorder.

The county’s team only made its way through around 100,000 records, finding about 400 of the thousands of defunct racist clauses that are tucked into documents related to ownership of homes and control of blocks and neighborhoods of the South Bay.

That’s when the county and Stanford University’s RegLab, a hub for research and development into how government agencies can perform core services more efficiently, partnered to bring the power of AI language models onto the job, significantly speeding up the process.

After curating racist covenant documents from seven counties across the nation, the RegLab researchers trained an open-source language model on those examples. They then put it to work, scanning 5.2 million Santa Clara County deed records from 1902 through 1980.

It took about a week.

“What our project really shows is there’s a very different and compelling path forward to achieving these kinds of tasks that don’t suffer the kinds of cost overruns that have historically really plagued government technology contracting,” said Daniel Ho, a professor and the director of the RegLab.

Los Angeles County, for example, outsourced the work to a contractor for about $8 million in a process expected to take about seven years to finish, Ho said.

Chiaramonte said the RegLab model helped Santa Clara County accelerate the process of flagging and mapping about 7,500 restrictive covenants. From there, the covenants are reviewed and sent to the county’s lawyers for final approval before a new, modified version of the deed is recorded. About 4,500 have been completed, and the original deeds remain unchanged for historical reference.

“It’s just amazing. I’m very thankful that this opportunity presented itself, and we’re able to work with them, Chiaramonte said. “And it appears that this language model tool that they have is extremely effective and has produced meaningful changes to how we could approach things in the future.”

The county employees who started the work will shift their focus to manually culling through the remaining records from 1850 to 1901 — most of which were handwritten — and digitizing newer records from after 1981.

Map by Stanford University’s RegLab

Faiz Surani, one of the co-authors of the research paper on the project, noted that the curation of examples and the training of the open-source model was the bulk of the front-end work, and it needed to be precise. The team trained the model to recognize not just simple keywords but also to identify a covenant even when a document scan is degraded, common strings of words and where in the document covenants are often located.

“If you ask ChatGPT to detect racial covenants, it’ll do a decent job out of the box,” Surani said. “The challenge is when you are going over 5 million, 10 million, 20 million records, you need to be virtually perfect, or else you’re going to be missing something or you’re going to be buried under a pile of false positives.”

Surani and Ho said the model has so far shown itself to be nearly 100% accurate in finding covenants in the records it searched. In all, the AI-based technology was able to save about 86,000 person-hours for the county.

The racist covenants and restrictions often included racial epithets. The covenants were less often seen in the very early 20th century because it was still legal to zone by race. After the nation outlawed that practice as unconstitutional in 1917, deed restrictions became more commonplace as a way to use private transactions to maintain segregation.

“The language in these covenants became more targeted and explicit. Deed records reveal widespread exclusion of specific ethnic groups, including African Americans, Chinese, Japanese, and other non-Caucasian communities. Terms such as ‘Negro,’ ‘Mongolian,’ and ‘colored’ were commonly employed to delineate the racial boundaries of acceptable property owners and tenants,” the RegLab’s research paper said.

Ho said he is optimistic the technology could be used to help governments look for other violations of California’s fair housing laws, including protections based on veteran status, family status, income and religion.

Surani said that as someone who identifies as Asian American, he was struck by the bluntness and banality of how the covenants were included in contracts.

“There will be one provision which says, you know, you have to install a sewage tank. The next provision, only Caucasians may live here. The next provision, you can’t construct an outhouse here,” he said.

Surani said another “gut punch” was how the research helped crystallize the widespread nature of the covenants.

“We find entire towns — not just neighborhoods — towns that were racially restricted from their founding,” Surani said, such as Redwood Estates, an unincorporated town along Highway 17 in the county.

There were heavy concentrations around Stanford and even a city-owned cemetery in San José with dozens of covenants allowing only white people to be interred.

Overall, Ho said the research showed that in 1950, about a decade after the peak use of covenants, about one in every four housing units in the county was under some sort of racially restrictive covenant.

He estimates that about 10 developers were responsible for roughly a third of all the covenants in the county, suggesting that a small group had a major influence on how Santa Clara County was plotted and built.

Some successful developers, like Joseph Eichler, chose not to include such covenants in their home tracts, “contrary to some historical scholarship, which notes that at that time, you would have lost business and would have gone out of business by not including that,” Ho said.

Walter Wilson, a co-founder of the Minority Business Consortium and an advocate for African Americans and Black people in the South Bay, said these long-unenforceable covenants were one of the biggest ways long-term wealth was concentrated in the hands of a few and laid the foundation for ongoing systemic discrimination.

“That still continues to this day by design,” Wilson said. “Among those people in those communities and the folks who control the politics, there’s almost an unwritten word, where they won’t even say it.

“But you don’t see very many Black people in Cupertino. You don’t see very many Latinos in Cupertino.”

He added: “California racism is the most dangerous in the world because it is just under the surface. It lies just under the law.”

Wilson said it’s exciting to see technology being used to address written discrimination but suggests the technology should also be targeted at current racist systems and practices.

“How is it addressing real discrimination that’s impacting people’s lives?” he said.