June 2 (Reuters) – Law professors overwhelmingly preferred answers drafted by AI over ones written by fellow professors, a new Stanford Law School study, opens new tab found, suggesting that the technology is ?capable of legal reasoning and that law students may benefit from AI ?tutoring.
Professors from 14 U.S. law schools developed a list of 40 questions representative of those first-year contracts students ask during faculty office hours. The professors wrote answers to the questions, and researchers ?had two AI platforms — Google’s Gemini 2.5 Pro and NotebookLM — also answer them.
The same ?professors blindly judged the short answers head-to-head and chose the AI-generated ones as most ?beneficial to students 75% of the time. The AI platforms performed just as ?well as the professor rated most highly in the study.
“We were frankly surprised by the ?magnitude of the results,” lead researcher and Stanford law professor Julian Nyarko said in an article on Stanford’s website, opens new tab about the study. “These weren’t just simple questions with obvious answers.”
The study comes as law schools ?and the legal profession grapple with how to incorporate rapidly evolving AI into teaching ?and law practice. Earlier studies have found that AI can pass the bar exam, earn A+ law school grades, ?and effectively ?grade law school exams.
A growing number of law schools are mandating AI instruction in students’ first year. But approaches vary. The University of California Berkeley School of Law recently adopted a new policy significantly curtailing how students may use AI in their academic work.
The new ?tutoring study suggests AI ?may have benefits on ?the teaching side. Rather than relying on peers or sporadic emails to instructors to answer questions, law students could utilize AI ?for on-demand answers with reliable results, according to the study.
Importantly, less ?than 4% ?of the AI-generated answers were flagged as “harmful” to student learning by the judges, compared with 12% of the professor-written answers.
“We find that, when evaluated by legal educators, AI tutors can ?offer ?high-quality, on-demand support that complements classroom instruction, and may ?broaden access to expert guidance,” said study co-author and Stanford researcher Alejandro Salinas in the Stanford article.
