Tech Explained: Here’s a simplified explanation of the latest technology update around Tech Explained: New AI and the Law course explores the capacity and limits of technology in the legal profession in Simple Termsand what it means for users..
In just a short time, with a well-written prompt, AI tools can produce documents with credible legal arguments.
The briefs appear polished — but are their claims sound and supported by the law?
A new course, “AI and the Law,” taught at Willamette Law by Associate Professor Terry O’Reilly aimed to find out last spring, as students examined AI legal tools and their potential applications.
When the dean’s office asked for proposals from faculty on a course about AI, O’Reilly thought that preparing an outline would be effective motivation for him to learn more about the subject, which was drawing intense interest since the release of ChatGPT 3.5 at the end of November 2022.
O’Reilly reviewed news articles, reports by law firms and accounting firms, entertainment industry bargaining agreements with AI clauses, technical reports and legal filings in cases involving AI. He also experimented with AI tools to prepare simple handouts for topics he understood well, assessing the current strengths and weaknesses of the tools.
Given the subject’s novelty, he was able to organize enough material for a course in about a week, in collaboration with the dean’s office. He made considerable adjustments before and during the course to keep pace with rapid developments in AI and in the AI tools for lawyers.
During the course, O’Reilly gave an overview of the technology of large-language models and the legal issues surrounding the development and applications of AI.
To evaluate the quality of AI legal output, students first reviewed the format and style of law firm work products: internal memos, client memos, legal opinions and legal briefs. They then surveyed AI tools designed for or adaptable to legal applications and experimented with methods of allocating tasks, structuring queries, and reviewing and refining AI output for proper formatting.
Their assignments simulated law firms using AI to draft documents on firm policies, a memo to clients identifying potential environmental law issues in acquiring industrial properties, and documents addressing legal issues arising out of the use of AI.
“The emphasis of the course was to obtain sound legal analysis using existing artificial intelligence tools. It is challenging for students — for anyone — to achieve this,” O’Reilly says. “The capacity of a particular AI tool to produce reliable legal documents sometimes varies even from session to session in the same week, plus new versions of a tool might respond differently to instructions than a prior version.”
Despite occasional setbacks with some of the tools, students appreciated their progress and exchanged ideas and discoveries about AI.
The course is scheduled to be offered again next year, but O’Reilly says he would be surprised if, in five years, it was still being offered without changes. The course could be obsolete if AI developments stall, or, if progress continues, it could be too much content for a single class.
Regardless, he recognizes the attention that AI requires. It’s prudent to appreciate its capacity and limits, he says.
“Even using sophisticated techniques in formulating AI instructions does not guarantee satisfactory output,” O’Reilly says. “In any event, each claim and cited authority still must be reviewed and confirmed — that’s nothing new and may never change. But with care and experience using a particular AI platform, it is often possible to obtain useful legal analyses or drafts — sometimes quickly, and of impressive quality. So, for better or worse, heads up.”