Debunking Some Myths About Generative AI
1.16.2025
(L-R): Bridget McCormack and Jennifer Leonard.
Two esteemed experts discussed the myths and the future of generative AI within the legal industry during Wednesday’s Presidential Summit, the marquee event of the New York State Bar Association’s Annual Meeting at the Hilton Midtown in New York City.
The program, entitled “Generative AI: Disrupting the Business and Practice of Law” featured the former Chief Justice of the Michigan Supreme Court Bridget McCormack, who is president and CEO of the American Arbitration Association, and Jennifer Leonard, founder of Creative Lawyers and professor at the University of Pennsylvania. They engaged in a back-and-forth discussion for nearly 90 minutes.
McCormack and Leonard addressed some myths about the technology. They include:
- The assumption that finding the right tools and following other firms’ examples is a sound strategy.
- The strategy that the IT department can implement AI on its own.
- The belief that the death of the billable hour is upon us.
Among the first questions lawyers ask her is how other firms are approaching AI. She said that nobody has yet to develop a roadmap that applies to everyone.
“People in the profession are accustomed to incremental change,” Leonard said. “However, this era really requires the adoption of fully new mindsets and the implementation of new frameworks for developing strategies that work in your specific firm. Some of those mindsets include entrepreneurial thinking and thinking about where the opportunities are for your firm.”
As for integrating the technology, McCormack said it is not necessary to have a large team to incorporate generative AI into a firm’s daily business practices.
“You can empower people in every group within your firm through licenses to different tools and start figuring out what are the efficiencies for operations and what are the ways these are going to impact your core services by having them all involved in your R&D effort. You don’t need a dedicated team of experts; nobody has those. This is a case that is uncomfortable for lawyers, but I believe you must collaborate across job functions and with people outside the profession to figure out what the future is going to look like,” McCormack said.
Leonard said the ultimate death of the billable hour through the growth of generative AI is not happening anytime soon, if at all. She added that it has been in decline for 20 years but has gone somewhat unnoticed because of the rise in rates has helped offset it.
“I don’t think we’ll see an era anytime soon where the billable hour is gone, but I do see an era where it becomes a smaller and smaller part of a law firms’ portfolio. I also am optimistic that we will enter an era in private practice where a lawyer’s value is separated from the number of hours she bills in a year and will really focus on the value of the service she is able to provide to the firm’s clients. I think that will be a refreshing change.”
Despite these myths, the technology does present challenges.
McCormack spoke about one of the most unique challenges incorporating generative AI technology — the “jagged frontier” coined by Wharton Professor Ethan Mollick, where outcomes of similar inputs may vary.
“None of us including the technologist who are building these (generative AI) models understand the limits of what it can do or what it can’t do until they get to those limits. You can try a test case for generative technology, and it might perform beautifully and then you imagine something similar that it can also do, and it doesn’t perform as well,” said McCormack.
Leonard offered her own thoughts on that point.
“The outcomes are not deterministic. We don’t know what will come out of any given prompt and if you play around with ChatGPT you can ask it the same question three seconds apart and it will give you different answers. We have much less control on the algorithm and much less ability to predict what will come as an outcome,” Leonard said.
McCormack wrapped up the discussion by explaining how the American Arbitration Association has incorporated some strategies. She said it is important for firms to break down silos and to have transparency so they can gradually innovate their generative AI strategies.
She said among the biggest challenges may be the legal profession itself.
“Adaption to technology tends to move slower for the legal profession, which I think is a good thing. The regulatory and compliance requirements are higher than other industries; there are high stakes in what we do, and clients expectations continue to grow,” McCormack said.
She said soon we will see a multi-model explosion and AI agents – generative AI platforms that you can ask to do things solely for yourself. However, that is a small step.
What’s after that. We’ll see early forms of robots within the next few years and Artificial Super Intelligence beyond that where the tools can do things better than the smartest human in any domain.
