top of page

Overruled by Algorithms: Embracing AI in Legal Practice

Writer: Colin LevyColin Levy

 

We are now at the stage of the AI revolution where even those of us living under a proverbial rock have heard of generative AI. The idea that a particular release version of a large language model could be a major global media event was a laughable idea until very recently, but AI has moved from research labs to our daily lives with remarkable speed.



In the legal space, we are constantly bombarded with marketing campaigns about productivity gains and improvements in the quality of our work. The promises vary, but include claims to supercharge this or that, to make a particular task instant or effortless, or to deliver game-changing efficiency gains across the board. This may all sound very odd coming from a person who has just joined a generative AI startup, but please bear with me!


I was called to the Bar almost twenty years ago. Ten years ago, I drifted away from private practice and, after the traditional mid-life crisis, towards technology. Like many in the space, tech represents the intersection of my deep professional skillset and a slight tendency towards neo-mania, with a twist of gadgetry obsession. I have lost count of the number of shiny gadgets that I have loved (and then consigned to landfill, sorry Mother Earth) over the years. While the urge to buy soon-to-be obsolescent electronic junk has faded as I age, my love of legal tech has only spiraled, and—to be frank—is now beyond all semblance of control.[1]


I first thought that legal tech would also have the beneficial side effect of allowing me to do less work and make more money, but since I drifted away from the wig and gown, I have been disabused of the rather naïve notion that this is a simple matter.[2] Along my journey, I have been fortunate enough to have been involved in global transformation projects alongside some of the biggest legal teams in the world. While I am not yet as grizzled as some of the towering figures in the ops and transformation space, I am at least tipping into the category of those who have their share of war stories, and when Colin asked me to share what he very generously described as “insights” or “thoughts”, I jumped at the chance. Thank you, Colin. Sidenote: if you want to see what generosity looks like, follow Colin on LinkedIn, or better yet, catch him at an event.


So, after a fabulously verbose introduction, the question we’ve all been asking: what on earth is going on, and what on earth can I actually DO?


What Does AI Eat For Breakfast?


We’ve all heard the phrase “culture eats strategy for breakfast”. Sometimes it’s said by someone looking for an excuse for lackluster planning, sometimes by a charismatic narcissist looking to paper over the behavioral cracks in their organization through which toxicity is seeping and oozing, and much more rarely, it’s said by a true leader.[3]


But if culture eats strategy for breakfast, what does AI eat? In my opinion, there is a non-zero chance that AI will eat culture. The technology is just so incredible that I believe that there is a non-zero chance that—absent a major or indeed global EU-style interventionist push—our existing culture will be eradicated. I mean that in the broadest possible, non-corporate sense.


This is not the cloud revolution, changing enterprise SaaS purchasing habits, and creating a new segment in an existing market. This is something else entirely. It will impact every single facet of life, and has the potential to completely up-end the assumptions on which we have built our professional world. To avoid worsening my chronic lack of brevity, I will deliberately leave this wide-ranging discussion, the possibility of AGI, the potential for an AI singularity, and other such topics for another time, potentially when I have a drink in hand.


My focus here, then, will be on legal culture exclusively. But I’m supposed to be talking about AI strategy, so why am I starting with culture? The extent to which we are merely products of our environment is debatable, but AI strategy considerations—and indeed any strategy considerations—should in my view start with an analysis of culture.


Corporate culture is a vast topic, which I am probably under-qualified to discuss. Legal culture is a little closer to my wheelhouse, but even that I would prefer to leave to the experts. Check out Charlotte Smith and her writing, for example. However, I have been asked to address the issue of AI in legal teams on sufficiently numerous occasions that I will accept if not the label of “expert” then at least the label of “non-moron”.


Lawyers are not famously convivial or congenial. They are a very more diverse group than stereotypes allow, but qualifying as a lawyer is to the right-hand side of the bell curve in terms of difficulty in most jurisdictions, and social status is similarly located. Type-A personalities abound, and parts of Big Law culture can be grind-centered. I don’t want to toss the baby out with the bathwater, as I have a deep love for the profession, and I am very proud of the time I spent prosecuting. I should also note that I am no less proud of the time I spent representing wealthy corporations and nation states in international arbitrations. The law has been my passion for a long time, and I like to think I can make a case for any aspect of it. Except trusts and estates. You folks are just straight up weird.


So, to return to our generalizations, we have a bunch of smart, driven, socially and professionally conservative knowledge professionals, faced with a novel technology that has a substantial chance of being better than all humans at reasoning and arguing on a timeline measured in months or maybe years, not decades.


What should we expect?


Fear. Lots of fear. When I show teams what AI can do, I make a joke that it takes thirty minutes to configure and deploy the instance, thirty days for associates to get up to speed, and then three years of therapy for the partners to get over it. They’ve just watched AI chew through ten thousand pages of loan agreements in ten minutes, smashing out verifiable data, and building an excellent first draft due diligence report. What do they see? Oftentimes, they just see billables evaporating.


Getting to the top is hard. Really hard. When you get there, it’s your turn to get rich, frankly. You’ve earned your time in the sun, it’s your turn to hold a bucket underneath the money faucet, and you probably don’t want to hear that “everything’s different now, Jim”. When their turn comes, some people tip into more extreme immobilism, while some just become a shade more conservative.

It takes a very special and courageous person indeed to see something new, recognize the potential, and immediately set to work on cannibalizing the business they just spent thirty years building. They are putting down the bucket and going to look for a way to tap the water main. There are too many to list, but early adopters of legal AI who are spending their very hard-earned cash are my professional heroes — even if they spend it with my competition!

 

Strategy Building


I’ve said this phrase, or a variation thereof, to rooms full of GCs or law firm partners more times than I can count: “Who you are determines where you are.” There’s a bit of elaboration, but that’s the punchline. Going on vacation? Which destination? Why? What do you want to do? What do you prefer? Who do you like to hang out with? Noisy bars or quiet cafés? I got a friend who likes horse-trekking vacations in Mongolia. Another who wants only goes to party destinations. Personally, I want a resort with nice weather and good food, a moderate distance from my house, fun stuff for the kids a must. We are different people and that determines where we end up.


Assuming you’re the leader, there’s hopefully a fit between who you are as a person, your values, and those of the organization you serve. If not, you probably don’t need to be told to dust off the CV and move as soon as you can. In the legal industry, this is the “type” of team you are. White-shoe firm with impeccable credentials and terrifying rates? Aggressive personal injury team with massive billboards? Dedicated local courthouse solo-warrior who hung her shingle in ’76 and never looked back?


In-house teams have the same question. Are you building for speed? Industrializing standard paper in a B2C industry with a website click-wrap agreement? Negotiating three agreements a year in a complex geopolitical context? Struggling to maintain alignment in your database of one million product codes? Somewhere in between?


You can’t hope to nail AI strategy if you haven’t got this straight. If you can point to a written and maintained version of your company’s values, a written legal team mission statement, and you can see tangible evidence of them in your daily professional life, that’s a good sign. If you need help with this, I’ll declare a conflict of interest and recommend that you call Emilie Calame, my former boss (and long-suffering wife).


Once you’ve figured out who you are, and you’re aligned with the organization that you serve, you need a destination. In-house teams may well have some clearly communicated corporate targets, OKRs and so on. Private practice targets tend to be equally clear and mainly financial. Bring that down to a more granular level and build a team target.


Then, take a look around. What’s your team like? How big is it? Average age? Willingness to experience discomfort? Adaptability to change? Moving a fifty-lawyer team isn’t the same ask as moving a three-lawyer team. Second part of the stock take: what process and tech do we already have? Microsoft org? Check for Power Automate licenses. Notion teams can build some pretty slick stuff with middleware (Zapier, Make, Airtable, Bubble, etc.).


Values, alignment, team status, destination, available resources for the journey. Now comes the fun bit. What do you need to do, to get where you want to go?

For some people, AI is a solution looking for a problem. Don’t be that person. If your problems include handling large volumes of unstructured data, extraction, analysis, markups, contract negotiation, document comparisons and so on, then there is undoubtedly a game-changing level of leverage an LLM away.


Step one, then: identify your current tasks. What are you doing on a regular basis? What do those tasks look, in detail? How many person-hours a month, which colleagues handle them? Fair warning: this is much, much harder than it sounds. Once you have a clear view of processes, run the “Five Whys” and dig into the underlying motivation. See if there’s something you can ditch, or handle in another way, or just optimize out of existence. Example: contract negotiations. Revisit your standard paper, do some deal post-mortems, and if you’re drowning in red ink on each deal make sure you know why, and then try to eliminate the problem. Eliminate the task if possible, optimize the survivors, then try to use technology to automate all of the optimized flows you have left. In that order.


Step two: among the tasks you’ve identified as rock-solid must-haves in your work life across the next year or so, which of them involve large volumes of data, repetitive work, similar documents, and other such “AI-tells”? A good rule of thumb is “AI is not for everything”. Again, this might seem strange from an AI-evangelizing professional from an AI company who has tied his career to the future of this technology, but I hope that you can see that is something to take seriously.


If your organization needs to push tens of thousands of SKUs into order forms from a CRM, and then run this data into an ERP system, performing complex math along the way, you probably want some integrations that pipe the data end-to-end, and not the probabilistic miracle machine that is an LLM. If, on the other hand, you have an increasing volume of work that requires you work with large volumes of unstructured data, analyzing, extracting and generating written work, your life is going to change very quickly. 


Step three: preparation. Let’s assume that you’ve identified a process that is unavoidable, optimized, and not automatable. In an in-house team, this might be contract negotiation, IP licence audit, or an employment contract audit. In a law firm this might be anything from an an M&A review, first pass reviews of written submissions from the other side, reviewing docs from a client, or even client onboarding.


Figure out how much this process is costing you. Anything under 10k a year is very unlikely to be on your radar. Anything under 100k might not be worth it. Touching business critical processes means risk, and it means decision-maker time. Price the risk as best you can, price the time it takes for your leaders to provide input, and then add the cost of change. Lost productivity, the valley of despair, and the cost of the tool itself (plus assistance with the change) will all add up very quickly.

Get this enormous number to the front of your mind and add a safety margin. Did you include project planning and general bandwidth? What about the extra business this shiny new toy might bring in? How safe are we feeling about these numbers?


Step four: project launch. We have identified something that AI can do, the fixing of which will generate not only a positive ROI, but one so large that once all other factors are taken into consideration, it meets the IRR criteria required by whichever body makes such decisions. The decision is made, your organization commits to action, and the fun begins! Scoping, RFIs, RFPs, POCs, pilot phases, roll outs and more. If you’ve got this far, building a culture of continual improvement shouldn’t be too taxing, so let’s say Step Four rolls into eternity…


Business As Usual?


These basics should give you a framework that allows you to identify opportunities in a given team at a given time, but it’s very much a view from the trenches. In my opinion, this kind of work can only deliver exceptional results when it is performed by someone with an excellent grasp of the larger strategic dynamics at play in the market. What’s the 30,000 foot view?


The nature of innovation is that it is not at all “business as usual”. It is a technology that changes things to such an extent that entire sections of the economy will die out. The whale oil industry, horse-drawn cabs, coal mining, human translators. They were all replaced in large part by machines. Are lawyers next?


No.


Caveat: the market for legal services baffles me. I see constant rate rises, above inflation increases across the board, almost constant complaining from in-house teams, ever-increasing numbers of lawyers qualifying to practice, and ever-increasing volumes of work being performed. Supply and demand don’t seem to be particularly well-correlated, in other words. To make matters better/worse depending on whether you’re buying or selling, buyers appear to be almost entirely insensitive to price.


Two-and-a-half predictions then. The half prediction is that given the economic incentives and social status involved, I don’t see the pull of the profession diminishing soon. That will help drive the first “real” prediction. Lawyers won’t disappear. More lawyers, better tooling, bigger addressable market. Jevon’s Paradox will play out in full, and as efficiency rises, overall revenues that flow to the sector will increase, not decrease.


Second “real” prediction. In no team does the advent of AI means business as usual. It is orders of magnitude better at very particular tasks, and this comfort zone is expanding rapidly. I believe that in some industries, a massive proportion of tasks are already well within the AI comfort zone. I include the legal industry here. Over time, what we traditionally considered to tasks reserved for human experts will be taken by AI.


Economics would suggest that the higher the proportion of such tasks in an industry, and the higher their cost to the wider economy, the more capital will rush to address this. My economic ignorance aside, didn’t a legal AI team out of the US just raise 300 million bucks?


Leaders in such industries have less time than anyone else to react. In fact, reacting is not enough. They must anticipate. This involves risk, and fortunately for us lawyers, we excel at risk analysis. What does this mean in very real terms? Get your hands dirty. Understand what an LLM is, what it does, what it can’t do. Understand what training data is, and what fine-tuning can and can’t do for model performance. Test things. Get an idea of what compromises might be made in the deployment of a system. Speed? Security? User-friendly interface? Make smart supply chain decisions. Is your vendor model agnostic? US-based? What about open source? What about self-hosting?


The answers to these questions vary across time and across industries. I don’t have the answers for you, but just asking the questions will put you to the right-hand side of the bell curve when it comes to AI strategy.

Good luck.

 

P.S. — as a bonus, here is a list of what my priorities would be as a GC or law firm partner:


GCs:

1.        Immediately draft AI use policy for employees

2.        Draft an AI supplier policy, particular eye on data reuse

3.        Go for low-hanging fruit (unstructured contract data, most likely)

4.        Re-invest every minute saved in more AI work.

Goal: a lean department that does nothing but strategize and handle BAU outliers. 

 

Law Firm Partners:


1.        AI vulnerability assessment: which parts of my practice are AI-tractable, and which are billed on a time basis? Which part of the market am I in? Mass-market, leave ASAP. You are fungible. Pick a specialism and differentiate if possible. Mid-market? AI price competition will be painful. Find a segment and see if you can build a super-reliable money-spinning use case where you can secure first-mover advantage. Use the extra money to try to keep your lead/move up-market. Elite? Secure the leading specialists. Knowledge capitalization is coming fast. If your model is sufficiently agile, fight harder for fewer elite hires.


2.        Invest in AI, reinvest each hour saved in improving the things clients love most: accessibility, face time, updates, those spontaneous meetings or lunches that overrun but where you figure out really important stuff together.


3.        In parallel, consider pushing the parts of my work that are AI-tractable towards fixed fees.


Goal: a firm with impeccable credentials and brand, top-tier work, “cash-cow” AI-powered workstreams, massive margins, unrivalled client care, strong BD and growth. 


Jonathan Williams is a recovering litigator and arbitration practitioner. He began his career as a prosecutor before switching to international arbitration and moving to Paris. A decade ago, he slid across into technology, where he has developed a deep expertise in solving the biggest problems for the biggest legal teams.


Innovation, technology, change management, strategic advisory work and generative AI have taken up a substantial portion of his professional life. The last five years he has spent working for Calame, the advisory team founded by his wife, Emilie Calame. He joined Legora at the end of 2024, where he is heading up their operations in France. He is based in Paris but travels extensively.


[1] By the way, that’s a deliberate em-dash, not an OpenAI one. I’m probably not using them correctly, but it is me that’s using them.

[2] As you may be able to tell, I have not, however, been disabused of the notion that sentence length or the excessive use of subordinate clauses is a proxy for intelligence. See also, excessive use of footnotes. Lawyers love footnotes.

[3] The ratio I’ve encountered in my career thus far is approximately 70:29:1. Better than hearing a leader say it, is watching a leader embody it. I’ve been lucky enough to see this at Calame and with my current employer.

bottom of page