At the recent ACC annual meeting, Adrian Goss—whom I first met a dozen years ago on a trip to Australia—told me he had uploaded some miscellaneous-provisions stuff from Practical Law and asked ChatGPT to redraft it so it complies with the guidelines in A Manual of Style for Contract Drafting.
In this 2023 blog post, I discuss the notion of doing that sort of thing. It quotes a tweet by Alex Hamilton saying he had tried it, and that the result “wasn’t great.” But because I have the full details from Adrian, I thought it worth revisiting this topic.
Here’s Adrian’s prompt and ChatGPT’s response:
Could you redraft a contract for me using the style recommendations contained in Ken Adams A Manual of Style for Contract Drafting?
Absolutely, I can help you redraft the contract. Please provide the current version of the contract you would like to revise, and specify any particular sections or clauses you want to focus on.
What ChatGPT had to offer is contained in this Word document, marked to show changes from the Practical Law original.
Here’s what comes to mind.
“Absolutely”
Regarding that “Absolutely,” it signals overeagerness of a sort that quickly gets smarmy in humans; in this technology, it suggests one is being played for a rube.
Has ChatGPT’s Large Language Model Assimilated MSCD?
Given ChatGPT’s self-assured response, you’d think that, yes, MSCD is in the ocean of text fed to ChatGPT. But I don’t know whether OpenAI is ripping creators off to the extent that it’s running books through a scanner. Perhaps ChatGPT knows me only from my articles and blog posts. Or it might have decided to ignore that Ken Adams guy and offer Adrian some generic changes.
ChatGPT’s Changes
The changes marked in Adrian’s document don’t demonstrate any particular fealty to MSCD.
ChatGPT’s markup eliminates (as shown in strikethrough) an instance of what I call “throat clearing” (see this 2020 blog post): “Each party undertakes that it shall not at any time disclose …” But it also ignores another instance of throat-clearing: “Each party agrees that it has no claim …” That’s not the MSCD way.
ChatGPT uses the simple present in language of policy relating to a contingent future event (“No variation of this agreement is effective …”); I would use will be.
And ChatGPT didn’t adjust overuse of shall, as in “the courts of England and Wales shall have exclusive jurisdiction”.
I could go on, but I’ll spare you.
Anyone who expected more doesn’t understand how ChatGPT works. It’s not as if it has somehow become the most adept student of my Drafting Clearer Contracts training. ChatGPT relies on a large language model (LLM). Here’s how Ethan Mollick describes (in this post) what LLMs do:
Large Language Models are, ultimately, incredibly sophisticated autocomplete systems. They use a vast model of human language to predict the next token in a sentence. …
When you give an AI a prompt, you are effectively asking it to predict the next token that would come after the prompt. The AI then takes everything that has been written before, runs it through a mathematical model of language, and generates the probability of which token is likely to come next in the sequence.
So if ChatGPT has in fact digested MSCD, it has done so without being able to distinguish MSCD’s expository prose from MSCD’s contract prose. Instead, all MSCD prose would be applied to determining a single set of probabilities, and the result would be a mish-mash.
Fixing Traditional Contract Text Is a Mug’s Game
But for the reasons I give in that 2023 blog post, it’s not realistic to expect ChatGPT to rehabilitate traditional contract prose, which is so dysfunctional that fixing it is laborious. Using traditional contract language as your starting point makes it less likely you’ll end up with something that’s optimally clear, concise, and relevant. It would be much simpler to make a fresh start. As I say in this blog post, “Instead of renovating a dilapidated building, usually it makes sense to tear it down and build something new that fits your needs.”
That’s why when I did my own analysis of the mediocrity that is Practical Law contract templates (see this blog post), I limited myself to pointing out the problems. You want to know what I would do instead? Check out Adams Contracts’ confidentiality agreement template.
The Real Fix
So GenAI won’t fix what’s wrong with contracts. I’ve said as much in several blog posts (go here for the most recent one, with links to the others). GenAI is just another flavor of the copy-and-paste machine. Do something else instead—something that will actually fix the problem. All it would take is expertise, intricate work, and a sense of civic virtue. Those happen to be deeply unfashionable notions, but I’m up for it.
Ken,
First, I am a lay person who swims in a see of lawyers pretty consistently and am often called upon to ‘proof’ for atty’s and especially paralegals; your writings have been a godsend for years!
I thought you might find interesting that the newest version of CGPT, while it misses some simplification opportunists, is pretty on point; to wit: (sorry, had to!)
Neither party is liable for any delay or failure to perform its obligations under this agreement to the extent that such delay or failure results from events beyond its reasonable control. If such delay or non-performance continues for ten days, the unaffected party may terminate this agreement by giving written notice to the affected party.
No variation of this agreement is effective unless it is in writing and signed by the parties or their authorized representatives.
(a) If any provision or part of a provision of this agreement is invalid, illegal, or unenforceable, it is deemed deleted, but that does not affect the validity and enforceability of the rest of the agreement.
(b) The parties shall negotiate in good faith to agree on a replacement provision that, to the greatest extent possible, achieves the intended commercial result of the original provision.
Just thought I would share!
~MP