Crowdsourced Mediocrity Is Still Mediocrity

I periodically do my best to dump cold water on the notion that one can crowdsource quality contract language. I did so in this 2010 post, in this 2011 post, and in this post and this post in 2013.

I now permit myself to do so again after reading this post on Open Law Lab, maintained by Margaret Hagan. It’s entitled “Githubbing Law: Open-Source Legal Doc Repositories,” and it’s a helpful survey of what’s on offer. Margaret sets the scene as follows:

[T]here are an increasing number of Legal Document Repositories, many of them now overlaid with a user-friendly interface that allows the user to take the standard document and fill in the designated fields with their own information.

Thus, the user can take a standard doc and make it her own by simply entering in a few pieces of information (that she likely has at her finger tips). Some of these document repositories are even hosted in part on Github, so that any other visitor who signs up for Github could fork these documents & customize them for her own.

The terminology can be confusing. In her title, Margaret uses the term “open-source,” but I suggest that she’s really dealing with crowdsourcing, in that she leads with the Github model. “Crowdsourcing” refers to collaboration initiatives, whereas I think of “open-source” as referring to distribution rather than production. (You can have open-source initiatives that don’t involve collaboration.)

When it comes to crowdsourcing contract language, the problem is the same as it has always been: contracts are too complex, and traditional contract language is too dysfunctional, for one to be able to put any faith in volunteer efforts to compile contract language. Heck, I can’t even put any faith in BigLaw offerings (see for example this 2012 post on DLA Piper’s “Document Factory”), so why should I trust volunteers? If you can’t rely on a source of contract language, you have to treat it as raw material, checking everything against your own understanding and doing, as necessary, your own research. You might as well go dumpster diving on EDGAR.

Margaret appears to acknowledge this with the first item in the wish-list that wraps up her post:

Could we have Expert Curators who give their reviews & recommendations of documents?

That seems unlikely. Systematically grooming contract language requires way too much effort.

About the author

Ken Adams is the leading authority on how to say clearly whatever you want to say in a contract. He’s author of A Manual of Style for Contract Drafting, and he offers online and in-person training around the world. He’s also chief content officer of LegalSifter, Inc., a company that combines artificial intelligence and expertise to assist with review of contracts.

8 thoughts on “Crowdsourced Mediocrity Is Still Mediocrity”

  1. Hi Ken, thanks for writing on this topic. Here are my responses!
    On the open-source/crowdsourcing question: I refer to sites that purport to offer legal documents for open use — for others to review, remix & deploy without charge — in a model inspired by Github’s open-source site for coders to view and borrow each others’ code.
    This bleeds into crowdsourcing, when the sites invite most of the public to create an account & contribute their legal documents (or edits) to the repository. Not all of the sites open themselves up for crowdsourcing. Some of them (especially those affiliated with law firms) have a limited set of documents, coming from a single source, but presented in a way that allows for open access.

    And on the question of mediocrity: the ideal certainly isn’t peer-to-peer legal advice on the Internet, in which people randomly borrow & remix documents from each other. But the peer-to-peer model is in demand. Research I’m doing on non-lawyers’ use of the Internet is showing me that there’s a certain type of legal user who would rather go to a site like Docracy (if not Yahoo Answers) to try to hack together a legal solution through free resources rather than hire a lawyer.

    So if there are these people seeking out free, non-lawyer legal services — and there are sites that purport to offer them with “open, free, and reliable” legal documents — I want to see what best practices could be to make these users smarter/more critical about the services they’re finding online. That’s where my thoughts on metrics & expert curators came in, as possible models for making these peer-to-peer sites more protective of users.

    I’d look forward to any thoughts you have on how to protect non-lawyers from themselves when it comes to online legal contracting services.

    • Hi, Margaret; it’s a pleasure to make your in-the-comments acquaintance.

      If the goal is quality and simplicity, someone has to pay something. It doesn’t have to be the user. It could be a foundation. It could be a trade group. But it has to be someone. Otherwise, the question will be which honorable failure to use.

    • A few comments:

      1/ If you want curated, reliable documents, go with professional document automation. It’s not for everyone, though.

      2/ Anyone, lawyer or not, who uses crowdsourced documents uncritically is a fool and has a fool for a client.

      3/ Lawyers who know what they’re doing often use crowdsourced documents as a starting point to save a little time and get some traction, especially if they or their firms don’t have a good form or precedent for the task at hand. The lawyers most likely to do this are ones who don’t do a high volume of similar contracts, but rather a lot of one-offs.

      4/ If open-sourcing were the answer, we’d all be using some form of Linux operating system, and we’re not. Nevertheless, the absolutely fantastic achievement that is Wikipedia — warts and all — suggests that no one can rule out the possibility that sooner or later we will see massive sources of good-or-better contract forms and precedents online, searchable, well-indexed, well-“commentated,” and free.

      5/ Ken is right that grooming a contract takes time, but imagine an army of people familiar with “MSCD style” grooming online forms and precedents a little each day for several years, and offering comments, and rating forms and each others’ comments. Surely the cream would rise to the top.

      6/ With the explosion of self-representing people, it’s probably better not to bemoan the, ahem, mixed quality of online precedents like EDGAR and others, but instead to upgrade legal resources for nonlawyers. I used to teach business law to undergraduates who were aiming at careers in businesses, so I know very well that smart, ignorant people can quickly become smart, knowledgeable people.

      • Hear hear!

        Leaving the hypothetical MCSD army aside, I’d take exception only with your comment on Linux.

        Linux and other open-source Unix operating systems dominate the server space. The greater part of the online services we all use are provided by Linux machines. Few of us run Linux directly, but I’d wager the majority of computing power serving any one of us is performed by Linux machines.

        Proprietary systems like Windows and OS X still dominate on personal devices, but those personal devices are increasingly “thin clients” relegated to playing nice with hardware and running a web browser. We’re starting to see Linux encroach on that space in the form of Android and ChromeOS.

  2. Hello Ken, thanks for pointing out the risks and problems with crowdsourced legal documents. Since my little GitHub-based project was mentioned in Margaret’s original article, please allow me to share some background.

    While the “legal docs Britannica vs Wikipedia” discussion is interesting, the implication of “GitHub for Lawyers” is more than that:

    1. Crowdsourcing of legal docs in the open source style. In my experience this means most work is done by an individual or small team, and there is a long tail of small contributors.

    Open source did not bring the end of professional programmers nor a world where everyone can code. In fact programmers have never been more sought after. The same might happen for professional lawyers.

    2. Better tools to edit legal documents. GitHub is used by many companies to build commercial software, not because it’s open source or free of charge (of which GitHub is neither), but because it’s a superior productivity tool.

    Last year I built a prototype to host legal docs on GitHub because, as a programmer, I am more productive using git than word processors to view file revisions and do collaborative editing. This recently got me thinking: If legal docs are viewed as source code, what other tools from software engineering would benefit legal doc authors?

  3. Systematically grooming contract language requires way too much effort.

    What if all the mechanical time-sinks—checking references, defined terms, keeping formatting consistent, etc.—were handled automatically? Any thoughts on where the cost breaks even, and review becomes viable?

    • The problem isn’t the ministerial stuff, it’s figuring out what to say and how to say it. For those who have a decent idea what they’re doing, it’s manageable. But I, for one, have no interest in doing it for free.


Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.