You are currently browsing the archives for July, 2010
These papers have been floating around for awhile now, but it’s worth reposting again (pdf alert):
Needless to say to anyone familiar with the research, it’s clear that the number crunching indicates that two simple things. A) judges render decisions in a predictable way over time based on a not-overly complex set of factors, and B) quantitative approaches tend to systematically beat out legal “experts” at predicting which way a case will go (i.e. experience does not necessarily win out over calculation).
Most of the work has been done with the Supreme Court, which is data rich, but unfortunately pretty difficult to apply for 99.9% of work being done by lawyers. But what’s exciting about judicial prediction is that you might eventually be able to turn entire parts of the legal business into a numbers game, particularly as processing cases become more automated.
Example: say you wanted to aggregate a sufficient caselaw to challenge a particular doctrine — we could work to identify and aggregate a portfolio of cases to test against judges that we know statistically tend to give a thumbs down on a particular argument/doctrine/issue. So long as the rate of this targeting happens faster than the random cases entering the system, you could engineer an appellate body of cases that support an eventual argument one way or the other. That’d be pretty neat (though expensive). Even if the strategy wasn’t activist in nature, in the very least a model like this could give a prediction around the support around one argument or another over time, given the average case inputs into the system.
Thinking through the infrastructure to set something like this up, so RRH is currently looking for datasets that might be relevant to doing this kind of processing. Stay tuned!
Brian Leiter’s nicely done critique of the current state of empirical legal studies (ELS) has stirred up a neat micro-flurry of comment in the past few weeks. The fascinating ELS project of taking statistical approaches to legal data (see e.g. here) as a way of approaching the system overlaps a bit on what RRH is pursuing in its work. So, it’s worth addressing how we want to do it differently here.
The real question — his first one in the post — that I think is most relevant, goes like this:
First, too much of the work is driven by the existence of a data set, rather than an intellectual or analytical point. But the existence of a data set then permits a display of technical skills, which is satisfying to those with a technical fetish. But for everyone else, the question remains: why does this matter? why should one care? and so on.
He’s right. Despite the awesomely badass work being done, the “big deal problem” is still the elephant in the room for all things having to do with merging quantitative approaches with the law. Why is this worth paying attention to?
It’s the opinion of the senior (robotic) partners here at the firm that the problem definitely isn’t — as some people have claimed — because of the academic quibblings of the political science community at the heart of ELS. And it isn’t either, as Leiter himself mentions, due to the fact that ELS loyalists are less quantitatively sophisticated than they could be (though that’s definitely a valid observation as well).
Instead, the problem has to do with the academic context in which ELS is trying to get involved. Empirical legal studies is trying to use quantitative methods to tackle an essentially non-quantitative discussion, what Leiter calls “the central normative and conceptual questions of legal scholarship and legal education” that undergird the established scholarly community. That’s a fundamental disconnect, and a losing battle to boot.
The real value of empirical work isn’t in tackling the “normative and conceptual.” The real immediate value is practical and positive — modeling the measurable realities of wins, losses, and everything in between. Like baseball statistics before the Oakland A’s — research has accumulated a huge gold mine of potential applications and insights, but currently lacks a community of tools and practitioners to deploy them. We need more of an engineering outlook: people experimenting with the models that have emerged from ELS and using quantitative methods to creatively design applied solutions to measurable questions and problems.
Though our senior partner Athena Cluster has always been skeptical of the praise and acknowledgment of mere humans — we’re pleased that Robot Robot and Hwang has been received so much great coverage since its launch.
Thanks guys! We’ll try not to let you down.
Been talking about it in abstract for awhile, but interested to finally see that we have a real, live case study of courts being hit with a type of denial of service attack through leveraging some simple automated legal systems, here in the debt collections world:
Collection law firms are able to handle such large volumes of cases because computer software automates much of their work. Typically, a debt buyer sends a law firm an electronic database that contains various data about consumers, including name, home address, the outstanding balance, the date of default and whether interest is still accruing on the account.
Once the data is obtained by a law firm, software like Collection-Master from a company called Commercial Legal Software can “take a file and run it through the entire legal system automatically,” including sending out collection letters, summonses and lawsuits, said Nicholas D. Arcaro, vice president for sales and marketing at the company.
No group has definitive statistics on debt collection lawsuits, but federal regulators, collection lawyers and judges say the numbers have increased and are straining the court system.
Interested in seeing how the courts respond to this type of tech going forwards — though largely it’ll just tend towards incentivizing more sophisticated automation/outsourcing rather than stuffing the genie back in the bottle, I think.
