Legal Engineering and Brian Leiter
Brian Leiter’s nicely done critique of the current state of empirical legal studies (ELS) has stirred up a neat micro-flurry of comment in the past few weeks. The fascinating ELS project of taking statistical approaches to legal data (see e.g. here) as a way of approaching the system overlaps a bit on what RRH is pursuing in its work. So, it’s worth addressing how we want to do it differently here.
The real question — his first one in the post — that I think is most relevant, goes like this:
First, too much of the work is driven by the existence of a data set, rather than an intellectual or analytical point. But the existence of a data set then permits a display of technical skills, which is satisfying to those with a technical fetish. But for everyone else, the question remains: why does this matter? why should one care? and so on.
He’s right. Despite the awesomely badass work being done, the “big deal problem” is still the elephant in the room for all things having to do with merging quantitative approaches with the law. Why is this worth paying attention to?
It’s the opinion of the senior (robotic) partners here at the firm that the problem definitely isn’t — as some people have claimed — because of the academic quibblings of the political science community at the heart of ELS. And it isn’t either, as Leiter himself mentions, due to the fact that ELS loyalists are less quantitatively sophisticated than they could be (though that’s definitely a valid observation as well).
Instead, the problem has to do with the academic context in which ELS is trying to get involved. Empirical legal studies is trying to use quantitative methods to tackle an essentially non-quantitative discussion, what Leiter calls “the central normative and conceptual questions of legal scholarship and legal education” that undergird the established scholarly community. That’s a fundamental disconnect, and a losing battle to boot.
The real value of empirical work isn’t in tackling the “normative and conceptual.” The real immediate value is practical and positive — modeling the measurable realities of wins, losses, and everything in between. Like baseball statistics before the Oakland A’s — research has accumulated a huge gold mine of potential applications and insights, but currently lacks a community of tools and practitioners to deploy them. We need more of an engineering outlook: people experimenting with the models that have emerged from ELS and using quantitative methods to creatively design applied solutions to measurable questions and problems.
I’ve been thinking about Leiter’s post for a while, and I’m glad to see others have as well.
I think the way you’ve restructured the question still doesn’t answer it.
The tricky part is creating those “measurable questions and problems” so that they would be meaningful or interesting.
Saying that ELS and quantitative analysis are good tools to answering the question: “does this policy work” is not enough.
The essence of Leiter’s critique is that the measurements by which we count for success or failure are currently too complicated and theoretical to be easily measured quantitatively. While giving obvious answers to these questions is easy, ELS should try to find measurements that are BOTH quantifiable and theoretically compelling. I agree with Leiter that few have done so.
Hey Ayelet — it’s been awhile, how goes it?
I think you’re making a fair point, and I’d definitely agree with what you’re saying. Part of the problem is essentially that the academic dialogue at present really doesn’t easily engage problems that are quantifiable in a compelling or useful way — as you mention they’re way too complicated and theoretical.
In some sense, my answer to that situation is a bit of a surrender and a bit of a redirect. It’s a surrender insofar as it interprets the fact that tons of brilliant academics can’t come up with simultaneously quantifiable and theoretically compelling work in ELS as an indication that it might truly be a dead end because of the academic norms/questions at work.
It’s a redirect insofar as I think there’s much lower hanging fruit that’s equally exciting with the work coming out. ELS doesn’t necessarily have to engage the highly complex/theoretical, and might actually be better benefited carving out its very own problem space.
Thoughts?
All’s well. Still a grad student, and it ain’t gonna change soon 🙂
I see your point about the “too complicated” questions, but I wonder how many of the lower- hanging fruit are actually hanging out there.
Criminologists, sociologists and political scientists have been dealing with socio-legal questions for a long time. A lot of what we theorists might call low-hanging fruit have already picked by these disciplines, which have taught us all the empirical data we currently know about the legal system (and we know a lot). So you need to find fruit that are both low and were not picked yet. I don’t know how many of those there are.
On a brighter note, I don’t think that the “sophisticated quantification” is a dead end.
I don’t think that it would be accurate to say that tons of brilliant academics tried to come up with great quantifiable measurements and failed. Most legal scholars until very recently didn’t aim at this direction at all. Until now, ELS was very marginal in law faculties, and most empirical folks found their home at one of the other disciplines grouped in Law and Society, which use their own distinct tools.
ELS is a very new endeavor in that regards. It asks lawyers to do quantitative work without using the well known disciplines. And as such a baby-discipline (this year at Yale they’re going to have the second annual meeting) it might be able to square the circle.
What do you say?