Peer Review and the Legal Academy

The story making the rounds late last week about a peer-review ring recently busted by academic publisher SAGE was fortuitously timed, as it coincided with the completion of my first referee’s report for a legal academic journal.  This led me to some (perhaps unoriginal) reflections on the state of legal scholarship.

I’ve been thinking about the problems of legal scholarship longer than I’ve been an actual legal scholar. My primary editorial contribution to the Harvard Law Review as a third-year law student was to serve as co-chair of a symposium that invited some fairly high-powered navel-gazing on the topic.  (My own contribution was in fact far less substantial than that of my co-chair Chris Anderson, who conceived the topic and took the lead in soliciting contributors).  One thing that strikes me now about this symposium is how little attention the panel gave to the question of peer review.  To be sure, it was mentioned, but mainly in the context of wariness (or celebration) over the increasing interdisciplinarity of the legal academy, its growing resemblance to other academic disciplines, and its increasing distance from its professional roots.

In 2002, when these essays were published, peer-reviewed legal journals were still firmly a minority interest, largely confined to law-and-economics.  But things have been changing.  Then, the JD-PhD legal interdisciplinarian was still a relative rarity; last year fully a quarter of all self-reported entry-level tenure-track hires held a doctorate in a non-law field.  Two years after his more measured and historically oriented comments at our symposium, Judge Posner lashed out at student-edited law reviews in print. Over time, scholars from other disciplines chimed in, arguments pro and con were reviewed at length, and prominent scholars registered their affiliation with the forces of peer review.  Of course, there was contrarian sniping and chin-scratching along the way.  And over time, the student-edited law reviews have proven remarkably difficult to displace–a case study in first-mover advantages, collective action problems, and switching costs.

During the course of my still-quite-brief career, I have watched the ground move under my feet towards a legal academy that more closely resembles other academic disciplines.  Like many law professors, I often say that if I were on the entry-level market today I’d never stand a chance of getting hired.  My lack of formal training in the non-legal disciplines that inform my scholarship is a big reason why (the savage economic pressures on legal education are another, perhaps more important, reason).   I have felt my lack of hands-on experience with the institutions and practices of non-legal disciplines more keenly with each passing year, and I’m always looking for ways to remedy that deficit.

So when I was recently asked to review a submission to a peer-reviewed law-and-economics journal for the first time, I agreed, albeit with some misgivings.  If the primary knock on the student-edited law review is that it entrusts editorial control–and particularly article selection–to students who lack the necessary experience, I wasn’t sure that a non-economist referee like myself was so vast an improvement. But I felt that the invitation was an opportunity, and that it was incumbent on me to at least try to make good on it.

Fortunately for me, my worries turned out to be unwarranted in this case.  Because the piece I was tasked with reviewing was not very well argued or supported, I felt very little difficulty discharging my duties as a referee.  Frankly, I was reminded of my days as a 2L law review editor, when the most junior members of the law review staff were burdened with the task of sifting through and summarizing the thousands of anonymous manuscripts submitted for publication each year, and deciding which ones to flag for closer review by a (more senior) articles editor.  This was not as difficult a task as it sounds, partly because HLR’s standards were very high, but mainly because the overwhelming bulk of the submissions were, to be blunt, crap.  It seems that the same may be true for at least some of the articles submitted to peer-reviewed journals.

This was just one assignment.  Lessons to be drawn from it are obviously limited.  But if my experience is in any way typical of the peer-reviewed law journal referee, I have to say I am far less certain about the arguments in favor of peer-reviewed journals than I would otherwise be:

• If peer review is supposed to serve some quality-assessment function, I don’t see why a significant part of that function cannot be carried out by conscientious law students.  I suspect we can rely on student law review editors, leaning on their faculty advisors when necessary, to filter out much of the chaff in legal academic writing, at least below a certain threshold.  And I think we can rely on what Will Baude calls “post-publication peer review” to do the rest.  Blogs and social media–and particularly purpose-built blogs like JOTWELL–seem to serve this purpose quite well.
• If the point of peer review is to improve the overall quality of scholarship, I’m unconvinced. I have found early circulation of drafts and participation in workshops to be quite adequate to the task of improving my works-in-progress based on feedback from my peers. (Admittedly, I work in a field–intellectual property–that is especially active, cooperative, and collegial.) Indeed, this disintermediated scholarly conversation appears to be the way the world is going (and not just on law faculties).  In the meantime, the imperative of “pitching” sophisticated ideas to a less-sophisticated audience of graduate students seems to me to incentivize better, clearer, more complete exposition of those ideas than would obtain in a private conversation among specialists.

Of course, both of these functions require initiative and humility on the part of authors, and effort and goodwill on the part of those involved in evaluating scholarship–both before and after publication.  These can be serious obstacles: nobody likes to volunteer for criticism; advising law review editors is an often thankless burden on busy faculty; and it can be difficult to promptly and thoroughly respond to legitimate requests for comment from colleagues–I’ve often failed in this regard myself.  But it seems to me that these obstacles would still exist in a world where formal peer review was the norm, they would just manifest themselves in different ways, and with less transparency (see the SAGE investigation I noted at the outset).

Conversely, if the legal academy did move wholesale to the peer-review model that prevails in other disciplines I think we would lose a key function that law reviews historically served (if only with varying success)–the education and professional socialization of the students who edit them.  I know that my service as a law review editor was far and away the most useful element of my preparation for life as a practicing lawyer and as a legal academic–far more useful than anything else I did as a law student, including my summer internships.  And I don’t see how this “perhaps most important” feature of the traditional law review gets replaced in a world where law professors rely entirely on one another for the editorial functions of the scholarly enterprise. For all the complaints these days about the impracticality of contemporary legal education and scholarship, I’d hate to see an institution I view as having great practical educational advantages abandoned for the uncertain promise of marginally better reputational metrics or marginally more structured professional engagement with pre-publication scholarship.

So while I’m happy to make room for peer-reviewed journals in the big and growing tent of legal scholarship, I don’t see any need for them to supplant student-edited reviews as the leading, let alone the only, journals in the field. The historical accidents that produced today’s legal academic journals have generated some extraordinary opportunities for many of our students; I for one find it very difficult to justify making those opportunities scarcer or less meaningful.

Tags: legal educationlegal scholarshippeer review