Rebuttal: LSAT’s predictive value despite diversity issues

By Robert Steinbuch

Aaron Taylor's recent piece here on diversity in law school admissions discusses racial disparities in outcomes in legal education.  While Taylor recognizes that this could be caused by "differences in applicant quality" and that there are "[v]ast racial and ethnic LSAT score disparities," Taylor contends that the "value [of the LSAT] is often inflated by law schools."  I present some additional data to consider. 

After offering a few studies in support of his thesis, Taylor states that "[s]tudies of empirical relationships . . . are uncommon in legal education. In the absence of this data, inflated assumptions about the LSAT’s power pervade, to the inequitable detriment of many applicants, most profoundly black applicants."

Taylor may be unaware of my coauthored 2016 study of admissions data from my school -- the University of Arkansas at Little Rock, William H. Bowen School of Law — which analyzed the correlation of LSAT and undergraduate GPA with bar passage.  It was published in the Texas Review of Law and Policy.  Specifically, that study evaluated Bowen data from 2005 to 2011; Taylor served as Bowen's admissions dean from 2006 to 2011.  

The underlying data from my article show the following distribution of bar passage at my school segmented by LSAT score bands for the time period analyzed:

The chart shows that the correlation between LSAT performance and Bar-exam passage is noteworthy.  For sure, this relationship is complex.  Indeed, evidence supports the conclusion that intra-school relative LSAT scores are even more important than absolute scores.  But, either way, LSAT scores are very useful.  

The problem with most anti-objective-factor evaluation advocates is that they typically don't offer a viable alternative to considering the index score (the LSAT score and undergraduate G.P.A.) of applicants.  Indeed, Richard Sander's seminal piece on the mismatch phenomenon, published in the Stanford Law Review, aptly describes the predictive value (35%) of the LSAT and undergraduate G.P.A. as "impressive."  And Sander highlights that no other predictor explained more than 5% of outcomes.  By way of comparison, describes Sander, cigarette smoking -- a known carcinogen -- explains a mere 4 percent of longevity.

There are certain indisputable truths about law-school admissions:  If a school enrolls a cohort of students with very low academic indices, as a group, their overall bar passage will be low; by examining the average academic indices for any sizeable group of students, we can determine their average performance with a fair amount of precision; and the predictive value of academic indices on bar-exam success doesn't appreciably vary across cohorts. 

A difficulty with focusing on enrollment outcomes in law schools that don't correspond to the cohorts' presence in the general population or the size of the group's application pool is that it suggests a tacit advocacy for the use of quotas in admissions.  But quotas rightly remain contrary to established Supreme Court precedent.

No single measurement, nor even the combination of the most common metrics, can fully predict which applicants will ultimately graduate law school and pass a bar exam.  But the absence of perfection quite obviously doesn't translate into a lack of usefulness.  To conclude otherwise would result — in the context of medicine, for example — in patients that wouldn't bother with virtually all testing (e.g., blood counts, EKGs, and X-rays).  In both contexts, that's bad medicine.