The problem with dumping the LSAT

by Rob Steinbuch

Last year the Supreme Court sent back to the Court of Appeals for the Fifth Circuit for further consideration Abigail Fisher's claim for racial discrimination against the University of Texas. Fisher took issue with Texas's affirmative action program, which admitted minorities with lower objective criteria than she attained — while denying her acceptance. This week the Court of Appeals didn't change its outcome.

Fisher remains, so far, without recourse for having been denied admission over minority candidates with less substantial academic records. Many expect that Fisher's next appeal will lead to the Supreme Court fundamentally changing how affirmative action operates. Until then, however, the existing system and law persists. But the data show that affirmative action is not the panacea that it was hoped to be.

Cognitive dissonance theory teaches us that when confronted with information we don't like, it is a natural human tendency to try to ignore those facts. It is, perhaps needless to say, a better choice to confront reality squarely even when we don't like the results.

Public longitudinal data from the school in which I'm a tenured professor of law (the University of Arkansas at Little Rock School School of Law) show a dramatic difference in bar passage rates for whites and African Americans. Multi-year data, publicly disclosed after the Arkansas Attorney General informed the school that its prior refusal to produce the material pursuant to a Freedom of Information Act request was erroneous, showed that African Americans failed the bar on their first try twice as often as whites: 40 percent versus 20 percent. My school's numbers are far from unique; indeed, they are better than some others'. And, of course, first-time bar passage rates are lower than lifetime rates, but first-time passage rates directly relate to the overall outcome. So, any cohort starting with a high failure rate has a steep hill to climb.

The regression analysis bears out the sound instinct that race is not a predictive factor; it confounds (overlaps) with LSAT scores and undergraduate grades. In other words, the average standardized test scores and undergraduate performance differed measurably between enrolled African American and whites students, likely because schools are seeking to reflect percentages of cohort representation in the general population notwithstanding a smaller available applicant pool, which further diminishes as one moves down the hierarchy of schools. These variations predictably correlated with first-time bar passage.

For those like me, who have consistently lauded the goals of affirmative action, it's most unpalatable to learn that admission preferences often harm the very underrepresented minorities these policies are designed to help.

The data nationwide quite convincingly show that LSAT scores and undergraduate grades are solid predictors of success in law school, as well as on the bar exam. For example, approximately one-third of law student success can be explained through LSAT scores and undergraduate GPAs alone.  And particularly poor scores are even more predictive than that.

This is quite impressive scientific prognostication. A 33 percent predictive power when forecasting human action is terrific. There are some, however, who suggest that such factors are unremarkable because their explanatory force is not closer to 100 percent. Such arguments are misplaced. This is not chemistry, after all, where the same few inputs always produce the identical outcome. Humans are immensely complex, and their future is inherently murky. If one can predict a full one-third of students' success by means of LSAT and GPA alone — factors present before the want-to-be attorneys even get to graduate school — that’s gold!

Yet the notion of rejecting standardized testing has unfortunately gained some traction in academia. The mischievously named "Fair Test" movement proclaims: “[m]ore than 800 four-year colleges and universities do not use the SAT or ACT to admit substantial numbers of bachelor-degree applicants.”  I'm most familiar with law-school admissions — being the longest continuously serving member of my school's admissions committee — and there are some differences between undergraduate and graduate entrance considerations, but the concern over rejecting useful information persists in both contexts.

I'm also particularly uncomfortable with academics eschewing standardized exams based on the assertion that those devices are somehow unfair, while all along administering tests in their own classes. I am skeptical of the allegation that the former are inherently flawed, while the latter are not.  I suspect it reflects the "bias blindspot," as some Princeton researchers have characterized it, wherein individuals claim they're well attuned to spotting biases in others, while ignoring their own.

Moreover, the better question is not whether to ignore objective metrics, but what to do when we reject these useful measures. Other proxies, to the extent they have the potential to be fruitful, are already part of our full-file review. Indeed, law schools have expended incredible efforts to afford all students the opportunity to articulate, describe, and demonstrate the unique skills they've developed, the particular challenges they've faced, and their public-service activities and goals. For example, at my school, I'm proud to say that for at least the almost-decade that I've been there, every single applicant has received a full-file review. Indeed, many applications are reviewed by several faculty members. We rightfully have no applications that are summarily dismissed based upon LSAT scores and undergraduate exams alone.

When people reject statistics as being insufficiently powerful, they bear the burden of explaining what the better alternative is. And, while there is solid evidence that LSAT and GPA can predict future academic success, I have never seen the evidence that a committee of law professors can predict success by rejecting objective indicators — no matter how smart and well-credentialed they are. (Compare the belief that police interrogators can detect lies better than chance with the scientific evidence that says the opposite.)

To be clear, objective measures shouldn't be the only factors considered. But such an assertion, if ever made, would be a red herring, as the objective data almost never are the only information evaluated in admissions. And it's not to say that in certain contexts, standardized exams can't be skipped.  For example, I would be willing to look most favorably upon a law-school applicant who graduated with a 3.5 GPA from the Massachusetts Institute of Technology with a major in mathematics who didn't take the LSAT. But that's a rather thin reed. My guess is that this student would wind up doing quite well on the LSAT in any event.

But the underlying idea that standardized exams (and relatedly, at times, undergraduate GPAs) should be largely overlooked in certain, often inconvenient, contexts persists in some circles.

For law schools, dropping the LSAT is not even an option. The American Bar Association, the accreditor of US law schools, won’t permit it. The ABA mandates that every accredited law school “shall require each applicant for admission as a first-year JD student to take a valid and reliable admission test [LSAT] to assist the school and the applicant in assessing the applicant’s capability of satisfactorily completing the school’s educational program.”

Indeed, the ABA section on legal education wrote in 2012:  "The [debate over whether] to delete what, for shorthand purposes, will be termed the LSAT requirement, does not suggest, and indeed should not be taken to express a lack of confidence in that instrument or in the wisdom of using a fair and valid objective test as one key measure in winnowing a class of entering students from each school’s applicant pool. The LSAT can and does provide a fair measure of first-year law school performance and correlates well with the final law school grade-point average, rank in class, and performance on bar examinations.  It helps a law school identify promising performers.  It also helps schools avoid the admission of those who are not capable of satisfactorily completing the law school program. Given the enormous investment of resources that individuals make in securing a legal education, the LSAT offers would-be students some predictability when they make such an important investment."

The ABA does, however, permit law schools to admit up to 10 percent of an entering class without requiring the LSAT for those applicants that come from the undergraduate component of the same school or for joint-degree students. But in either case, these applicants must have taken another standardized entrance exam and have scored above the 85th percentile on the other exam, and these same applicants must have either ranked in the top 10 percent of their undergraduate classes or earned an undergraduate GPA of at least 3.5, both for six semesters or more. If it were up to me, I would happily waive the LSAT for all such applicants to my school, because those other criteria restrict the applicant pool to an elite academic few.

The denigration of objective measures perhaps is related to the conflict resulting from the intersection of culture, politics, and facts. A recent New York Times article referenced this phenomenon in the context of global warming, as has Vice President Gore in his so-titled work, "An Inconvenient Truth."

With the restrictions on affirmative action imposed by the Supreme Court, advocates of elevating race considerations in admissions appear to be searching for a way to accomplish their goals without subjecting themselves to the unfavorable gaze of an unsympathetic bench. Eliminating race-neutral objective measures may be one such way. That's unfortunate. Academics, in particular, should strive for full transparency of all preferential admission policies.  Only then can we ensure success in improving outcomes for all groups.