We have revised our Best Law School rankings that appeared in the February issue of The National Jurist to account for two errors and better data from RateMyProfessors.com.
The University of Chicago and Boston College initially brought to our attention that their RateMyProfessors grades were incorrect. They, along with other schools, also pointed out that non-law professors were included in the tabulation of the scores.
This led to a thorough review of every law school’s grade for this data point, as well as a review of the overall GPA. We compared RateMyProfessors grades with each school's current list of faculty, as available on their website. The new scores are based on current law faculty only.
After the update, 33 percent of RateMyProfessors grades did not change; 16 percent shifted to “no grade” on account that there were fewer than 40 evaluations completed for the school; 33 percent changed by less than a grade; 14 percent changed by more than a grade but less than two grades; and eight schools — or 4 percent — changed by more than two grades.
These schools, in order of the size of the change, were: University of Chicago, Georgia State, University of Texas, Boston College, Northwestern, Wake Forest, University of Southern California and Duke University. All of these schools saw their overall GPA improve.
The error with the University of Chicago grade was related to a data entry mistake on our part. Even if we had not reviewed the RateMyProfessors data for all schools, the University of Chicago's placement would have improved dramatically. The rest of the changes are related to the more thorough and accurate review of the RateMyProfessors data.
We deeply regret these errors and are taking several steps to ameliorate the erroneous information. This includes, but is not limited to, republishing the correct data in the digital and iPad version of the February issue; and publishing the correct data in the March print edition of The National Jurist.
Several critics have suggested that we exclude the RateMyProfessors data altogether because it was either not widely used by law students or tended towards more negative reviews. We considered this option but chose to retain this data set. There were more than 35,000 evaluations completed of current law professors, with an average of 175 per school. The average was significantly higher for schools with more than 40 evaluations — 246. Also, the average score is 3.54 on a 5-point scale, with almost all law schools falling between 3.0 and 4.0. There are studies that are helpful in better understanding the usefulness of RateMyProfessors.com scores. They are listed at the end of this story.
We also identified that we had made a calculation error with Notre Dame Law School. The correction significantly improved that school’s overall GPA.
We deeply regret the errors and our failure to exclude non-law professors from the original RateMyProfessors scores. We take accuracy and fairness very seriously and pride ourselves on producing the highest quality content for our readers. We fell short of our standards in this case and are taking all necessary steps to correct the errors, be responsive to our readers, and ensure we do not repeat similar mistakes in the future.
We appreciate the many suggestions for improving the study in the future and invite further comments.
-The National Jurist Editors
Research papers on RateMyProfessors.com:
April Bleske-Rechek and Kelsey Michels. “RateMyProfessors.com: Testing Assumptions about Student Use and Misuse.” Practical Assessment, Research & Evaluation 15, No. 5 (May 2010)
Theodore Coladarci, Irv Kornfield, “RateMyProfessors.com Versus Formal In-class Evaluations of Teaching,” Practical Assessment, Research & Evaluation (May 2007)
James Felton, Peter Koper, John Mitchell, Michael Stinson. “Attractiveness, Easiness and Other Issues: Student Evaluations of Professors on RateMyProfessors.com.”
Scott Jaschik, “Validation for RateMyProfessors.com?” Inside Higher Ed, April 25, 2008
James Otto, Douglas Sanford Jr., and Douglass Ross. “Does ratemyprofessor.com really rate my professor?”, Assessment & Evaluation In Higher Education 33, No. 4 (August 2008)
Michael Sonntag, Jonathan Bassett and Timothy Snyder. “An Empirical Test of the Validity of Student Evaluations of Teaching Made on RateMyProfessors.com,” Assessment & Evaluation in Higher Education (July 2008)