Saturday, April 12, 2008

Was the Supreme Court in Grutter Misled About the Success of Michigan’s Race-Based Admissions Policies?
Gail Heriot

This is troubling. 

Back during the trial before Judge Bernard Friedman in Grutter v. Bollinger, Richard Lempert, Eric Stein University Professor of Law and Sociology,  testified as an expert on behalf of his employer, the University of Michigan Law School.  At issue, of course, was the constitutionality of the Law School’s admissions policy, which granted very substantial affirmative action preferences to black and Hispanic students.  Lempert assured the court that, despite these preferences, Michigan affirmative action beneficiaries had overwhelmingly been successful in passing the bar.  I have little doubt that Justice Sandra Day O’Connor found this information comforting in arriving at her decision upholding the University of Michigan’s authority to engage in discrimination for racial diversity’s sake.

“Our study finds that Michigan, just not to put too fine a point on it, Michigan graduates pass the bar. It doesn't matter, really, whether you're a minority or whether you're white. ... [I] think there might have been a statistically significant difference favoring whites, but it was substantively sort of completely trivial. It was like 95 percent of minorities and 98 or something or 99 percent of whites.” Grutter Trial Transcript at 57-58.

He testified similarly before the U.S. Commission on Civil Rights in 2006:

“I and colleagues did a major study of Michigan’s graduates, affirmative action graduates over a 27-year period. ... [W]e took special care to check for non-response bias and found considerable evidence that this was not a serious concern. ...  But, for example, at Michigan in the 1970s, 98.5% of our respondents graduated and passed the bar.  In the 1980s, 95.1 percent; 1990s, 96.1%, pretty much the same as our whites.” USCCR Briefing Transcript at 19.

On both these occasions, Lempert was testifying to the ultimate bar passage rate (defined as passage within four years), not the first-time rate.  And his figures were meant to be the bar passage rate for law school graduates who take the bar exam, not the rate for those who simply start law school (some of whom, of course,  never finish or, if they do, never take the bar exam).  Even so, the figures seemed a bit high.  Evidence from the largest data base available on law student bar passage rates nationwide painted a less cheerful picture.  And flaws in Lempert’s methodology suggested that, contrary to his testimony, non-response bias was a very serious concern.   

Now evidence has apparently come to light in connection with the litigation over Proposal 2 that suggests that the Michigan experience may have been seriously overstated.  According to UCLA law professor Richard Sander (who acted as an expert witness for Proposal 2 supporters in the recent litigation over that initiative), evidence obtained in discovery shows that the bar passage rate for white Michigan graduates is indeed very high–97.2% is the figure he quoted me.  But for black graduates, Sander reports that the figure is only 77%.  These figures are for ultimate bar passage and should therefore be directly comparable to Lempert’s figures. The figures for first-time takers would be lower–although I do not know by how much.

What accounts for the strikingly different figures between Lempert’s testimony and the data that Sander analyzed?  I cannot say.  I note that Lempert’s testimony refers to “minority” and “white” rates, while Sander contrasts “black” with “white.”  But that would not be enough to cause such a stark discrepancy.  I also note that Sander’s figures are for recent years, and Lempert’s are for the 1980s and 1990s, but the notion that the gap in bar passage rates suddenly ballooned in the post-Grutter world seems rather implausible.  Differing lengths of time in defining "ultimate bar passage" are also unlikely to cause such a large gap in the figures. If Sander’s figures are accurate, it really doesn’t look good for Lempert (and, in fairness, if Lempert’s figures turn out to be the more accurate, it won’t look good for Sander).

By the way, when intervenor Eric Russell sought further discovery to confirm Sander’s preliminary findings, defendant University of Michigan refused to comply.  The issue was instantly mooted when Judge David Lawson upheld Proposal 2.

| Permalink

TrackBack URL for this entry:

Listed below are links to weblogs that reference Was the Supreme Court in Grutter Misled About the Success of Michigan’s Race-Based Admissions Policies?
Gail Heriot


This affirmative action hooplah must be enormously to the advantage of those white students who do get into Law School, since a large chunk of the class will never, apparently, give them much competition in their careers. Handy, that.

Posted by: dearieme | Apr 12, 2008 9:13:05 AM

why wouldn't the non-black minority graduation rates be enough to distort the results? By including minority groups with presumably high pass rates, but who weren't necessarily the beneficiaries of affirmative action admissions, Lempert would have given a very misleading, if not outright false, depiction of how minority affirmative action students were doing. Not that anyone would ever attempt to manipulate statistics or testimony to help one's cause, right?

Posted by: steve sturm | Apr 15, 2008 10:16:30 AM

This suggests Lempert included non-black minorities who were not the beneficiaries of affirmative action admissions (Asians maybe?) in order to bolster the argument for affirmative action. This seems to me to be somewhat misleading, but not beyond the permissable bias of an expert witness. Opposing counsel should have drilled him on the numbers and clarified for the record exactly to what he referred.

Posted by: USD Law Student | Apr 21, 2008 5:38:11 PM

I attempted to replicate Professor Sander's work, which he said at a debate at NYU involved coding race from pictures in the UM Law School's freshman “Facebook.” There are problems with this mode of racial identification, as numbers of students don’t appear in the Facebook, picture quality is not always great, and the race of some students cannot be unequivocally identified. Moreover, the number of black students taking the bar and the number failing it are so small that a few coding errors can lead to large percentage differences. I don’t know if these caveats were made clear to Professor Heriot when Prof. Sander told her his results, and other important information also seems not to have been conveyed.

By adding some information and with the above caveats it is easy to defend the data I testified to in Grutter. Prof. Heriot suggests some possible reasons for the discrepancy, but her generous effort to explain it nevertheless misses the key point which may be why she is ultimately skeptical of my testimony. Prof. Sander was working with bar passage data from the years 2004-2006. Those who failed the bar in July 2006 had no chance to retake it within the time for analysis and those who failed in the winter of 2006 had only one chance, a chance I do not recall anyone having taken. This applies not just to black students, but to whites, Asians and Hispanics as well. If one simply looks at black graduates who took the bar during 2004 or 2005 and did not pass the first time, one finds that by 2006, 90.9% of them had passed a bar. Moreover, this is likely to be a low estimate of eventual bar passage since someone who failed for the first time in one of the states for which Professor Sander and I had data (which covers about 80% of UM Law’s bar test takers) may have retaken the bar and passed it in a state for which we did not have data. Add to this the fact that even within the data we have there are instances of people, white and black, passing the bar as late as a 7th try, and the fact that bar passage standards have been raised in many states, and there is no reason to question my testimony in Grutter, based on a study done with David Chambers and Terry Adams, that Michigan's affirmative action minorities (which does not include Asians) had about a 95% bar passage rate.

I might add that the Facebook data do not suggest that affirmative action has anything to do with the results Sander reports. Latinos, who benefited from affirmative action at Michigan, seem to have done as well or better on the bar than Michigan's white students, while the first time bar success of Michigan's Asian law students, who as a group did not benefit from affirmative action, is much like that of Michigan's black students (in two of the three years first time bar passage is almost identical).

I might also add that contrary to what Prof. Heriot writes non-response bias was not a very serious concern in my study with Adams and Chambers because we took great pains to explore this possibility. Indeed, I think this is a major reason why the Sociology of Law Section of the American Sociological Association awarded the article its prize for the best sociology of law article published over a two year period. Anyone interested in what we wrote and the care we took to ensure against non-response bias should read the original article. It was in the Spring 2000 issue of the journal Law & Social Inquiry. Professor Sander comments on our article in the same issue, and we respond to some of his criticisms, including his concern with sample non-response bias. Alternatively, if one clicks on Prof. Heriot’s link to “serious concerns” please read to the end because Professor Sander raises his concerns there and I answer them. One point made in the exchange is worth reiterating because it relates to the issue Prof. Heriot raises. He estimates that overall about 77% of Michigan’s black students in the sample Adams, Chambers and I worked with passed the bar. (This number is coincidentally the same as the figure Prof. Heriot gives for Sander’s current estimate, and I wonder if she has confused the two.) Yet the Bar Passage Study found that about 75% of all black test takers passed the bar. The rate of blacks starting law school in the BPS data who passed the bar is much lower because of drop outs. However, like their white counterparts almost all blacks who start at Michigan graduate. Thus Sander is arguing that Michigan’s black students, whose credentials must place them in the top 5% to 10% of all blacks attending law school have a bar passage rate no greater than the average bar passage rate of black students no matter what law school they could get into and graduate from. This is preposterous. The fact that Michigan’s black students do considerably better in passing the bar than the average black law school graduate is no more surprising than the fact that Michigan’s white graduates do considerably better at passing the bar than the average white law school graduate. In short there is no reason in Sander’s earlier analysis or in the information he recently gave Prof. Heriot to believe that Sandra Day O’Conner or the U.S. Supreme Court were given a misleading picture of how UM’s black law school graduates fare on the bar.

Rick Lempert
Eric Stein Distinguished University Professor of Law & Sociology
University of Michigan

Posted by: Richard Lempert | May 21, 2008 3:53:01 PM