Over the last two semesters, our lab, the Peace and Justice Psychology Lab, has tried to replicate a finding from a 2007 article (Sommers & Norton, 2007) that showed a racial bias in excluding jurors during jury selection. They showed that black potential jurors were struck from jury selection more than white jurors, a finding that generalized from college students to law students and to attorneys. This finding matched the findings of a report issued last year that documented race bias in jury selection in Louisiana over the last decade.
We contacted Dr. Sommers to ask about his methods, and had a good conversation with him. One of the comments he made was that we needed to pilot test our juror profiles extensively to find ones that were effective in making the jurors undesirable. We did that (but only tested four different profiles). We asked him for the images he used in his study, but it was a long time ago and he couldn’t find his exact materials.
So, we set about selecting juror images from a database of facial images – one black and one white – that were matched on gender, age, expression, attractiveness, etc. We settled on two images of middle-aged women.
The replication failed spectacularly: There was no difference between the rates of striking black and white jurors. If we want to interpret our finding optimistically, we might say that students at Southern Arkansas University are impervious to racial bias. Realistically, I believe the replication failed because the materials were not identical. That tells us that the effect has something to do with factors other than race. For example, it is possible that Sommers & Norton used images that were not matched on attractiveness, and the black juror’s image simply was less attractive. Sommers used male juror profiles and images, and there is a pervasive (replicated) effect that black men are typically viewed as more threatening than white men by research subjects, which may not have affected our participants’ impressions of the female jurors. If we replicated with male jurors, we might get the same effect.
The one finding we did get was that people high in Social Dominance Orientation (SDO) (which is a preference for hierarchical power structures, traditional social roles, and status quo among other things) struck the jurors (of both races) at a higher rate than those low in SDO. I believe that this reflects a gender role preference consistent with SDO. To wit, the jurors were women, middle aged, with no children, who had advanced degrees and a job as journalist. This may violate the gender roles that our culture typically assigns to women as homemakers, child bearers, etc., and those high in SDO might be more sensitive to this violation of social roles and want to “punish” the women. We have not directly tested this hypothesis, and I am not sure we will try.
I was motivated to blog about this because the social psychologist Jay Van Bavel wrote an opinion piece for this Sunday’s New York Times about the replication crisis. He addresses the difficulty with conducting direct replications of previous findings in social psychology because of the effect of context: there are lots of factors going on in the social environment that may be related to whether a replication fails or succeeds. But, I think that helps us understand that the effect is not as robust as only one, unreplicated, research finding might suggest.
Because it is hard to recreate the exact conditions of the original research.
Source: Why Do So Many Studies Fail to Replicate? – The New York Times
Sommers, S. R., & Norton, M. I. (2007). Race-based judgments, race-neutral justifications: Experimental examination of peremptory use and the Batson challenge procedure. Law and Human Behavior, 31, 261–273.
