PROFESSOR Bartlett has written a bold article pushing back against what might be called inchoate or half-hearted empiricism.1 The half-hearted empiricist recognizes the value of empirical evidence to help solve a legal problem but, for whatever reason, fails to acknowledge the complexity and uncertainties of the evidence and as a result offers haphazard prescriptions. Professor Bartlett's article demonstrates what a whole-hearted commitment to empiricism looks like: it involves an engagement with primary sources rather than a reliance on secondary sources (or tertiary sources in the form of law review summaries of secondary sources), a review of research relevant to a problem rather than a review of a subset of research focused on one particular aspect of a problem, and a struggle to find usable prescriptive lessons in a literature that ranges from basic-level research with little obvious real-world application to applied research that can be so situation-specific that its generalizability can be questioned. Whole-hearted empiricism is hard, messy work, as well as a frustrating undertaking. Unlike theory- or model-based approaches to policy that self-consciously simplify the world and avoid many empirical complications, an evidence-based approach must confront empirical complexities and try to find meaning in the midst of ever-changing empirical evidence, imperfect studies, and contradictory findings located among the fragmented social sciences. Consequently, rarely will consensus exist on the prescriptive meaning of the assembled evidence. It should thus not be surprising that, after saying a few words on the dangers of half-hearted empiricism, I raise some questions about the empirical conclusions and recommendations offered by Professor Bartlett and suggest that her attempt to foster good intentions might more usefully be focused on firms rather than managers.
A LITTLE EMPIRICISM CAN BE A DANGEROUS THING
The present concern among employment law scholars about implicit bias as a cause of workplace discrimination no doubt arises from good intentions: the desire to inform legal policy using the latest empirical evidence on why minorities and women may continue to lag behind white males on various aggregate indicators of workplace outcomes. When prominent psychologists, sociologists, and science writers proclaim unconscious biases against minorities and women to be a likely cause of these workplace disparities, and when these empiricists partner with prominent legal scholars to provide helpful overviews of the relevant empirical research in top law reviews,2 it is understandable why Professor Bartlett and other legal scholars start from the assumption that much workplace discrimination today involves subtle behaviors motivated by implicit bias. Where Professor Bartlett departs from some others working in this area is in going beyond the empirical research on implicit bias to ask how this work fits with empirical research on motivation and the debiasing of workplaces. From a review of this additional research, Bartlett concludes that some of the prescriptions offered by legal scholars are likely to be ineffective against implicit bias and, more distressingly, counterproductive by undercutting the intrinsic motivation of managers to act without bias. If Bartlett is right, then these well-intentioned scholars are advocating policies that not only may fail but may make matters worse.
The potential negative consequences of inchoate empiricism become more serious when such empiricism influences legal opinions. For instance, the dissenters in Ricci v. DeStefano embraced the use of assessment centers as a selection tool that they considered likely to have less adverse effects than written tests of the kind used by the New Haven Fire Department.3 What these justices failed to acknowledge, however—perhaps because they limited their research review to the incomplete information contained in the appellate record—is that there is little systematic evidence that assessment centers reliably reduce adverse impacts on African American candidates. A recent meta-analysis (or systematic quantitative review) of twenty seven studies examining subgroup differences in assessment center evaluations found "that Black-White mean differences for assessment centers are not as small as has been suggested by a number of researchers" and concluded that much more research is needed on this important topic.4 Thus, if municipalities turn to assessment centers in hopes of avoiding the problems experienced by the City of New Haven, they may be sorely disappointed.
Half-hearted empiricism may be a function of deference to empirical experts pushing promising theories, the effort to say something fresh in the crowded legal scholarship market, or the opportunistic use of studies that support one's preferred causal or remedial theories. Whichever is the case, the result is likely to be the same: important qualifications to the empirical research will be neglected and inapt prescriptions offered. A sincere commitment to the formulation of evidence-based antidiscrimination policy requires a hard look at a wide range of empirical research on both the causes of discrimination and its cures, and it requires that scholars understand the limits of this research to avoid oversimplifying complex problems that may have context-dependent or yet-to-be-found solutions.
A LOT MORE EMPIRICISM MAY STILL NOT BE ENOUGH
Of course, whole-hearted empiricism is no guarantee of good policy either, given the contested and incomplete nature of much of the empirical evidence. Consider the key, interrelated empirical claims in Professor Bartlett's discussion of how to regulate workplaces to reduce the impact of subtle, even unconscious, biases on human resource decisions: (1) stronger legal prohibitions against discrimination are likely to crowd out intrinsic motivations to treat all workers equally, leading to greater overall discrimination because of the difficulty of proving discrimination in a world of subtle discriminatory behaviors motivated by stereotypes and prejudicial attitudes operating beneath or at the fringes of consciousness;5 (2) recent research points to effective means to reduce subtle discrimination that will not crowd out positive intrinsic motivations.6 A closer look at the underlying research raises questions about whether Professor Bartlett's proposed workplace policies will work as advertised.
Drawing on the literature on norm internalization and intrinsic motivation, Bartlett argues that strong legal sanctions against subtle, unconsciously-motivated discrimination may be seen as unfair and intrusive, may compromise managers' feelings of self-determination with respect to fair treatment, and may thus "crowd out" intrinsic motivations to act without bias.7 Bartlett proposes that workplace interventions be designed to "crowd in" positive intrinsic motivations by creating situations that facilitate positive intergroup interactions, encourage the adoption of egalitarian norms in these interactions, and permit managers and co-workers to feel that they acted fairly because it was the right thing to do rather than to avoid punishment. Of course, the devil is in the details for the organization or policymaker looking for guidance on how to make this broad proposal a reality. And when we turn to the research that Bartlett discusses to guide the formulation of policy, it is not clear that it provides sufficient or consistent guidance.
Much of the research on the reduction of implicit bias that Bartlett discusses has not yet been shown to be effective in workplaces, as Bartlett acknowledges when noting the potential value rather than the proven track record of this work. Reliance on largely experimental work is necessary because surprisingly little research has examined what actually works inside organizations: "Whereas there has been a great deal of research on the sources of inequality, there has been little on the efficacy of different programs for countering it. . . . We know a lot about the disease of workplace inequality, but not much about the cure."8 Ideas that look good in the laboratory when tested on college students making low-stakes or hypothetical decisions in limited interactions may run into considerable complications when moved to real workplaces. A perennial problem in human resource management is convincing practitioners to embrace practices that have not been field tested and shown to be successful within their industry.9 To the extent that many of the practices Bartlett recommends would need to be adopted voluntarily rather than imposed on a company, the evidence that Bartlett presents is unlikely to be persuasive to many HR managers.
Furthermore, it is not clear that all of the research that Bartlett touts fits with her crowding-in thesis. In particular, the organizational research by Kalev and Dobbin finds that the centralization of authority and responsibility for diversity planning and the imposition of affirmative action plans are among the most effective means to improving the number of women and African Americans in private sector management.10 These practices involve external pressure and oversight of managers by others inside the firm. It is certainly possible that successful diversity officers and affirmative action plan administrators impose their directives in ways that do not interfere with ordinary managers' feelings of autonomy and competence or intrude on managers' intrinsic motives to enhance diversity, but that remains an open question. Professor Bartlett's article offers good reasons to be cautious about legal proposals that employ the simple economic logic that increased external sanctions can compensate for the low probability of detection and punishment of unconsciously motivated discriminatory behavior. Such sanctions may cause some managers to increase their supply of unbiased behavior, but they may backfire in some cases and cause managers who would otherwise act fairly to reduce their supply of unbiased behavior. Unfortunately, the net effects of increased external sanctions are difficult to estimate from the existing evidence.
Professor Bartlett focuses her inquiry on how legal sanctions may affect managerial norm internalization and intrinsic motivation, but her argument can usefully be scaled up to the level of the firm. As I see it, the central crowding-out problem in antidiscrimination law today is that even those companies that may want to engage in the sorts of self-critical review and program evaluation necessary to determine whether their antidiscrimination and diversity-enhancement policies are working as desired have strong externally imposed incentives not to do so. Evidence generated by in-house audits and organizational culture reviews may be subject to discovery, adopting evidence-based policies proven to reduce workplace disparities will not immunize companies from attacks by plaintiff experts who can always find some aspect of a personnel process to criticize, whether merited or not, and there are no formal legal benefits from engaging in self-critical review.11 Companies are in the best position to detect and correct workforce disparities. We need to create a set of legal rules and policies that reward internal monitoring and self-correction and that penalize deliberate ignorance about the discriminatory impact of a company's personnel policies.12 In short, we need to consider how best to crowd in positive intrinsic motivations and crowd out negative intrinsic motivations at the firm level, perhaps more so than at the managerial level.
Preferred citation: Greg Mitchell, Good Scholarly Intentions Do Not Guarantee Good Policy, 95 Va. L. Rev. In Brief 109 (2010), http://www.virginialawreview.org/inbrief/2010/02/28/mitchell.pdf.
[Back to this issue]