Expert Mining and Required Disclosure
In the social sciences, “data mining” sometimes refers pejoratively to the repetitive use of classical statistical methods to find “evidence” that results from only random variation. Various aspects of evidence and civil-procedure law disincentivize data mining by expert witnesses in federal civil litigation. But as many authors have noted through the years, resourceful attorneys can do data mining’s dirty work by hiring multiple experts, asking each to provide an expert report on the same issue, and then put on the stand only the one who provides the most favorable report. This practice is often referred to as “expert shopping” or “witness shopping.” To emphasize its analogousness to data mining, though, I will use the term “expert mining.”
Nothing in the Federal Rules of Evidence or the Federal Rules of Civil Procedure (jointly, “the Rules”) prevents expert mining; what little case law exists is mixed. It is often observed that our adversarial system induces situations in which both sides predictably hire fancy experts who predictably testify to opposite effect. In part because of the tendency of trials to devolve into such a battle of the experts, many have argued that our system should at least make more use of court-appointed (and thus putatively neutral) experts, if not use them exclusively. However, there has been little use of Federal Rule of Evidence 706, which allows courts to use such experts, and there seems little likelihood of change on the horizon.
Judge Richard Posner has suggested a less sweeping solution to the specific problem of expert mining. Posner advocated requiring lawyers who call an expert witness “to disclose the name of all the experts whom they approached as possible witnesses before settling on the one testifying. This would alert the jury to the problem of ‘witness shopping.’” Posner suggested that if one party’s testifying expert were the only one it hired, while the other party’s testifying expert were, say, its twentieth, then fact finders would be able to draw the “reasonable inference” that the latter’s case must be weaker (otherwise why hire so many experts?). Professor Christopher Robertson has proposed a more substantial reform. His proposal is based on the double-blind matching of experts to litigants, but for my purposes its key feature is that it, too, relies on disclosure of the number of experts retained to eliminate expert mining.
My primary objective here is to assess disclosure’s ability to realize the promise of (i) inducing parties to acquire expert evidence, while (ii) eliminating parties’ ability to obscure the informative value of that evidence via expert mining. I argue that while required disclosure surely reduces the allure of expert mining, it generally does not eliminate the use of multiple experts. I then point out that if we can count on parties to disclose truthfully, then a party’s use of expert mining is observable, so that a combined policy of required disclosure and exclusion of evidence obtained through expert mining would be feasible and would eliminate the incentive to use expert mining. But it is not obvious that such a combined disclosure-exclusion policy is desirable. When coupled with required disclosure, “expert mining” is really just the reporting of evidence gathered from multiple experts. To the extent that additional fully disclosed expert testimony increases the fact finder’s information, we can expect a beneficial increase in accuracy. On the other hand, expert evidence is costly; in addition, changes in its use could also change the pattern of settlement and litigation, with potentially unpredictable effects on both the extent of litigation and primary behavior. Thus, I conclude that it is ex ante unclear whether disclosure-exclusion or just disclosure would be a better policy reform.