Jury Consultants: To What Extent Does Methodology Matter?

Risk & Insurance had an interesting article about “scientific perspective” in predicting jury verdicts. The premise of the article is that the quality of jury consultants varies wildly because different jury consultants use different methodologies. In the litigation world, there are no barriers to entry for those who seek to be jury consultants. The only thing you really need to do is put the words “jury consultant” on your business cards. As a result, many jury consultants are “amateurs in terms of their training” to predict jury behavior.

The author believes – I think correctly – that accident and malpractice lawyers often make choices based on who the lawyer likes (and respects) as opposed to the jury consultants’ credentials and background in the science of predicting juror behavior.

As a result, the variable quality of jury consultants leads to mixed results on the efficacy of jury verdict research. Accordingly, settlement decision makers (plaintiffs’ lawyers, defense lawyers and adjusters) often question the reliability of research when coming up with a dollar figure for dispensing with a case, and end up instead making major decisions on gut instincts (see also: George W. Bush).

The author, Courtroom Sciences’ National Director of Litigation Consulting, George Speckart, contends that these gut decisions create losses that are more expensive than the costs of scientific jury research.
In summary, the article is saying that jury consultants are a lot more reliable than most lawyers think, and if you have had bad results, it might be you were using a jury consultant who was not using solid methodology.

Certainly, the author, as a jury consultant, has a horse in this race. And he casually throws out $50,000 as a number to get you where you need to be to make a prediction as to value, which I think is beyond what should be necessary. But the logic of the article is compelling, particularly for those who have had a previous bad experience with jury consultants. For these lawyers, the author makes a good case that mock juries are still worth considering if you are handling a serious accident, product liability or medical malpractice case.

Updated:
  • I definitely agree that, without a methodology that has a foundation in real science (from the manner in which your respondents are recruited to the logistics of the day), you cannot rely on the data you get from a particular research study.

    I do not agree, though, with the price tag Mr. Speckart put on a quality jury research. It all depends on what it is you want to accomplish. For some quality jury consulting firms, 50K will get you a good research exercise, but you can also get a good exercise for less than that, depending on your goals.

    Here is the important thing to know: If a jury consultant is telling you that he or she can “predict” your outcome with a “more expensive” research, run the other way. Fast.

    The truth of the matter (coming from a jury consultant) is that there is no way to duplicate ahead of time the idiosyncrasies of the eight-to-twelve people who will eventually sit on a jury panel or predict how they will make heads or tails of damage numbers (especially related to intangible numbers like pain and suffering).

    Related to damages, though, I have found that jury research is predictive of the most important damage drivers in a case: juror emotion. In other words, jury research, when done well, can give you a glimpse into how jurors will express sympathy toward an injured plaintiff and whether there are issues in the case that will mitigate that sympathy.

    In addition, jury research I have conducted also has closely reflected how jurors perceive the tangible damage analyses (i.e., medical costs, life care plans, lost wages, etc.) presented by each side when that detailed damage information is presented at the research. In other words, plaintiff damage numbers perceived overreaching in research are typically perceived the same at trial. The same concept tends to apply to defense damage numbers perceived as penny-pinching or uncaring.

    Related to verdict decisions, I have found that the key arguments/facts that favorable jurors focus on in research are just about identical to the key arguments/facts that jurors have focused on at trial.

    In essence, jury research is a valuable tool to get a sense of a case from juror perspectives in terms of, inter alia, strategies that resonate or fall flat, pitfalls you may fall into, characteristics of jurors who will find against you no matter what you say, and the level of emotion elicited by a particular story. It can only do these things when the research is conducted with competent science (especially if you are trying to get a valid juror profile).

    Jury research, no matter who is conducting it, does not as reliably predict actual outcome, though, especially as related to damage numbers. It is the other components of the damage evaluations (the impact of damage accelerators and mitigators) that should be considered in the settlement equation, not necessarily the numbers themselves.

  • The $50,000 number never appears in the article. I don’t know where this came from.

    In evaluating jury research, one needs to determine the practitioner’s knowledge, ability and qualifications. Those who use services say “well he has a Ph D in psychology.” Studying what? Vandalism? Chasing rats through mazes? Pain management?

    Would you go to an opthamologist for a kidney disorder just because he is a “doctor”?

    There is an established body of research on methodology in prediction of behavior (which is what assessing exposure is, since it is a jury’s behavior that we are talking about). All that needs to be done is to ascertain whether this background exists.

    Yet no one ever does, but they are still willing to make determinations as to whether jury research is predictive.

  • Ron Miller

    Actually, $50,000 does appear in the article. It is you quoting someone using that number which leads the reader – or me anyway – to assume you approve the number and are putting it out there. If he had said $1,000 or $500,000, I don’t think you would have included that quote without comment. By quoting it, you seem to be tacitly approving it unless you were referring to more than one case.

    But we are thin slicing too much here. I obviously liked your article a lot which is why I pointed my readers to it.

    If you have another number you want to throw out other than 50, I’d love to hear what you think the appropriate range should be.

    Thanks for your comment (and yours Dr. Gomez).

  • OK I see what has happened: That was the quote from the insurance adjuster ($50,000). Mock trial costs are actually given in the article at $40,000, but even that number is misleading because the flat fee we use is in the 20’s. You can get up to 40 with out-of-pocket expenses: The example in which the case was proposed for settlement at $750,000 and then settled at $400,000 because of the mock trial research was in New England and we had to travel across country to get there, pay jurors in an expensive city, etc, thus final costs of $40K (and savings of $350,000).

    Notice people still talk about “expensive” when you have just shown how much money they saved. This generally comes from the knee-jerk assumption that if you say mock trials “predict” then you are selling snake oil (and by the way Ron I appreciate your very fine comments and understand that you were generally praising the article). But you can see just in this thread how the notion of prediction is a threat to many people (and many lawyers, who want control over the client).

    There is over 30 years of evolution in research methodology behind this development, with a basis from another 40 years of research on the prediction of behavior, so to simply reject prediction as “impossible” sounds a lot like the days of Ptolemy and Copernicus. In any event no one is claiming perfect prediction. As the founder of the thread correctly pointed out at the beginning, all that is being claimed is that the research beats guessing by a country mile, and the result is a lot of money that gets saved.

Contact Information