I love that you're exploring what kinds of probabilistic rules law does, can, and should use, because I think the topic is important and neglected. Most of your analysis is thoughtful and compelling. My one objection is that you seem to have left out the most important reason for trying to make laws mostly return results that line up with our pre-existing moral intuitions: we as a society rely on a sense that the law is "just" or "fair" in order to enforce the law. You can get away with occasional minor deviations, but if the law diverges too widely or too often from what people think is fair, then they won't be adequately motivated to follow the law. There's a little bit of intellectual motivation to follow the law out of an explicit belief that law-following will result in better outcomes for society, but mostly people follow the law out of an intuitive or emotional sense that the law embodies justice. When people stop feeling that the law is a good approximation for justice, they stop supporting law enforcement.
There may well be uses of statistical evidence that offend people enough that even though we can't come up with general principles that rule them out (without also ruling out evidence we want to allow, such as motive evidence), allowing them would sufficiently undermine faith in the system as a whole that we'd be better off ruling them out ad hoc.
Interesting post. For street stops, the standard established in Terry v. Ohio is reasonable *articulable* suspicion, which rules out purely statistical evidence, no matter how overwhelming. My reading of this is that we do not want innocents to be punished simply because they belong to a group whose members are often guilty. I think this principle is admirable.
Further discussion of the Terry decision and related issues here:
Thanks for the comment! I'm a big fan of your substack.
I'm happy with having some restriction like that on street stops, though likely for slightly different reasons. If you think about the effects of a policy like "stop and frisk" in New York, my sense is that even if it can be defended on narrow fairness grounds, it also degrades the relationship between police and the communities they're policing, makes it less likely for members of minority communities to report crimes to the police, and so on.
I think those sorts of reasons for wanting a reasonably high bar for stops are slightly different than the more abstract, general principle you're relying on. Even if we demand "articulable" suspicion, it's still often going to be the case that innocents are searched in part because they belong to a group whose members are often guilty. Generic eyewitness evidence provides a clear illustration of the point; in a neighborhood where most of the crimes are committed by young Black men, innocent young Black men will be more likely to "fit the description" of a criminal suspect while also being close in time and place to the scene of a crime. So it would be *very* hard to craft reasonable policies that would allow police to follow up on eyewitness reports that wouldn't disproportionately ensnare innocent young Black men, at least in stops and searches.
I admit it's a fine line I'm trying to tread here. I want to concede that it's reasonable to worry about effects on community trust of criminal justice policies (whether concerning police behavior, or standards of evidence), and to accept some tradeoff between accuracy and community trust, while denying that it's at all feasible to demand that the rates with which innocent people are ensnared in some aspects of the criminal justice system be equal across demographic groups; no reasonable set of policies could ensure the latter constraint was met.
Totally agree about the points you made regarding trust and witness cooperation. In the language of the paper I linked to, these general equilibrium effects can make statistical discrimination self-defeating. But I don't think the courts rely on this kind of instrumental reasoning when insisting that suspicion be articulable and individualized. There is an aversion to using purely statistical evidence for any purpose under any circumstances. This appeals to me but I should think more deeply about why.
The suspect-fits-description case is considered sufficiently individualized, even though (as you say) it disproportionately burdens those belonging to groups with high rates of criminal offending.
By the way, I just upgraded your subscription to lifetime comp in case you ever want to leave a comment on my substack! All my posts are available in full to everyone but I restrict comments just to keep out trolls.
First comment - I haven’t seen this sort of justification of state decisis before! The usual defense I’ve seen says that once a rule is set in one case, the reason other cases should be decided the same way is that giving people knowledge and certainty about the law is more valuable than the benefits of fine-tuning the law. This seems importantly different from the Chesterton’s fence defense - it’s not that we think that the fact of it having once been decided this way is evidence that it’s good, and therefore we should keep it, but rather that the fact of it having once been decided this way *makes* it good to keep it, even if there had been a better way for it to have been decided the first time.
In any case, the Chesterton’s fence defense seems stronger when we are talking about an old precedent rather than one decided last year and not tested since then. And I’m a bit suspicious of Chesterton’s fence arguments for objects and rules that were invented by a specific person at a specific time - they tend to be better when we are talking about a tradition of practice that has been built up over generations by many people (which common law as a whole is, though individual decisions are not).
Strikingly, much of the same intuitive discomfort with relying on statistical evidence can come up outside of the law, or, for that matter, outside of other contexts where an institutional design approach would be at home. Perhaps you think such cases -- cases of individuals deciding whom go blame, whether to withdraw reciprocity, etc.-- should be handled differently from a theoretical perspective? But I'm inclined to think that it's basically the same issue and it should be handled by the same solution. Or maybe you are sympathetic to something like rule consequentialism, which extends the institutional design approach to moral constraints generally?
You guessed it! I am sympathetic to something like rule consequentialism, and I like your characterization of it. I'm not sure whether I need to be a rule consequentialist at a fundamental level, as various different foundational approaches to morality can end up endorsing closely related forms of reasoning. I'm thinking of Parfit's "triple theory" idea that the most attractive versions of rule consequentialism, contractarianism, and Kantian deontology all end up coinciding. I'm not sure that he's right about that; but neither am I sure that he's not.
Interesting! FWIW, rule consequentialism (and similar views, like some versions of contractualism) seems implausible to me precisely because the institutional design approach seems out of place when it comes to assessing ordinary moral choices. The kinds of factors that are relevant to institutional design (e.g., the ease with which a rule can be remembered or disseminated) will often seem like the 'wrong kind of reason' for individual choices.
Another thought about the post: you note sympathy with the view that institutions (like legal procedures) lack non-instrumental value. What this means depends on how you construe the idea of institutions having noninstrumental value. I think there is a weak and a strong way to understand the thesis you are denying here. The strong way takes, say, the rule of law or property to be fundamentally valuable, i.e., valuable in a way that does not derive from other goods that these things serve. I agree that that seems implausible. The weak way, by contrast, merely asserts that institutions can have value over and above their causal consequences. That seems very plausible to me. E.g., being in some degree of control over your life, or being able to rationally rely on others to do what they say they will-- these seem like reasonable candidates for things that are valuable over and above their causal consequences. And these goods, in turn, are (in many cases) partly constituted by legal and other social institutions. Standards of proof (and, more generally, legal protections against false conviction) could be valuable in a similar way.
I think I agree about the "wrong kind of reason" thought, but I'd want to interpret it differently. Here's an example of the kind of view I'm broadly sympathetic to. Hume's take on virtue, as I understand it, is broadly consequentialist. We regard as virtues those character traits we approve of, and we approve of them on broadly consequentialist grounds. But you might still think--I'm not sure if he talks about this--that ordinary moral motivation is (appropriately) insensitive to that whole superstructure of justification. E.g., a virtuous person will be directly motivated by various considerations--e.g., of gratitude, justice, or what have you--even though the reasons we think, on balance, that it's good to be motivated by those considerations are broadly consequentialist ones. This the kind of Williams, "one thought too many" idea.
On the latter point, I think I agree. For example, stability and predictability might be valuable independently of the fact that you can only make long term plans (which achieve independently valuable results) in stable, predictable environments. Do you think that makes a difference to the dialectic here? That is, do you think we can identify some plausible good that is constituted by standards of proof that limit the use of statistical evidence, but not by standards of proof that don't? "Legal protections against false conviction" seems far too broad a category to capture what sort of value we might be looking for; for any identifiable subspecies of evidence (e.g., motive evidence, opportunity evidence, DNA evidence), we could ban its use by prosecutors and characterize that as a protection against false conviction.
Dan, I was going to try and summarize what I thought about this stuff six years ago, which I think I essentially agree with you, but when I looked up the talk I gave, turns out, its not that easy to summarize so I just posted the whole thing because....well, what else am I going to do with this talk? Anyways, if its too long to read, gist is: statistical/individualized evidence is Gettierology; what counts as justice, i.e., retributive or utilitarian, selects for different epistemic standards along orthogonal dimensions. https://open.substack.com/pub/hiphination/p/knowledge-risk-and-separation-of?r=i44h&utm_campaign=post&utm_medium=web&showWelcomeOnShare=false
Looking forward to checking it out! I suspect I'll be sympathetic. Part of my inclination towards the simple threshold story (with no special treatment for statistical evidence) is the thought that trying to vindicate all out intuitive responses to the various cases in the literature is just as hopeless as trying to do it in the trolley problem, or trying to analyze knowledge in more basic terms (ie, getteriology). Basically, I think either you're going to bite some bullets, or you're going to have a set of legal principles that look extremely ad hoc/incoherent. I prefer to bite the bullets.
I might have a surprising intuition here - I think it's totally reasonable to use statistical evidence in Blue Bus and Prison Yard, but not in the Hmong case. My worry here is that Hmong people aren't actually more likely to be opium smugglers, they've just been convicted or scrutinized more because of bias in law enforcement. In fact, this is the kind of thing I worry about every time that I hear that "95% of [serious crime] is committed by [specific ethnic group]". Maybe the narcotics division is just staffed with Hmong-haters (which might also explain why they're arresting people that they don't have strong evidence against other than statistics).
So that fleshes out my intuition - I'm okay with Blue Bus and Prison Yard because those statistics are obviously reliable, but I'm not so sure about the Hmong statistic. I'm alright with using stats in criminal justice in principle, there just needs to be some way to verify its reliability. But I think that's thorny and difficult in practice, as any statistician will tell you. So that might be one reason people are leery of it, while being happy to embrace DNA evidence (which is easy to see is reliable).
So one issue that's related to the ones in the post, but which I'm not really dealing with, is how much judges should get to filter evidence before juries get to see it. In the US, it's already quite a lot--there's not some general rule that as long as evidence is in principle relevant, juries can consider it. Rather, there are all sorts of rules that judges are supposed to apply to make sure juries are seeing evidence that's probative, not too prejudicial, etc.
So you might think the statistical evidence in the Minneapolis case just isn't all that probative, and potentially prejudicial, and shouldn't be allowed into jury deliberations just because you don't trust juries to think rationally about it.
As it turns out, my instincts--maybe hard to reconcile with a kind of Chestertonian respect for precedent, I know!--is to be pretty skeptical of the thicket of evidential rules that have judges trying to limit what sorts of evidence juries can think about. I'd rather a default that juries get to see the evidence, and both prosecution and defense get to make their respective cases for how it should be weighed. So your argument seems to me an argument that a defense attorney should get to make, rather than an argument that should explain why the evidence isn't allowed to be presented in the first place. But I admit that's pretty different from how American courts tend to work!
This is one place where I'm glad we have the probative/prejudicial balancing test. Even if statistics of disparate crime rates by ethnic group are reliable, I'd be worried about how they interact with prejudices held by jury members. And then there's the separate issue of needing to build public trust in the criminal justice system, and how these statistics would affect people's perception that the system is rigged against them.
But outside this one issue, I very much agree that law should be less scared of stastical evidence.
Yeah I tend to think given the background probative/prejudicial balancing test, even without specific bans on statistical information, statistics about crime rates and ethnicity would rarely clear the bar. But in a special case like the Minneapolis one, where there's an ethnic group which is a very small proportion of the population and which dominates a particular category of crime (which means more probativeness), and people don't have strong stereotypes (so less prejudicial effect), I think it's not too hard to think they could be admissible even within that balancing framework.
Yes, I think my worries would be assuaged if the defense attorney can just whip out research that shows the Hmong stat is prejudicial or otherwise flawed. Thinking harder about it, that's not so different from when they say "that eyewitness testimony was vague" or "being around the crime scene proves nothing; my client was just in the wrong place".
I still don't want every trial involving a person of color to involve a lot yelling about racial crime statistics, but I think in principle I'm okay with this.
Fascinating, but does the law have a moral interest in treating people as individuals? That might explain my (and others?) greater reluctance to allow statistical evidence against the Hmong and the prisoners, compared to corporate civil defendants.
I love the policy engineering approach, tempered by conservative attention to precedent. But beyond accuracy and effective application, do we also want to engineer policies with moral constraints? And does being treated as an individual count as a legitimate one?
I think it turns out to be really hard to spell out what's involved in "treating people as individuals" that rules out prosecutors bringing up statistical evidence about the demographics of opium smugglers in Minneapolis, without ruling out the more general practice of drawing (tentative, probabilistic) inferences about an individual by subsuming them under patterns that include other individuals.
E.g., suppose somebody is accused of some gruesome, serial-killer style murder. Should the prosecution be able to point to evidence that as a child they tortured animals, wet the bed quite late past five, and frequently set fires, (google "Macdonald triad") and that these behaviors are very disproportionately common among people who go on to be violent sociopaths as adults? Or would that amount to not treating them as an individual? (Because it's evidence about what *other* people that exhibit those behaviors went on to do?)
Or should prosecutors be able to point to gang tattoos--which are perfectly legal to get--as corroborating evidence (*not* as the whole case!) that someone was engaged in criminal activity? Or would that amount to not treating them as an individual, because gang tattoos only have evidentiary force because of what *other* people with similar tattoos have done in the past?
Once you start asking questions like these, I think it's hard to see what's going on in the Minneapolis case as anything but a more explicit and numerical version of a practice that's utterly ubiquitous.
1. Yeah, the rule consequentialist definitely needs a move like that. But I'm not sure it's entirely kosher. When a moral theory cites some fact as explaining why some act would be wrong to perform, I'm inclined to think that those facts should also be reasons not to perform the relevant action. After all, the question "why must I PHI?" seems to be asking for both an explanation for the duty and a reason to PHI.
2. It might make a difference; I'm not sure. My inchoate suspicion is that the kinds of safety from false conviction we care about might have a modal structure that makes sense of the intuitive repugnance of purely statistical evidence. Could it be as simple as this: we don't just want to minimize the chances that we will be falsely convicted, we want to be in a position to *know* we won't be falsely convicted (assuming we are in the good case)? Probably this is a view already out there in the (enormous and still growing) literature...
There are views in the literature that try to justify a ban on statistical evidence in terms of knowledge. Usually it's not about knowing ex ante you won't be convicted, but knowing ex post that the accused is guilty, but I think the idea is pretty similar.
To my mind, the idea that we should value knowledge of this sort loses a lot of plausibility once we see how different it is from just treating false convictions as bad outcomes that we want to minimize. If you tell me there are two scenarios, and in one of them I know I won't be falsely convicted, and in the other I don't, I'll prefer the first. But if you then tell me that despite that difference, in both of them I have an equal, small chance of being falsely convicted, that preference will evaporate, and I'll also wonder just what you mean by "knowledge" and why you think it matters.
It's hard to comment on made up examples, because they're so often undertermined. In both of these cases the rules are subject to gamesmanship. If one bus company runs a majority of buses the other buses can afford to be less meticulous. If having one unidentified prisoner not involved in an attack renders the rest unpunishable it behooves them to make sure one uninvolved prisoner is present.
On the “statistical evidence” point, I’m sympathetic to what I believe is David Enoch’s view, that the relevant distinction is something like sensitivity. If the presence and force of the evidence isn’t counterfactually sensitive to whether the person did it, in either direction, then it’s not useful for establishing that this person did it. The blue bus evidence is completely insensitive - it doesn’t depend in any way on the recklessness. The prison yard case is slightly sensitive - if the person was involved, then the evidence would have been 98/100 rather than 99/100 if they hadn’t. And if the person wasn’t involved then the evidence would have been 100/100 rather than 99/100 if they had. (It would seem less sensitive if we were dealing with 990/1000 rather than 99/100.)
Using demographic factors as evidence is both insensitive in this way, and also creates a deeper sort of apparent unfairness, in that certain people can’t avoid having this kind of evidence for or against them, not just by modifying their criminal behavior, but in any sort of way.
Also, whenever there's generic eyewitness evidence (eg. someone looked to be about 6'2'', 200 lbs) lots of innocent people can't avoid having the evidence apply to them.
Yeah, that seems right, which is why it wouldn't be good policy to have a general ban on demographic factors as evidence. But the eyewitness evidence is at least somewhat sensitive.
Consider a situation where 90% of the buses are blue buses and 10% are green, and every instance of reckless driving is observed by an eyewitness who gets the correct color 80% of the time and the incorrect color 20% of the time. If the two bus companies are equally reliable, and the court always sides with the eyewitness and ignores the background information, then the blue company will be held liable for 74% (=72%+2%) of the incidents and the green company will be held liable for 26% (=18%+8%) of them. The blue company has a substantial incentive to improve their recklessness, and although the green company's liability mainly depends on the blue company's recklessness, they do have some incentive to clean up as well.
But if you let the background information is allowed to overwhelm the eyewitness, then blue has some incentive to get better, but green does not.
The discussion in the post about the rarity with which you can predict, ex ante, that only statistical/profile evidence will be available was meant to address cases like this. I know this isn't exactly that--because it's a case where there is non-statistical in addition to statistical evidence--but I think it's close enough; it's a case where green can predict, ex ante, that if statistical evidence is allowed, then they'll never be held liable. I think it's pretty plausible that those cases are quite rare, if they exist at all; in more typical cases, it will be an open possibility that if you commit a tort/crime, there will be evidence linking you directly to it that's powerful enough to overwhelm any statistical evidence to the contrary. Certainly if we're imagining actual buses, it's always a live possibility that somebody sees a license plate.
In cases like that, I conjecture that allowing statistical evidence in will tend to improve accuracy without sacrificing deterrence.
Motive and opportunity might be critical features of an explanation given for why the crime was committed by a particular person. He was in the prison yard, prone to starting fights when slighted, and hated the guard who was murdered is a solid foundation for explaining why a particular person committed a crime. He was in the prison yard is not sufficient to explain a person’s guilt in the same way even if their presence makes it highly likely that they committed the crime.
I love that you're exploring what kinds of probabilistic rules law does, can, and should use, because I think the topic is important and neglected. Most of your analysis is thoughtful and compelling. My one objection is that you seem to have left out the most important reason for trying to make laws mostly return results that line up with our pre-existing moral intuitions: we as a society rely on a sense that the law is "just" or "fair" in order to enforce the law. You can get away with occasional minor deviations, but if the law diverges too widely or too often from what people think is fair, then they won't be adequately motivated to follow the law. There's a little bit of intellectual motivation to follow the law out of an explicit belief that law-following will result in better outcomes for society, but mostly people follow the law out of an intuitive or emotional sense that the law embodies justice. When people stop feeling that the law is a good approximation for justice, they stop supporting law enforcement.
I think this is an important caveat.
There may well be uses of statistical evidence that offend people enough that even though we can't come up with general principles that rule them out (without also ruling out evidence we want to allow, such as motive evidence), allowing them would sufficiently undermine faith in the system as a whole that we'd be better off ruling them out ad hoc.
Interesting post. For street stops, the standard established in Terry v. Ohio is reasonable *articulable* suspicion, which rules out purely statistical evidence, no matter how overwhelming. My reading of this is that we do not want innocents to be punished simply because they belong to a group whose members are often guilty. I think this principle is admirable.
Further discussion of the Terry decision and related issues here:
https://onlinelibrary.wiley.com/doi/full/10.1002/pam.22527
Thanks for the comment! I'm a big fan of your substack.
I'm happy with having some restriction like that on street stops, though likely for slightly different reasons. If you think about the effects of a policy like "stop and frisk" in New York, my sense is that even if it can be defended on narrow fairness grounds, it also degrades the relationship between police and the communities they're policing, makes it less likely for members of minority communities to report crimes to the police, and so on.
I think those sorts of reasons for wanting a reasonably high bar for stops are slightly different than the more abstract, general principle you're relying on. Even if we demand "articulable" suspicion, it's still often going to be the case that innocents are searched in part because they belong to a group whose members are often guilty. Generic eyewitness evidence provides a clear illustration of the point; in a neighborhood where most of the crimes are committed by young Black men, innocent young Black men will be more likely to "fit the description" of a criminal suspect while also being close in time and place to the scene of a crime. So it would be *very* hard to craft reasonable policies that would allow police to follow up on eyewitness reports that wouldn't disproportionately ensnare innocent young Black men, at least in stops and searches.
I admit it's a fine line I'm trying to tread here. I want to concede that it's reasonable to worry about effects on community trust of criminal justice policies (whether concerning police behavior, or standards of evidence), and to accept some tradeoff between accuracy and community trust, while denying that it's at all feasible to demand that the rates with which innocent people are ensnared in some aspects of the criminal justice system be equal across demographic groups; no reasonable set of policies could ensure the latter constraint was met.
Thanks Daniel.
Totally agree about the points you made regarding trust and witness cooperation. In the language of the paper I linked to, these general equilibrium effects can make statistical discrimination self-defeating. But I don't think the courts rely on this kind of instrumental reasoning when insisting that suspicion be articulable and individualized. There is an aversion to using purely statistical evidence for any purpose under any circumstances. This appeals to me but I should think more deeply about why.
The suspect-fits-description case is considered sufficiently individualized, even though (as you say) it disproportionately burdens those belonging to groups with high rates of criminal offending.
By the way, I just upgraded your subscription to lifetime comp in case you ever want to leave a comment on my substack! All my posts are available in full to everyone but I restrict comments just to keep out trolls.
First comment - I haven’t seen this sort of justification of state decisis before! The usual defense I’ve seen says that once a rule is set in one case, the reason other cases should be decided the same way is that giving people knowledge and certainty about the law is more valuable than the benefits of fine-tuning the law. This seems importantly different from the Chesterton’s fence defense - it’s not that we think that the fact of it having once been decided this way is evidence that it’s good, and therefore we should keep it, but rather that the fact of it having once been decided this way *makes* it good to keep it, even if there had been a better way for it to have been decided the first time.
In any case, the Chesterton’s fence defense seems stronger when we are talking about an old precedent rather than one decided last year and not tested since then. And I’m a bit suspicious of Chesterton’s fence arguments for objects and rules that were invented by a specific person at a specific time - they tend to be better when we are talking about a tradition of practice that has been built up over generations by many people (which common law as a whole is, though individual decisions are not).
Agreed on both points.
Strikingly, much of the same intuitive discomfort with relying on statistical evidence can come up outside of the law, or, for that matter, outside of other contexts where an institutional design approach would be at home. Perhaps you think such cases -- cases of individuals deciding whom go blame, whether to withdraw reciprocity, etc.-- should be handled differently from a theoretical perspective? But I'm inclined to think that it's basically the same issue and it should be handled by the same solution. Or maybe you are sympathetic to something like rule consequentialism, which extends the institutional design approach to moral constraints generally?
You guessed it! I am sympathetic to something like rule consequentialism, and I like your characterization of it. I'm not sure whether I need to be a rule consequentialist at a fundamental level, as various different foundational approaches to morality can end up endorsing closely related forms of reasoning. I'm thinking of Parfit's "triple theory" idea that the most attractive versions of rule consequentialism, contractarianism, and Kantian deontology all end up coinciding. I'm not sure that he's right about that; but neither am I sure that he's not.
Interesting! FWIW, rule consequentialism (and similar views, like some versions of contractualism) seems implausible to me precisely because the institutional design approach seems out of place when it comes to assessing ordinary moral choices. The kinds of factors that are relevant to institutional design (e.g., the ease with which a rule can be remembered or disseminated) will often seem like the 'wrong kind of reason' for individual choices.
Another thought about the post: you note sympathy with the view that institutions (like legal procedures) lack non-instrumental value. What this means depends on how you construe the idea of institutions having noninstrumental value. I think there is a weak and a strong way to understand the thesis you are denying here. The strong way takes, say, the rule of law or property to be fundamentally valuable, i.e., valuable in a way that does not derive from other goods that these things serve. I agree that that seems implausible. The weak way, by contrast, merely asserts that institutions can have value over and above their causal consequences. That seems very plausible to me. E.g., being in some degree of control over your life, or being able to rationally rely on others to do what they say they will-- these seem like reasonable candidates for things that are valuable over and above their causal consequences. And these goods, in turn, are (in many cases) partly constituted by legal and other social institutions. Standards of proof (and, more generally, legal protections against false conviction) could be valuable in a similar way.
I think I agree about the "wrong kind of reason" thought, but I'd want to interpret it differently. Here's an example of the kind of view I'm broadly sympathetic to. Hume's take on virtue, as I understand it, is broadly consequentialist. We regard as virtues those character traits we approve of, and we approve of them on broadly consequentialist grounds. But you might still think--I'm not sure if he talks about this--that ordinary moral motivation is (appropriately) insensitive to that whole superstructure of justification. E.g., a virtuous person will be directly motivated by various considerations--e.g., of gratitude, justice, or what have you--even though the reasons we think, on balance, that it's good to be motivated by those considerations are broadly consequentialist ones. This the kind of Williams, "one thought too many" idea.
On the latter point, I think I agree. For example, stability and predictability might be valuable independently of the fact that you can only make long term plans (which achieve independently valuable results) in stable, predictable environments. Do you think that makes a difference to the dialectic here? That is, do you think we can identify some plausible good that is constituted by standards of proof that limit the use of statistical evidence, but not by standards of proof that don't? "Legal protections against false conviction" seems far too broad a category to capture what sort of value we might be looking for; for any identifiable subspecies of evidence (e.g., motive evidence, opportunity evidence, DNA evidence), we could ban its use by prosecutors and characterize that as a protection against false conviction.
Dan, I was going to try and summarize what I thought about this stuff six years ago, which I think I essentially agree with you, but when I looked up the talk I gave, turns out, its not that easy to summarize so I just posted the whole thing because....well, what else am I going to do with this talk? Anyways, if its too long to read, gist is: statistical/individualized evidence is Gettierology; what counts as justice, i.e., retributive or utilitarian, selects for different epistemic standards along orthogonal dimensions. https://open.substack.com/pub/hiphination/p/knowledge-risk-and-separation-of?r=i44h&utm_campaign=post&utm_medium=web&showWelcomeOnShare=false
Looking forward to checking it out! I suspect I'll be sympathetic. Part of my inclination towards the simple threshold story (with no special treatment for statistical evidence) is the thought that trying to vindicate all out intuitive responses to the various cases in the literature is just as hopeless as trying to do it in the trolley problem, or trying to analyze knowledge in more basic terms (ie, getteriology). Basically, I think either you're going to bite some bullets, or you're going to have a set of legal principles that look extremely ad hoc/incoherent. I prefer to bite the bullets.
I might have a surprising intuition here - I think it's totally reasonable to use statistical evidence in Blue Bus and Prison Yard, but not in the Hmong case. My worry here is that Hmong people aren't actually more likely to be opium smugglers, they've just been convicted or scrutinized more because of bias in law enforcement. In fact, this is the kind of thing I worry about every time that I hear that "95% of [serious crime] is committed by [specific ethnic group]". Maybe the narcotics division is just staffed with Hmong-haters (which might also explain why they're arresting people that they don't have strong evidence against other than statistics).
So that fleshes out my intuition - I'm okay with Blue Bus and Prison Yard because those statistics are obviously reliable, but I'm not so sure about the Hmong statistic. I'm alright with using stats in criminal justice in principle, there just needs to be some way to verify its reliability. But I think that's thorny and difficult in practice, as any statistician will tell you. So that might be one reason people are leery of it, while being happy to embrace DNA evidence (which is easy to see is reliable).
So one issue that's related to the ones in the post, but which I'm not really dealing with, is how much judges should get to filter evidence before juries get to see it. In the US, it's already quite a lot--there's not some general rule that as long as evidence is in principle relevant, juries can consider it. Rather, there are all sorts of rules that judges are supposed to apply to make sure juries are seeing evidence that's probative, not too prejudicial, etc.
So you might think the statistical evidence in the Minneapolis case just isn't all that probative, and potentially prejudicial, and shouldn't be allowed into jury deliberations just because you don't trust juries to think rationally about it.
As it turns out, my instincts--maybe hard to reconcile with a kind of Chestertonian respect for precedent, I know!--is to be pretty skeptical of the thicket of evidential rules that have judges trying to limit what sorts of evidence juries can think about. I'd rather a default that juries get to see the evidence, and both prosecution and defense get to make their respective cases for how it should be weighed. So your argument seems to me an argument that a defense attorney should get to make, rather than an argument that should explain why the evidence isn't allowed to be presented in the first place. But I admit that's pretty different from how American courts tend to work!
This is one place where I'm glad we have the probative/prejudicial balancing test. Even if statistics of disparate crime rates by ethnic group are reliable, I'd be worried about how they interact with prejudices held by jury members. And then there's the separate issue of needing to build public trust in the criminal justice system, and how these statistics would affect people's perception that the system is rigged against them.
But outside this one issue, I very much agree that law should be less scared of stastical evidence.
Yeah I tend to think given the background probative/prejudicial balancing test, even without specific bans on statistical information, statistics about crime rates and ethnicity would rarely clear the bar. But in a special case like the Minneapolis one, where there's an ethnic group which is a very small proportion of the population and which dominates a particular category of crime (which means more probativeness), and people don't have strong stereotypes (so less prejudicial effect), I think it's not too hard to think they could be admissible even within that balancing framework.
Yes, I think my worries would be assuaged if the defense attorney can just whip out research that shows the Hmong stat is prejudicial or otherwise flawed. Thinking harder about it, that's not so different from when they say "that eyewitness testimony was vague" or "being around the crime scene proves nothing; my client was just in the wrong place".
I still don't want every trial involving a person of color to involve a lot yelling about racial crime statistics, but I think in principle I'm okay with this.
Fascinating, but does the law have a moral interest in treating people as individuals? That might explain my (and others?) greater reluctance to allow statistical evidence against the Hmong and the prisoners, compared to corporate civil defendants.
I love the policy engineering approach, tempered by conservative attention to precedent. But beyond accuracy and effective application, do we also want to engineer policies with moral constraints? And does being treated as an individual count as a legitimate one?
I think it turns out to be really hard to spell out what's involved in "treating people as individuals" that rules out prosecutors bringing up statistical evidence about the demographics of opium smugglers in Minneapolis, without ruling out the more general practice of drawing (tentative, probabilistic) inferences about an individual by subsuming them under patterns that include other individuals.
E.g., suppose somebody is accused of some gruesome, serial-killer style murder. Should the prosecution be able to point to evidence that as a child they tortured animals, wet the bed quite late past five, and frequently set fires, (google "Macdonald triad") and that these behaviors are very disproportionately common among people who go on to be violent sociopaths as adults? Or would that amount to not treating them as an individual? (Because it's evidence about what *other* people that exhibit those behaviors went on to do?)
Or should prosecutors be able to point to gang tattoos--which are perfectly legal to get--as corroborating evidence (*not* as the whole case!) that someone was engaged in criminal activity? Or would that amount to not treating them as an individual, because gang tattoos only have evidentiary force because of what *other* people with similar tattoos have done in the past?
Once you start asking questions like these, I think it's hard to see what's going on in the Minneapolis case as anything but a more explicit and numerical version of a practice that's utterly ubiquitous.
1. Yeah, the rule consequentialist definitely needs a move like that. But I'm not sure it's entirely kosher. When a moral theory cites some fact as explaining why some act would be wrong to perform, I'm inclined to think that those facts should also be reasons not to perform the relevant action. After all, the question "why must I PHI?" seems to be asking for both an explanation for the duty and a reason to PHI.
2. It might make a difference; I'm not sure. My inchoate suspicion is that the kinds of safety from false conviction we care about might have a modal structure that makes sense of the intuitive repugnance of purely statistical evidence. Could it be as simple as this: we don't just want to minimize the chances that we will be falsely convicted, we want to be in a position to *know* we won't be falsely convicted (assuming we are in the good case)? Probably this is a view already out there in the (enormous and still growing) literature...
There are views in the literature that try to justify a ban on statistical evidence in terms of knowledge. Usually it's not about knowing ex ante you won't be convicted, but knowing ex post that the accused is guilty, but I think the idea is pretty similar.
To my mind, the idea that we should value knowledge of this sort loses a lot of plausibility once we see how different it is from just treating false convictions as bad outcomes that we want to minimize. If you tell me there are two scenarios, and in one of them I know I won't be falsely convicted, and in the other I don't, I'll prefer the first. But if you then tell me that despite that difference, in both of them I have an equal, small chance of being falsely convicted, that preference will evaporate, and I'll also wonder just what you mean by "knowledge" and why you think it matters.
I definitely see the force of that line of thought. Need to think more about it. Anyway, thanks for the interesting post and discussion!
It's hard to comment on made up examples, because they're so often undertermined. In both of these cases the rules are subject to gamesmanship. If one bus company runs a majority of buses the other buses can afford to be less meticulous. If having one unidentified prisoner not involved in an attack renders the rest unpunishable it behooves them to make sure one uninvolved prisoner is present.
On the “statistical evidence” point, I’m sympathetic to what I believe is David Enoch’s view, that the relevant distinction is something like sensitivity. If the presence and force of the evidence isn’t counterfactually sensitive to whether the person did it, in either direction, then it’s not useful for establishing that this person did it. The blue bus evidence is completely insensitive - it doesn’t depend in any way on the recklessness. The prison yard case is slightly sensitive - if the person was involved, then the evidence would have been 98/100 rather than 99/100 if they hadn’t. And if the person wasn’t involved then the evidence would have been 100/100 rather than 99/100 if they had. (It would seem less sensitive if we were dealing with 990/1000 rather than 99/100.)
Using demographic factors as evidence is both insensitive in this way, and also creates a deeper sort of apparent unfairness, in that certain people can’t avoid having this kind of evidence for or against them, not just by modifying their criminal behavior, but in any sort of way.
Also, whenever there's generic eyewitness evidence (eg. someone looked to be about 6'2'', 200 lbs) lots of innocent people can't avoid having the evidence apply to them.
Yeah, that seems right, which is why it wouldn't be good policy to have a general ban on demographic factors as evidence. But the eyewitness evidence is at least somewhat sensitive.
Consider a situation where 90% of the buses are blue buses and 10% are green, and every instance of reckless driving is observed by an eyewitness who gets the correct color 80% of the time and the incorrect color 20% of the time. If the two bus companies are equally reliable, and the court always sides with the eyewitness and ignores the background information, then the blue company will be held liable for 74% (=72%+2%) of the incidents and the green company will be held liable for 26% (=18%+8%) of them. The blue company has a substantial incentive to improve their recklessness, and although the green company's liability mainly depends on the blue company's recklessness, they do have some incentive to clean up as well.
But if you let the background information is allowed to overwhelm the eyewitness, then blue has some incentive to get better, but green does not.
The discussion in the post about the rarity with which you can predict, ex ante, that only statistical/profile evidence will be available was meant to address cases like this. I know this isn't exactly that--because it's a case where there is non-statistical in addition to statistical evidence--but I think it's close enough; it's a case where green can predict, ex ante, that if statistical evidence is allowed, then they'll never be held liable. I think it's pretty plausible that those cases are quite rare, if they exist at all; in more typical cases, it will be an open possibility that if you commit a tort/crime, there will be evidence linking you directly to it that's powerful enough to overwhelm any statistical evidence to the contrary. Certainly if we're imagining actual buses, it's always a live possibility that somebody sees a license plate.
In cases like that, I conjecture that allowing statistical evidence in will tend to improve accuracy without sacrificing deterrence.
What do you think about motive and opportunity evidence?
Motive and opportunity might be critical features of an explanation given for why the crime was committed by a particular person. He was in the prison yard, prone to starting fights when slighted, and hated the guard who was murdered is a solid foundation for explaining why a particular person committed a crime. He was in the prison yard is not sufficient to explain a person’s guilt in the same way even if their presence makes it highly likely that they committed the crime.