The goal of social sciences and social psychology is to understand human behavior in the real world. Experimental social psychologists use laboratory experiments to study human behavior. The problem with these studies is that some human behaviors cannot be studied in the laboratory for ethical or practical reasons. Police shootings are one of them. In this case, social scientists have to rely on observations of these behaviours in the real world. The problem is that it is much harder to draw causal inferences from these studies than from laboratory experiments.
A team of social psychologists examined whether police shootings in the United States are racially biased (Are victims of police shootings more likely to be not White (Black, Hispanic). This is an important political issue in the United States. The abstract of their article states their findings.
The abstract starts with a seemingly clear question. “Is there evidence of a Black-White disparity in death by police gunfire in the United States?” However, even this question is not clear because it is not clear what we mean by disparity. Disparity can mean “a lack of equality or a lack of equality that is unfair (Cambridge dictionary).
There is no doubt that Black citizens of the United States are more likely to be killed by police gunfire than White citizens. The authors themselves confirmed this in their analysis. They find that the odds of being killed by police are three times higher for Black citizens than for White citizens.
The statistical relationship implies that race is a contributing causal factor to being killed by police. However, the statistical finding does not tell us why or how race influences police shootings. In psychological research this question is often framed as a search for mediators; that is, intervening variables that are related to race and to police shootings.
In the public debate about race and police shooting, two mediating factors are discussed. One potential mediator is racial bias that makes it more likely for a police officer to kill a Black suspect than a White suspect. Cases like the killing of Tamir Rice or Philando Castile are used as examples of innocent Black citizens being killed under circumstances that may have led to a different outcome if they had been White. Others argue that tragic accidents also happen with White suspects and that these cases are too rare to draw scientific conclusions about racial bias in police shootings.
Another potential mediator is that there is also a disparity between Black and White US citizens in violent crimes. This is the argument put forward by the authors.
When adjusting for crime, we find no systematic evidence of anti-Black disparities in fatal shootings, fatal shootings of unarmed citizens, or fatal shootings involving identification of harmless objects.
This statement implies that the authors conducted a mediation analysis, which uses statistical adjustment for a potential mediator to examine whether a mediator explains the relationship between two other variables.
In this case, racial differences in crime rates are the mediator and the claim is that once we take into account that Black citizens are more involved in crimes and involvement in crimes increases the risk of being killed by police, there are no additional racial disparities. If a potential mediator fully explains the relationship between two variables, we do not need to look for additional factors that may explain the racial disparity in police shootings.
Readers may be forgiven if they interpret the conclusion in the abstract as stating exactly that.
Exposure to police given crime rate differences likely accounts for the higher per capita rate of fatal police shootings for Blacks, at least when analyzing all shootings.
The problem with this article is that the authors are not examining the question that they are stating in the abstract. Instead they are conducting a number of hypothetical analyses that start with the premises that police officers only kill criminals. They then examine racial bias in police shootings under this assumption.
For example, in Table 1 they report that the NIBRS database recorded 135,068 sever violent crimes by Black suspects and 59,426 violent crimes by White suspects in the years 2015 and 2016. In the same years, 475 Black citizens and 1168 White citizens were killed by police. If we assume that all of those individuals killed by police were suspected of a violent crime recorded in the NIBRS database, we see that White suspects are much more likely to be killed by police (1168 / 59,426 = 197 out of 10,000) than Black suspects (475 / 135068 = 35 out of 10000). The odds ratio is 5.59, which means for every Black suspect police kills over 5 White suspects. This is shown in Figure 1 of the article as the most extreme bias against White criminals. However most other crime statistics also lead to the conclusion that White criminals are more likely to be shot by police than Black criminals.
This is a surprising finding to say the least. While we started with the question why police officers in the United States are more likely to kill Black citizens than White citizens, we end with the conclusion that police officers only kill criminals and are more likely to kill White criminals than Black criminals. I hope I am not alone in noticing a logical inconsistency. If police doesn’t shoot innocent citizens and they shoot more White criminals than Black criminals, we should see that White US citizens are killed more often by police than Black citizens. But that is not the case. We started our investigation with the question why Black citizens are killed more often by police than White citizens. The authors statistical analysis does not answer this question. Their calculations are purely hypothetical and their conclusions suggest only that their assumptions are wrong.
The missing piece is information about the contribution of crime to the probability of being killed by police. Without this information it is simply impossible to examine to what extent racial differences in crime contribute to racial disparities in police shootings. And therewith it is also impossible to say anything about other factors, such as racial bias, that may also contribute to racial disparities in police shootings. This means that this article makes no empirical contribution to the understanding of racial disparities in police shootings.
The fundamental problem of the article is that the authors think they can simply substitute populations. Rather than examining killings in the population of citizens, which the statistic is based on, they think they can replace it by another population, the population of criminals. But, the death counts apply to the population of citizens and not to the population of criminals.
In this article, we approached the question of racial disparities in deadly force by starting with the widely used technique of benchmarking fatal shooting data on population
proportions. We questioned the assumptions underlying this analysis and instead proposed a set of more appropriate benchmarks given a more complete understanding of the context of police shootings
The authors talk about benchmarking and discuss the pros and cons of different benchmarks. However, the notion of a benchmark is misleading. We have a statistic about the number of police killings in the population of the United States. This is not a benchmark, it is a population. In this population, Black citizens are disproprotionally more likely to get killed by police. That is a fact. It is also a fact that in the population of US citizens more crimes are being committed by Black citizens (discussing the reasons for this is another topic that is beyond this criticism of the article). Again, this is not a benchmark, it is a population statistic. The author now use the incident rates of crime to ask the question how many Black or White criminals are being shot by police. However, the population statistics do not provide that information. We could also use other statistics that lead to different conclusions. For example, White US citizens own disproportionally more guns than Black citizens. If we would use that to “benchmark” police shootings, we would see a bias to shoot more Black gun-owners than White gun-owners. But we don’t really see that in the data because we have no information about the death rates of gun owners, just as the article does not provide information about the death rates of criminals and innocent citizens. Thus, the fundamental flaw of the article is the idea that we can simply take two population statistics and compute conditional probabilities from these statistics. This is simply not possible.
The authors caution readers that their results are not conclusive. “The current research is not the final answer to the question of race and police use of deadly force” In fact, the results presented in this article do not even begin to address the question. The data simply provide no information about the causal factors that produce racial inequality in police shootings.
The authors then contradict themselves and reach a strong and false conclusion.
Yet it does provide perspective on how one should test for group disparities in behavioral outcomes and on whether claims of anti-Black disparity in fatal police shootings are as certain as often portrayed in the national media. When considering all fatal shootings, it is clear that systematic anti-Black disparity at the national level is not observed.
They are wrong on two counts. First, their analysis is statistically flawed and leads to internally inconsistent results. Police only kill criminals and are more likely to kill White criminals, which does not explain why we see more Black victims of police shootings. Second, even if their study had shown that there is no evidence of racial inequality, we cannot infer that racial biases do not exist. Absence of evidence is not the same as evidence of absence. Cases like the tragic death of Tamir Rice may be rare, and they may be too rare to be picked up in a statistic, but that doesn’t mean they should be ignored.
The rest of the discussion section reflects the authors’ personal views more than anything that can be learned from the results of this study. For example, the claim that better training will produce no notable improvements is pure speculation, and ignores a literature on training in the use of force and its benefits for all citizens. The key of police training in shooting situations is for police officers to focus on relevant cues (e.g., weapons) and to ignore irrelevant factors such as race. Better training can reduce killings of Black and White citizens.
This suggests that department-wide attempts at reform through programs such as implicit bias training will have little to no effect on racial disparities in deadly force, insofar as
officers continue to be exposed after training to a world in which different racial groups are involved in criminal activity.
It is totally misleading to support this claim with trivial intervention studies with students.
This assessment is consistent with other evidence that the effects of such interventions are short lived (e.g., Lai, 2017).
And once more the authors attribute racial differences in police shootings to crime rates and they ignore that the influence of crime rates on shootings is their own assumption and not an empirical finding that is supported by their statistical analyses.
Note that this analysis does not blame unarmed individuals shot by police for their own behavior. Instead, it highlights the difficulty of eliminating errors under conditions of uncertainty when stereotypes may bias the decision-making process. This difficulty is amplified when the stereotype accurately reflects the conditional probabilities of crime across different racial groups.
Like many articles, the limitation section is not really a limitation section, but the authors pretend that these limitations do not undermine their conclusions.
One potential flaw is if discretionary stops by police lead to a higher likelihood of being shot in a way not captured by our crime report data sets. If officers are more likely to stop and frisk a Black citizen, for example, then officers might be more likely to enter into a deadly force situation with Black citizens independent of any actual crime rate differences across races. Online Supplemental Material #5 presents some indirect data relevant to this possibility. Here, we simply note that the number of police shootings that start with truly discretionary stops of citizens who have not violated the law is low (*5%) and probably do not meaningfully impact the analyses.
There are about 1000 police killings a year in the United States. If 5% of police killings started without any violation of the law, this means 50 people are killed every year by mistake. This may not be a meaningful number to statisticians for their data analysis, but it is a meaningful number for the victims and their families. In no other Western country, citizens are killed in such numbers by their police.
The final conclusion shows that the article lacks any substantial contribution.
Conclusion
At the national level, we find little evidence within these data for systematic anti-Black disparity in fatal police deadly force decisions. We do not discount the role race may play in individual police shootings; yet to draw on bias as the sole reason for population-level disparities is unfounded when considering the benchmarks presented here. We hope this research demonstrates the importance of unpacking the underlying assumptions inherent to using benchmarks to test for outcome disparities.
The authors continue their misguided argument that we should use crime rates rather than population to examine racial bias. Once more, this is nonsense. It is a fact that Black citizens are more likely to be killed by police than White citizens. It is worthwhile to examine which causal factors contribute to this relationship, but the authors approach cannot answer this question because they lack information about the contribution of crime rates to police shootings.
The statement that their study shows that racial bias of police offers is not the only reason is trivial and misleading. The authors imply that crime rates alone explain the racial disparity and even come to the conclusion that police is more likely to kill White suspects. In reality, crime rates and racial biases are likely to be factors, but we need proper data to tease apart those factors and this article does not do this.
I am sure that the authors truly believe that they made a valuable scientific contribution to an important social issue. However, I also strongly believe that they failed to do so. They start with the question “Is there evidence of a Black-White disparity in death by police gunfire in the United States?” The answer to their question is an unequivocal yes. The relevant statistic are the odds of being killed by police for Black and White US citizens, and these statistics show that Black citizens are at greater risk to be killed by police than White citizens. The next question is why this disparity exist. There will be no simple and easy answer to this question. This article suggests that a simple answer is that Black citizens are more likely to be criminals. This answer is not only too simple, it is also not supported by the authors statistical analysis.
Scientists are human, and humans make mistakes. So, it is understandable that the authors made some mistakes in their reasoning. However, articles that are published in scientific journals are vetted by peer-review, and the authors thank several scientists for helpful comments. So, several social scientists were unable to realize that the statistical analyses are flawed even though they produced the stunning result that police officers are 5 times more likely to kill White criminals than Black criminals. Nobody seemed to notice that this doesn’t make any sense. I hope that the editor of the journal and the authors carefully examine my criticism of this article and take appropriate steps if my criticism is valid.
I also hope that other social scientists examine this issue and add to the debate. Thanks to the internet, science is now more open and we can use open discussion to fix mistakes in scientific articles much faster. Maybe the mistake is on my part. Maybe I am not understanding the authors’ analyses properly. I am also not a neutral observer living on planet Mars. I am married to an African American woman with an African American daughter and my son is half South-Asian. I care about their safety and I am concerned about racial bias. Fortunately, I live in Canada where police kill fewer citizens.
I welcome efforts to tackle these issues using data and the scientific method, but every scientific result needs to be scrutinized even after it passed peer-review. Just because something is published in a peer-reviewed journal doesn’t make it true. So, I invite everybody to comment on this article and my response. Together we should be able to figure out whether the authors’ statistical approach is valid or not.
The authors did not show that crime rates proportionally contribute to fatal police shootings, or that police mostly kill criminals, but they are reasonable priors anyway (the latter one can be easily checked, does the data show otherwise?). Given those priors (and therefore restricting ourselves to two theoretical alternatives: either systemic anti-black bias, which causes police to shoot more blacks regardless of their innocence, or the absence of systemic anti-black bias), the study raises the posterior of no systemic anti-black bias. So the study is not completely rubbish, although it did not preclude an anti-black bias theory coupled with an explanation of why police kill black criminals less often, despite harboring anti-black bias.
I think you are not understanding the flaw of the study. it is simply not possible to make claims about racial bias, by substituting one population (US citizens) with another population (Homicide suspects). What the authors would need to show are the actual odds of being shot given involvement in crime or innocence, but they don’t have this information. Thus, they lack the proper empirical data to make evidence based claims. This means they published results that look scientific, but are not. We simply do not know to what extend the crime rates contribute to the disproportional killing of Black citizens. It is of course possible that it fully explains it, but raising this possibility is one thing, to claim proof is another.
Hi, Uli:
Thanks for engaging with us on this work – I know it’s important to you (and to us) that we get a good understanding of this question. I’ll try to work through some general points and then reply to some specifics.
I. General Critique
I think one point that might be useful is to explore the idea of a benchmark in more detail. You state:
“There is no doubt that Black citizens of the United States are more likely to be killed by police gunfire than White citizens.” To support this claim, you note that the odds ratio for being shot by police is 2.5, showing an anti-Black disparity. (FYI, it’s 2.5 and not 3.3 as that is for unarmed shootings only.)
You also state,
“However, the notion of a benchmark is misleading. We have a statistic about the number of police killings in the population of the United States. This is not a benchmark, it is a population. In this population, Black citizens are disproprotionally more likely to get killed by police. That is a fact.”
But there is something missing here, which is that when you say “There is no doubt that Black citizens of the United States are more likely to be killed,” you are choosing a benchmark. You don’t get away from the benchmark problem by looking at population-levels odds ratios. Instead, you’re making the implicit claim that the entire population is the relevant benchmark. When you say, “Black citizens are more likely to be killed” what you are actually saying is “Black citizens are more likely to be killed given the benchmark of population proportions” or equally, “Black citizens are more likely to be killed given the expectation that all people will be shot in accordance with their population proportion.”
Using population-level benchmarks requires the assumption that people from all groups have equal exposure to those situations where deadly-force can be used by police, or for which deadly force is relevant. We know this is false (more on this below), hence the population benchmark is misleading.
By way of analogy, consider the following four hypothetical examples:
*We want to understand the disparity in males being more likely to be killed by police than females (something like 95% of fatal police shootings are of men), wondering whether police might be biased in perceiving threat from males but not females for the same behavior. Suppose that men, despite making up 49% of the population, make up 99% of people who do dangerous things like shoot handguns at the police or swing bats at them. Is the relevant benchmark 49% or 99%? What would you expect, that men should be shot in accordance with their population proportions or in accordance with being in those situations where deadly force is relevant? If men make up 95% of fatal police shootings, do you take this as evidence of sex bias in the decision to shoot?
*We want to calculate the odds of being in the army, by racial group. In order to be in the army (in the U.S.) you are to be 18 years of age. Different racial groups have different distributions of ages. Is the relevant comparison the overall proportion of citizens from a given racial group or the overall proportion of citizens who are 18+ from a given racial group?
*We want to understand something about gender and becoming a Physics major, when women make up ~20% of physics majors. One possibility is to compare this 20% to the 51% of women in the general population. However, women actually make up ~70% of all undergraduates. You need to be an undergraduate to major in physics. Is the relevant comparison 51% or 70%?
*We want to know whether the best cancer treatments are being provided evenly across racial groups. We find out that 15% of patients receiving the top treatment are Black. Black citizens are about 15% of the total U.S. population. If we find out that Blacks make up 50% of all cancer patients, is there a racial disparity in administration of the best treatment?
In all cases, the relevant population of interest is not the entire U.S. population and computing odds ratios on this value will be misleading. In these four examples, it is clear that the population proportion fails as a benchmark because there are clear assumptions needed for using population proportions that are obviously false, e.g., people without cancer do not try to get cancer treatment.
In the case of police shootings, the same applies. Except in rare cases, fatal police shootings happen in the context of violent crime. (No one disputes that rare cases do occur… more on this below.) They are not events that occur equally across all situations. If people from different racial groups encounter the police (in situations for which deadly force is relevant) at different rates, then using population proportions as a benchmark will be misleading.
The question is not, “Should I look at the population-level odds ratios or benchmark on some value?” The question is, “On what value should I benchmark?” because the population level is itself a benchmark. As we laid out in the paper, there are strong reasons to believe the population-level benchmark is a poor one.
You state “The authors continue their misguided argument that we should use crime rates rather than population to examine racial bias. Once more, this is nonsense.” Would it be nonsense to compare female physics majors to the percent of female undergraduates? Cancer treatment rates to those with cancer? It’s the same analysis and logic across all these cases.
Broadly, this issue has a long history in criminal justice and benchmarking on some other value is a standard practice. For example, let’s take Dr. Goff’s words (https://policingequity.org/images/pdfs-doc/CPE_SoJ_Race-Arrests-UoF_2016-07-08-1130.pdf):
“Population benchmarks provide only a crude method for estimating disproportionality. They allow for an inference that force is being used in a manner that is disproportionate to presence in the general population, but do not allow for a clear inference as to whether the force is disproportionate to presence in any particular area or to legitimately provocative behavior. A more direct, albeit still limited, proxy for level of provocative behavior would be actual offending. The closest available index of offending is arrest rates. While arrest rates are confounded by racial bias in policing practices, they provide a closer approximation than simple resident population.”
Moreover, Dr. Goff’s own analyses confirm exactly what we claim: no anti-Black disparity (in fact, anti-White disparity) in fatal shootings once violent crimes are taken into account, i.e., once shooting rates are benchmarked on violent crime. (Using a different method, this is also what Fryer found). (As a side note, the overall analyses on all kinds of use of force by Goff show the same patterns.)
You describe what we’re doing at times as “substituting” one group for another, but this isn’t how we see it. Instead we’re working through an analysis of the conditions surrounding deadly force use and then asking, given these conditions, what is the most appropriate benchmark?
You state: “This is not a benchmark, it is a population. In this population, Black citizens are disproportionally more likely to get killed by police. That is a fact.” This is somewhat correct but somewhat incorrect. Again, when you say, “Black citizens are disproportionately more likely to be killed” you mean that “according to the benchmark of the overall population, Black citizens are more likely to be killed.”
II. Violent crime as a benchmark
In various ways you question whether violent crime is really the key factor for fatal police shootings. One way to first address this is to note that past work analyzing fatal police shootings makes it clear that violent crime is the relevant context for shootings. The choice of violent crime is not random, and of course leads directly the point that population benchmarks are misleading (given that people from different groups are not involved in violent crime to the same degree).
You say “While we started with the question why police officers in the United States are more likely to kill Black citizens than White citizens, we end with the conclusion that police officers only kill criminals and are more likely to kill White criminals than Black criminals.” This is not quite right on several accounts. First, we aren’t starting with the question of why police officers are more likely to kill Black citizens. That isn’t a starting point in the sense of “this is the known fact to be explained.” As described above, we start by noting that the greater odds for Black citizens are true when you benchmark on population values. However, this benchmark is a poor one, given (1) what we know about the nature of fatal police shootings and (2) what we know about the exposure of people from different groups to these situations.
Second, you say “we end with the conclusion that police officers only kill criminals,” not only do we never say this, but we clearly state (e.g., in the abstract) that we when break down the shootings to look specifically at those cases where citizens are not aggressive, i.e., are not violent against police, the data are too uncertain to draw firm conclusions.
Third, we are not saying that police are more likely to kill White criminals. What we are saying is that, given rates of exposure to police in violent crime situations, there is not evidence of anti-Black disparity in fatal shooting rates.
“The fundamental problem of the article is that the authors think they can simply substitute populations. Rather than examining killings in the population of citizens, which the statistic is based on, they think they can replace it by another population, the population of criminals. But, the death counts apply to the population of citizens and not to the population of criminals.” This is not quite correct. The counts apply to whatever subsample is the relevant subsample. We aren’t substituting populations, we’re using an analysis of the nature of deadly force decisions to determine which benchmarks are most appropriate. You say “killings in the population of citizens, which the statistic is based on” but as I noted above it’s only based on the population because you’ve implicitly chosen the population as the relevant benchmark. It is exactly this choice that the paper argues is inappropriate.
This misinterpretation is seen again when you say, “The missing piece is information about the contribution of crime to the probability of being killed by police.” This actually is not a piece of missing information. It is the case that the far majority of fatal police shootings involve criminal contexts. Here’s the first 10 shootings of Black or White men in 2015:
1. police respond to 911 call of domestic dispute, man shot by police when he emerged from the house pointing his gun at officers
2. police respond to 911 call of armed man, man threatened to shoot officers and was shot when he refused to drop his gun
3. police respond to 911 call of violent dispute, man was tased but did not comply with orders, reached for waistband and was shot
4. man pulled over in traffic stop, gun was discovered during pat down, he fought with the officer and was shot while reaching for his gun
5. man parked in restricted police lot, showed gun when officers made contact and was shot (suicide by cop — left suicide note)
6. man identified by police as robbery suspect while driving stolen vehicle, man opened door and fired his gun at officers
7. police respond to 911 call of “man armed with a gun threatening bar patrons and assaulting his girlfriend,” man shot when he refused to obey commands to drop weapon
8. police respond to 911 call of man threatening his son, was shot when man took out gun and pointed it at officers
9. police carrying out warrant for arrest of man accused of sexual assault of a teenage girl, man shot when he aimed his gun at officers
10. police respond to 911 call of home invasion, man stabs police dog when police arrive, was shot when he raised his knife over his head toward officers
This is not a random sample of U.S. citizens nor is it a random sample of daily life nor even is it a random sample of policing situations. It is heavily skewed toward violent crime situations and people from different racial groups do not occupy such situations at the same rate. This is the foundational fact on our paper was based.
What is clear from the above examples (and also more systematic and full analyses by Nix et al. 2017, Johnson et al. 2019), is that violent crime is the context of fatal police shootings. Yes, shootings of unarmed and nonviolent citizens do occur, but they are a small minority of cases. Somewhere between 85-90% of citizens shot by police are armed at the time of being shot, and of the roughly 800 black and white citizens shot by police in 2015, only 45 could be defined as “not aggressing” against the police or other citizens at the time they were shot.
Do shootings outside the context of violent crime happen? Absolutely yes (see below). The point is that these are rare.
It’s not as though there are no data on this question, as you suggest. It isn’t a piece of missing information. To show this, you can just rerun the analyses removing those cases where violence wasn’t an issue. The results are not going to reveal anti-Black disparity.
“And once more the authors attribute racial differences in police shootings to crime rates and they ignore that the influence of crime rates on shootings is their own assumption and not an empirical finding that is supported by their statistical analyses.” Again, it is not our assumption – violent crime is the relevant context for fatal police shootings as analyses of such shootings have shown.
III. When it’s not violent crime
One thing that we are in agreement on is the possibility of racial disparity in unarmed shootings and shootings of citizens who are not acting aggressively. You are right to point out that not all shootings involve people currently in violent criminal situations, and we agree with you – which is why we did the analyses we did and were explicit up front that the data are too uncertain to draw strong conclusions. That is because these cases, while absolutely tragic, and thankfully very rare.
IV. Additional Points
You state “The statistical relationship implies that race is a contributing causal factor to being killed by police.” It does not. For example, in an earlier submission of the manuscript, we explicitly noted that race may have nothing to do with fatal shootings and instead it is all driven by social class differences, with race as a third variable. This is also why we didn’t frame the paper as “a search for mediators” and why I don’t think that’s the right way of approaching the question. At no point do we suggest we did any mediation analysis and so claims otherwise are not correct.
That is why, in the abstract, we clearly and unambiguously state that we’re doing benchmarking analyses: “This is commonly answered by comparing the odds of being fatally shot for Blacks and Whites, with odds benchmarked against each group’s population proportion. However, adjusting for population values…”
You state, “Instead they are conducting a number of hypothetical analyses that start with the premises that police officers only kill criminals.“ This is not quite correct on three counts. First, it doesn’t require as a premise that police only kill criminals. It requires that criminal contexts, and in particular violent crime contexts, are the relevant situation for officers to be making deadly force decisions. The question is then the degree to which people from different groups occupy those situations. Second, this is why we break down the analysis in terms of shootings of armed v. unarmed & not aggressing individuals. We can redo the analyses removing all those cases of people who were not engaged in crime. This is a small number and will not impact the results. Finally, I don’t know what a “hypothetical” analysis is – we’re taking exactly the analysis done by everyone who calculates odds on the population benchmark and then repeating that with different benchmarks.
But more important, this illustrates a main point that is missing: The failure to appreciate the fact that most fatal police shootings – the far, far majority – happen in the context of violent crime. That is exactly why the correct benchmark is in terms of exposure to the police in violent crime situations, i.e., benchmarking against violent crime rates and not population values.
“We could also use other statistics that lead to different conclusions.” Of course this is true. You would then need to justify why that statistic is the relevant statistic. We justify the crime rate statistic as the relevant one because that is the context of fatal police shootings.
“There are about 1000 police killings a year in the United States. If 5% of police killings started without any violation of the law, this means 50 people are killed every year by mistake. This may not be a meaningful number to statisticians for their data analysis, but it is a meaningful number for the victims and their families.” Of course this is correct and no one debates that having a family member killed by the police in such a manner is devastating.
A final slight correction; you state: “However, the notion of a benchmark is misleading. We have a statistic about the number of police killings in the population of the United States. This is not a benchmark, it is a population.” We are not saying the number of police killings is a benchmark. It’s the population of comparison that’s a benchmark. The number of killings is a static quantity, it doesn’t change across analyses.
—-
I agree that the topic is important and that any shooting of citizens under conditions where the citizen is not a threat is tragic and, though rare, may have a large impact on police-community relationships. I appreciate you engaging with the work, but as laid out above in my view some of the main criticisms that you offer are a bit off the mark and may not “fundamentally” undermine the work in the way you suggest.