Associate Prof @UMBCPubPolicy. Formerly @UMichSPH. Econ PhD @UMich. Health and economic policy to fight HIV, TB & COVID19. All my takes are evidence-based.
Headlines in @Bloomberg @business and @FT reporting an "89.4% reduction in transmission" from the Pfizer vaccine are *wrong* because the Israeli pre-print overstated its main results. Let me explain. 1/8 pic.twitter.com/VOScW8srf0
Haas et al. use data from the national surveillance system which includes test results from people who had PCR testing for COVID19. The issue is that vaccinated and unvaccinated people are not testing at the same rates. So cases in unvaccinated people are overestimated. 2/8
Unvaccinated people are required to be tested when returning from travel or after contact with a confirmed case, vaccinated people are not. Unvaccinated people are also likely to be more vigilant about contracting or spreading COVID19, so they'd test more often. 3/8
If testing rates differ between vaccinated and unvaccinated people then we shouldn't interpret the *observed* difference in cases as the *actual* difference in cases. Many asymptomatic cases among the unvaccinated will go undetected because of lower testing rates. 4/8
This means that the estimate of 89.4% (95% CI 88.8 – 89.9%) is higher than the true (currently unknown) value. How much higher? We need more evidence to know for sure. We expect there to be a reduction in transmission, we just need to know by how much. 5/8
So why did @Bloomberg & @FT report the wrong number? Because the study overstated its (non-peer-reviewed) results in the abstract and news outlets reported that uncritically. News outlets should have asked one or more experts to read the study and weigh in first. 6/8
In fact, the pre-print included this caveat: "Israel’s testing policy exempts fully-vaccinated persons from required testing when returning from travel abroad or if they are in contact with a confirmed case. Thus, this could lead to overestimating VE against infection." 7/8
Thank you to everyone who has been posting my thread in the replies to all the RTs of the misleading headline!!
As is often the case the wrong information gets thousands of RTs and the correction just a few. That's why we need to vet before RTing.
@DrEricDing @DrEricDing I read the preprint and my thread below explains that the 89.4% is an overestimate. Please consider deleting this tweet and/or retweeting updated information. Thank you!
@dandrezner I'm going to suggest deleting and reposting. Thousands of RTs for those who reported the headlines uncritically and the corrections only see a fraction of that.
The paper's estimate was biased:
My mentions are full of languages from around the world as my thread gets retweeted.
Estimating #COVID19 vaccine efficacy is a question of global importance. It’s imperative that we get it right. twitter.com/ZoeMcLaren/sta…
@jburnmurdoch @PHE_uk @PHE_uk relies on real-world testing behavior so would likely overstate reduction in symptomatic infections because vaccinated people get tested less often. (See Haas Israeli study in attached thread.)
@chrisvanderveen We have new evidence this week. Biased estimates yes, but good evidence that reduction in transmission is substantial since bias adjustment unlikely to reduce 85-90% estimates to zero. More evidence on the way.
@KelseyTuoc @dylanmatt Uncritically reporting results from observational pre-print studies on transmission can be misleading because they have several sources of bias at play. Most pernicious is sample selection into getting vaxxed. Worth considering. (Overall great piece!) twitter.com/zoemclaren/sta…