The Updated Meta-Analysis of E-Cigs & Health Outcomes: Still Critically Flawed

A highly-criticized meta-analysis of e-cigs &health outcomes was just updated. Not only does it still perpetuate critical flaws in the underlying studies, but it's interpreted with double-standards.

The History: The Original Meta-Analysis

Longtime tobacco control activist Stanton Glantz and colleagues published a meta-analysis of e-cigarettes and health outcomes two years ago (Glantz et al., 2024). It first performed a systematic review of all published studies on e-cigarette use and some health outcome, then grouped the papers by health outcome (e.g. cardiovascular disease (CVD), chronic obstructive respiratory disease (COPD), etc.) and did a meta-analysis to find the “pooled effect” across individual studies for each disease outcome.

The problem is, a meta-analysis is only as good as the underlying papers, and virtually all of the underlying papers are based on population survey data, where e-cigarette use is correlated with (in separate papers) each and every health outcome that was measured. As those of you who follow my research critiques know, this is probably the most common category of flawed paper, for the following reasons:

• Confounding by cumulative smoking exposure. Since the vast majority of e-cigarette use is by people who currently or formerly smoke, and given how highly harmful cigarettes are to all sorts of health outcomes, any association is at least mostly due to prior smoking history. Essentially, lingering effects of cigarette smoking are blamed on e-cigarette use.

• The above issue is not resolved by “adjusting for smoking status.” Most researchers realize on some level that this confounding exists, but then only crudely adjust for it (by putting into the model a variable for whether they currently smoked, formerly smoked, or never smoked). But this is not nearly sufficient: clinical health outcomes scale with duration and intensity of smoking, and for people who quit, health risks decline with time since quitting. I don’t have an official estimate, but I can recall only a handful of papers (out of hundreds of papers of this type) that adjust for smoking history reasonably well.

• Failing to exclude reverse-directionality. Since many health conditions have a risk gradient that increases with age, and/or require decades of smoking, it’s possible (or even likely) that the health condition might have developed first, before the person ever tried e-cigarettes (which are relatively more recent). In fact, Drs. Rodu and Plurphanswat analyzed PATH data — one of the few studies that collected information on age of diagnosis — and found that in participants who reported being diagnosed with one of these conditions (COPD, emphysema, myocardial infarction, stroke) and who used e-cigarettes, 93-97% developed the respective condition before they started vaping. Some of this could be reverse causality, if the health condition motivated the person to switch to vaping. Obviously, e-cigarettes cannot have caused the health condition if the health condition developed first, but unfortunately most of these types of papers do not rule out this reverse-causality.

So not surprisingly, since a large majority of these types of papers have all the flaws above, a meta-analysis that pools together flawed studies will perpetuate the underlying flaws (a “garbage in, garbage out” situation). But because it’s a meta-analysis, it is touted as a higher level of evidence. This is what happened in Glantz et al. 2024, which claimed to find “harm” from e-cigarettes in nearly every disease category that was examined.

Critiques of the Original Meta-Analysis

Several groups of prominent academic researchers strongly criticized the Glantz et al., 2024 meta-analysis. Two letters to the editor (below) by separate academic groups were published in the same journal shortly after publication (full exchange here; paywalled but I will share the full text privately):

• Cummings et al. made the points above about confounding by cigarette smoking and reverse temporality, and also added that there is a lack of evidence for a dose-response effect between e-cigarette use and health outcomes (i.e., the meta-analysis lacks an important criterion for causality):

• Hartmann-Boyce and Livingstone-Banks are two key authors in the ongoing Cochrane review finding e-cigarettes to be effective for smoking cessation; their criticism pertains to Glantz et al. 2024’s evaluation of bias in the underlying studies (a standard part of meta-analyses which is meant to reduce the garbage-in, garbage-out challenge). Essentially, Glantz et al. rated all 107 included studies as “low risk of bias overall” which should not be possible, given the severe limitations of the underlying datasets:

• Cohen & Cook published a “best-practices” paper on how to better analyze observational data on e-cigarette use and health outcomes, using Glantz et al. 2024 as an illustrative example of what not to do. In addition to pointing out the issues with confouding-by-smoking and unknown temporality, Cohen & Cook also re-analyze Glantz et al.’s results and find lower risk when comparing e-cigarettes vs. smoking.

• Rodu et al. also published a criticism in a different journal, reiterating some of the points above and also pointing out that Glantz et al.’s disease categories were illogical. For example, erectile dysfunction (ED) was grouped with much more serious conditions in the CVD category, and ED’s much stronger association may have “carried” the entire category of CVD. Similarly, respiratory symptoms (which could be subclinical or simply due to seasonal infection) were grouped with COPD.

• Lee & Farsalinos also published a criticism in a different journal, making many of the same points above, and (similarly to Cohen & Cook above) re-analyzed some of the data to show that e-cigarettes have lower risk than cigarette smoking.

• Edited Feb 22 to add: Polosa et al. published yet another criticism in a different journal,making some of the points above about illogical groupings of disease categories, reliance on cross-sectional studies (and therefore the reverse-directionality issue), residual confounding even in longitudinal studies, in addition to the larger point that meta-analyses “create the illusion of coherence where none exists.”

• There are also many other comments on PubPeer for the Glantz et al. 2024 meta-analysis.

The Updated Meta-Analysis: Same Old Flaws

Glantz & Oliveira da Silva 2026 is an update to the above meta-analysis that adds 18 new studies and is published in a different journal.

Other than adding 18 new studies, there are two minor improvements, but the long list of critical flaws above remains. The two minor improvements are:

• Removing the since-retracted Patel et al. article on stroke outcomes (I covered this retraction in a previous post) from the analysis. The previous meta-analysis on “stroke” outcomes showed no difference between e-cigarette use and smoking, which was interpreted as evidence for harm. Removing this especially flawed article from the updated meta-analysis actually shows a significantly lower risk of stroke with e-cigarette use vs. smoking (OR=0.62), but see below on how that’s interpreted.

• The updated article more comprehensively presents all possible comparisons (exclusive e-cig use vs. exclusive smoking; exclusive e-cig use vs. non-use; dual use vs. exclusive smoking) rather than other academics needing to do informative re-analyses in published criticisms.

The updated meta-analysis groups the results by each of these comparisons [emphasis mine]:

Comparing e-cigarette use with cigarette use, the ORs (95% CIs) for metabolic dysfunction (1.00 [0.91-1.09]) and oral disease (0.89 [0.78-1.02]) were not different from 1.0. The ORs (95% CIs) for cardiovascular disease (0.76 [0.58-0.99]), stroke (0.62 [0.47-0.82]), asthma (0.84 [0.74-0.95]), chronic obstructive pulmonary disease (0.55 [0.40-0.76]), and fetal growth (0.64 [0.44-0.92]) were ≤1.0.

Pooled ORs for dual use versus cigarette use were increased for all outcomes (range, 1.22-1.42) except fetal growth (0.99).

Pooled ORs for e-cigarette use, compared with nonuse of e-cigarettes, were increased for all outcomes (e-cigarette range, 1.24-1.53) except fetal growth (1.20). Dual use was associated with increased ORs for all outcomes (1.49-3.17).

Studies had a low risk of bias.

Essentially, exclusively vaping has lower odds of some health risks than exclusively smoking, but has higher odds than non-use. Meanwhile, dual use has higher health risks than exclusive smoking. A lot of the overall conclusion hinges on dual use having higher risks than exclusive smoking (more on that later).

The other fundamental flaws from the original meta-analysis still apply:

• Lingering effects from cigarette smoking, or effects that scale with the level of ongoing smoking, are blamed on e-cigarette use.

• People who developed a smoking-related health condition and then switched to vaping, are counted against vaping, because most of the papers stop at finding an association between vaping and the health outcome, without being able to (or without bothering to) know which one occurred first.

• Odd and illogical disease groupings (e.g. counting erectile dysfunction with much more serious heart conditions, and respiratory symptoms with much more serious COPD).

• Grading flawed studies as “low risk of bias”; the underlying studies do have critical flaws, and that did not change in this update.

Selective Interpretation and Double Standards

Since these flaws are so standard in the individual papers, I was not surprised to see them again in this updated meta-analysis. But there are new errors introduced in the interpretation. First, let’s take a deeper dive into the results (colored highlights are my markup):

The green highlights show possible harm reduction: they are cases where exclusive e-cigarette use has significantly lower odds of the health condition than exclusive smoking. It’s almost all of the categories, and the effect sizes are quite large (almost a 50% reduction in the case of COPD). (I should acknowledge that these data are still flawed, e.g. might be confounded by younger age of e-cigarette users. But since Glantz & Oliveira da Silva interpret these causally, let’s accept that for the sake of argument.)

How is this good news interpreted?

“disease outcomes [of e-cigarette use are] indistinguishable from or approaching cigarette use” (Glantz & Oliveira da Silva 2026)

This is very dismissive and gives the impression that there’s no difference between e-cigarette use and smoking. There is no difference for two outcomes (metabolic dysfunction and oral disease), but that’s not the same as finding harm. More importantly, there are statistically significant and large reductions for CVD, stroke, asthma, COPD, and fetal growth.

This is a clear double-standard because the magnitude of possible “risk reduction” is similar or larger to the possible “harm signals” (e-cigs vs. no use in yellow above; and dual use vs. exclusive smoking in blue).

For example, take the results for CVD (putting all of the other flaws above aside, for the sake of argument). E-cigs show a 24% lower risk vs. cigs (OR=0.76; green), while e-cigs show a 24% higher risk than no use (OR=1.24; yellow) and dual use shows a 29% higher risk than exclusive smoking (OR=1.29; blue). So (again, taking this at face value for the sake of argument), the risk decrease is exactly the same magnitude as the purported risk increase. Yet Glantz dismisses one set of results as “indistinguishable from or approaching cigarette use” but interprets the other set as a different enough risk difference to worry about.

This a pretty clear case of a double-standard when interpreting these results. If a statistically significant 24% reduction in risk is not recognized as a benefit, then logically, a 24% increase shouldn’t warrant concern. And of course in reality, the 24% “increase” in risk is very inflated because of the major flaws above.

Why Not Be Cautious about Possible Harm? The Importance of Risk Perceptions

Some people might think the double-standard is warranted because of the precautionary principle: if e-cigarette use vs. non-use poses possible harms (someone might thing), then why not focus more heavily on that than on the possible risk reduction?

The problem with this thinking is, people who smoke cigarettes face severe health risks, especially if they have been diagnosed with one of these conditions, and they can immediately and substantially benefit from switching to e-cigarettes. By dismissing possible benefits of switching to e-cigarettes, and hyping the possible harms (which are in reality, most likely explained by prior or ongoing smoking), it scares people away from trying or continuing to use e-cigarettes.

A large majority of the general public, of adults who smoke, and even of healthcare providers, have misperceptions about the harms of nicotine vs. combusted tobacco. Misperceptions can influence whether people who smoke try e-cigarettes, switch to them, and keep using them after they stopped smoking (Kim et al., 2022; Selya et al., 2023). Academics that have been quite critical of e-cigarettes have recently called for correcting misperceptions so that older adults who smoke are more willing to switch to e-cigarettes (Snell et al., 2025). While some papers have (unethically) called for more messaging on the harms of e-cigarettes to deter youth use (Skran 2026), this is having harmful effects too, as other work has documented young people who switched from e-cigarettes to cigarettes because all the alarmism has made them incorrectly think cigarettes are safer (Banjo et al.. 2025).

This updated meta-analysis perpetuates these harmful misperceptions by minimizing and dismissing the possible “harm reduction” signal of e-cigarettes and perpetuating critical flaws that blame the lingering or ongoing effects of cigarette smoking on e-cigarette use.

Further, the conclusion of this article is not about protecting non-users, it actively discourages e-cigarettes’ use for harm reduction:

“E-cigarettes should not be promoted as a safer alternative to cigarettes.” (Glantz & Oliveira da Silva 2026)

Conclusion

This updated meta-analysis further perpetuates all the severe flaws of its underlying papers and (being a meta-analysis) presents itself as higher-quality evidence. Specifically, lingering effects from prior smoking and/or ongoing effects from current smoking are blamed on e-cigarette use, and most papers make no attempt to exclude people whose health conditions occurred before they ever tried e-cigarettes.

Despite these systematic flaws, exclusive e-cigarette use still shows a possible “harm reduction” signal that is statistically significant and meaningful in size (e.g., 45% lower odds of COPD). Yet this good news is dismissed and misrepresented as “indistinguishable from or approaching cigarette use”, while similarly-sized differences are emphasized where they can be misrepresented as causing harm. The article concludes with a blanket denial of e-cigarettes’ harm reduction potential.

This article will continue to fuel misperceptions that are harming people by driving them to return to, or start using, cigarettes.

Some Closing Thoughts

What can we do? Post-publication criticisms only go so far and have a low chance of success (see my recent post on this). Nevertheless, I continue to be involved in these efforts, but I’m only one person.

I wish journals would raise their standards on observational data analysis. Some general guidelines exist (e.g. STROBE) but since there are so many special considerations in nicotine/tobacco research, journals specializing in this field should be imposing additional field-specific standards, e.g.:

• Analyses must adjust for pack-years of smoking history and (for people who quit) how long they’ve been quit.

• Analyses must stratify by smoking status; if e-cigarettes have a causal effect, it should be evident at all levels of smoking status (current, former, and never), not just spuriously within former smokers (as is often the case).

• Do not publish analyses that don’t remove from the analysis people who developed the health condition before they ever used e-cigarettes.