New Research Flaw Unlocked?

In my weekly review of new nicotine/tobacco research, I just saw two examples of a critical flaw that to my knowledge is a brand new error. (The two papers have a common author.)

In my weekly reviews of ~30-70 new papers in PubMed on nicotine/tobacco, I often see the same flaws week after week which I covered in previous content (GFN session and Part 1 of State of the Academic Research). It’s rare that I see something new, but that happened this week with two new papers (which are at least partly related, as they both list clinical epidemiologist Danbee Kang at Sungkyunkwan University as an author)

The two papers are similar: both analyzed data from a national health insurance dataset in Korea, focused on patients who smoked cigarettes and had a certain diagnosis at baseline, then looked at whether relevant health outcomes at ~5 years differed based on whether they continued smoking, switched to e-cigarettes, or quit nicotine entirely at ~1 year after baseline. Essentially:

• Paper 1: Kim et al. 2026, examined patients with COPD at baseline, and examined several outcomes at the ~5-year follow-up, including lung cancer, COPD exacerbation, major adverse cardiac and cerebrovascular events (MACCE), and all-cause mortality.

• Paper 2: Song et al. 2026, examined patients with hepatitis B at baseline, and examined outcomes of hepatocellular carcinoma (HCC; a type of liver cancer) at the ~5 year follow-up.

The Critical Flaw

The critical flaw is that the “EC switchers” group still includes cigarette smoking. “Switched” usually means they switched completely away from cigarettes. Here, it doesn’t mean “fully switched,” or even “mostly switched”; it just means they used e-cigarettes, and they don’t distinguish from people who switched completely versus dual-used. The latter group are still at risk for ongoing worsening of health because of their cigarette smoking, and including these people in the “EC switchers” group blames this on e-cigarettes.

Additionally, even I don’t think the baseline sample was necessarily exclusively smoking. It’s hard to tell because the sample inclusion criteria aren’t described thoroughly, but I think the baseline sample includes dual users. This further biases the comparison because it combines people who switched or quit quickly after baseline, with those who might have been dual-using for some time before baseline (and who might be “stuck” in a dual-use state, who tend to have heavier smoking history and more difficulty switching, and who, due to the error above, count as “EC switchers”).

Harm Reduction Prevails Anyway

Despite the pessimistic bias of including dual-use in the “EC switchers” group, they actually did have favorable health outcomes compared to the exclusive smoking group, demonstrating harm-reduction:

• Paper 1 (COPD) found that those using ECs after 1 year had significantly lower odds of MACCE after 5 years (adjusted hazard rate (AHR) = 0.80) which was a similar reduction in risk found in those who quit all nicotine (AHR = 0.85).

  • Other outcomes (COPD exacerbations and all-cause mortality) didn’t show a significant difference, but that’s likely because of the main flaw here, that smoking is occurring in both comparison groups.

  • However there were additional analyses run (that I think were added only during peer review; see below) that separate out the complete EC switchers from the dual-users. None of those reach statistical significance because of the small group size (in some cases, too small for reliable statistics), resulting in wide confidence intervals. The non-significance is used to dismiss a difference, but the effect sizes tell a different story. The table below is my compilation of the paper’s Table 3, and it shows that with the exception of lung cancer, all other outcomes have a similarly-sized reduction for complete switchers and complete nicotine abstainers:

    

• Paper 2 (liver cancer) found that those using ECs after 1 year had significantly lower odds of developing liver cancer after 5 years (AHR = 0.78), which was also exactly the same risk reduction found in those who quit all nicotine (AHR = 0.78).

Bonus: Evidence of Good Peer Review?

Paper 1, as of the time of writing this post, still does not have the final formatted version up on the website; it has the final submitted version (after the last peer review) while the formatting is being done. This is notable because when an author submits a revised paper, they have to highlight the text that was actually changed, to help the reviewers verify whether the changes are sufficient.

All analyses of the true complete EC switchers (in the table above) are in red font in the pre-formatted version of the paper, suggesting these were not in the original paper but were added because of peer review. This is a success of peer review because the paper would have been almost completely uninformative without it (except for the risk reduction in MACCE despite the confounding by ongoing smoking).

There’s something gaslighty about this, though, because the authors describe the analyses of true EC switchers as “co-primary analyses” which seems to imply the authors had thought of the idea themselves and planned these analyses from the start.

But, Paper Narratives Remain Negative

Paper 1 should have presented the analysis with the “true” EC switchers as the main analysis, especially after a diligent peer reviewer forced them to do it, since this is the only informative analysis in the paper. But it doesn’t; the abstract mainly refers to the flawed analysis where they call the comparison “EC switchers” but allow ongoing smoking to occur.

Paper 1 also is swift to dismiss non-significant p-values as “not providing comparable… benefits” compared to complete quitting. But this falsely implies zero benefit, when there was possible benefit according to the effect size (AHR ~ 0.8) that did not reach significance because of the small numbers.

Paper 2 does acknowledge the risk-reduction of switching to e-cigarettes, which was exactly the same magnitude of risk reduction as complete abstinence from nicotine (AHR = 0.78). In fact, the authors did follow-up analyses testing persistent quitters (i.e., remained abstinent from all nicotine at both follow-ups) with EC switchers, and found no significant difference and a similar effect size (AHR = 0.64 and 0.73, respectively), meaning switching to ECs and persistent nicotine abstinence are associated with similar risk reduction for liver cancer.

But then the authors in Paper 2 go on to recommend against switching to ECs anyway! Why? Based on conflating reach with effectiveness and making invalid comparisons:

Only 12% of EC switchers eventually quit smoking while 61% of initial quitters remained quit at follow-up…

Conclusions: Complete tobacco cessation should remain the primary strategy given superior behavioral sustainability, with no significant difference in HCC risk reduction versus switching.

Translation: Switching to ECs was associated with the same risk reduction in liver cancer as complete nicotine abstinence, but we recommend against switching to ECs because it’s not as stable/sustainable.

But obviously there’s a bias where some people found it easier or were more motivated to stop using nicotine entirely, and those people are more likely to stay abstinent. The EC switchers, on the other hand, didn’t even have to stop smoking entirely to be in this group! If they have dual-used for a long time, the chance of quitting entirely is lower, but it doesn’t mean they can’t reduce their risks by switching to ECs. This is the group who could benefit most from ECs.

Conclusion

Hopefully this is a lone flaw from one network of researchers, and will not become more widespread. Nevertheless, it’s something to keep an eye out for, that sometimes “EC switchers” means “EC switchers and dual users,” in future research and in any media coverage these studies get.