The State of Academic Research on Nicotine, Part 1

Common Flaws in Nicotine/Tobacco Research

This is a multi-part series on content that I first presented in my Michael Russell Oration at the Global Forum on Nicotine conference in Warsaw, June 2025 (full video). This is a high-level and wide-ranging synopsis of: (Part 1, here) the pervasive, but often preventable, flaws in research on nicotine and tobacco; (Part 2) how problematic incentive structures in academia have contributed to the state of the research; and (Part 3) what we can do about it. This is my own perspective, based on ~10 years in academia and ~5 years as a consultant to industry.

10,000-Foot View of the Research

Those of you who are well-versed in THR might want to skim or skip this section. But for my readers who are new to this area, I should cover the basics.

Research on nicotine/tobacco spans a wide range of disciplines, from chemistry and toxicology (to detect harmful exposures) to research on cells/living tissues and animals, to surveys of people, to randomized controlled trials, to population-level modeling. (My expertise is on the human- and population-level end).

Nicotine/tobacco products exist along a continuum of risk (some describe it as more of a “cliff” than a “continuum”), with combustion being the key determinant of risk. Cigarettes are harmful because of the combustion, not because of the nicotine (even though the nicotine is what can cause dependence).

Source: Global State of Tobacco Harm Reduction, 2023 briefing paper

When talking about the health harms of nicotine/tobacco products, there are two questions that are often conflated:

1. How harmful are e-cigarettes relative to cigarettes?

2. How harmful are e-cigarettes in absolute terms?

Note: I am focusing here on e-cigarettes because this is what most of the research focuses on. Nicotine pouches are newer and the research has only started, but everything I say below applies as much or more to pouches, which are most likely even lower risk than e-cigarettes.

On the first question of relative risk, it’s not a question anymore. Biomarkers of potential harmful exposures are substantially lower in people who exclusively use e-cigarettes compared to those who exclusively smoke cigarettes (e.g., see this article from my Pinney & Juul colleagues analyzing the national US PATH study). Biomarkers are a proxy for harm and not everyone buys the idea that they are relevant to human health outcomes, but results are similar when looking at more direct clinically-relevant outcomes. People who used to smoke but switched completely to e-cigarettes improved their respiratory symptoms (Goniewicz et al., 2020), including people with chronic obstructive pulmonary disease (COPD; Polosa et al. 2020). The National Academies of Sciences, Engineering, and Medicine, in a massive evidence review in 2018, concluded that e-cigarettes “are likely far less harmful” than conventional cigarettes ("Public Health Consequences of E-Cigarettes Report), and the UK Office for Health Improvement & Disparities concluded that “vaping poses a small fraction of the risks of smoking” (2022 Evidence Update).

On the second question of absolute risk, there is some (though much lower) direct risk of consuming nicotine alone, even in noncombustible form. However, some of this science is less well-established, especially when it comes to questions of clinical relevance to humans under realistic use conditions involving long-term and exclusive use of non-combustible products:

• Chemistry and toxicology studies show that some types of harmful emissions can be detected in heated tobacco products and e-cigarettes. However, these levels are not always quantified against established safety thresholds. Worse, many of these studies overestimate harmful exposures that would not occur in realistic conditions, e.g. operating e-cigarettes in a way that would be aversive to human users. (Note, for these studies I rely on the expertise of Roberto Sussman).

• Animal studies sometimes show negative health outcomes after exposure to nicotine or e-cigarette vapor. However, the animals are often forcibly exposed to massive doses of nicotine, which may not reflect self-controlled nicotine use under realistic settings, and health outcomes may not translate to humans.

• Human data are limited because there are very few people who use only noncombustible nicotine (e.g. most people who use e-cigarettes either currently smoke or used to smoke) for a long enough time to evaluate health outcomes (e.g. ~30-year latency period for lung cancer). However there is some informative data from Sweden, where snus (a noncombustible oral product) use has been common for a while, showing that rates of smoking-related disease are lower than the rest of the European Union (Smoke Free Sweden report). Additionally, e-cigarettes have been around long enough to evaluate respiratory symptoms; I recently published a systematic review with CoEHAR that found only mild symptoms (coughing/wheezing) to be a possible risk of e-cigarette use in people who never smoked (Caci et al. 2025; Selya et al. 2025).

Prof. Neal Benowitz, MD, recently gave a conference presentation summarizing the research on direct effects of nicotine (NTSC 2024 presentation). Nicotine alone might pose some health risks: reproductive toxicity, possible cardiovascular harms (especially in those who already have cardiovascular disease), and dependence (if you consider this a health harm), but data are unclear in some cases, and Prof. Benowitz emphasizes that these risks are “undoubtedly much lower than cigarette smoking.”

Hence tobacco harm reduction (THR): for people who would otherwise smoke cigarettes, separating the nicotine from the combustion and using a noncombustible nicotine product (e-cigarettes, pouches, etc.) can substantially reduce their harm.

State of the Research with Respect to THR

Back to the state of the research, THR is a very divisive issue. By number, the studies opposing THR dominate, based in large part on the research examining absolute risks: that nicotine is not completely safe and is addictive (though see the Addiction Ontology which notes that it’s better to use the term “dependence” rather than “addiction” when there’s no significant harm), as well as concern about uptake by youth and non-users. On the other hand, research supporting THR focuses on noncombustible products having lower harm relative to cigarette smoking and the fact that people often substitute one product for another while continuing to use nicotine.

The research field is also divisive with respect to industry involvement. Traditional “big tobacco” companies have been unwelcome in certain research forums for many years for their past bad behavior (misrepresenting their research on health harms and addictiveness of smoking). There is still strong hostility and distrust of industry in the research field — and this has extended to industry for noncombustible products as well.

Industry employees are not permitted to attend or present at some conferences, and this list is growing. Many journals have a formal policy against publishing manuscripts that were funded, in whole or in part, by industry, and this list too is growing. Conflicts of interest can pose a problem for research, but there are established methods of handling them: full disclosure of any perceived conflicts and reasonable skepticism based on the merits of the research. In nicotine/tobacco research, however, conflicts of interest are often used as a reason to remove platforms for some science, rather than as something to disclose and consider in interpreting the science.

Extending these bans to the noncombustible industry as well is unfortunate because A) these industries must conduct rigorous science for any hope of getting their products authorized by the U.S. FDA through the PMTA process, so why shouldn’t industry share this science and strengthen it through peer review? and B) noncombustible products are important part of the solution; in fact, maybe the biggest part, considering that e-cigarettes are the most commonly used method to stop smoking in both the US (Foxon & Niaura, 2025) and UK (Jackson et al. 2025).

Worse, the ostracization doesn’t stop at studies directly funded or people directly employed by industry. For example, as a consultant, I am not able to publish my client-funded work in many journals, but I even have trouble publishing my unfunded side projects that my clients did not ask for, pay for, nor are even aware of. I know of cases where academics who have never taken any industry funding have been rejected or banned because they had a co-author who took money from industry (“industry-adjacent”), or had funding that was two or three degrees away from tobacco industry (note: which is the same degree of separation as academic grants), or once gave free advice to someone affiliated with industry. I’ve even seen academic research criticized for making arguments used by industry (a guilt by association tactic, since these arguments were not addressed on their merits) or for “serv[ing] exploitative industry interests.”

Overall, this research field has become very polarized and toxic. It has gotten so extreme that several junior researchers published an editorial on the negative impact on their career development as well as the detriment to the scientific integrity of the field (Carroll et al. 2020). The scientific integrity is indeed suffering, as I explore below.

Common Flaws in THR Research

Every week as a pro-bono activity, Clive Bates and I informally review all the new research articles on nicotine/tobacco published in PubMed, which amounts to ~30-70 studies every week.

Week after week, we see the same fundamental flaws in many studies. We presented a session at the Global Forum on Nicotine conference in 2024 titled “Rating the Evidence: Good and Bad Science” (video) where we compiled the 10 most common flaws and solicited audience feedback. We are in the process of refining and writing up a “how-to” guide on interpreting and thinking critically about nicotine/tobacco research. Our current working list of 10 is below (but subject to change):

I won’t go into an exhaustive explanation of all 10 flaws here since we are still refining these and have already discussed them in the 2024 GFN session, but I will discuss some examples of what I mean.

Below are two representative examples, chosen mainly because they were recent in my memory. These are not even particularly bad papers, but that’s the point - they are very representative of the majority of new studies coming out each week. I do not really consider them problematic enough to write a Rapid Response commentary on the journal’s website or a PubPeer comment (as some people have asked me after my GFN talk) because these issues are pervasive in this field and I don’t believe a formal criticism wouldn’t have any tangible effect.

Side note: it’s also unintentional that both of these examples are published in the journal Tobacco Control — one of the highest-ranking journals in this field by standard metrics (e.g. impact factor 4.7). I only realized both of my examples were in Tobacco Control after I picked them.

Example Article 1: Nicotine strength in snus and pouches in Finland

Summary of Article 1

This article by researchers at the Finnish Institute for Health and Welfare analyzed 1) the prevalence of snus and nicotine pouch use among the general Finnish population (ages 15-69 years) and 2) the nicotine strength of the products used (self-reported by participants). This was based on data from a national survey in 2022.

The main results were:

• 29% of people ever used snus

• 14% ever used pouches

• ~30% of people who used one of these products didn’t know the nicotine level

The conclusions in the narrative centered on the authors’ concern about nicotine levels (including the lack of knowledge about nicotine levels):

• “Not knowing the nicotine level… was common among the Finnish population having used the products”

• “The nicotine levels and addictiveness of oral tobacco and nicotine products should be regulated”

This is the gist of the article and it contains at least 4 of our 10 common flaws.

Common Flaws in Article 1

• Flaw #2: Ignores continuum of harm. Snus and nicotine pouches are both very low on the continuum of harm, yet the narrative of the article is based on concern about people using these products, when (as reported in this very article) current smoking (highest on the continuum of harm) was much higher (22.8%).

• Flaw #4: Missing counterfactual. Implicit in the narrative is that if pubic health efforts (e.g. capping nicotine content) could prevent people from using snus or pouches, then they’d completely stop using nicotine. But what if they would smoke cigarettes instead (which I believe is more likely)? Either way, the public health implications are completely different depending on the counterfactual (what people would be doing otherwise), which should be thoroughly discussed in the article. But the authors do not explicitly state their assumptions, let alone acknowledge other possible assumptions.

• Flaw #5: Questionable definitions. There are many definitions of “use,” ranging from ever-use to established and daily use; these different definitions have very different health implications as I discussed in a commentary (Selya et al. 2024). The current article focuses on ever-use, the broadest possible measure: this inflates the number of “users” because it includes everything from daily use for an extended duration down to people who tried the product once years ago and never used it again (which poses negligible health risks). These issues are directly confirmed in the current article, which reports that ~60% of ever-use is “tried once or twice” and only ~25% is “current use” (i.e. used at all in the past month); this percentage drops further for daily current use (as opposed to occasional current use).

• Flaw #7: Inappropriate policy recommendations. The article’s conclusions call for regulations on nicotine levels in snus and pouches. But as Jukka Kelovuori pointed out to me, the data were collected in a year when both products were already illegal in Finland! Snus were and still are banned, but are often smuggled into the country; and nicotine pouches are now allowed to be sold, but this was after the survey year of 2022. The article does not reconcile their recommendations with the reality of Finland’s policy: even if nicotine levels could be regulated on an already-illegal product, why would they expect to work when a complete ban hadn’t? The kicker, as Patrick Strömer pointed out, is that the main concern in this article (that Finnish snus consumers often don’t know the nicotine content) was probably caused by Sweden’s tobacco control efforts: Sweden (a main source of snus smuggled into Finland) had previously banned labeling the nicotine content on snus, so no wonder Finnish snus users don’t know the nicotine content!

• Bonus flaw: Illogical analysis. While not quite common enough to make our list of 10, a key result was that “the use of snus [is] associated with the use of stronger snus.” This doesn’t make sense conceptually (I’d guess there’s something wrong with how missing data were handled, e.g. it doesn’t make sense to ask non-users this question, but the analysis seems to include data from non-users). Is the corollary that not using snus is associated with lower-strength snus? Something hasn’t been thought through here.

Example Article 2: Tobacco marketing exposure and tobacco use

Summary of Article 2

This article by US academic researchers analyzed a longitudinal survey of Texas starting in elementary school (~11 years) and continuing into young adulthood (~22 years). Participants were asked whether they remember seeing product marketing at tobacco retail outlets (TRO) (both cigarettes and e-cigarette marketing was asked about separately) and whether they used the corresponding product in the past month. Results showed a correlation:

• Self-reported exposure to cigarette TRO marketing was associated with past-month cigarette smoking (odds ratio [OR]=1.05).

• Self-reported exposure to e-cigarette TRO marketing was associated with past-month e-cigarette use (OR=1.06).

Key conclusions drawn by the authors:

• “Exposure to TRO marketing increased the odds of tobacco product use for youth ageing into young adulthood.”

• “TRO marketing restrictions are needed to reduce tobacco use among youth and young adults.”

Common Flaws in Article 2

• Flaw #3: Unfounded causal conclusions. The results are interpreted as a one-way causal relationship in which marketing exposure at an earlier time point causes some youth to use the product at a later time. But these observational data can’t establish causality, a fact which authors usually at least do lip service to (by stating it as a limitation), but that wasn’t done here. Nearly all studies of self-reported marketing exposure have the viable alternative explanation that youth who are interested in, or already using these products, are more likely to notice and remember seeing such marketing (attention and recall bias). (For studies on social media exposure, which wasn’t included here, there is another reverse-causality explanation, which is that social media users seek out and interact with content they’re already interested in, and the algorithms reinforce this content).

• Flaw #5: Questionable definitions and statistics. On variable definitions, TRO was defined as a composite variable that included simply visiting gas stations or convenience stores (along with the more relevant variable of noticing advertising in those stores). On statistics, the main results have almost comically-weak effect sizes (odds ratios of 1.05-1.06, when 1.00 is no difference). Odds ratios are a way to quantify relative (not absolute) differences, and when the base rate is low (i.e. most youth do not smoke or vape), relative differences inflate the absolute differences (a point demonstrated nicely by Sun et al. 2023). So when odds ratios are small, absolute differences must be minuscule. Since the paper did not report the absolute differences in smoking or vaping across TRO exposed vs. non-exposed groups, I played around with this R package and found (if I used it correctly) that if cigarette smoking is 5.000%, then those who reported cigarette marketing exposure would smoke at a rate of 5.002%. These are, for all intents and purposes, null results.

• Flaw #7: Inappropriate policy recommendations. The conclusions call for marketing restrictions to reduce youth use, but this recommendation implicitly assumes that it’s a causal, one-way relationship, which is not supported by the actual data (as I note above). If this association is not causal, then this action would not be effective. If attention and recall bias explain the results, then this would only prevent youth who already use these products, or are interested in them, from seeing marketing in convenience stores, and there’s no reason to think that would reduce use. Additionally, there are already heavy regulations on TRO marketing to youth, and the manuscript doesn’t address how these recommendations are different or would be expected to have an additional effect over what’s already in place. Finally, there’s no consideration of potential harmful indirect effects of this recommendation, such as a lost opportunity to make adults who smoke aware of or interested in lower-risk nicotine products.

Conclusions from Part 1

In Part 1 of this series, I hope I have shown convincingly that the nicotine/tobacco research field is in an unfortunate state, both with respect to the polarization and hostility and with respect to research integrity. I believe the two are related and that confirmation bias results in researchers interpreting ambiguous findings to support their “side” without critically thinking about their findings and conclusions. This problem is compounded when peer reviewers are on the same “side” and the flaws go unchecked and unchallenged. In my judgment, these problems are especially severe on the anti-THR side, but not exclusively; for example I’m seeing more lower-quality research on the pro-THR side as the tide turns on some issues.

In Part 2, I will explore the question “How did it get this way?” And my answer will go into the academic research system and the structural incentives that I believe contribute to the state of the nicotine/tobacco research.