A tobacco study says ‘lower’: lower what, compared with whom?

01

An endpoint's date can differ from the title's date

Gale and colleagues' 2021 paper reports up to 180 days of heated-tobacco use. Yet Table 2's formal comparison for total urinary NNAL is at day 90: under the planned analysis, an exposure endpoint significant at day 90 was not retested for significance at day 180. NNAL is a marker of exposure to the tobacco-specific nitrosamine NNK, not a count of new lung cancers. Reading ‘180 days’ and ‘lower NNAL’ together does not create an observed reduction in cancer over 180 days. A machine-emission result would be another endpoint again, not this person's measured exposure.

[1]

02

Randomisation does not cover every group or every analysis

In that open-label UK trial, healthy smokers not intending to quit were randomised to continue cigarettes or use the specified heated-tobacco product. Smokers intending to quit entered the cessation group without randomisation; never-smokers were another group. It is not a randomised comparison of all four choices. Analyses also used per-protocol and biochemically compliant populations: 97 of the 127 product-group participants reaching day 180 met the study's CEVal compliance criterion. Individual biomarker analyses could contain fewer valid observations. Do not call that selected result the experience of everyone allocated, or extend exclusive-use findings to continued cigarette use alongside it. The manufacturer funded the study; that disclosure prompts scrutiny of design/reporting, not an automatic verdict on each measurement.

[1]

03

A review's colours are not effect sizes

Braznell and colleagues' 2025 review searched through 5 December 2024. Its 40 interventional studies were mostly five days or shorter; 32 had high and eight unclear risk of bias. Because biomarker data were too heterogeneous to pool, the authors plotted effect directions without using their size or statistical significance. A favourable colour therefore cannot be read as a large, certain clinical benefit. Adverse events were analysed separately; no statistically significant difference against a comparator is not proof of equal safety. The review identifies important uncertainty, not a safe winner or a count of diseases prevented.

[2]

04

Write a bounded conclusion instead of naming a winner

Fictional reading exercise: replace ‘This product is safe because the markers fell’ with ‘In this study, specified markers changed in specified analysis groups; allocation, adherence, timing and clinical-endpoint limitations remain.’ Keep the actual comparator attached. Lower than continued smoking does not mean equal to abstinence, and not measured or not detected does not mean absent. Before combining studies, match the product generation, exposure pattern, endpoint, units and follow-up; an omitted fact stays unresolved rather than being filled in from another abstract.

There is no safe tobacco product, and non-users should not start. This exercise is not advice to switch products or choose treatment. Personal health and quitting questions need qualified local help; NHS Better Health offers support information in England. No tobacco history, laboratory result or health information needs to be entered or sent here.

[1][2][3][4]

05

What to keep in mind

• Complete the comparison sentence before believing ‘lower’.
• Check actual allocation, analysis population and endpoint time.
• A direction plot or non-significant result cannot certify a safe product.