How to read a quit-smoking success rate

01

Find the count hidden behind the headline

For a binary outcome, the numerator counts people meeting the stated criterion; the denominator identifies the population used in that calculation. CONSORT asks trial reports to give the analysed numbers, available data at each outcome time and each group’s result. An enrolment total is not necessarily the denominator of every later percentage.

Keep ‘assigned’, ‘received the intervention’, ‘followed up’ and ‘analysed’ separate. If an advertisement says only ‘25% success’, the absent counts are a reporting gap, not permission to assume who was included.

[1]

02

One fictional cohort, two denominators

Invented numbers, not a study: 100 people enrol. At the stated assessment, 80 have outcome data: 20 meet the same fixed criterion and 60 do not. The other 20 have no outcome data. Dividing the 20 observed qualifying outcomes by all 100 gives 20%; dividing them only by the 80 with data gives 25%.

These are two calculations on the same cohort, not success rates for two interventions. The numerator has not increased. If the first calculation treats missing outcomes as not meeting the criterion, that is an analysis convention; it does not establish that those 20 people actually smoked. Neither calculation is automatically the unbiased answer.

03

Make the word ‘success’ do some work

A report should define the outcome and its assessment time. Check whether it counts stopping smoking, reducing consumption, attending visits or satisfaction; those are different outcomes. Also check whether the displayed figure is the prespecified primary outcome or a secondary result.

For a cessation result, keep the visit date separate from the behaviour window. ‘Assessed at six months’ might mean no smoking in the preceding seven days, rather than continuously for six months. Product scope and verification also affect whether two percentages describe the same thing.

[2][4]

04

Missing results need a method, not a guess

Look for how many results are missing in each group, why they are missing, the stated analysis method and any sensitivity analyses under different assumptions. Restricting analysis to available observations, imputing an outcome and using a statistical model are not interchangeable choices.

CONSORT explains that missing-data methods rely on assumptions. Simple replacement rules are not guaranteed to be conservative or unbiased. A missing person is not a measured outcome; a more complete-looking percentage cannot make that uncertainty disappear.

[3]

05

‘Twice as many’ and ‘ten percentage points’

Another arithmetic example, entirely fictional: two genuinely comparable groups each contain 100 people; 20 meet the endpoint in one and 10 in the other. Their proportions are 20% and 10%. The absolute difference is 10 percentage points; the relative ratio is 2. Neither means an extra 20 percentage points.

This invented contrast has no confidence interval, randomisation evidence or real intervention behind it. It demonstrates units, not effectiveness. In an actual trial, seek the absolute and relative comparison with its uncertainty, not just an impressive relative headline.

06

Leave uncertainty attached to the result

An estimated difference should come with its precision, commonly a confidence interval. A wide interval leaves more uncertainty about the size of the difference. Its presence does not repair biased follow-up or incompatible outcomes. Results from selected participants and a particular service context are not a personal forecast.

Before using a rate to choose care, bring the original report to a qualified local professional and ask: does this population, support setting, outcome and follow-up answer my question? That discussion is different from asking a website to calculate your own chance of quitting. In England, the NHS local Stop Smoking Service directory is a starting point for finding a service to contact; it does not establish that a published percentage applies to you.

[1][5][6]

07

What to keep in mind

• Save n/N, outcome, time and missing-data rule together.
• A response-only rate is not the same quantity as an all-enrolled rate.
• Compare like with like, and keep absolute difference, relative effect and uncertainty distinct.

08

Common questions