Risk of bias in a quit-smoking study: follow one result, not a paper's reputation

01

Name the result before reading its colour

A six-month estimate of assignment to additional support differs from an estimate restricted to people who attended it. Record what was compared, the definition of the outcome, the follow-up and the analysed population. Assignment and adherence are different questions; assessing the latter needs appropriate methods, not simply deleting non-attenders.

Some information, such as how the allocation sequence was concealed, can be shared across results in the same trial. Missing data, measurement and the selection of an analysis can differ. An early result's reassuring judgement therefore does not automatically carry over to a later result.

[1][2]

02

Practice: thirty absent from an analysis does not mean thirty unmeasured

Everything in this exercise is invented. A trial assigns 300 adults to a leaflet, two offered conversations and optional reminders, and 300 to the same leaflet and conversations without reminders. An independent central system reveals assignment only after enrolment is confirmed. The planned question is the effect of assignment on self-reported absence of cigarette smoking in the preceding seven days, assessed at six months.

In group A, 12 people have no six-month answer; another 18 answer but are excluded because they attended neither conversation. Group B has 8 without answers and 10 further answerers excluded for the same attendance rule. The headline analysis thus uses 270 people in A and 282 in B. The table separates the two reasons rather than describing everyone omitted as lost to follow-up.

The interviewer is unaware of assignment, but participants know whether reminders were offered. The report mentions several smoking measurement windows; the dated analysis plan is not available in the materials supplied. No smoking event counts, effect estimate or actual bias rating are supplied here.

A useful reading note would say: ‘The headline excludes measured non-attenders as well as people without outcomes; I need the analysis relevant to assignment, the missing-data reasons and the measurement and selection explanations.’ It would not say ‘48 people failed to quit’ or assign a colour from these counts alone.

03

Locate the cause, not just an omission

RoB 2 examines five possible routes: the randomization process, deviations from intended interventions, missing outcome data, outcome measurement and selection of the reported result. Randomization requires both a random sequence and protection against foreseeing the next assignment. A baseline difference compatible with chance is not itself evidence of systematic bias; a balanced table does not repair an exposed allocation sequence.

For an assignment effect, examine whether departures caused by the trial context could affect outcomes, and whether the analysis preserves the assigned comparison. Not attending an offered intervention is not automatically such a departure. Excluding eligible people with measured outcomes because of attendance is an analysis issue in the deviations domain. Outcomes genuinely unavailable belong in the missing-data domain: consider reasons in each arm, possible dependence on the unobserved outcome and the assumptions and sensitivity analyses. Do not count the same exclusion as both problems.

For measurement, ask who supplies the outcome, whether methods are comparable across arms and whether knowledge of assignment could influence it. A blinded interviewer does not make a participant-reported answer blinded. Neither ‘open-label’ nor ‘self-report’ alone supplies the complete judgement; the outcome and the tool's questions matter.

For selection, compare the specific measurement and analysis with intentions finalized before unblinded outcome data were available. Unexplained differences deserve checking, not invented motives. Choosing a favourable reported estimate from several eligible analyses is different from an entire outcome never being reported; the latter also requires review-level examination of missing evidence.

[1][2][3]

04

Read the reasons behind low, some concerns or high

These RoB 2 categories are judgements, not percentages or points to average. Overall low risk requires all domains to be low for that result. A high-risk domain makes the result high overall; multiple concerns may also be serious enough for high overall risk. Keep the reasoning, especially when assessors override a proposed algorithmic judgement. Without enough information, state what is unresolved instead of inventing a complete assessment or a direction of bias.

Funding interests warrant their own disclosure checks, but a sponsor's identity is not a sixth RoB 2 score or proof of truth or falsehood. Look for the concrete mechanism. Low bias risk does not settle precision, applicability or certainty of the whole evidence base, and high risk does not prove an intervention has no effect.

You can follow these public report fields without submitting your smoking history or health information. Personal support choices belong with a qualified local professional. NHS information lists different UK service routes; readers elsewhere should use their own local health services, not assume the same arrangements.

[1][2][3][4]

05

What to keep in mind

• Attach a judgement to a defined result and analysis.
• Separate unavailable outcomes from available outcomes excluded by a rule.
• Preserve domain explanations; do not average colours or infer motives.