A quit-smoking app reports 55% active users. What does that actually show?

01

Reconstruct the invented dashboard

All people and numbers here are fictional. One arm of an imagined study offers 100 adults the same unbranded software version alongside ordinary support. The other arm receives the same ordinary support without this software; no outcome results for either arm are supplied. Here, ‘active at week four’ means at least one user-initiated interaction on days 22–28. Assume logs for this event are complete. A separate research team seeks an answer to a smoking-status question at six months, regardless of app activity.

Of the 55 people meeting the activity definition, 40 answer that later question and 15 do not. Of the 45 not meeting the activity definition, 20 answer and 25 do not. The activity percentage is 55/100 = 55%; the question-response percentage is (40+20)/100 = 60%. The table does not disclose what any answer said about smoking.

The 20 inactive responders show why stopping app use is not the same as disappearing from the study. The 15 active nonresponders show the converse. A faithful headline is ‘55% met the week-four activity definition; 60% supplied the six-month smoking answer.’ ‘55% quit’ and ‘all inactive users resumed smoking’ are unsupported. The 40/55 proportion describes responses among active people, not a substitute denominator for the entire assigned arm.

02

Give the software event its literal name

Find the event definition, observation window, denominator and log coverage. Downloading, opening a screen, completing an exercise and staying signed in are different measures. A notification being delivered is not necessarily a user interaction; a long session may include idle time. ‘Retention’ without these details is ambiguous.

Distinguish a recorded absence of a qualifying event from an unavailable log. Technical capture errors, changes in event definitions or a new software version can alter a metric without a corresponding behavioral change. Usage data are useful delivery information, not something to discard, but label what was actually measured.

[1][5]

03

Keep the smoking-outcome record separate

The outcome needs its own tobacco-product scope, abstinence window, assessment time and collection or verification method. Read how follow-up was sought for people who stopped using the software as well as those who continued. An app event cannot silently fill in a missing smoking answer.

Nonusage and loss to outcome follow-up may overlap but are not interchangeable. Inspect missing outcomes and their handling in both assigned groups. Treating missing answers as smoking is an analysis assumption when used, not a discovery from app inactivity and not an automatic guarantee against bias. Sensitivity analyses should show how reasonable alternative assumptions affect the result.

[1][2][3][5]

04

Which comparison did the study actually make?

If access to a defined package was randomly assigned, an appropriate assignment-based analysis addresses offering that package versus its comparator. It does not automatically isolate the effect of an extra session or one feature. Restricting the comparison afterward to frequent users can lose the protection of random assignment; missing outcome data remain a separate problem.

Frequent and infrequent users can differ in motivation, available time, support and early progress. Earlier progress or difficulty may also change later use. Check the order of measurement before treating the association as a cause. Statistical adjustment depends on measured factors and assumptions; it does not certify that all confounding has disappeared. More sophisticated adherence or mechanism analyses need additional methods and assumptions, not a simple users-only ranking.

[3][4]

05

Read the version, then leave the dashboard behind

Attach a study to its actual version, content, reminders, human support and comparator. A redesign, another product or a shared ‘quit-smoking app’ label does not automatically inherit its findings. This page attributes no cessation or health effect to a named app, and needs no reader's usage logs or smoking history.

For personal support, the NHS lists routes that differ across England, Scotland, Wales and Northern Ireland. Check the relevant local service or qualified professional. Your app activity is not a test of deserving help, and a dashboard is not a treatment-selection tool.

[1][5][6]

06

What to keep in mind

• Name the event, time window and denominator.
• Nonuse is not loss to research follow-up or proof of smoking.
• Check the separately measured outcome and actual comparison.