How we handle numbers

How a study here is chosen, designed, run, reviewed and published

A programme funded by a company that sells into the market it studies has to publish its method before anybody should take its findings seriously. Method is what makes a result checkable by someone who does not trust us, and not trusting us is a reasonable starting position.

This page is the full account. It is long on purpose. If you intend to work with us, disagree with us in print, or decide whether a finding published here is worth citing, this is the page that should settle it.

The conflict, stated first

State of Ads is funded and staffed by Blowtrumpet, which builds advertising technology and holds a commercial position on several of the questions studied here. Specifically:

• Blowtrumpet benefits commercially if interruptive formats are shown to carry costs that interactive formats do not.
• Blowtrumpet benefits commercially if advertising is shown to work without processing personal data about the people who see it.
• Blowtrumpet is damaged commercially if game-based and interactive formats turn out to carry wellbeing costs of their own, or if contextual approaches measure materially worse than behavioural ones.

All three are live empirical questions and we do not know the answers. Writing them down in that order is deliberate: a reader can now check whether our published findings happen to line up with our commercial interest every single time, which would itself be evidence of a problem.

The structural answers. Ethical approval and data custody sit with the collaborating institution, not with us. Analysis plans are fixed before data exists. Collaborators keep authorship and conclusions, and no future collaboration is ever made conditional on a result. The funding disclosure appears on every output. Null and unfavourable results are published on the same footing as everything else. None of this makes the conflict disappear; it makes the conflict visible and expensive to act on.

Principles

Write the question down before you look. A hypothesis formed after seeing the data is a description of the data, not a test of anything.

Measure with instruments somebody else built. A metric invented in-house that happens to favour the inventor is not evidence.

Publish what you find. Including nothing. Including the opposite of what you expected.

Say what you cannot conclude. Most of the useful sentences in a study are about its limits.

One claim, one boundary. Our privacy claim is no viewer personal data — not minimised, not anonymised, not collected — and separately Blowtrumpet is a data fiduciary for creator and business-contact data. Precision on that boundary is a condition of publishing here, including where it makes the claim sound smaller.

The lifecycle

Stage	What happens	Who decides
1 · Question	A question is proposed, scoped and prioritised	Programme, openly, against published criteria
2 · Design	Hypotheses, conditions, sample, instruments and analysis plan are fixed	Lead researcher with methods review
3 · Ethics	Approval sought from the collaborating institution’s ethics committee	The institution, independently of us
4 · Registration	Design and analysis plan are registered publicly, with a timestamp	Lead researcher
5 · Collection	Data is gathered by the research partner under their framework	Research partner
6 · Analysis	The registered plan is executed; anything else is labelled exploratory	Lead researcher
7 · Review	Methods review inside, plus a reviewer independent of Blowtrumpet	Reviewers, who cannot change a conclusion
8 · Publication	Open access, with data and code where consent and licensing allow	Authors
9 · Afterlife	Corrections, replications, challenges, version history	Public, through a documented route

1 · Choosing a question

Where questions come from. Researchers proposing work, people inside Blowtrumpet who hit something they cannot answer, creators and agencies raising a problem at a roundtable, students with a thesis that overlaps, and policy people who need evidence that does not exist yet.

What gets prioritised. We weigh four things, in this order:

1. Would the answer change a decision? A finding nobody would act on is a publication, not research.
2. Is it genuinely open? If the literature already answers it, the useful contribution is a replication in an Indian context, and we say that is what it is.
3. Can it be answered honestly with the access we have? Ambition that outruns the available data produces a weak study and a strong press release. We would rather decline.
4. Does it risk embarrassing us? This one raises priority rather than lowering it. A programme that only studies questions flattering to its funder is advertising.

What we do not study. Anything requiring covert observation of people. Anything requiring identifiable data about viewers, which our own systems do not hold and will not start holding for a study. Anything whose design depends on deceiving a vulnerable group. Competitive intelligence dressed as research. And any question where a published answer would materially help somebody build a more invasive tracking system.

2 · Designing the study

Hypotheses and predictions are written as falsifiable statements before any data is collected, with the direction of the predicted effect stated. “We expect a difference” is not a hypothesis.

Comparison conditions are the point. A study of an interactive format against no advertising at all tells you little. The comparison that matters is against the conventional format the industry actually runs, with matched exposure time and matched content wherever that is possible, and an honest note where it is not.

Sample size is reasoned about in advance, from the smallest effect that would be worth acting on rather than from the number of people we can conveniently reach. Where a study is underpowered for its question — which happens, particularly early — the registration says so and the write-up repeats it. An underpowered null result is not evidence of absence and we never report it as one.

Instruments are validated ones wherever they exist. For attention, recall and emotional response we use established scales and task designs from the published literature, cited by name and version, rather than bespoke constructs. Where no validated instrument exists — which is the honest position for several wellbeing questions in advertising — we say so, describe what we built, publish it, and treat the first use as exploratory rather than confirmatory.

Blinding is used where it is feasible: subjects not told which condition is the one under investigation, and analysts working on coded condition labels. Where blinding is impossible because the formats are visibly different, we state that and treat demand effects as a named limitation rather than an afterthought.

3 · Ethics and the people in a study

Approval sits outside this company. Any study involving people runs under the ethical approval of the collaborating institution’s ethics committee or institutional review board. We do not constitute our own ethics committee, and we do not run human-subjects work without that approval, however small or informal the study looks.

Work is designed to the Declaration of Helsinki and, in India, to the ICMR National Ethical Guidelines. Where a study spans jurisdictions, the stricter standard applies.

Informed consent is obtained by the research partner, in a language the person actually reads, covering what will happen, how long it takes, what is recorded, who will hold it, how long it is kept, what will be published, and the fact that they may stop at any point.

Withdrawal is unconditional. Someone may stop mid-session or withdraw afterwards up to the point of analysis, with their data destroyed and no reason required and no consequence to anything they were paid.

People are paid for their time, at a rate set before recruitment and not contingent on completing the study or on producing any particular response. Travel and data costs are covered. Payment is never structured so that stopping early costs someone money, because a withdrawal right you have to pay for is not a right.

Under-18s. No study involving minors runs without specific ethics approval for that population, guardian consent alongside the young person’s own assent, and a design reviewed for whether the question could be answered without involving them at all. Given that this programme studies attention, compulsion and wellbeing, that last test is applied strictly.

Distress protocols. Wellbeing and mental-health work carries a written protocol: what the session lead does if someone becomes distressed, when a session is stopped, and what support information is offered. It is approved with the study and it is published with it.

Debriefing. Everyone is told what the study was actually about once their involvement ends, including where a cover task was used, and is given a route to ask questions afterwards.

Nobody is named, quoted or identifiable in a publication without separate written consent, which remains withdrawable. Closed roundtables run under the Chatham House Rule. No attendee list or contact detail is ever published, shared or used for marketing.

4 · Data

Who holds what. Where a study needs data about individuals, the research partner collects and holds it under their own ethical framework and their own legal basis. Blowtrumpet does not receive identifiable data about study subjects. What comes to us is aggregated or de-identified output, and the agreement says so before recruitment starts.

Our operating systems are not a back door. Blowtrumpet processes no viewer personal data — there is no tag, pixel, SDK, cookie, device identifier or viewer-side request anywhere in the advertising path, because creative is composited locally before broadcast. No study changes that. We will not instrument our production systems to collect viewer data for research, and a proposal that depends on it is declined rather than negotiated.

On the DPDP Act, 2023. The Act contains an exemption for processing necessary for research and statistical purposes, subject to prescribed standards. We do not build studies on that exemption as a way around consent. Human-subjects work here runs on informed consent obtained by the partner, which is a higher bar than the law may require and the right one for research that will be published under our name.

Anonymisation, described accurately. Most research data is pseudonymised rather than anonymised, and the difference matters: a key exists somewhere. Each study states which it is, who holds any key, and when the key is destroyed. We do not describe pseudonymised data as anonymous.

Retention and destruction. Raw data is kept for the period the ethics approval specifies, held by the partner, and destroyed on schedule. Analysis files supporting a publication are retained longer so that findings remain checkable. Both periods are stated in the study.

Security. Data under a study is encrypted at rest and in transit, access is limited to named researchers on the approval, and transfers happen through agreed channels rather than email attachments.

5 · Measurement

Several of our research areas suffer from a shortage of agreed measures, which is precisely why loose claims survive in this industry. Our rules:

Define before measuring. Every published study carries its operational definitions: what counted as attention, over what window, measured how. A reader must be able to disagree with the definition without having to guess it.

Prefer measures with a literature. Established attention, recall, affect and cognitive-load instruments, cited by name, over anything invented for the occasion.

Wellbeing is not a single number. Where we report a wellbeing outcome we report the instrument, the subscales, and the uncertainty, not a composite score that hides them.

Commercial and wellbeing outcomes are reported together. The point of the programme is that these are measured on the same page rather than in separate documents produced by separate departments with separate incentives.

A metric is never changed after seeing results. If the pre-registered measure produces an awkward answer, the awkward answer is what gets published, with a note on what we would measure differently next time.

6 · Analysis

The analysis plan is registered before data collection and is executed as written: the primary outcome, the statistical tests, how missing data is handled, what counts as an outlier and on what rule, and any planned subgroup breakdowns.

Anything not in that plan is labelled exploratory in the write-up, every time, without exception. Exploratory findings are presented as the hypotheses they are, not as results.

Effect sizes and intervals, not just significance. A p-value alone tells a reader almost nothing they need. We report the size of the effect, its uncertainty, and whether it is large enough to matter in practice — which is often the more damaging question for an interesting result.

Multiple comparisons are corrected for and the correction is named. Running twenty comparisons and reporting the one that cleared a threshold is how this industry’s statistics usually go wrong.

Null results are written up and published, in the same format and the same place as positive ones. A programme whose published record contains no null results has either been extremely lucky or has a filing problem.

Code and derived data are published where consent and licensing allow, so that the analysis can be re-run rather than taken on faith.

7 · Review

Methods review inside the programme happens at design stage and again before publication, and is about method rather than conclusions.

An external reviewer independent of Blowtrumpet reads any study making a substantive claim. Independent means no current commercial relationship with Blowtrumpet and no stake in the result. Reviewers are named in the publication where they agree to be.

What a reviewer can require: a clearer statement of method, acknowledgement of a limitation, a correction, a re-analysis, or a weaker claim where the data does not support a strong one.

What nobody can require, including us: a changed conclusion, a softened finding, a delay for commercial convenience, or the removal of a result. Blowtrumpet has no veto over findings published here, and the absence of that veto is the single most load-bearing sentence on this page.

8 · Publication

Open access by default, published under Creative Commons Attribution so that anybody may reuse the work with credit.

Every output carries the funding and conflict disclosure. Not in a footnote at the end — in the document, visibly, every time.

Preprints and registration links accompany publication, so that the registered plan can be compared with the final paper by anyone who wants to check whether they match.

Authorship is by contribution, recorded with the CRediT taxonomy so that it is clear who designed, who collected, who analysed and who wrote. Guest and gift authorship are not offered and are declined when offered to us.

Collaborators keep their authorship and their conclusions. We publish what they found.

Data and materials are made available to other researchers on request wherever consent and licensing allow, including the instruments, the stimuli and the analysis code.

Embargoes are used only to coordinate a release with a co-author or an institution, never to manage commercial timing.

9 · Work that is not a human-subjects study

Not everything here is an experiment, and the standard differs. Each output says which kind it is, on its first page.

Surveys and interviews with industry people run with consent and attribution rules, and report their sampling frame and response rate honestly. An industry survey is not a representative sample of an industry and we do not write it up as one.

Desk research and literature synthesis states its search strategy, its inclusion criteria and what it excluded.

Economic analysis of how value moves between an advertiser’s budget and a creator’s account states its data sources, its assumptions and its sensitivity to those assumptions. Where figures come from Blowtrumpet’s own operations we say so explicitly, because a single company’s book is not an industry.

Opinion and commentary is labelled as opinion and is not presented as a finding.

10 · Correction, replication and challenge

Corrections are made in the open, with a dated note recording what changed and why. Where a correction changes a conclusion, it is republished as a corrected version rather than edited quietly. Our handling of corrections, withdrawal and end-of-life follows the [Editorial Guidelines](/editorial-guidelines) and is aligned with COPE practice on corrections and retractions.

If you think a finding here is wrong, write to research@stateofads.com with the reasoning. We answer within ten working days. If you are right, we correct it and say who found it. If we disagree, we say why, and you are free to publish the exchange — we will link to it from the study.

Replications are welcome, including hostile ones. We will supply materials, instruments and code to anyone attempting to replicate a result, with no requirement to share findings with us first and no right of review over what they publish.

Every study carries a version history. What changed, when, and why.

11 · What this method cannot do

Setting out limits is part of the method rather than a disclaimer attached to it.

We are a small programme in an early stage. Sample sizes are modest, most studies are single-site, and a first result here should be read as a reason to run a bigger study rather than as a settled answer.

Our own operational data is not an industry sample. It describes one company’s book, in one market, under one model. Where we use it, that limit is stated.

Observational work does not establish cause, however suggestive the pattern, and we do not let a correlation become a recommendation between the abstract and the press summary.

We cannot fully correct for who we are. A researcher working with an industry funder may unconsciously frame questions favourably, and no disclosure fully solves that. The reason external review, pre-registration and published data exist here is that our good intentions are not sufficient evidence of anything.

Some questions we care about are not yet answerable with instruments anybody trusts, particularly around long-run wellbeing effects. We would rather say that than publish a number.

This page is versioned

The method itself is a document under revision, and changes to it are recorded. Where a study was run under an earlier version, the study says which version it followed. Suggestions for improving it are genuinely wanted — methodologists in particular.

Method version 1.0 · 16 October 2026

Research collaboration and challenges	research@stateofads.com
Corrections and withdrawal requests	editorial@stateofads.com
Programmes, events and roundtables	programmes@stateofads.com
Media	media@stateofads.com

[Read the research](/research) · [Editorial Guidelines](/editorial-guidelines) · [Care & Impact](/care-and-impact)

*State of Ads is the industry engagement programme of Blowtrumpet Technologies, South Delhi. State of Ads is a trade mark of Blowtrumpet Technologies. Research published here is funded by Blowtrumpet and published whatever it concludes.*