Creative A/B testing: define a hypothesis before advertising
Two different pieces can spark a good creative discussion and a comparison that is hard to interpret. If the image, headline, offer and audience all change together, the result does not reveal which change mattered. Before asking an agency for several versions, define the question the test aims to answer.
This evergreen guide presents a form for preparing creative A/B tests. The aim is to turn “let’s discover which ad is better” into a bounded investigation. There is no standard number of people, fixed duration or promise that the experiment will end with a winner.
Creative A/B testing: write a hypothesis explaining the change
A hypothesis connects a creative choice to a response you intend to observe. Instead of “the blue ad will work,” try: “Showing the product in use, while keeping the rest of the piece unchanged, may make the proposition more understandable and increase the proportion of clicks to the product page.” This remains a hypothesis, not a confirmed prediction.
The official Google Ads experiments guide recommends a hypothesis linked to the business objective, testing one variable at a time and choosing metrics before starting. It also warns that changes to the base campaign during the comparison may make interpretation harder. Google Ads: planning experiments
For the client, the next question is: if the hypothesis is not supported, what do we learn? One possible answer is that the presentation change, under the observed conditions, did not provide enough evidence to change the choice. That conclusion is more useful than attributing a preference to the audience that the test could not demonstrate.
Choose the unit of change
If the variable is photography, keep the same headline, call to action and commercial terms. If you want to compare two complete messages, record that the difference is the overall creative proposition. In that case, do not later conclude that the result came specifically from the color or one headline word.
Consider the fictional store Mesa Clara, which sells organizers. Version A presents the item in isolation; version B shows the same item on a desk in use. The form keeps the offer and destination the same and defines photography as the main change. This is a planning simulation without campaign figures or real results.
Form field | Illustrative definition |
Question | Does an in-use photograph change the response to the ad? |
Versions | A: isolated product. B: product in its usage context. |
Preserved elements | Text, commercial terms and destination page. |
Primary metric | Click-through rate, if that is the approved objective. |
Limit of the conclusion | The metric does not prove sales or a lasting preference. |
Owner | Professional configuring and monitoring the comparison. |
Agree on how versions will be compared
The creative form does not replace experiment setup. Ask the media manager to explain how audience, distribution and delivery conditions will be handled. Publishing one piece one week and another in a different week, without controlling contextual changes, does not automatically attribute the difference to the creative.
When the setup does not allow a suitable comparison, describe the work as exploratory observation. There may still be learning about questions or how the piece functions, but the conclusion must respect that limitation. Do not give the material a more precise name than the process performed.
In Google Ads, the basic A/B test mode for video assets is currently restricted to video reach campaigns using efficient reach and video view campaigns. Other types have their own paths. Confirm the applicable option before promising a particular tool. Google Ads: video experiment options
Define what would make the team stop or review the test
Before activating the versions, record failures that would invalidate interpretation: an offline page, changed commercial terms or an identified measurement error. These situations require operational handling. Leaving them unrecorded to “avoid touching the test” can produce a comparison that has already lost its reference.
Also agree on how the number of observations and result uncertainty will be assessed. This decision depends on the metric, the difference you aim to detect and the comparison design. The supplier should explain the approach used instead of presenting a universal sample size or stopping as soon as one version leads.
Read the result alongside the original question
Do the compared versions match the approved form?
Did the main change remain identifiable during delivery?
Did changes to the offer, audience, destination or measurement affect interpretation?
Does the report show the chosen metric and comparison limitations?
Is there sufficient basis for a decision, or is the result inconclusive?
Which conclusion can be applied without extrapolating beyond the observed period and context?
An inconclusive result does not require inventing a winner. The team can keep the current choice, reformulate the hypothesis or decide the cost of continuing is unjustified. Record the reason. This preserves learning and prevents an aesthetic preference from later being presented as a statistical conclusion.
When commissioning the pieces, provide the question and variable with the brief. Creative work then has a function in the experiment, and analysis receives a clear reference for what changed. To discuss media planning, explore Briccitly’s traffic management service. Briccitly traffic management
Comments