What Is Ad Testing?
Ad testing is showing an ad to people from your target audience before it runs, and asking them what they noticed, what they understood and how it changed their view of the brand. It is also called ad pre-testing or copy testing. The aim is to find out, before you pay for media, whether the ad gets its message across and whether one version works better than another.
Once an ad is live, the platforms tell you how many people saw it and clicked. They do not tell you what people took away from it, and by the time the numbers come in, the budget is spent. A survey-based ad test answers the questions that come first: do people get the message, do they believe it, and does it make them think better of the brand?
This guide covers what ad testing measures, the questions to ask in order, how to compare two or more versions, how many respondents you need and how to read the results. It is part of our concept testing guide, which covers testing ideas, ads and products before launch.
What Ad Testing Measures
Measure | What it tells you | Why it matters |
|---|---|---|
Message takeaway | What people think the ad is saying, in their own words | An ad that is liked but misunderstood sells the wrong thing |
Clarity | How clearly the ad gets its message across | Unclear ads are forgotten |
Relevance | Whether the ad feels meant for the viewer | Irrelevant ads are skipped |
Brand impact | Whether the ad improved or hurt the impression of the brand | The reason most brand ads exist |
Credibility | Whether people believe the claim | A claim nobody believes does not persuade |
Ad character | The words people would use to describe it | Shows tone problems, such as boring or confusing |
Recognition | Whether people have seen the ad before | Used for ads already running |
Message takeaway comes first for a reason. Most ad tests that go wrong are not about liking. People enjoy the ad and come away with the wrong idea, or with no idea at all.
When to Test an Ad
Before production: test the script, storyboard or a rough version. Changes are cheap at this stage.
Before launch: test the finished ad, or two finished versions, against each other.
While it runs: check recognition and whether people are tiring of it.
After the campaign: measure what the campaign changed with a brand lift study.
This article is about the first two: testing before the money is spent.
Ad Testing Survey Questions, in Order
These are the questions in the SegmentOS Ad Testing template, in order. Replace [brand] with your brand.
1. How familiar are you with [brand]? (I'm a regular customer / I've purchased from them before / I'm aware of them but haven't purchased / I've never heard of them before)
Then the ad is shown: an image, a video or the copy, as people would see it. The questions that follow are about recall and recognition:
2. Have you seen this advertisement for [brand] before today? (Yes / No)
3. Where have you seen [brand]'s advertising most recently? (TV / Streaming, Social media, Online / digital banner, Out-of-home (billboard, transit), Print / magazine, I haven't seen it before)
Message and communication:
4. In your own words, what is the key message of this advertisement? (Open text)
5. How clearly does this advertisement communicate its message? (1 Very unclear to 5 Very clear)
6. How relevant is this advertisement to you personally? (1 Not at all relevant to 5 Highly relevant)
An attention check comes next: "Quality check: For this question only, please select the third option from the list below to show you are reading carefully." Then brand impact:
7. After seeing this ad, how has your impression of [brand] changed? (1 Much more negative to 5 Much more positive)
8. How credible does this advertisement feel? (1 Not at all credible to 5 Very credible)
9. Which words best describe this advertisement? Select up to 3. (Memorable / Relevant / Believable / Engaging / Clear / Inspiring / Boring / Confusing / Misleading / Offensive)
10. What feedback would you give to improve this advertisement? (Open text)
Why the order matters. The familiarity question comes before the ad, so it is not influenced by it, and it lets you split the results by customers, people who know the brand and people who do not. The open question about the key message comes before the rating questions, so people describe what they took away before the ratings suggest what to look for.
For an ad that has not run yet, questions 2 and 3 mostly measure false recognition: people who say they have seen an ad that has never aired. Keep them if the ad looks like your past advertising, because a high "yes" means the new ad is easy to mistake for an old one. Otherwise you can remove them.
How to Compare Two or More Versions
Show each version to a separate, matched group of people, so nobody sees more than one. This is a monadic design, and it is the fair way to compare ads: once someone has seen version A, they judge version B against it. Our guide to monadic testing explains the design and how to keep the groups matched.
Change one thing between versions where you can: the headline, the opening scene, the offer, the visual. If the versions differ in everything, the test tells you which ad won but not why. For a framework to plan several rounds of variants, see our guide to creative testing.
How Many Respondents You Need
150 per version is the minimum for reliable percentages.
200 per version is the practical standard.
More if the versions are close. At 200 per version, two scores need to be about 9 points apart before the gap is statistically significant at 95% confidence.
Two versions at 200 each is 400 respondents. Test with people who match the audience the ad will run to, not with your team or your existing customers alone. The sample size calculator shows the margin of error for your numbers.
How to Read Ad Test Results
Code the key message answers. Read the open answers to question 4 and count the share who play back the message you intended. Take an example with illustrative numbers: if 45% of 200 people describe the intended message, the true figure is probably between about 38% and 52%. If most people describe something else, fix the ad before anything else, whatever its other scores.
Report top-two-box scores. For questions 5 to 8, report the share who answer 4 or 5. "61% find the ad clear" is easier to act on than an average of 3.8.
Look at the negative words. In question 9, a small share choosing "Confusing" or "Misleading" matters more than a large share choosing "Memorable." Those words point to the change to make.
Split by familiarity. An ad that works on people who already know the brand but not on people who have never heard of it may be relying on what they already know. If the ad is meant to win new customers, read the results of people who answered "I've never heard of them before" on their own.
Compare with your own past tests. No single benchmark fits every category and channel. Your best reference is the scores of your past ads, tested the same way, with the same questions. Keep every test so the next one has something to compare against.
Common Mistakes in Ad Testing
Asking about liking only. An ad can be liked and still fail to communicate. Ask for the message in people's own words.
Rating questions before the open question. The ratings suggest the answer.
Showing several versions to the same people. Each version needs its own group.
Testing with the wrong audience. Colleagues and loyal customers are kinder than the market.
Testing the finished ad only. By then changes are expensive. Test scripts and rough cuts too.
Test an Ad in SegmentOS
The SegmentOS Ad Testing template asks the 10 questions above, with the attention check, and has a place to insert your ad image or copy before you launch. To compare versions, create one study per version from the template and give each the same audience. It is a Premium template: publishing a study from it needs the Premium plan. To reach your target audience rather than only your own customers, the Pro plan adds a verified panel of more than 30 million respondents in 127 countries, with the cost of each study shown before you launch.
For a template that covers a product idea and its ad together, see our concept and ad testing survey templates. To test a single headline or tagline, see message testing.
Key Takeaways
What it is: showing an ad to your target audience before it runs, and measuring what they understood and how it changed their view of the brand.
Start with the message: ask what the ad is saying in people's own words before any rating question.
Measure five things: message takeaway, clarity, relevance, brand impact and credibility.
Compare versions fairly: one version per group of people, about 200 people per version.
Read the negatives: a few people calling the ad confusing or misleading is the clearest sign of what to fix.


