Ad Testing: How to Test an Ad Before You Spend on It

Ad Testing: How to Test an Ad Before You Spend on It

Ad Testing: How to Test an Ad Before You Spend on It

Written by

Written by

Patricio Luna

Patricio Luna

Published on

Published on

What Is Ad Testing?

Ad testing is showing an ad to people from your target audience before it runs, and asking them what they noticed, what they understood and how it changed their view of the brand. It is also called ad pre-testing or copy testing. The aim is to find out, before you pay for media, whether the ad gets its message across and whether one version works better than another.

Want to check this for your own idea?

Free to start. No card needed.

Once an ad is live, the platforms tell you how many people saw it and clicked. They do not tell you what people took away from it, and by the time the numbers come in, the budget is spent. A survey-based ad test answers the questions that come first: do people get the message, do they believe it, and does it make them think better of the brand?

This guide covers what ad testing measures, the questions to ask in order, how to compare two or more versions, how many respondents you need and how to read the results. It is part of our concept testing guide, which covers testing ideas, ads and products before launch.

What Ad Testing Measures

Measure

What it tells you

Why it matters

Message takeaway

What people think the ad is saying, in their own words

An ad that is liked but misunderstood sells the wrong thing

Clarity

How clearly the ad gets its message across

Unclear ads are forgotten

Relevance

Whether the ad feels meant for the viewer

Irrelevant ads are skipped

Brand impact

Whether the ad improved or hurt the impression of the brand

The reason most brand ads exist

Credibility

Whether people believe the claim

A claim nobody believes does not persuade

Ad character

The words people would use to describe it

Shows tone problems, such as boring or confusing

Recognition

Whether people have seen the ad before

Used for ads already running

Message takeaway comes first for a reason. Most ad tests that go wrong are not about liking. People enjoy the ad and come away with the wrong idea, or with no idea at all.

When to Test an Ad

  • Before production: test the script, storyboard or a rough version. Changes are cheap at this stage.

  • Before launch: test the finished ad, or two finished versions, against each other.

  • While it runs: check recognition and whether people are tiring of it.

  • After the campaign: measure what the campaign changed with a brand lift study.

This article is about the first two: testing before the money is spent.

Ad Testing Survey Questions, in Order

These are the questions in the SegmentOS Ad Testing template, in order. Replace [brand] with your brand.

1. How familiar are you with [brand]? (I'm a regular customer / I've purchased from them before / I'm aware of them but haven't purchased / I've never heard of them before)

Then the ad is shown: an image, a video or the copy, as people would see it. The questions that follow are about recall and recognition:

2. Have you seen this advertisement for [brand] before today? (Yes / No)

3. Where have you seen [brand]'s advertising most recently? (TV / Streaming, Social media, Online / digital banner, Out-of-home (billboard, transit), Print / magazine, I haven't seen it before)

Message and communication:

4. In your own words, what is the key message of this advertisement? (Open text)

5. How clearly does this advertisement communicate its message? (1 Very unclear to 5 Very clear)

6. How relevant is this advertisement to you personally? (1 Not at all relevant to 5 Highly relevant)

An attention check comes next: "Quality check: For this question only, please select the third option from the list below to show you are reading carefully." Then brand impact:

7. After seeing this ad, how has your impression of [brand] changed? (1 Much more negative to 5 Much more positive)

8. How credible does this advertisement feel? (1 Not at all credible to 5 Very credible)

9. Which words best describe this advertisement? Select up to 3. (Memorable / Relevant / Believable / Engaging / Clear / Inspiring / Boring / Confusing / Misleading / Offensive)

10. What feedback would you give to improve this advertisement? (Open text)

Why the order matters. The familiarity question comes before the ad, so it is not influenced by it, and it lets you split the results by customers, people who know the brand and people who do not. The open question about the key message comes before the rating questions, so people describe what they took away before the ratings suggest what to look for.

For an ad that has not run yet, questions 2 and 3 mostly measure false recognition: people who say they have seen an ad that has never aired. Keep them if the ad looks like your past advertising, because a high "yes" means the new ad is easy to mistake for an old one. Otherwise you can remove them.

How to Compare Two or More Versions

Show each version to a separate, matched group of people, so nobody sees more than one. This is a monadic design, and it is the fair way to compare ads: once someone has seen version A, they judge version B against it. Our guide to monadic testing explains the design and how to keep the groups matched.

Change one thing between versions where you can: the headline, the opening scene, the offer, the visual. If the versions differ in everything, the test tells you which ad won but not why. For a framework to plan several rounds of variants, see our guide to creative testing.

How Many Respondents You Need

  • 150 per version is the minimum for reliable percentages.

  • 200 per version is the practical standard.

  • More if the versions are close. At 200 per version, two scores need to be about 9 points apart before the gap is statistically significant at 95% confidence.

Two versions at 200 each is 400 respondents. Test with people who match the audience the ad will run to, not with your team or your existing customers alone. The sample size calculator shows the margin of error for your numbers.

How to Read Ad Test Results

Code the key message answers. Read the open answers to question 4 and count the share who play back the message you intended. Take an example with illustrative numbers: if 45% of 200 people describe the intended message, the true figure is probably between about 38% and 52%. If most people describe something else, fix the ad before anything else, whatever its other scores.

Report top-two-box scores. For questions 5 to 8, report the share who answer 4 or 5. "61% find the ad clear" is easier to act on than an average of 3.8.

Look at the negative words. In question 9, a small share choosing "Confusing" or "Misleading" matters more than a large share choosing "Memorable." Those words point to the change to make.

Split by familiarity. An ad that works on people who already know the brand but not on people who have never heard of it may be relying on what they already know. If the ad is meant to win new customers, read the results of people who answered "I've never heard of them before" on their own.

Compare with your own past tests. No single benchmark fits every category and channel. Your best reference is the scores of your past ads, tested the same way, with the same questions. Keep every test so the next one has something to compare against.

Common Mistakes in Ad Testing

  • Asking about liking only. An ad can be liked and still fail to communicate. Ask for the message in people's own words.

  • Rating questions before the open question. The ratings suggest the answer.

  • Showing several versions to the same people. Each version needs its own group.

  • Testing with the wrong audience. Colleagues and loyal customers are kinder than the market.

  • Testing the finished ad only. By then changes are expensive. Test scripts and rough cuts too.

Test an Ad in SegmentOS

The SegmentOS Ad Testing template asks the 10 questions above, with the attention check, and has a place to insert your ad image or copy before you launch. To compare versions, create one study per version from the template and give each the same audience. It is a Premium template: publishing a study from it needs the Premium plan. To reach your target audience rather than only your own customers, the Pro plan adds a verified panel of more than 30 million respondents in 127 countries, with the cost of each study shown before you launch.

For a template that covers a product idea and its ad together, see our concept and ad testing survey templates. To test a single headline or tagline, see message testing.

Key Takeaways

  • What it is: showing an ad to your target audience before it runs, and measuring what they understood and how it changed their view of the brand.

  • Start with the message: ask what the ad is saying in people's own words before any rating question.

  • Measure five things: message takeaway, clarity, relevance, brand impact and credibility.

  • Compare versions fairly: one version per group of people, about 200 people per version.

  • Read the negatives: a few people calling the ad confusing or misleading is the clearest sign of what to fix.

What do you want to learn?

Say it in a sentence. SegmentOS writes the survey, and real people answer it.

Free to start. No card needed.

In this article

No headings found on page

Stop guessing. Ask your actual buyers.

Stop guessing. Ask your actual buyers.

Build a study, reach verified respondents in your market, and get answers back in days.

Free plan available · No card required

Frequently Asked Questions (FAQ)

Frequently Asked Questions (FAQ)

What Is Ad Testing?

Ad testing is showing an ad to people in your target audience before it runs and asking what they understood, how clear and believable they found it, and how it changed their impression of the brand. It is also called ad pre-testing or copy testing.

What Questions Should You Ask in an Ad Test?

Start with what people think the key message is, in their own words. Then ask how clear and relevant the ad is, how it changed their impression of the brand, how credible it feels, which words describe it, and what they would improve.

How Many People Do You Need to Test an Ad?

About 200 people per ad version, and at least 150. To compare two versions, show each to its own group of about 200. If the versions are close, you need more people to tell them apart.

What Is the Difference Between Ad Testing and Brand Lift?

Ad testing happens before launch and tells you whether an ad communicates and which version is better. A brand lift study happens after the campaign and measures what it changed in awareness, consideration or purchase intent.

What Is Copy Testing?

Copy testing is another name for ad testing: measuring how people respond to an ad before it runs. Here "copy" means the whole ad, its pictures and sound as well as its words.

Insights & Updates

Explore articles, resources, and ideas where we share updates about the product, thoughts on technology, and lessons learned while building along the way.

Insights & Updates

Explore articles, resources, and ideas where we share updates about the product.