Benchmark · Creative effectiveness

What makes an AI-generated ad convert? We scored the formats head to head.

By Orcool · June 30, 2026 · 6 min read

Why we ran this

Making an ad with AI is no longer the hard part. A script goes in, a talking-head video comes out, and you can do it before lunch. The open question, the one that decides whether any of this matters, is whether those ads actually work once real money sits behind them.

Volume is cheap now. Winners are still rare. So we stopped arguing about it and put a number on it.

The setup

Same app. Same offer. Same audience. Two ways to build the video.

The first was a popular AI-UGC tool: paste a script, pick an avatar, get a clean talking-head ad. The second was an agent pipeline that builds the ad from real signals first, actual user reviews, the hooks competitors are running, and the format patterns that have been holding up in the category, then writes the testimonial around those.

Both were scored blind on ABCD: Attention, Branding, Connection, Direction. It is the framework YouTube's own research ties to creative lift, so it is a fair, neutral referee that neither format was tuned to game.

signals what real users say  →  format what already wins  →  ad built, not prompted

The result

Same product, same length, same faces-on-camera premise. A 27-point gap on a framework built to predict whether an ad earns its spend.

What the gap was actually about

Here is the part worth keeping. The two ads were close on polish. Lip-sync, resolution, pacing, all fine on both. The difference lived in three places.

1. The first three seconds

The single-tool ad opened the way scripts want to open: a friendly greeting, a bit of setup, then the point. The agent-built version opened on the point, a specific frustration pulled straight from a real review. On a feed you get roughly one second to earn the second one. That opening was most of the Attention gap.

2. A claim a person would say

Generic praise reads as an ad. A user saying "I stopped typing notes with my thumbs" reads as a person. The agent version scored higher on Connection because it borrowed the actual words users use, not the words a brand wishes they used.

3. One clear next step

Direction is the least glamorous letter and the one the single-tool ad dropped. It ended soft. The testimonial ended on a single, concrete action. No menu, no hedge.

The winning ad was not better made. It was better informed. It knew what real users said and what the category was already rewarding before a single frame was rendered.

What this means if you buy media

Score before you spend. A creative-effectiveness score turns "which of these six do we run" from a hallway debate into a ranked list. It will not pick your winner for you, but it will tell you which ads are built like winners and which are quietly weak at the hook.

The hook is the product. Most of the gap here was decided in three seconds. If your AI workflow spends its effort on avatars and none on the opening line, you are polishing the part almost nobody watches.

Formats beat prompts. A better prompt gives you a nicer version of the same idea. Building from signals, real reviews and proven structures, gives you a different, stronger idea. That is the lever, and it is the one a single generation step cannot pull.

Frequently asked questions

Methodology note: this was an internal head-to-head run over five weeks on a single voice-to-text app, scored blind on the ABCD framework. The client is anonymized and the competing tool is described by category. A single test is a signal, not a law. We are publishing the method so you can weigh the result for yourself.