- Founders' Newsletter
- Posts
- CAC
There's growing chatter in the DTC community that A/B testing can drive up your CAC and, because of that, testing is bad.
I'm far from an ads expert, but I do think the conclusion people are drawing is wrong.
And it feels wrong in a way that impacts growth: Even if A/B testing raises your CAC, that tells you almost nothing about whether your testing program is working.
The math makes this clearer than any argument I can make in writing.
Say you're running a test and you send 400,000 visitors to it, backed by $600,000 in ad spend. Because you're splitting traffic evenly, each variant gets 200,000 visitors and $300,000 in spend. The cost to get a visitor to your site is effectively the same for both groups — the ad auction doesn't know which landing page they're heading to.
| Control | Variant A | Variant B price ↑ | |
| Test setup | |||
| Total visitors | 400,000 | ||
| Total ad spend | $600,000 | ||
| Per variant | |||
| Traffic split | 50% | 50% | 50% |
| Visitors | 200,000 | 200,000 | 200,000 |
| Ad spend allocated | $300,000 | $300,000 | $300,000 |
| Conversion rate | 2.00% | 2.50% | 1.80% |
| AOV | $100 | $90 | $120 |
| COGS + fulfillment / order | $15.00 | $14.50 | $15.00 |
| Outcomes | |||
| Total orders | 4,000 | 5,000 | 3,600 |
| Total revenue | $400,000 | $450,000 | $432,000 |
| Total gross profit | $340,000 | $377,500 | $378,000 |
| Revenue per visitor | $2.00 | $2.25 | $2.16 |
| Gross profit per visitor | $1.70 | $1.89 | $1.89 |
| The metric everyone watches | |||
| CAC | $75.00 | $60.00 ↓ | $83.33 ↑ |
| The metrics that actually matter | |||
| Total contribution after CAC | $40,000 | $77,500 ↑ | $78,000 ↑ |
| Contribution / order | $10.00 | $15.50 ↑ | $21.67 ↑ |
Variant A converts at a higher rate and generates more total revenue and gross profit on the same ad spend. CAC drops from $75 to $60. This looks like a clean win by every conventional measure.
But now look at variant B. Our AOV is 20% higher than the control (driven by higher prices) and conversion rate drops to 1.8% — fewer people buy. CAC goes up to $83, which is the worst we see in this example.
What actually happened, though, is that gross profit per visitor hit $1.90 and total contribution after CAC hit $78,000, both higher than either alternative. Contribution per order is $21.67, nearly double the control.
So, while your CAC went up, you made more money per customer, more efficiently.
This is the ratio that actually matters: gross profit per dollar of ad spend.
CAC is a downstream abstraction of it, and a misleading one when conversion rate and AOV move in opposite directions, which is exactly what happens when you test prices or high-conviction page redesigns.
If you can make more gross profit per dollar of ad spend, then you can spend more money to acquire more customers.
Dave Rekuc, a long-time customer of Intelligems, illustrated this for me a few years ago. He found an offer that increased gross margin by 5%, and then took that efficiency and allocated it into his advertising budget, allowing him to reach a larger audience and acquire more customers faster. And because he was acquiring more customers faster, those gross profit dollars were growing at a faster rate, meaning his ad budget was growing, too.
He called it a compounding win. This is what testing can deliver.
So, yes, some of your gains may get eaten up by higher CAC, but in many cases plenty of your gains will be left over. And that leftover is now money you have to increase your budget and grow faster.