Perfect

“Where should I start?”

I hear this question a lot from folks who are considering Intelligems specifically and price/offer testing more generally. And while I’m always happy to answer it (we’ll get to that), I think the question is actually reflecting uncertainty about what comes after starting: “What should I do next?”. I can relate. 

My cofounder Adam and I have talked about a newsletter for nearly two years (since our first one, designed for family, friends, and investors, wound down). Each discussion has usually started with—you guessed it—“Where should we start?”

The answer for us shouldn’t be too hard: We have a Notion board with 38 content ideas on it. Eight of them have even been started. But none of them have stuck, because we’ve wanted to find the absolute perfect starting point.

Here’s the thing about the perfect starting point, though: I’ve come to think that the effort to find it is really just a symptom of trying to solve the whole problem at once—instead of serializing the problem and solving it over time. (Which is why this newsletter is finally here—even though we haven’t figured out a name for it yet.)

This same psychological dynamic seems to be at play when I speak to founders who REALLY want to know where to start testing.

They want to solve the whole thing at once. 

But that’s basically an impossible task—and I can’t tell you how many times I’ve seen those same folks completely lose their interest in testing if their first test “loses”. Sometimes, they get a quick win and it spins up a positive feedback loop that gets them to their next test. I understand it, but it’s always struck me as a very funny dynamic.¹

On average, though, a test has between a 20% and 40% chance of “winning.” So, even if a single test could solve the whole problem (it can’t), those founders are starting from a place where their motivation is destined to wane. They WILL experience losing tests, it’s just a matter of when.

So—back to the original question—where should you start?

The answer to serializing the problem, and solving it over time, step by step, lies in the P&L.

Putting aside the initial motivation for testing, your P&L is your testing roadmap, because it shows you where your business is strong and where your business is weak.

When it comes to profit growth experimentation (which is what price/offer testing aligns to), it’s less about optimizing for clicks and more about engineering profits. It’s business testing from a marketing standpoint. So, you just start where your P&L says you’re weak.

For example: Maybe your margins are strong, but your AOV is behind benchmarks.

If that’s the case, maybe you start by testing whether you can raise your free shipping threshold. And then you can work testing various components of a bundling strategy.

After you improve AOV, you can move on to whatever is the next biggest area for improvement in your P&L. Your order of operations are tied to what will deliver the biggest impact for your business.

When you frame it from the perspective of your P&L, your roadmap sort of writes itself. It gives you which themes to tackle in which order. You just fill in the blanks with ideas related to each theme.

If you do that, “What’s next?” is always right in front of you. And that makes starting a lot less intimidating.

— Drew

1   BTW - recently heard a war story from someone doing AB testing in the early 2010s that one of the major players would actually bias the data on a customer’s first test in order to try to get them a winner≈for exactly this reason. I hope it’s apocryphal—that would be total malpractice to fudge the data—but it is pretty funny and just shows that this is something burned into our nature.