5 A/B testing mistakes that are killing your email optimization

After analyzing hundreds of A/B tests, these are the mistakes I see repeatedly.

1. Testing too many variables at once

Change ONE thing per test. Subject line OR send time OR design — not all three. Otherwise you cannot attribute results to any specific change.

2. Insufficient sample size

You need at least 1,000 recipients per variant for statistically significant results. Testing with 200 per variant is basically noise.

3. Not waiting long enough

Many marketers call a test after 2 hours. Wait at least 24 hours — some subscribers open email the next morning.

4. Ignoring downstream metrics

Open rate is not the only metric that matters. An emoji subject line might get more opens but fewer conversions. Always measure what matters to your business.

5. Not documenting learnings

Every test should add to a shared knowledge base. "Personalized subject lines improve open rates by 12%" is infinitely more useful than re-running the same test next quarter.