The Rules of a Credible Test
One Variable at a Time
Test subject line OR opening line OR CTA — never all three. Otherwise you don't know what caused the difference.
Split the Same Segment
Send both variants to the same audience, same day, same time. Comparing fintech founders to healthcare execs isn't a test.
Big Enough Sample
Minimum 200 recipients per variant. Under that, random noise dominates the signal. 500+ is much more reliable.
Run for a Full Week
People open on different days at different times. A one-day test misses the tail.
Measure the Right Thing
Testing subject lines? Measure opens. Testing CTAs? Measure clicks. Testing overall copy? Measure replies.
What's Worth Testing
Subject Line
Highest ROI test. Length, personalization depth, question vs. statement, lowercase vs. sentence case.
Opening Line
The second-most-read line. Test personal reference vs. value-first vs. mutual connection.
CTA Style
"Worth a 15-min chat?" vs. "Open to a quick call next week?" vs. "Should I send a 2-min Loom instead?"
Send Time
Tuesday 10am vs. Thursday 2pm. Bigger effect than most people expect, especially for opens.
Follow-Up Cadence Spacing
3-5-13 vs. 3-7-14. Longer gaps sometimes improve reply rates without hurting volume.
What's Not Worth Testing
Font sizes, colors, or button styles inside plain-text outreach — the differences are noise-level for cold email.
Subject line emoji vs. no emoji when your list is B2B execs. You already know the answer.
Anything on a sample smaller than 200 recipients. You'd need a 40%+ lift to be statistically confident, which is unrealistic.
Multivariate tests (3+ variants at once). You'll rarely hit the sample size needed to distinguish them.
Reading Results Honestly
A 25% lift on 100 recipients could easily be random. The same 25% lift on 1,000 recipients almost never is.
Look for consistent direction across multiple tests, not one big win. A subject-line pattern that beats the control 4 out of 5 times is real; a single 40% winner might be a fluke.
MyProspectPro's dashboard shows per-campaign open and reply rates side-by-side. Save winners as your new control and test against them.
Do
- Isolate one variable per test.
- Randomize which contacts get which variant.
- Run tests until at least 200 recipients hit each variant.
- Save winning variants as your new baseline.
- Log every test — assumption, variants, sample size, result.
Don't
- Don't call a winner on 30 recipients per side.
- Never change multiple variables — you'll never know what caused the lift.
- Skip tests on wildly different segments; compare apples to apples.
- Don't test forever on a low-volume list — you'll never get significance.
- Never let a small-sample "win" override qualitative judgment.
Pre-Send Checklist
- 1Hypothesis written in one sentence.
- 2One variable defined; everything else held constant.
- 3≥200 recipients per variant, randomly assigned.
- 4Success metric decided in advance.
- 5Test runs for a full 7 days before calling it.
- 6Result logged even if inconclusive.
Frequently Asked Questions
What if I only send 100 emails a week?
Skip formal A/B tests. Instead, iterate qualitatively — read replies, look for patterns, refine copy monthly. Small lists don't produce statistical certainty.
How do I split my list evenly?
In MyProspectPro, create two campaigns targeting the same segment with different templates. Split by even/odd recipient count or randomize the segment tag.
How long before results are trustworthy?
Seven days minimum, ideally 10–14. Opens keep trickling in for a week after send; replies for two.
Should I A/B test the follow-up emails too?
Yes — follow-ups often outperform first touches, so testing them compounds. Same rules apply: one variable, same segment, enough sample.
Stop Guessing. Start Testing.
MyProspectPro handles the sending, tracking, and follow-ups so you can focus on the message.