The test shows whether changing the cart CTA from “Continue” (A) to “Continue to Secure Checkout” (B) improves conversion.
- Dataset: 300 users (150 per variant), binary outcome
converted(0/1) - Primary metric: Conversion rate (purchase completed)
- Unit of randomisation: user
- Test: Two-proportion z-test, two-sided, α = 0.05
File: ab_test_data.csv (300 rows)
| column | type | description |
|---|---|---|
| user_id | int | anonymous user id |
| variant | text | A (control) or B (test) |
| converted | int | 0 = no purchase, 1 = purchase |
Top-line
| Variant | Conversions | Users | Conversion |
|---|---|---|---|
| A (control) | 24 | 150 | 16.00% |
| B (test) | 41 | 150 | 27.33% |
Effect
- Absolute lift: +11.33 pp
- Relative lift: +70.8% (27.33% vs 16.00%)
Significance (two-sided)
- z = 2.382
- p-value = 0.0172
- 95% CI for (p_B − p_A): [+2.10 pp, +20.57 pp]
- Decision: Ship Variant B (statistically significant and practically meaningful)
External verification: ABTestGuide (two-sided, 95%), A=24/150 vs B=41/150 → p = 0.0162, z = 2.4053, uplift +70.83%.
Variant B (“Continue to Secure Checkout”) outperformed the control with a conversion rate of 27.33% vs 16.00% — an absolute lift of +11.33 percentage points (+70.8% relative).
The effect is statistically significant (two-sided z-test, p = 0.0172, 95% CI [+2.10 pp, +20.57 pp]).
Decision: roll out Variant B.
Interpretation
- The stronger, reassurance-led CTA likely reduced hesitation at the cart step.
- The confidence interval suggests a real, positive improvement even at the lower bound (~+2 pp).
TL;DR: Variant B increased conversion from 16.00% to 27.33% (+11.33 pp, p = 0.0172).