Test Report - Checkout Page Style Test

Owned by Tim Shuvaloff
Jul 31, 2018

Test Purpose

This test was aimed at determining if two different designs would beat control in terms of CVR. San Diego and Vancouver produced relevant designs, Vietnam developed them.

With that said, the following were the hypotheses for each variable:

  • CVR - Simplified Checkout and Brightcheck will show CVR increases
  • Paypal Ratio; Simplified Checkout will increase the ratio, Brightcheck will stay the same
  • Decline Rate; no significant difference
  • AOV; due to the Paypal hypothesis, Simplified Checkout will show decreased AOV 
  • Average Quantity - Simplified Checkout will show lower avg quantity, again, due to Paypal hypothesis

Test Variables

CVR, AOV, Average Quantity, Decline Rate, Paypal v. CC Ratio

Control - https://www.mindinsole.com/en/order.html

Simplified Checkout  - https://www.mindinsole.com/en/order-split.html

BrightCheck  - https://www.mindinsole.com/en/order-bc.html

Summary of Results

Increase/Decrease in CC v. Paypal Ratio

As you can see; Simplified checkout increase Paypal usage, which is in-line with our hypothesis.

The end result is that simplified checkout would typically result in a lower AOV, and brightcheck in a higher one, but you'll see below that this is not necessarily the case.

Increase/Decrease in AOV

BrightCheck - 10% increase in AOV, due to Average Quantity

Simplified Checkout - 5.7% increase in AOV, due to Average Quantity AND Warranty Take Rate


Increase/Decrease in CVR

No statistically significant difference between Control and Simplified Checkout

Brightcheck performed 6% worse in CVR relative to Simplified Checkout or Control.


Conclusions

The main purpose of this test was CVR; in that regard, Control wins. 

However, the unexpected AOV improvements are valuable; as such, combining both the improvement AOV, and the lack of change in CVR; Simplified checkout wins.

Next test is changing order of Paypal vs. CC