Test Purpose
The purpose of this test was to see if altering the UX experience on the order page with a stronger by providing a clearer visual aid on for unit selections would reduce confusion could increase clarity for the customer and increase improve CVR.
This concept went through 2 phase of testing:
Test V1 - Selections made by the customer where highlighted in yellow.
Test V2 - Selections made by the customer where highlighted in yellow + Arrow
Test Variables
CVR / AOV / Step1 Decline Rate. AOV
Summary of Results
...
TEST V1
TEST V2
Reference
Variables controlled in this test:
Device - Arrow element was displayed on Test V2 on mobile to keep it consistent with desktop.
Site - We controlled the site by using the same order page for both tests.
Results:
Note:
...
Overall both test had a negative effect on CVRVisitor sample sizes were too small on desktop to be useful, so for this report we will be focusing on mobile performance.
All variations failed to out perform the control CVR. There was a decrease in AOV for both test, which could be due to customers having better idea of what they were selecting. Decline rates also spiked during both tests, after further review there is no clear reason why this happened, and further testing would be required to understand. I would request a follow up test, perhaps trying a different colour to highlight the selections (light grey) while keeping the yellow box / arrow reserved for highlighting special deals.