Page Comparison

• Every week, a CRO team member generates CRO Weekly Report (sample) identifying where traffic volume is, and what the strengths and weaknesses are per brand. We start with simple KPIs. This CRO Weekly Report is sent to Idea Group, and is composed of the following:
  - CVR, Profit Margin Health (Bad, Okay, Good), Upsell Take rates by product, purchases demographics, Presale/Int/Checkout CTR
  - Problem severity list by priority using categorization labels

• Ideas are to be submitted via Form/Criteria containing the below informationprescribed template to split-test-team@dfo.global:

1. Hypothesis – IF we change X, then Y will happen, because of Z

2. Business Value Estimate – what is your predicted impact on the KPIs involved?

3. Brand Specific? If so, which Brand – some ideas can be tested anywhere where the is traffic, some ideas are brand specific

4. Sample Pages, Idea source, etc. – any assets that may be useful to the CRO Team Member

• Pre-processing

1. Each request needs to be categorized as below:

   1. CVR

   2. AOV/Margin

   3. Remarketing

   4. Scale/Volume

   5. Processing

2. Confirmation of business value estimates

...

2. Submission System – this system outputs a “voting” rubric to the Evaluation Group. Evaluation Group is required to same day, vote binary whether they think the test will be successful or not. Prioritization will be first set based on “votes”, and if other ideas that week have the same number of votes => business value will be used to prioritize by CRO Management

   1. Voting results and submission success will be used to evaluate whether someone’s future ideas will be prioritized, and in which manner

   2. Evaluation Group membership will be determined by prediction

3. Evaluation System – Evaluation Group votes same day on the test. The members can say, yes or no. There are no abstentions.

CRO Team

• Voting System:

1. What ideas makes it onto our queue - the evaluation group will vote on each idea, and if an idea gets the majority vote (3+), it will be part of the week’s queue, up to 15 ideas per week.

2. Idea Prioritization -CRO Managementcollates output of Evaluation Systemand assigns priorities to each individual CRO Team Member. 

   Rank by category

   based on the below metrics:
   - Margin/AOV
   - CVR
   - Volume
   - Processing
   - Customer Satisfaction
   - Other

   Rank by Contribution

3. Impact Evaluation

greatest

business value
- Impact evaluation
- Greatest positive votes

...

• CRO Management provides design and development a list of priorities on a weekly basis, with a Thursday management call as support. These deliverables are based on component output of CRO Team Prioritization

• Creative and Development Managers assign out and execute upon the Asset Creation, informing and cooperating the CRO Team Member of any questions, changes, issues, etc. When content is live – CRO Team Member must be informed.

...

• CRO Team Member QAs and launches the test, or informs Creative and FE Managers of any issues or missing assets relating to Asset Creation

• Testing and Daily Updates – Live Update Google Sheet is updated at the end of each day, with basic results and a summary of what is live. Each stakeholder is responsible for checking this themselves

• Final Results Reporting – this has two components

  1. Weekly Report – Friday’s weekly report to inform general stakeholder group

  2. Immediate Test Report – to be sent to CRO Management

• CRO Management and CRO Team Member determine whether result is Brand specific, or Golden Standard. Test result is approved per Golden Standard System, and executed on that basis