View Source

Idea Group – Designers, Copywriters, and Split Test Team
Evaluation Group – Tim, Bruce, Jordan, Jeff, Sang
CRO Team Member – Katie, Paul, Vanessa (unofficial)
Creative and Development Managers – An, Ngoc, Sang, Angus, Jonathan
CRO Management – Bruce, Sida, Tim

note

Monday - split test team send out weekly report
Thursday - deadline for ideas submission, evaluation of ideas start on same day
Friday - EOW report for this week's tests

Request Layer (Idea Submission)

Every week, a CRO team member generates CRO Weekly Report (sample) identifying where traffic volume is, and what the strengths and weaknesses are per brand. This CRO Weekly Report is sent to Idea Group, and is composed of the following:
- CVR, Profit Margin Health (Bad, Okay, Good), Upsell Take rates by product, purchases demographics, Presale/Int/Checkout CTR
- Problem severity list by priority using categorization labels
Ideas are to be submitted via Form/Criteria containing the below prescribed template to split-test-team@dfo.global:

Hypothesis – IF we change X, then Y will happen, because of Z
Business Value Estimate – what is your predicted impact on the KPIs involved?
Brand Specific? If so, which Brand – some ideas can be tested anywhere where the is traffic, some ideas are brand specific
Sample Pages, Idea source, etc. – any assets that may be useful to the CRO Team Member

Evaluation Layer (Idea Evaluation)

Voting System:

What ideas makes it onto our queue - the evaluation group will vote on each idea, and if an idea gets the majority vote (3+), it will be part of the week’s queue, up to 15 ideas per week.
Idea Prioritization - CRO Management collates output of Evaluation System and assigns priorities to each individual CRO Team Member based on the below metrics:
- Margin/AOV
- CVR
- Volume
- Processing
- Customer Satisfaction
- Other business value
- Impact evaluation
- Greatest positive votes

Asset Development

CRO Management provides design and development a list of priorities on a weekly basis, with a Thursday management call as support. These deliverables are based on component output of CRO Team Prioritization
Creative and Development Managers assign out and execute upon the Asset Creation, informing and cooperating the CRO Team Member of any questions, changes, issues, etc. When content is live – CRO Team Member must be informed.

Execution of Test Ideas

CRO Team Member QAs and launches the test, or informs Creative and FE Managers of any issues or missing assets relating to Asset Creation
Testing and Daily Updates – Live Update Google Sheet is updated at the end of each day, with basic results and a summary of what is live. Each stakeholder is responsible for checking this themselves
Final Results Reporting – this has two components
1. Weekly Report – Friday’s weekly report to inform general stakeholder group
2. Immediate Test Report – to be sent to CRO Management
CRO Management and CRO Team Member determine whether result is Brand specific, or Golden Standard. Test result is approved per Golden Standard System, and executed on that basis