Internal - CRO Team Procedural Notes

Owned by Sida Jiang (Unlicensed)
Last updated: May 29, 2020
3 min read

Idea Group – Designers, Copywriters, and Split Test Team
Evaluation Group – Tim, Bruce, Jordan, Jeff, Sang, Krishna, Marco
CRO Team Member – Katie, Paul, Vanessa (unofficial)
Creative and Development Managers – An, Ngoc, Sang, Angus, Jonathan
CRO Management – Bruce, Sida, Tim

Monday - split test team sends out weekly report to evaluation group and idea group
Thursday - deadline for ideas submission, evaluation of ideas start on same day
Friday - EOW report for this week's tests

Request Layer (Idea Submission)

Request Layer (Idea Submission)

  • Every week, a CRO team member generates CRO Weekly Report (sample) identifying where traffic volume is, and what the strengths and weaknesses are per brand. This CRO Weekly Report is sent to Idea Group, and is composed of the following:
    - CVR, Profit Margin Health (Bad, Okay, Good), Upsell Take rates by product, purchases demographics, Presale/Int/Checkout CTR
    - Problem severity list by priority using categorization labels

  • Ideas are to be submitted via Form/Criteria containing the below prescribed template to split-test-team@dfo.global:

  1. Hypothesis – IF we change X, then Y will happen, because of Z

  2. Business Value Estimate – what is your predicted impact on the KPIs involved?

  3. Brand Specific? If so, which Brand – some ideas can be tested anywhere where the is traffic, some ideas are brand specific

  4. Sample Pages, Idea source, etc. – any assets that may be useful to the CRO Team Member

  • Sida to gather the ideas and inputs them as Jira tickets

Evaluation Layer (Idea Evaluation)

Evaluation Layer (Idea Evaluation)

  • Voting System:

  1. What ideas makes it onto our queue - the evaluation group will vote on each idea, and if an idea gets the majority vote (3+), it will be part of the week’s queue, up to 15 ideas per week.

  2. Idea Prioritization - CRO Management collates output of Evaluation System and assigns priorities to each individual CRO Team Member based on the below metrics:
    - Margin/AOV
    - CVR
    - Volume
    - Processing
    - Customer Satisfaction
    - Other business value
    - Impact evaluation
    - Greatest positive votes

  • Tim to finalize and assign out the top 5 priority tickets for each CRO

Asset Development

Asset Development

  • CRO Management provides design and development a list of priorities on a weekly basis, with a Thursday management call as support. These deliverables are based on component output of CRO Team Prioritization

  • Creative and Development Managers assign out and execute upon the Asset Creation, informing and cooperating the CRO Team Member of any questions, changes, issues, etc. When content is live – CRO Team Member must be informed.

Execution of Test Ideas

Execution of Test Ideas

  • CRO Team Member QAs and launches the test, or informs Creative and FE Managers of any issues or missing assets relating to Asset Creation

  • Testing and Daily Updates – Live Update Google Sheet is updated at the end of each day, with basic results and a summary of what is live. Each stakeholder is responsible for checking this themselves

  • Final Results Reporting – this has two components

    1. Weekly Report – Friday’s weekly report to inform general stakeholder group

    2. Immediate Test Report – to be sent to CRO Management

  • CRO Management and CRO Team Member determine whether result is Brand specific, or Golden Standard. Test result is approved per Golden Standard System, and executed on that basis