Split Test Team Deliverables and Outputs

Owned by Tim Shuvaloff
Last updated: Sept 17, 2019

Please reference "Types of Analyses" here to follow along with the requirements.

Team Deliverables and Workflow

  1. Make sure that launch requirements are followed and ready. These requirements should be in each optimization plan as soon as the Brand is assigned to the Brand Analyst.
    1. Keyword Analysis (example here), Customer Avatars (example here and here), and Basic Competitor Funnel documentation should be done prior to launch as well

  2. With the Brand Ambassadors, develop a comprehensive link list to put in the CRO Google Drives. BAs are to maintain this list, CRO areto use this list as reference on when to add HotJar assets, keep track of the Top 5 Affiliates on the offer and their funnels (including any presale assets that they developed themselves). The presale assets that are successful, should be analyzed for testing ideas and angles for future internal development

  3. Develop a Competitive Analysis and Opportunities Framework in the CRO Google Drives for each brand
    1. Competitive Analysis - What are our top competitors' (funnel) links? What are they doing (and why do you think they are doing it) that we can split test?. This will also help you determine brand new checkout split tests to send to design and development (as 1/3 checkouts should be developed from competitive analysis)
    2. Opportunities Framework - Identify basic strengths and weaknesses from Types of Analyses...analysis, and develop testing hypothesis around them and other insights. This will help keep track of priorities, prevent over-analysis on issues already identified, and help guide the Optimization Plans
  4. Brand Analysts are to develop a "Weekly Report" Tracking sheet with the below information:

    1. Offer Description
    2. List of Creatives and their IDs in Everflow
    3. List of Presales and their targets
    4. Top Funnels list
    5. Relevant split test results for their week and follow-up actions
    6. Split tests live, and next Top 3 optimization plan items
    7. Strengths/Weaknesses reporting - where are we sending traffic that is performing poorly, where are we not sending traffic that isn't? What are our traffic opportunities?
    8. Example of weekly report can be found here

  5. Perform Reviews along below schedule:
    1. WEEKLY - Weekly Report
    2. WEEKLY - 3 new testing ideas per brand, based on opportunities. Testing ideas are to be added to the Optimization Plan, and ticketing is to be ensured to be completed on the BA end each week.
    3. TWICE MONTHLY - Updates to Keyword Research and Angles document, as well as Competitive Analysis. Determine new angle and avatar for presale development.
    4. DAILY - review Web Analytics, Qualitative, and Mouse Tracking Analysis (heatmaps and scrollmaps)
    5. TWICE WEEKLY - review screen recordings, and perform heuristic and technical analysis
    6. DAILY - Update any live split test results into the "live tests" google sheet, and move any concluded tests to their respective CRO folder's google sheet - "Split Test Results Summary" (example here)
    7. DAILY - Make sure Optimization plan is chugging along, has new additions, and that you are aware of development tickets and ETAs. Once done, move to completed table, and launch the test in Google Optimize.


Summarized Team Responsibilities

Each brand should have below documents:

  • "Link List" - containing all offer links, their Everflow IDs, whether they have HotJar or not, the Top 5 Affiliates on the brand and what funnels they're running along with presale links they are running as well. 
  • Split Test Results Summary per Brand (and updating of brand independent Current Live Tests document in parent CRO folder)
  • Keyword Analysis and Customer Avatar Documentation (only needs to be reviewed every two weeks when coming up with new potential angles/avatars)
  • Basic Competitor Funnel list and documentation
  • Opportunities Framework
  • Optimization Plans in Confluence

Team's responsibilities are as follows:

  • Per item #5, standard review scheme
  • 3 split test ideas per week with tickets
  • Maintenance of above documentation
  • Weekly reports to traffic teams per Item #4
  • Creating new HotJar accounts and maintaining for new links
  • Launching and Reviewing split tests in Google Optimize