CRO Memo to Idea Group

Open

To increase the efficacy of split testing, formalize and standardize procedures, and enhance cross-functional collaboration – we would like to open our split testing workflow.
This memo here will outline the basic standards and procedures moving forward for split testing. To briefly summarize:

1. The people on this e-mail will be receiving a curated report every week

2. Using the weekly report, the people on this e-mail, using a template, can submit ideas to split-test-team@dfo.global

3. The ideas will be voted on by an expert panel every Thursday

4. On Friday, Vietnam will receive a priority queue with tickets. If your idea is in that ticket – you will work with the CRO-in-charge to execute on the idea

5. Test will run – results will be reported to the Eval group in detail, the person with the idea originally, and summarized in the following week’s CRO report

6. If needs to be added to a Gold Standard, it will be

Any feedback can be sent to Sida and I directly. If you are to participate in this system, I expect you to know the rules and abide by them – or at the very least, provide constructive and meaningful feedback. This is not a perfect system, but we are taking a prototyping methodology to see if this concept has merit. Bruce, the Split Test Team, and I are very excited to take this next step, and we hope you are as well.

Idea group – anyone can submit ideas if they follow this submission template to split-test-team@dfo.global. The template is as follows:

• Idea – What you want to do

• Hypothesis – predict the result in the format of “By doing X, outcome would be Y, because of Z”

• Business Value Estimate – will it address CVR/Profitability/Scaling/Customer Satisfaction or Other Business Value?

• Brands – include the brands this needs to be tested on, if there is no specific brand, write general

• Additional Assets – include screenshots, URLs, or any reference points

To help create these ideas, the CRO team will be sending a weekly report on an ongoing basis. This will be a very guided starting point to help formulate ideas and enhance collaboration, and eventually develop a more robust understanding and reporting capability. You can choose to use this guide or not, but at minimum, you are expected to know what is running.

For example, let us say that you notice that Oshenwatch EN has a low average order value. You see that this is our highest traffic checkout page, and you notice that hey – why don’t we run a contest, and see if that increase conversion rate?

• Problem – I would like to run a contest give-away for $10,000 for two weeks, to see if the conversion rate increase outweighs the $10,000 giveaway

• Hypothesis – If we run a contest, it will increase conversion rate, because contests have been shown to increase user engagement and impulse buying patterns

• Business Value Estimate – I expect CVR to increase by 10%, and at 2500 sales a day at a $80 AOV, that would be an additional $20,000 in revenue a day

• Brands – Oshenwatch

• Additional Assets – skipping for the example

When you submit these, it is cool to make mistakes. If we do not have enough to go on, we will let you know.

Afterwards, the idea will be evaluated by an expert committee to determine priority. Every week, there will be a certain amount of tickets in the priority queue – the person who came up with the idea, will be consulted, involved, and given the final reporting accordingly.

• Ideas are to be submitted with the prescribed template to split-test-team@dfo.global

My expectations for this democratized system from creative personnel is that you support the person with the idea, in executing that idea in the way they want. Such respect will be afforded to you for you ideas and is expected in reverse situations.