|
By Using Control Group Testing, You Can Modify Your Approach to Gain a Greater ROI
As budgets tighten, marketers are increasingly expected to justify every dollar they spend. Thus, it is increasingly important to ensure that you are making more effective use of your marketing dollars. In the 1:1 print marketing space, one of the best ways to
do that is to test, test, test.
But what are best practices for testing? Do you tweak one program to another, trying different variables to see what works? If so, how do you determine which differences in results are due to the change in variables and which are due to other factors, such as any subtle changes to the customer database or the impact of other factors, such as weather, political events or holidays, on recipient behavior?
These things are unknowable. That’s why you should always do your testing within the program itself. You do this by using control groups.
Is It Worth the Cost?
Say you are doing a mailing of 10,000 pieces, fully variable by text and image. You’re using five variables, including name, address, gender, age and income level. For every additional variable, there is an incremental cost, but you aren’t sure which variables are most important. You want to err on the side of caution, but you also want to be sure that you spend your dollars well.
Out of 10,000 pieces, you decide to set aside 1,000 pieces for testing. The bulk of the mailing - 9,000 pieces - uses your original set of variables. Then you separate out 500 pieces to be fully variable but to exclude the variable of gender. You separate out another 500 pieces with no variables, sending it as a traditional, static mailer to create a baseline.
Let’s say the response rate is 8.6% on the primary campaign. You compare that to the static control group, which received a 1.4% response rate. Since all other factors are equal, this tells you that the personalized campaign created a lift of 7.2 percentage points over static mail. What role did gender play? You look at your second control group. The response rate was 6.8%. So gender created a lift of 0.4 percentage points.
The Number Crunching Begins
How much additional value did that 0.4% bring? That depends on the value of the product and what percentage of respondents converted to sales. If you are selling oral bouquets, that additional 0.4% might not justify the added expense of the fifth variable. But if you are selling automobiles, it might put the ROI over the top. This is why, at each stage, it’s critical to know your costs so you can calculate the return on investment of each variation.
So on your next 1:1 campaign, consider adding control groups. When you do, here are some best practices to incorporate:
1. Include a baseline static control group, if possible.
2. Select large enough test groups to be statistically valid (100–200 pieces).
3. Use a random number generator to select your test groups so that they statistically mirror the larger universe.
4. Track your costs and the additional return at each stage.
The more you learn, the more powerful and cost-effective your programs will become. So track your progress. You might be surprised how quickly this additional intelligence will impact your bottom line.
<< Return to Main Menu
|
