Control groups are an important part of any good marketing campaign. What are they? Essentially, they are small samples of your target population that are held back from your campaign, and their performance is used to test for an uplift in response (number of customers placing an order), and value (the value of sales made by those customers) against the customers you targeted – this tells you and other stakeholders if your marketing has been a success, and puts a value on the uplift achieved or not.
There are a number of important points to consider when selecting control groups. The first key decision to make is the size of your control group – this is very important. The size of your control group will influence how statistically significant your resulting campaign uplifts are. For example, if you were to launch a social media campaign to 100,000 customers, but decided to only have 100 customers in your control group, then no matter how impressive your results are, it won’t give you a statistically significant result, and that confidence that you’re heading in the right direction! Significance testing can be done by T-testing the population sizes against expected uplifts – we will dive into this at a later date.
The second important point is to ensure ‘no bias’ is present in either group. It’s no good picking all your weaker customers as your ‘control’ and then hailing a mighty success when you measure the performance of your stronger target group. The exact same criteria needs to be used to pick your control group as your target group.
The third point to consider is the spread of the control group population across the segments of your target customers. What this means is that when you’re performing a customer selection using a number of different segments and selection criteria (As is often the case), it’s important to ensure that those selected to be part of the control group are representative of these different criteria. The primary way that we would recommend to ensure this is to use a random sample (check that the method you use is indeed random – EG, not based on customer ID, age, geographic or any other factors that may introduce bias), and then once you have a sample, to test their metrics (value, frequency, average basket size, length of account etc) to check that your sample control group is a fair and reasonable representation of your selection.
A question we often get asked is ‘can we retrospectively pick our control groups’ – in other words, can we look back and pick a population of customers not marketed to use as a test? The answer to that is unfortunately both ‘yes’ and ‘no’, but in most cases ‘no’. Let me expand on that – as has been stated above, the control groups need to be an accurate and fair reflection of what may have happened if you had not run the campaign that you have run. In most cases, it’s nearly impossible to select customers that are exactly identical in every way – many businesses simply do not have the data available. Sales values are the easy part, but do you have marketing exclusion flags stamped historically on your database for example? For ‘prospect lists’ EG lists of potential customers this is even more difficult. My advice is to always pick your control groups at the same time as your target groups. Ensuring good, statistically significant results is the bedrock of good marketing practice.
Contact us for information on the ‘customer selections’ elements of our data marketing offer.