What is a Control Group and how does it work?
Control groups aid in measuring the effectiveness of marketing campaigns by excluding a particular set of users from receiving a specific campaign and measuring its impact on revenue/conversions against users who receive it. Unlike A/B testing, where users experience variations of your message, control groups bucket your target audience into two groups – users who receive a campaign and those who don’t. By doing so, you can measure the impact of each group on conversions and understand the incremental value your marketing campaigns provide.
To understand how it works, let’s consider an example. Suppose you run an E-commerce store and to promote the launch of a new menswear collection, you decide to run a promotional marketing campaign to drive conversions. Your target segment consists of 10,000 users and using a control group of 10%, 1000 users will NOT receive the promotional campaign.
Further, to measure a control group’s impact, KPIs such as conversions, website or page visits, revenue, and more should be considered to understand the impact or uplift. In our example above, if you select conversions as your KPI, then comparing conversion rates between 9000 who receive the campaign vs 1000 users who don’t will help you understand the uplift your promotional campaign was able to create.
Why is it necessary?
By measuring the difference between a target group and a control group, you can understand how effective your marketing effort is against a measured KPI (such as conversions, purchases, or more). Without it, you would never know if your marketing efforts are creating a positive or negative impact.
By using Control Groups, you answer some critical questions, such as:
- Do push notifications promoting Black Friday Sale contribute to higher revenue?
- Which channels helped me get maximum conversions over the last quarter?
- Do email promotions help increase newsletter sign-ups?
Introducing WebEngage’s Universal & Custom Control Group
While your WebEngage dashboard supports applying control groups at a campaign level, we introduced Universal and Custom Control Group to address the need to assess the performance of multiple campaigns over time.
Universal Control Group: It creates a control group from the entire user base on your WebEngage dashboard and can be applied to multiple campaigns and journeys. The advantage of using this control group is that your control group users remain the same, i,e. you exclude the same set of users from multiple campaigns across channels. Unlike campaign-level control groups, where users within your control group change for each campaign you create, Universal Control Group help you standardize and measure the impact of your marketing activities on overall revenue.
Custom Control Group: It creates a control group that is mutually exclusive from the Universal Control Group and can be helpful to measure the impact of a seasonal or specific set of campaigns. By doing so, you can create a set of users that don’t already exist in your Universal Control Group and measure the impact of seasonal promotion campaigns such as the Diwali sale or Black Friday.
To know how to set it up and measure impact, read our documentation.
Why the need for a Universal Control Group?
Besides helping you measure the true impact of your marketing campaigns across channels and over a period of time, there are two more reasons why a Universal Control Group is necessary –
- Reduce cannibalization: Analyzing the campaign effectiveness over a defined KPI helps you identify whether you have cannibalized your target audience. For example, you sent two promotional campaigns to the same audience over 15 days (applying a universal control group for both campaigns). While analyzing the result, you find that the conversion rate for the control group is higher than the target group for the second campaign. Therefore, indicating audience cannibalization as the customers who did not receive the campaign ended up making a purchase nonetheless.
- Avoid overcommunication with your customers: Users are increasingly distracted by the sheer volume of communication. And if your communication is the source of uninstalls and unsubscribes rather than conversions, then a control group can help validate and keep message frequency in check.
How to select the relevant Control Group?
Let’s use some examples to understand when to use which control group.
Best practices to start with
Control Group helps you get the necessary insight into the real effectiveness of your marketing campaigns. However, it is not advisable to use them in every campaign. Simply because every missed interaction with a part of your users is potential revenue which is being missed. Therefore, here are some best practices to help you identify when to use control groups and when NOT to.
When to use:
- While running consecutive offers/ promotions: It would be a good idea to understand if consecutive offers or promotions are effective in getting conversions.
- Seasonal campaigns such as Halloween or Diwali: Creating and using a Custom Control Group can help understand the overall impact of marketing communications in driving conversions for seasonal promotional campaigns.
- Renewal or activation journeys: It would be helpful to measure the effectiveness of renewal communications across channels through stand-alone campaigns or journeys and optimize strategy accordingly.
When NOT to use:
- Sending transactional campaigns: Transactional campaigns such as payment completion and payment failure do not require a control group. These are important messages which must be communicated to the user.
- Collecting user feedback through surveys: Collecting user feedback, say, after a purchase, would not require a control group as all relevant customers need to receive this communication.
- Onboarding or educational campaigns: It is important that all new users receive some onboarding or educational information about your brand/services. Therefore, a control group is not necessary for such communications.
Things to watch out for in Universal Control Group
When setting up universal control groups, it’s crucial to consider key factors that ensure accurate and valuable results. By recognizing these challenges, you can establish a control group that delivers reliable insights and assists in forming data-based decisions.
Maintaining consistency
It’s essential to maintain consistency in your approach when using a universal control group for campaigns. Stick to the same standards when choosing members for the control group and evaluating their progress. Any variations in your approach can lead to misleading results and make comparing the impact of different campaigns difficult.
Monitoring behavior changes
Keep a close eye on any changes in the behavior of your control group members over time. If you observe a change in their engagement or buying patterns, it could suggest that external influences are impacting their decisions. In such instances, you might have to revise your analysis or consider revising your control group to ensure that it accurately reflects your desired target audience.
Take your control group analysis to the next level with the help of WebEngage’s Customer Data Platform.
Balancing short-term and long-term impact
Universal control groups are excellent for measuring the incremental impact of your marketing campaigns over an extended period. However, it’s essential to balance short-term and long-term results. Keep an eye on how your campaigns affect results, such as conversions and revenue, and their impact on customer loyalty and lifetime value in the long run.
Considering the opportunity cost
Using a universal control group means you are refraining from sending marketing messages to some of your audience. While this is necessary for accurate measurement, it’s important to consider the opportunity cost of not engaging with these individuals. Continuously evaluate the size of your control group and the length of your assessment to ensure you’re not missing out on significant revenue opportunities.
Communicating with stakeholders
Creating a universal control group can be quite complex, requiring involvement from various organizational stakeholders. Therefore, it’s crucial to convey the objectives, approach, and anticipated results of your analysis to all involved parties. Doing so will ensure backing for your efforts and facilitate better decision-making based on the insights you gather.
Wrapping up
Universal and Custom Control Group empowers marketers to make data-driven decisions by analyzing the impact of their marketing campaigns on overall revenue. We hope you try out this feature and share your feedback. If you need more assistance, get in touch with your Customer Success Manager or reach out to product@webengage.com to get started.
FAQs
What is a control group in marketing?
In marketing, a control group is a subset of your target audience intentionally excluded from receiving a specific marketing campaign or treatment. By comparing the behavior and responses of the control group to those who received the campaign (the test group), marketers can measure the incremental impact of their marketing efforts on key metrics such as conversions, revenue, and customer engagement.
How to select test and control groups?
When selecting test and control groups, you must ensure that both groups represent your target audience and are large enough to provide statistically significant results. Follow these steps to select your groups:
1. Define your target audience based on relevant criteria such as demographics, behavior, and preferences.
2. Randomly assign a portion of your target audience to the control group, typically around 10-20% of the total audience.
3. Assign the remaining audience members to the test group, which will receive the marketing campaign or treatment.
4. Ensure that there is no overlap between the test and control groups to maintain the integrity of your analysis.
What is a control group example?
Let’s say an e-commerce company wants to measure the impact of a promotional email campaign on sales. They select a control group of 1,000 customers who will not receive the promotional email, while the remaining 9,000 customers (the test group) will receive the email. By comparing the purchase behavior of the two groups, the company can determine the incremental impact of the email campaign on sales.
What is an example of a test group and a control group?
Imagine a mobile app developer who wants to assess the effectiveness of a new onboarding flow in improving user retention. They create two groups:
– Test Group: 50% of new users are shown the new onboarding flow.
– Control Group: The remaining 50% of new users experience the existing onboarding flow.
The app developer can determine whether the new onboarding flow positively impacts user retention by comparing the retention rates of the test and control groups over a specified period.
Take a demo with WebEngage today to implement these use cases, boost sales, and skyrocket your revenue.
Harshita Lal
