What is a Control Group and how does it work?
Control groups aid in measuring the effectiveness of marketing campaigns by excluding a particular set of users from receiving a specific campaign and measuring its impact on revenue/conversions against users who receive it. Unlike A/B testing, where users experience variations of your message, control groups bucket your target audience into two groups – users who receive a campaign and those who don’t. By doing so, you can measure the impact of each group on conversions and understand the incremental value your marketing campaigns provide.
To understand how it works, let’s consider an example. Suppose you run an E-commerce store and to promote the launch of a new menswear collection, you decide to run a promotional marketing campaign to drive conversions. Your target segment consists of 10,000 users and using a control group of 10%, 1000 users will NOT receive the promotional campaign.
Further, to measure a control group’s impact, KPIs such as conversions, website or page visits, revenue, and more should be considered to understand the impact or uplift. In our example above, if you select conversions as your KPI, then comparing conversion rates between 9000 who receive the campaign vs 1000 users who don’t will help you understand the uplift your promotional campaign was able to create.
Why is it necessary?
By measuring the difference between a target group and a control group, you can understand how effective your marketing effort is against a measured KPI (such as conversions, purchases, or more). Without it, you would never know if your marketing efforts are creating a positive or negative impact.
By using Control Groups, you answer some critical questions, such as:
- Do push notifications promoting Black Friday Sale contribute to higher revenue?
- Which channels helped me get maximum conversions over the last quarter?
- Do email promotions help increase newsletter sign-ups?
Introducing WebEngage’s Universal & Custom Control Group
While your WebEngage dashboard supports applying control groups at a campaign level, we introduced Universal and Custom Control Group to address the need to assess the performance of multiple campaigns over time.
Universal Control Group: It creates a control group from the entire user base on your WebEngage dashboard and can be applied to multiple campaigns and journeys. The advantage of using this control group is that your control group users remain the same, i,e. you exclude the same set of users from multiple campaigns across channels. Unlike campaign-level control groups, where users within your control group change for each campaign you create, Universal Control Group help you standardize and measure the impact of your marketing activities on overall revenue.
Custom Control Group: It creates a control group that is mutually exclusive from the Universal Control Group and can be helpful to measure the impact of a seasonal or specific set of campaigns. By doing so, you can create a set of users that don’t already exist in your Universal Control Group and measure the impact of seasonal promotion campaigns such as the Diwali sale or Black Friday.
To know how to set it up and measure impact, read our documentation.
Why the need for a Universal Control Group?
Besides helping you measure the true impact of your marketing campaigns across channels and over a period of time, there are two more reasons why a Universal Control Group is necessary –
- Reduce cannibalization: Analyzing the campaign effectiveness over a defined KPI helps you identify whether you have cannibalized your target audience. For example, you sent two promotional campaigns to the same audience over 15 days (applying a universal control group for both campaigns). While analyzing the result, you find that the conversion rate for the control group is higher than the target group for the second campaign. Therefore, indicating audience cannibalization as the customers who did not receive the campaign ended up making a purchase nonetheless.
- Avoid overcommunication with your customers: Users are increasingly distracted by the sheer volume of communication. And if your communication is the source of uninstalls and unsubscribes rather than conversions, then a control group can help validate and keep message frequency in check.
How to select the relevant Control Group?
Let’s use some examples to understand when to use which control group.
Best practices to start with
Control Group helps you get the necessary insight into the real effectiveness of your marketing campaigns. However, it is not advisable to use them in every campaign. Simply because every missed interaction with a part of your users is potential revenue which is being missed. Therefore, here are some best practices to help you identify when to use control groups and when NOT to.
When to use:
- While running consecutive offers/ promotions: It would be a good idea to understand if consecutive offers or promotions are effective in getting conversions.
- Seasonal campaigns such as Halloween or Diwali: Creating and using a Custom Control Group can help understand the overall impact of marketing communications in driving conversions for seasonal promotional campaigns.
- Renewal or activation journeys: It would be helpful to measure the effectiveness of renewal communications across channels through stand-alone campaigns or journeys and optimize strategy accordingly.
When NOT to use:
- Sending transactional campaigns: Transactional campaigns such as payment completion and payment failure do not require a control group. These are important messages which must be communicated to the user.
- Collecting user feedback through surveys: Collecting user feedback, say, after a purchase, would not require a control group as all relevant customers need to receive this communication.
- Onboarding or educational campaigns: It is important that all new users receive some onboarding or educational information about your brand/services. Therefore, a control group is not necessary for such communications.
Universal and Custom Control Group empowers marketers to make data-driven decisions by analyzing the impact of their marketing campaigns on overall revenue. We hope you try out this feature and share your feedback. If you need more assistance, get in touch with your Customer Success Manager or reach out to firstname.lastname@example.org to get started.