Ask Fuse: Testing (Part 1)
What are the most critical elements in setting up a test?
Testing with Purpose
Tests should be driven by your program’s annual goals & strategic priorities — where do you need to move the needle the most? Identify testing opportunities that can inform strategies designed to maximize the chance of achieving those goals.
Establish Your Hypothesis
All tests start with your hypothesis — what do you believe will be the outcome from this test? Identify the metric that you think will be impacted by this test. If you are recommending a long-term test that spans multiple campaigns, determine key milestones for analysis throughout the testing period to determine if your hypothesis is playing out as expected.
Pre-Assessing Results
Consider if the lift needed for your test to be a winner is realistic and achievable before dedicating resources to development of the test. For example, if your test has to beat the control response rate by 300% to be considered a winner, this is likely not the right test to pursue and determining that upfront can save time and money while keeping you focused on tests that have a better chance at moving the needle.
Get Your Audiences Right
You will need a control audience to align with your test audience, and both of those audiences should be selected in the same way. If you are testing on a subset of your full universe, your control sample should be pulled from the exact same subset as the test (with representative sample sizes) to ensure you can get a clear read. Ensure your quantities are large enough to achieve statistical significance while also minimizing risk across the entire campaign.
Consider the Impact of Roll-Out
Make sure your test & control audiences are representative of everyone you plan to roll-out your win. Because rolling out to audiences that were not represented in a test, introduces risk to overall performance.
If you’re looking for assistance in testing or want to learn more of our expert insights, click here to connect!