Geo-experiments Part 1: How to Design Your Geo Test?

Welcome to the world of geo-experiments, a word that appears to be coming from geographers. But it doesn’t.

It is where the science of marketing meets the art of analytics.

In today’s digital age, where privacy regulations are reshaping the advertising landscape, geo-experiments have emerged as a powerful tool for savvy marketers to navigate this dynamic terrain.

Picture this: You’re responsible for allocating a substantial chunk of your company’s budget to various advertising channels like Facebook, Google, TV, or catalog ads. Your mission?

To figure out how much of an impact these ads truly make on your bottom line.

In essence, you need to answer the question: “If we were to pull the plug on our ads, what would we stand to lose?” That’s where geo-experiments come into play.

In this series, we’ll delve into the intricacies of geo-experiments. We’ll cover the essential steps, strategies, and tools that can help marketers like you gain a deeper understanding of your advertising’s true impact. Whether you’re a beginner or an expert, the world of geo-experiments has something for everyone.

This is Part 1, where we talk about designing your geo experiments.

The Basics of Geo-Experiments: Navigating the Advertising Landscape

Geo Test 101

At its core, a geo test, or geo-experiment, is a marketing research method that allows you to analyze the real impact of your advertising efforts within specific geographic markets. In simpler terms, it’s like running a controlled science experiment in the world of advertising.

Here’s how it works: you choose a particular geographic area or market where you plan to alter your advertising strategy in some way. This might involve increasing or decreasing your ad spend in that area, targeting a different audience, or even halting your advertising altogether. You then only closely monitor the changes in sales, sign-ups, downloads, or any other relevant key performance indicators (KPIs) within this market. This data helps you understand the true effect your advertising has on your desired outcomes.

Why Should You Care?

If you’re a marketer, you should care about geo tests because they can be your guiding light in the complex and ever-changing landscape of advertising. Here’s why:

1. Accuracy in Measurement: In the age of privacy regulations and the diminishing reliability of traditional measurement methods, geo tests provide an ethical and accurate way to measure the true impact of your ads. They don’t rely on invasive tracking or complex attribution models, which makes them invaluable.
2. Data-Driven Decisions: Geo tests empower you to make data-driven decisions. Instead of guessing the impact of your advertising, you can have clear, actionable insights that guide your budget allocation and advertising strategies.
3. Cost Savings: By understanding the actual impact of your ads, you can optimize your advertising budget. You’ll know exactly where to allocate your budgets, avoiding wastage and maximizing your return on investment.

The Role of Geo Tests in Measuring Advertising Impact

Now, you might wonder how geo tests fit into the grand scheme of measuring advertising impact. Traditional methods like multi-touch attribution (MTA) are losing their effectiveness due to stricter privacy regulations. Geo-tests fill this gap.

They serve as a reliable tool to measure casual impact or “incrementality.” This concept is at the heart of advertising impact assessment. It helps you answer a fundamental question: How much of your KPIs can be attributed to your advertising efforts, and how much would you have achieved without them?

They enable you to differentiate between the audience that was exposed to your ads and the audience that would have acted the same way without them.

Designing Your Geo Test

Designing your geo test is like setting up the foundation for a successful experiment. It’s where you decide the who, what, and how of your test. Depending on your level of expertise, you can approach it from a beginner, intermediate, or advanced perspective.

1. Beginner-Level Design

If you’re new to geo tests, it’s wise to start with a straightforward approach. Here’s what it involves:

Market Selection Strategy: At the beginner level, you’ll rely on readily available data from advertising platforms. You might choose test markets based on demographics, sales data, or other basic criteria. However, this approach has some limitations. Your results may not be entirely reflective of your broader marketing and sales performance because they could include “outlier” markets, ones that don’t behave like your typical audience.

Market Size: When it comes to market size, you’ll often opt for a 50/50 split. This means about half of your business will fall into a test geography. While this can give you a clear picture of your advertisings impact, it comes with the risk of substantial sales loss in the test market.

Testing Volume: With the 50/50 market split, you’ll typically run one test at a time, as running multiple tests in the same market can lead to inconclusive results due to contamination. You’ll need to make several assumptions about the comparability of your test and control groups, which can be a challenge.

2. Intermediate-Level Design

Once you’ve gained some experience, you can level up your test design:

Market Selection Strategy: Here, you’ll move towards matched-market testing. It involves selecting test/control market pairs that closely correlate with your national sales for specific business KPIs. This approach significantly reduces the chances of outlier markets skewing your results.

Market Size: With this level of testing, you won’t need to test such a large percentage of your business. Tests can be conducted in smaller, more precise markets–between 10-25% of your country. This reduces sales loss and lets you run multiple tests simultaneously without contamination risks.

Testing Volume: Intermediate-level marketers can handle multiple tests at once, thanks to more focused market selection. However, designing the test, deploying it, and analyzing the results will still require substantial manual effort.

3. Advanced Design

When it comes to advanced-level design for geo-tests, the approach takes a significant leap in precision and efficiency:

Market Selection Strategy: At the advanced level, market selection is a highly refined process. It leverages big data analysis, machine learning, and predictive models based on control markets. This allows for a deep understanding of markets that closely resemble your national sales for specific business KPIs. The advanced design ensures your test and control markets are nearly identical in behavior. This reduces the risk of having outliers that could skew your results.

Market Size: Market size becomes remarkably small. Advanced testing can operate with as little as 5-10% of markets in a treatment group. This means minimal risk to the business, with far fewer sales foregone during the testing period. The ability to conduct multiple tests simultaneously is maximized without any concerns of contamination.

Testing Volume: Advanced design enables a significant increase in the number of tests you can run concurrently. By leveraging fully automated processes, including rapid deployment and reporting, you can execute tests with the highest level of certainty and efficiency. This level of testing requires less time, effort, and resources, and results are always accurate and precise.

Here’s a table summarizing how the three approaches differ while designing geo-experiments:

In the next part

As you move on to Part II, we’ll explore the critical phase of deploying your geo test. This is where your test becomes a reality and where errors and inefficiencies could potentially impact the success of your experiments. Well compare manual deployment, semi-manual deployment, and the advanced realm of automated deployment, uncovering the strategies that make your tests truly actionable.

Meantime if you want to see how you can design a geo experiment on Lifesight, sign up for a free demo today.