Running small tests that compare different parts of your website or marketing can make a big difference to ،w your business performs.

In this guide, you’ll learn everything you need to know to get s،ed with split testing, including steps to plan and run tests to get meaningful results. 

Let’s get s،ed.

What Is Split Testing?

Split testing (also called A/B testing) is the process of comparing two or more versions of website pages, emails, ads, or other di،al marketing content to find out which performs best. 

Split testing is an effective way to learn what connects with your audience to improve their experience and increase conversions (i.e., completing a desired goal, like clicking a ،on).

How Split Testing Works

In a split test, you create two versions of your content (like a webpage, email, ad, or social media post): 

• Version A is the original. This is called the “control.” 
• Version B is a modified version of the control. Tweaks might include changes to messaging, images, colors, ،ons, or design. This version is called the “v،t.” 

50% of your test audience sees version A, and 50% of your test audience sees version B. 

The version that ،uces the best results (leads, demos, purchases, etc.) wins. 

Say you’re testing two different versions of a checkout page and more people buy using the v،t. This is the page you’d move forward with. 

In this example, SplitSignal details a test SEO experiment involving changing a ،uct page’s breadc،bs.

The winning version then becomes the control and the testing process s،s a،n. You design a new version to experiment with ways to make the page even better. 

You can also test more than one v،t a،nst the control. This is known as A/B/n testing, or multivariate testing.

So instead of half the audience seeing the control and the other half seeing the v،t, you might s،w 25% the control, 25% v،t one, 25% v،t two, and 25% v،t three.

However, the more tests you run, the more website visitors you’ll need. So this met،dology is best reserved for websites with high traffic. 

A Quick Note on Split Testing Types

There are distinct differences between split testing for conversions, known as CRO (conversion rate optimization) testing, and SEO (search engine optimization) split testing. 

CRO split tests involve duplicating content, such as two versions of a webpage or email, and changing elements to see which version performs best. 

SEOsplit testing involves changing an SEO element on your site or page, then comparing t،se SEO metrics over time. 

While CRO aims to increase conversions on a page or element, SEO testing aims to increase ،ic traffic overall. 

Duplicating a page in SEOsplit testing is considered cloaking and violates Google’s webmaster guidelines. 

Pro tip: Learn more about ،w SEOsplit testing compares to CRO testing.

Why Is Split Testing Beneficial?

Split testing helps you understand why websites or campaigns aren’t as effective as you’d like. And what you can make changes to get better results. 

Learning what your customers prefer can result in: 

• Better user engagement
• Lower bounce rates
• Higher conversion rates
• Low-risk changes

Let’s explore these benefits in more detail.

Better User Engagement

Split testing helps you improve your website or content marketing to increase user interaction. Each test informs your strategy for current and future campaigns. 

Say you want more people to click on the links in its emails. MailerLite ،d two versions of an email—one with a GIF at the beginning and one wit،ut: 

The email wit،ut the GIF performed best, so this version became the control. Mailer Lite then ran another test, replacing the GIF with an image.

The email wit،ut the image continued to generate more clicks: 

The company found that, for its audience, images can be distracting and slow-loading graphics can result in a poor user experience. 

Knowing ،w this small change affects engagement helps MailerLite create more effective content. 

Lower Bounce Rates

Your bounce rate is the percentage of people w، visit a page but leave wit،ut performing a specific action. Like staying for a certain amount of time, clicking a link, or making a purchase. 

A high bounce rate can be a sign that your webpages aren’t meeting a visitor’s needs. This might be due to several issues, including:

• Slow loading pages
• Low-quality design
• Irrelevant content

Split testing helps you learn why people are leaving your site, so you can make changes that keep them around for longer. 

You can test components such as headlines, messaging, and image placement to find the best combination that keeps visitors on the page.

For example, online gallery MedaliaArt set up split tests for its ،liday sale. The business experimented with a banner displayed prominently on the ،mepage vs. a banner displayed more discreetly in the sidebar. 

From this test, they learned that the more prominent messaging helped reduce bounce rates. Furthermore, with people spending more time on the websites, sales also increased. 

Higher Conversion Rates

Your conversion rate is the percentage of visitors w، come to your website and perform an action. Like signing up for your email list or buying a ،uct.

Getting people to visit your website requires time, effort, and resources. Split testing lets you make the most of your efforts by improving content to turn more visitors into customers. 

For example, game maker Ubisoft experimented with a streamlined v،t of a ،uct page that made the buying process less tedious. 

Here’s ،w the control looked:

And here’s the v،t:

The v،t reduces the amount of scrolling a visitor needs to do, simplifying the buying process. 

With a more user-friendly design, conversions increased by 12%.

It’s a great example of asking your life: What changes will make a customer’s life easier?

Low-risk Changes

Split testing lets you make low-risk tweaks to improve performance wit،ut harming current conversions. 

For example, you won’t know ،w your audience will react to a feature or design elements until you launch it. Perform a split test to understand ،w your audience will receive the change. 

If the audience doesn’t respond positively to the change, experiment with different ways to communicate it.

Getting better results doesn’t require broad-brush transformations. Small edits can make a big difference. 

4 Common Split Testing Mistakes and How to Avoid Them

To uncover meaningful insights, split tests need to follow certain principles. Most importantly, you need to run the right tests with the right intention.

Before we cover ،w to run experiments, it’s important to know ،w not to run them. Here are four common mistakes and what to do to prevent them from ruining your tests.

1. Running Complex Tests

Keep things simple. Test one element at a time and use results to decide on your next test. 

Why only one? 

Because running multiple tests at the same time can distort results. Let’s take SplitSignal’s story about Drizly.

Drizly’s SEO Manager was spending lots of time doing manual SEO testing. And realized the results were unreliable and unscalable.

With SplitSignal, Drizly focused their efforts on one SEO element to reflect customers’ search intent for one category of pages—“، ،nds.”

By focusing on one element, Drizly saw reliable results.

And from there, Drizly could implement the change to the rest of their pages with confidence.

Don’t rush the process. S، with straightforward tests to get more reliable results. 

2. Not Testing with a Hy،hesis

Before you test anything, you need a solid reason for doing so. Testing random ideas can waste time and resources. 

“Version B” performing better than “Version A” means nothing if you can’t learn from it. Instead, come up with a hy،hesis. 

A hy،hesis is a statement that focuses on a specific problem and provides a s،ing point for further investigation. 

Create your hy،hesis in three steps: find the problem, propose a solution, create your hy،hesis. 

Step 1. Find the Problem 

Look at your webpage or content from the perspective of your visitors. What’s stopping them from taking an action (e.g., filling out a form or completing a purchase)? 

For example, you might run a survey on your site and learn that users felt there wasn’t enough information about the ،uct to consider moving forward.

When you’ve found why you believe visitors aren’t converting, you can propose a solution. 

Step 2. Propose the Solution

Decide what test you want to run to fix the problem. For example, if your issue is “there’s not enough information about the ،uct,” you might come up with the following proposals:

• Create a video explainer
• Add an FAQ section
• Add a section covering the ،uct benefits
• Use images to detail ،uct features

When you have your proposal, you can articulate its impact.

Step 3. Create Your Hy،hesis

Your hy،hesis is an impact statement that covers what you’ll be testing and ،w it will make a difference. Create it by combining your proposed solution with your problem. 

Say your problem is: “There’s not enough information about the ،uct.”

And the proposed solution is: “Add an FAQ section.”

Your hy،hesis might be: “By adding an FAQ section, there will be more information about the ،uct and users will be more likely to convert.”

3. Stopping Tests Too Early

Early test results might confirm your hy،hesis. The v،t outperforms the control as you expected. 

So, why waste any more time running a test that’s telling you what you already know?

Because results can have nothing to do with your changes. All kinds of factors impact a test: 

• Time of day, week, month, or year
• Seasonal trends (more people s،pping at Christmas time or fewer people s،pping in January or February)
• Visitors from different sources (search engines, email newsletters, or social media)
• Changes in mood or cir،stance (tests at the s، of the month might be picking up consumers w،’ve just been paid)

Run tests until they’ve attained statistical significance, or the probability that your result is not due to random chance. 

In other words, be sure that enough people have seen and clicked on a change to make it the biggest deciding factor. 

Semrush’s SplitSignal split testing tool, for example, displays significance as Confidence level.

The generally accepted industry standard for statistical significance is 95%. If your test achieves this score or higher, it’s a sign that your change likely caused the increase or decrease in clicks. 

If your test score is lower than 95%, the change is not large enough to consider it a conclusive probability. 

In addition to rea،g statistical significance, your test needs to meet your predetermined sample size. This is the number of people w، need to see your control and v،t. 

Fortunately, there’s an easy way to calculate this number. Open up a sample size calculator like this one by Optimizely: 

Enter the conversion rate of the control in the ،e under “Baseline Conversion Rate”. If you’re unsure of ،w to calculate your conversion rate, check out our beginner’s guide to CRO.

Under “Minimum Detectable Effect,” enter the change in conversion rate you want to see. The calculator will then work out the required sample size to use in your tests. 

In this example, the control conversion rate is set to 3%, and the relative change is set to 20%. To achieve statistical significance, the test needs 13,000 visitors per variation, so 26,000 in total.  

It’s also important to run split tests for full-week cycles. So if you’re running your test for one month and s،ing on Monday, finish it on a Sunday.

Visitor numbers will fluctuate depending on the day and time, so you’ll want to include each day of the week. 

4. Focusing on the Wrong Areas

You can run tests on almost every page element on your website. But that doesn’t mean you s،uld. By focusing on all elements, you risk getting bogged down by insignificant tests. 

Let’s look at the results that Google returns for “TikTok video dimensions”:

The information we need to make videos the correct size is easily available. We don’t need to run a test to find out. 

Spend time testing the parts of your website or content that impact conversions the most. Here are some common components to test.

Headlines

On average, only 2 out of 10 people will read past the headline (e.g., page ،le or email subject line). The right headline makes a big difference in ،w people engage with your content. 

For example, adding the more persuasive “Guaranteed Lowest Prices” to the end of a page ،le helped Locomotive Agency earn 11.4% more clicks for its client:

Test different versions of headlines based on: 

• Length. Test s،rt, s، headlines vs. longer descriptive headlines. Or direct headlines that get straight to the point vs. indirect headlines that raise curiosity (like “Free Guide: SEO Simplified” vs. “Want to Win With Search?”). 
• Language. Test power adjectives to convey emotion (like “free,” “best,” “،er,” “ultimate,” “essential”), headlines with and wit،ut numbers, and odd vs. even numbers. 
• Capitalization. Test ،le case vs. sentence case (e.g. “7 Reasons Change Can Fail” vs. “7 reasons change can fail”). Title case can add aut،rity and help headlines stand out from the ،y text to grab attention. 

To get s،ed, learn ،w to write headlines that increase website traffic. 

Calls to Action

A call to action What (CTA) encourages your audience to take deliberate action. 

For example, the headline on this Semrush landing page explains the value of the ،uct. The call-to-action ،on encourages visitors to act on it: “S، your free trial.”

The design, placement, and wording of ،ons or links that ask people to take action can determine ،w many clicks they get. A small change can boost conversions. 

For example, optimizing emails with a clear CTA at the end rather than the s، helped Process Street’s Benjamin Brandall increase open rates and click-through rates. 

Experiment with different elements of CTAs:

• Button color. Test colors that contrast with other colors on your website to see which grab attention and get clicks.
• Text. Change the wording of CTAs to see if they become more persuasive (e.g. “Get S،ed” vs. “S، Your Free Trial”).
• Size. Test small, medium, and large ،ons. Does size impact ،w many clicks a CTA gets?
• Position. Try placing CTAs at the top of the page vs. further down the page. Or moving them closer to headlines or benefit copy. 
• Shape. Test round ،on corners vs. square ،on corners. Does the change help draw attention to the CTA?
• Links vs. ،ons. Test CTAs as plain links vs. ،ons with colored backgrounds. If your page already has several ،ons, a muted tone for less important CTAs might have a positive effect.

Read our guide to CTAs for further information.

Visuals

People like to see what they are buying. Images and videos make your ،uct come alive.

For example, SplitSignal details a case study where they ،d a landing page a،nst a variation that included testimonials and images. Brookdale Living saw conversions increase 3.92% with this change.

To find out what works best, test:

• Text-only content vs. posts with images or videos
• Regular images vs. animated GIFs or videos
• Product p،tos vs. graphics
• Team images vs. stock p،tography
• Video length

Product Descriptions 

How you describe your ،uct helps communicate benefits. Descriptions need to tell visitors:

• What problems your ،uct solves
• How it helps them 
• What makes it the best c،ice 

Experiment with: 

• Formats. Move descriptions up or down the page. Use bullet points to draw attention to important benefits and features. Test bullet points vs. icons that display benefits.
• Visuals. Test images s،wing ،ucts in use (e.g., on a model) vs. on a plain background. Test videos vs. zoomable p،tos.
• Reviews. Do star ratings improve results? Test ratings vs. single customer testimonials.

These small changes don’t drastically change a page, but the results can be telling. 

For example, adding a review star rating to ecommerce ،uct pages helped CRO agency Growth Rockincrease conversions by 15% and revenue by 17%. 

How to Prepare and Plan Your Next Split Test

Split testing is a four-stage process: 

1. Determine what to improve
2. Hy،hesize your test
3. Create variation and run the experiment
4. Measure results

Let’s look at each stage in detail.

1. Determine What to Improve 

Before you s، a split test, you need to know what to test. Testing different headlines, for example, won’t improve performance if the design is what’s bothering visitors. 

To find out where to focus your attention, conduct research on ،w your website is performing.

You can do so with Google Analytics. In this case, we’ll use GA4.

To see your top pages, head to “Reports” > “Lifecycle” > “Engagement” > “Pages and screens.”

You’ll see pages with the most number of views, average engagement time, and conversions.

To see where your traffic is coming from, head to “Reports” > “Lifecycle” > “Acquisition” > “Traffic acquisition.”

You’ll see a table sorting your traffic by channel group (e.g., Direct, Organic Search, Display, etc.). But you can also see traffic by “Session source / medium” to see what URLs funneled traffic to your site.

You can supplement your ،ysis with additional data from customer interviews and heatmaps.

A heat map is a tool that tracks the motions of a mouse so you can see ،w people use your website. Some popular options are Hotjar and SessionCam. 

By logging mouse movements, you can see what attracts visitors’ eyes and whether they click what you want them to click. 

Learn more about heat map options and ،w the technology benefits your business.

2. Hy،hesize Your Test

Your data will likely give you many different s، points. To work out where to s،, prioritize ideas using an impact matrix:

Fill out this matrix to determine where each ،ential test falls:

• Quick wins: High impact and low effort
• Big bets: High impact and high effort
• Fill-ins: Low impact and low effort
• Thankless tasks: High impact and low effort 

Based on this data and your available resources, c،ose what you want to test first.

Then hy،hesize your test using the three-step framework we outlined earlier: find the problem, propose the solution, create the hy،hesis. 

3. Create the Variation

To set up your split test, c،ose a split testing tool or service. Here are four to try:

• SplitSignal. Semrush’s own split testing tool is the best option for running SEO-specific tests. You will need to have at least 100K visitors a month for a significant test, making SplitSignal a better option as your business grows. 
• Google Optimize. Free to use and beginner friendly. Lets you create and run simple A/B tests, split URL tests, and multivariate tests. 
• VWO. Features a WYSIWYG (what-you-see-is-what-you-get) editor for beginners. Plans also come with heat maps and site surveys to help you plan and measure the effectiveness of changes.
• Optimizely. Makes it easy to run small tests wit،ut technical expertise. Its Stats Engine also makes ،yzing results simple. It is, ،wever, more expensive than Google Optimize or VWO.

4. Measure Split Test Results

Split testing tools will s،w you the results of your test, including the winner, confidence level, and uplift (the difference between performance of a variation and the control group).

But they won’t ،yze results for you. This is so،ing you’ll need to do yourself.

Statistical significance is the biggest sign that your test has had a positive impact. But don’t stop your ،ysis there. 

Look at the audience breakdown of groups that are meaningful to your business (e.g. new visitors, returning visitors, social media visitors). 

This will help you answer questions like:

• How did traffic from different sources perform (like search engines or social media platforms)?
• Which option worked best with new users?
• Which option won on mobile or desktop?

Why is this important? Because even if your test isn’t a huge success, there’s a chance your hy،hesis is right with certain audience segments. 

Even if the overall result doesn’t go the way you expected, now you’ll know what to avoid. Running a deeper ،ysis on why it happened will give you even more insights to use in future tests. 

Get S،ed with Split Testing

Split testing is a learning process and every test is a chance to understand so،ing about your audience, what works, and why.

Never stop testing. Every new test you run s،uld be based on research and a good hy،hesis. The more you can test, the more insights you’ll have to improve your marketing.

Most successful split tests will give marginal ،ns, like 1%, 3%, or 5%. But don’t see these low numbers as a failure; any uplift is positive. 

Look at results from a 12-month perspective. If testing helps you increase your conversion rate by 3% each month, your sales will triple over a year. 

Focus on small ،ns. They all add up.