2.6 How to A/B test SEO changes

In this article:

  1. About SEO A/B tests
  2. How to set up an A/B test
  3. Prepare control group(s)
  4. Import rules files
  5. Publish to staging
  6. Publish to production
  7. Validate results
  8. Running multiple A/B tests simultaneously

About SEO A/B Tests

The tagging feature of RankSense allows us to track two or more groups of pages and compare their organic performance.

For example, we routinely test the messaging in meta descriptions to improve click-through rate. In these tests, we create one or more templates for the meta descriptions. We apply the same template to every page in a group so that we keep consistency within a group.

One group is left unchanged to serve as the “control.” At the end of the test, the control will help us to determine if any increase or decrease that we saw in the test group(s) was due to our test, or if it might have been from something that effected the entire site, like a seasonal fluctuation.

You may be familiar with typical marketing A/B testing where two versions of a page exist, and users may be routed to one or the other.

However, in SEO, we do not have the ability to show one version of a page to some users and another to other users. The version that is found by the search engine crawler is the one that will be seen in the search results. Therefore, we must test two or more groups of pages and compare their performance – the more similar the groups, the better.

How to set up an A/B test

For each type of change that you want to test, you will need one group. RankSense supports up to four test groups. Keep in mind that you will want to leave some pages unchanged for a comparison.

Prepare a rules sheet for each group.

You may want to use the comment column to label the groups “Experimental Group A,” “Experimental Group B,” and so on. We will use these labels later in the software.

Prepare control group(s)

Decide whether to use one or two control groups, and prepare a sheet for each control group. On a control group sheet, you should just have the URLs listed, and nothing else (other than the header row). The other columns will be ignored by the software when you tag them as a control group later on.

You can optionally use the comment column to label these as “Control Group A” or “Control Group B”.

Import rules files

If you have published rules before, this is a similar process, but there is an extra step.

Connect your Google Drive if it isn’t already.

Go to Settings > SEO rules. Click “+ File.” Input the URL of a test group’s rules sheet in the first box. Do not click “+ Import” yet.

Below “Solution,” click “Add tag” and you will see a list of options. Choose “Experiment Group A” if this is an experimental group (i.e. not a control group).

Optionally, you can add other tags, for example “Optimized meta description” from the Solutions tags, or “Low quality meta descriptions” from the Issues tags. You can also choose the type of pages (i.e. product pages, blog pages) from the Affected tags.
01%20tags

Once you have selected your tags, click “+ Import.” This will load your rules into the app but not publish them.

Publish to staging

We recommend publishing to staging first to ensure the rules are working correctly. On the right side, select Staging.

The rules that have been loaded in the previous step will be shown under “Hide unpublished changes.” The number of rules that have been added are next to the green plus sign. Click “Show” beside that number to see all of the specific changes.

Click “Publish,” then verify your changes on staging.

Publish to production

Choose the production tab, check that the log is correct, then click Publish. You can verify the changes are live by simply visiting the site. You may wish to use our Chrome extension as a convenient way to check SEO meta data.

Validate results

RankSense can be used to check when search engines have crawled the test pages.

Finally, use RankSense to validate that your changes were effective. You can monitor the performance in terms of organic clicks in real time.

Running multiple A/B tests simultaneously

RankSense can technically be used to run more than one A/B test at the same time. Since RankSense allows up to four experimental groups and two control groups, you can use tags to manage multiple tests. You will simply use different tags for the new test.

It is important that your experimental groups do not overlap in URLs, or one A/B test could skew the results of the other.

Technically, you can use the same pages in the control group, but we would only recommend this if the pages in the control group are similar to the pages in both tests. If you decide to use the same exact pages as the control group, it is not necessary to re-publish the control group when you set up the second test. You can simply use the tags that are already in place.

Go back to main page