How to A/B test SEO changes

This is one of the advanced features available to sites that are managed by an agency. If you need a recommendation for an agency that uses RankSense, please contact us!

About SEO A/B Tests

The tagging feature of RankSense allows us to track two or more groups of pages and compare their organic performance.

For example, we routinely test the messaging in meta descriptions to improve click-through rate. In these tests, we create one or more templates for the meta descriptions. We apply the same template to every page in a group so that we keep consistency within a group.

One group is left unchanged to serve as the “control.” At the end of the test, the control will help us to determine if any increase or decrease that we saw in the test group(s) was due to our test, or if it might have been from something that effected the entire site, like a seasonal fluctuation.

You may be familiar with typical marketing A/B testing where two versions of a page exist, and users may be routed to one or the other.

However, in SEO, we do not have the ability to show one version of a page to some users and another to other users. The version that is found by the search engine crawler is the one that will be seen in the search results. Therefore, we must A/B test groups of pages and compare their performance – the more similar the groups, the better.

How to set up an A/B test

For each type of change that you want to test, you will need one group. RankSense supports up to four test groups. Keep in mind that you will want to leave some pages unchanged for a comparison.

Prepare a rules sheet for each group.

You may want to use the comment column to label the groups “Experimental Group A,” “Experimental Group B,” and so on. We will use these labels later in the software.

Prepare control group(s)

Decide whether to use one or two control groups, and prepare a sheet for each control group. On a control group sheet, you should just have the URLs listed, and nothing else (other than the header row). The other columns will be ignored by the software when you tag them as a control group later on.

You can optionally use the comment column to label these as “Control Group A” or “Control Group B”.

Import rules files

If you have published rules before, this is a similar process, but there is an extra step.

Connect your Google Drive if it isn’t already.

Go to Settings > SEO rules. Input the URL of a test group’s rules sheet. Do not click “+ File” yet.

Below “Solution,” click “Add tag” and you will see a list of options. Choose “Experiment Group A” if this is an experimental group (i.e. not a control group).

Optionally, you can add other tags, for example “Optimized meta description” from the Solutions tags, or “Low quality meta descriptions” from the Issues tags. You can also choose the type of pages (i.e. product pages, blog pages) from the Affected tags.

Publish to staging

We recommend publishing to staging first to ensure the rules are working correctly. On the right side, select Staging.

Click “Show log” to show the rules that have been loaded in the previous step. Then click “Show” beside the type of rule to see all of the specific changes.

Click “Publish,” then verify your changes on staging (article coming soon).

Publish to production

Choose the production tab, check that the log is correct, then click Publish. You can verify the changes are live by simply visiting the site. You may wish to use our Chrome extension as a convenient way to check SEO meta data.

Check results

We typically allow up to 2 weeks for search engines to pick up your changes, and several weeks to collect data. The Insights tab will show you the performance while the test progresses.

Under “What would you like to learn?” select “Other” to get access to a custom report.

Select the tags “Experimental Group A”, “Control Group A”, etc to display the performance for the pages you have tagged.

You can adjust the date range using the options above the graph. You can also select the search engine that the visitors are coming from.