Google likes to keep search professionals on their toes and in January, the SERPs on desktop were changed to include favicons on organic listings and a bold Ad label next to paid listings.

This caused a lot of controversy amongst SEOs and who worried the visual similarity of paid and organic listings would negatively affect each other. Google later removed the favicons from organic listings.

But the black Ad label still remains for paid ads. So that got us thinking: had the new-style SERPs had an effect on the click-through rate of paid listings? Before there was a distinction between ads and organic listings but now it’s more subtle. Would the untrained eye be more easily duped?

Methodology

To test this, we had a look at data from 7,927 accounts and compared clicks, impressions, and CTR data on the accounts between two periods:

• 12 December 2019 – 12 January 2020
• 13th January 2020 – 13 February 2020

These periods were chosen as the months before and after the SERP changes that took place on 12th January 2020.

We calculated the averages (mean) and standard deviation of every metric and percentage change, as well as a 5% margin of error.

The data and relevant calculations can be obtained from Google Sheets.

What we discovered

From the 7,927 accounts we analysed between the aforementioned periods, we discovered the following (with a 5% margin of error):

• A 0.03% decrease in CTR (±3.75%)
• A 6.29% increase in clicks (±1.53%)
• An 8.03% increase in impressions (±1.68%)

Given the controversy and thoughts from the search community, this negligible change was a surprise. What’s more, the fact the change was a decrease rather than an increase – albeit a small one – was also surprising. When the SERP design was changed on mobile in May 2019, click rates showed increases of 17% to 18% for companies placing their advertisements on the SERPs from May 24 to 30th stretch, in comparison, with the May 17 to 23rd slots.

Thoughts from our PPC team

I asked our PPC Manager Sophie Logan and Head of Paid Media Mark Neale (who escaped Team Rocket captivity) about their thoughts.

“I think user behaviour takes a while to change, so whilst people may be noticing these SERP changes, they are continuing to use Google as normal. This may change over time, especially with the wider public becoming more aware of the presence of advertising in their search results, but for now, I’d be surprised if people opted to go for organic results over paid simply out of protest.”

Sophie

“I’m a little surprised that the difference in CTR was so small but that goes to show how observant users are when it comes to search. I think the user has become less concerned on the source of the advertising and more focused on a fast resolution to their need in that moment. As we have seen over the years, the key to successful CTR is through answering the user with good quality copy and solutions to their query.”

Mark

Evaluation

No data study is absolutely perfect. Three considerations for future analysis:

• We could have segmented the data by other metrics to gain further insights – there may have been other factors that affected the stats and segmented the data by location, sector, or currency, for example, may have shown these more prominently.
• We could have checked the next month period to avoid confirmation bias – the analysis only looked at two month periods. A look at more periods could potentially confirm or refute the findings.
• The time period we used could have been impacted by the festive period – as some businesses may have reduced their ad spend during Christmas, that could have affected the data.

Are the new SERPs a dark pattern?

On 24th January, Patricio Robles wrote an article for Econsultancy entitled “Is Google’s updated SERP design a dark pattern?”. He questioned Google’s decision to change the SERP design and asked whether it helped searchers.

“Around the web, discussion and debate about the design has been significant, with many pointing out that the design makes it very difficult to distinguish organic listings from paid listings.”

Dieter Bohn asked whether Google’s SERPs were still trustworthy in an article for The Verge (published on the same day as Robles’s for Econsultancy). He called them “paid occlusion” rather than paid inclusion, stating:

If Google is willing to visually muddle ads, how long until its users lose trust in the algorithm itself? With this change, Google is becoming what it once sought to overcome: AltaVista.

AltaVista was a pioneering search engine established in 1995 which lost popularity due to Google (as many did in that time). The reference to seeking to avoid becoming AltaVista related to the search engine’s primitive style of web page listings, with no visual differentiation.

Conclusion

Of course, the data we gathered was a small sample of all the ads that appeared on Google’s SERPs between December and February. Therefore, our findings aren’t conclusive. But because of how strong opinions were about the new SERP design and accusations of Google’s use of dark patterns, it’s interesting to see such a nominal change in CTR.

As new SERP designs appear to favour ads, that would lead to more impressions and more clicks as users become confused about what’s an ad and what’s not. There’s also a possibility that CTRs aren’t as reliable as they used to be if more users click on ads by mistake, leading to wasted ad spend and ROAS (return on ad spend). Who knows what the future will bring for organic and paid listings and the users who have to tell the difference.