Do You Think Better QS = Better Results

A study of 27 small business accounts
A squirrel comparing a nut to an abacus.

Professionals who’ve spent any time in paid search – at an agency or in-house – can tell you how much of a big deal Quality Score (QS) is to us in the industry.
We base our strategies on it.
We lie awake at night wondering how to improve it.
We use it as a justification for nearly every change we make.
Recently, Larry Kim at Wordstream took another look at our favorite obsession in his post “Revisiting the Economics of Quality Score: Why QS Is Up to 200% More Valuable in 2013.”
He isn’t the only one. If you do a Google search for any PPC firm + “quality score,” you’ll find veritable ass-loads of blog posts on it:

Screen cap of Quality Score + agency

But nearly every study done on QS measures its impact on just one metric: Cost per Click (CPC).
It’s an important stat, don’t get me wrong – but looking at only CPC doesn’t matter much to our clients at the end of the day.

Why? For one, cheap traffic isn’t always good traffic. For a lot of businesses, the terms worth getting clicks on are the ones with the highest competition and the most cost-ineffective CPCs.
My point is: Our bosses and our clients don’t care about CPCs in and of themselves. They want more leads. They want cheaper leads. They want more-cheaper leads!
 
So, with that in mind, I wanted to look at QS under two different lenses: Conversion Rate (CR) and Cost per Acquisition (CPA).

The QS Study

 

We anonymously looked at 27 accounts with the following parameters:
  • Had to be small businesses – after all, they stand to gain (and lose) the most from paid search
  • Had to have conversion tracking enabled
  • Keywords in the account had to have at least 1 impression
  • Keywords with QS below 2 were ignored as there wasn’t enough volume to be statistically significant
 That gave us a keyword base of ~40,000 in the study and an impression base of ~26,000,000.
We analyzed performance on that group over calendar year 2012.

The Results by QS

 

The first question we wanted to answer with this study:

Did getting a higher QS lead to better CR?

 

The answer might surprise you:

Chart of conversion rate by Quality Score.

No, not really.

It makes sense that there’s a huge jump going from 2 to 3 QS. That’s basically the difference between your ads showing up often and not showing up at all.

But improving from 3 all the way up to 9 has literally no Conversion Rate benefit!
You only see massive benefit in Conversion Rate moving up from 9 to 10.
The next question was:

Did getting a higher QS lead to better CPA?

 

This answer was less surprising:

Chart of Cost Per Action by Quality Score

Yep. It’s a massive deal.
QS can cut your conversion costs by over 90% improving from 2 to 10.
But even going from 6 to 10 can improve your conversion costs by over 50%!
Here’s the data table, if you’d like to just stew in the numbers for a minute here:

Data Table for Quality Score

 

QS Study Takeaways

The question you really need to be asking – QS aside – is:

 

What are your client’s goals?

Do they want more conversions?

Being QS obsessed, or doing bid adjustments based on QS might not be the way to go if the client just wants a better conversion volume.
In fact, this study saw a greater CR at QS4 than any other besides QS10.

Do they want cheaper conversions? 

This is where QS has to be your absolute mission. The benefits to CPA by even incremental QS improvements were incredible.
Improving your QS by 1 meant an average CPA reduction of 22%!
+