AdWords QS is BS

Published on November 24th, 2006 by Jeremy Chatfield

Google has just announced a number of the factors that affect Landing Page Quality Score in the Inside AdWords Blog. This article needs heavy revision in the light of Google’s new information. The main thrust – that QS is composed of several different factors, some of which affecting initial MinCPC and others affecting longer term MinCPC, meaning that a single number is unrepresentative, remain true. If anything, the count of different factors has increased, as there is now a new QS for the Top Position box, and I expect that we’ll discover one for mobile adverts, too.

Google has introduced us, this year (2006), to the concept of the Quality Score and asserted that the Quality Score controls advertising performance on AdWords. Various Paid Search experts expound on the Quality Score and its’ effects.

I think that an “AdWords Quality Score” is arrant nonsense. There is no single magic number that Google calculates to determine when, where and for how much they show your adverts and respect your keywords. It is an obfuscation that potentially allows Google the freedom to choose advertisers they like, without industry insight or oversight. The blurring of a multitude of controllable and reportable internal processes into an unguessable and unrevealed Quality Score allows Google to affect the paid search industry without the industry being able to react, positively or negatively, and damages the ability of advertisers to investigate and improve their performance.

On a longer time scale, the hidden factors that contribute to this notion of a Quality Score is a sign of Google’s growing arrogance and a threat to Google’s dominance in paid search.

What Does Google Control?

I’ll focus on keyword search for simplicity, but similar processes apply to content match. This means that any special scores for keyword search are probably mirrored for content match, doubling the count of specific “quality scored” factors. [See "Updates", at the end].

When you submit a keyword and an advert to Google, it is immediately subjected to an automated test:

Do the keywords appear in the advert?

If they do, then you get an immediate grant to use the lowest MinCPC available for that keyword, for the country and geotarget you have chosen, subject to a keyword history for that account and for the Destination URL. Your advert, with a few exceptions, will then start running on Google Search Pages and is marked for two further reviews. One is automated, and the other is human mediated. Your advert can run on Google’s own properties, without running elsewhere, without a Landing Page check – though reports from some advertisers suggest that this can be systematically withdrawn by Google, probably as a consequence of using certain (unpublished) words in adverts.

Google is clicking on my adverts!

At some point, an automaton, the Google AdsBot, visits your landing page and determines whether it thinks your landing page is conceptually related to the keyword, and considers a bunch of other factors… undocumented factors. Depending upon this automated review, your MinCPC for each keyword may be adjusted. If adjusted, the change is usually upwards. This is the cause of the oft-voiced:

“My adverts were running at $0.02 and now Google wants to charge $5.00″.

At this point, the advert run is still restricted to the Google Search Pages. Note that this is also usually the cause of complaints that “Google is clicking on my adverts”. Google doesn’t bill for those clicks. We’re very sure of that. We measure web server log file clicks and we know what the editorial review clicks look like – and there’s no matching click fee in Google’s bills.

Applying the Editorial Guidelines

A further, human mediated review, is conducted. This involves a person considering the page using Google’s Editorial Guidelines. People spot things that automation can’t. If the AdsBot doesn’t spot that you are using AdWords Arbitrage, then the human reviewer probably will. This is the first and only opportunity for your keyword, advert and landing page to be considered as an entirety. It’s also the point at which Google can use recently documented (2007-09-19) and other undocumented factors.

Potentially, this is the point at which Google can apply unfair competitive advantage or penalty. All without external notification, insight or oversight, because Google obscures the multiplicity of factors behind the Quality Score. Take a look at this thread on the AdWords Help Forum about a non-specific answer for a rejected advert, for example.

The results of this most comprehensive review may be a raised or a reduced MinCPC. Google also reviews adverts and keywords, during the life of the campaign. I don’t know any specific factors for triggering these reviews, but some accounts can go for years without a comprehensive review and others seem to get re-reviewed after a few months. You can spot these reviews in the web server log files (more visits by the AdsBot and other Google-homed unbilled visits); the only time that most advertisers become aware of these aperiodic reviews is when low MinCPC keywords on a previously stable account suddenly jump in price for no explained reason.

What about CTR?

You’ll notice that the entire focus so far has been on the minimum cost per click. Not the auction. There’s no interaction between the MinCPC-focused review-based components of Quality Score and the performance based components of Quality Score. So, at a minimum, QS could be divided into Review-Based Factors and Performance Factors.

Google’s auction is complex. Some people describe it as closed. I think it is a little more complex than is described in this otherwise excellent description of the second price auction. I suspect that Google has a minimum acceptable revenue for a search. That minimum page revenue boundary helps to keep adverts off searches where users don’t click on adverts, because they are really doing research.

The basic auction is probably CPM based. Advertisers get rank based on the total revenue to Google over 1,000 impressions.

Where’s the Quality Score in this description? Position isn’t dependent on your landing page. It isn’t dependent on automated or human review of keywords and landing pages. It is entirely determined by search user response to the advert that is served, and the revenue that activity yields to Google. End of story.

There is one point of overlap. If you achieve a high CTR, then your MinCPC may be reduced.

This performance based component (high CTR driving down MinCPC) can cause complex interactions. For example, if you have a new keyword with a new advert, you could be assessed an $0.22 MinCPC for failing to precisely match keyword and advert. A high CTR could result in the MinCPC being progressively reduced to $0.01. However, except under rather unusual conditions, a very low MinCPC and a very high CTR rarely result in you being able to effectively use the low MinCPC. The rare condition is when you are also asserting trademark restrictions on searches, so your company is the only company on that keyword – you can still be assessed a $5.00 MinCPC based on review factors, which you could reduce to $0.01 through high CTR.

If there’s more to the story, then it is almost certainly a fudge factor. So there’d better not be an unannounced and human adjustable factor to this. If there is, then Google’s position as a near-monopoly provider of internet advertising will probably trigger, at least in Europe, state level investigations into unfair practices.

But who can tell? This hidden Quality Score is not revealed. Are you subjected to a 50% price uplift? Can you tell? How would you know you were, or weren’t? Why is your company given a competitive disadvantage? Very, very scary, not knowing, isn’t it?

It’s the economy, stupid.

The risk to Google of a single obfuscated “Quality Score” is that advertisers are not told what specific guideline they have breached. Google hides information from advertisers that is crucial to improving success. If Google were more open about the factors, advertisers could improve their adverts. Undoubtedly some bad eggs will use that knowledge to sneak through adverts that shouldn’t be there, but the penalty of the Quality Score is that well intentioned advertisers aren’t kept informed.

This, I think, is illustrative of Google’s embedded cultural attitude to advertisers. Advertisers are viewed with suspicion by Google. We are treated as if we are all ready and willing to abuse the system. There is no trust extended from Google to advertisers. And that means that information is kept back, just in case it could be abused.

The little fact that we pay for this system, that our advertising keeps the Googlopoly growing, has been lost. Our customer support services are delivered by undertrained and overworked staff, with the simplest questions taking weeks to answer. Information about, for example, whether robots are automatically excluded from our bills, is not given to advertisers on the spurious grounds of privacy. This issue is related to obfuscated Quality Scores, because it shows that Google has a history and predilection to abuse advertisers trust.

What are the privacy rights of a search engine spider?

Search engine spiders, unlike click fraudsters, have few reasons to hide. Google runs a number of robots, some of which are suspected to conceal their origins in order to look for SEO cloaked pages. But in general, most SE spiders explain exactly what search engine they come from. The User Agent field (reported by web browsers when they request a page), will carry some information naming the spider and usually giving a URL so you can read about it and learn how to block it using a Robots.txt file.

Advertisers do not control the blocking of spiders, publishers do. Blocking granularity is at page level. So publishers, even if they wanted to, could not stop an identified spider from following links into an advert. If a spider clicks on the advert, is it billed? Why doesn’t Google publish a list of spiders that are definitely excluded from billed clicks? If you look for information about this, you’ll see a wall of privacy concerns is thrown up. But a self-identified spider, from a well known IP address is clearly not a human with privacy to protect. So what is going on?

It is not privacy concerns.

It appears to be Google enforcing an asymmetry in information, which advantages Google and disadvantages the advertiser. I do spend time thinking about information security. I can’t, and perhaps it is a failure of my imagination, but I just can’t come up with any good reason to with hold the list of robots that Google excludes from click bills, automatically. If I could think of a reason that would advantage a click fraudster, I’d be happier about Google keeping this private.

If Google excludes robots from AdWords costs to AdSense publishers, that’s not something that advantages a click fraudster. Reducing bills by excluding robot clicks does penalise Google. We advertisers and agencies pay Google for high quality clicks, not to sign up every publisher and bill us for clicks from robots. Click bill reduction by omitting robots is a responsibility for Google, and assurance that they are doing so should be regarded as a perfectly normal business activity.

Now? Investigating web server log files and matching that analysis to Google’s invalid click rates consumes agency time, which could be reduced with simple and non-threatening disclosures. Stuff like that chews time that could have been spent on getting better adverts and keywords. Worse is the industry wide claim that getting bill reductions for click fraud is a lengthy and often fruitless exercise. I can certainly believe that many claims are wrong – it is easier to tell your boss that the poor performance of a paid search campaign is a result of click fraud, than that you wasted some budget on clicks you really shouldn’t have bought. Personal experience says that even with solid web server log file evidence of problems, paid search vendors respond slowly and fail to correct the defects in their systems, causing a recurring agency and advertiser cost.

Asymmetry in information holding, which destroys trust in the system, does not ultimately work to the benefit of Google or advertisers and their agencies. It is equivalent to a failure to act on bribery and corruption charges, and creates economic damage.

Advertiser defection

The consequence of information hiding will be that Google will start to suffer from advertiser defection. We are already slanting our clients’ spend more towards Yahoo!Search and MSN AdCenter, simply because of factors surrounding Google’s approach to advertisers. AdWords API usage fees on top of advertising costs, lack of answers to basic questions increases costs while we conduct experiments to determine the right way to do things rather than relying on a trustworthy answer, etc.

While we continue to get a good ROI from Google, these issues are only minor. As the competition for effective advertising increases, as is inevitable for such a brilliant medium, advertisers will defect to systems that offer them better value. Value isn’t just measured in Cost Per Click and a straight ROI for the client. It is measured in the total cost of doing business with Google, such as timely information and the feeling that I just can’t face trying to get a usable and reliable answer out of Google… I’d rather just do the experiment and add the cost to the bad side of the ROI equation.

If my agency overheads for dealing with Google, and additional stupid fees like the one for the AdWords API are added, then my cost base is consequently raised relative to that for dealing with Yahoo!Search Marketing/Overture. The ROI for Overture becomes more attractive and more money goes that way. Google’s targeting accuracy is high, but the cost of using it is high – learning and effectively applying that knowledge does take more time than on simpler systems. So Google has a built-in competitive disadvantage, more than compensated for by its’ reach and quality – for now.

Google has to learn to love its’ employers – advertisers – as much as it now loves search users. It is simple economics, guys. Time for a cultural revolution. Please?

Updates

Andrew Goodman (author of one of the better AdWords books and instigator of the SEM2 forum) has an interview with Nick Fox of Google confirming some of this. For now. Maybe. This interview also has the statement that “if we give too much information about the process to the bad guys, they’ll turn around and use that to circumvent the process“. This says, to me, that Google is more concerned to stop the bad guys than help the good guys. Kind of comforting, in a wierd, warped and annoying way. How many years and how many millions of client spend do you have to do, before you aren’t considered latent evil?

Richard Ball of Apogee provoked a single sentence about content match to be expanded – see his apropos comment.

I used the unnecessarily pejorative “pontificate” in the first published version, when referring to other commentators. I’m now using “expound” as I need a word that says that these people are using their experience to interpret QS, not based on anything of any clarity from Google. Many of the commentaries appear to match most of the evidence we’ve seen. Few seem to portray QS as an industry problem in transparency and verification. Perhaps they feel that Google would withdraw support services if they were more critical? Some private emails have suggested that we’re “brave” for voicing criticisms of Google.

Added some personal experience about pursuing reimbursement for invalid clicks and more about robots and the relevance to Google’s information hiding approach (triggered by some private email). There is a Nobel prize in economics that was awarded to someone who investigated market pricing in asymmetric information systems. I must look that up…

2006-12-16
Google have collated some of their material and added new pages about the Quality Score.

2007-09-19 Google have added more information about content that lowers Landing Page QS.