AdWords Search Query Reports

Quite a few people have written about using the fairly new Search Query Reports, and the sometimes surprising ways in which Google can extend Broad Match. Here’s another insight into AdWords. Look at the first two clips of screenshots, and notice that the impression volume reported is radically differently on keyword search.

This first screen shot shows the left hand side of a Search Query Report from May 2nd to August 31st (the entire period available at the time of taking this image), for one AdGroup with a collection of keywords in Broad, Phrase and Exact Match. I wanted to increase the proportion of exact and phrase matches that attracted clicks.

Notice that the report is pretty much useless. For all 29 impressions reported, there is not a single search query. Everything has been aggregated as “other unique queries”. That’s really not helpful.

Were these queries similar to the keywords? Were they brand names for a competitor (something that is likely to yield low CTR)? Failing to pass on the searches, cuts out some really useful information.

Impression Mismatch

This next screenshot is part of the AdWords UI, showing, for the same AdGroup, for the same date range, the count of impressions and clicks.

24 clicks - that matches. But the Search Query Report shows 29 impressions, for an 83% CTR. The Summary shows 128 impressions, meaning a 19% CTR. Where did those other 99 impressions come from? Are they, perhaps, Domain Parking, and Error Page and other “non-keyword-search” searches?

Here’s a hazard. Under conditions that I can’t reliably reproduce (or I’d have reported it to Google as a bug), you sometimes get a difference between what you are shown in the user interface and what you are shown in downloaded reports. When you see a mismatch like this, the answer is sometimes quite easy to resolve - check the downloaded version.

You do need to watch out. There’s a big “gotcha” in here. If you have content match enabled, then that will add impressions that are shown in the summary, that won’t show up in the keyword search data. That’s quite expectable. As a matter of practice we like to disentangle keyword search and content match, so it is rare for us to be in a situation where these could be confused.

In this case, there is a gap still present in the downloaded version of the report. The Campaign tab, AdGroup summary, shows more impressions than the Search Query report shows. Why?

Let’s see if there are any other tools that Google offers that might yield some insight into where these impressions have come from. How about a Placement report? A Placement Report from 1st June to 31st August (the maximum range possible at this date), shows 0 impressions. Had this been a content matched campaign, Google could (if I had checked the Specials box) have told me that traffic was going to an unusual category of content match, but advertisers are not explicitly told when their impressions are sent to non-keyword-search pages, on Google’s whims.

Other Free Reports From Google

I’ve now used Search Query Reports, and AdGroup Reports, and Placement Reports, all from within AdWords. Is there any other type of report that might help identify where these 99 impressions, none of which generated a click, might come from? Unlikely - because services outside Google should only know about clicks - people who have ended up on the site. The only people who really should know where these impressions have come from are Google.

How about Google Analytics? I have that enabled for this account. And I have used Urchin Tracking tags, built with the tag builder… Oh good grief, that’s even worse. When set to the date range for May 2nd to August 31st, it shows only 2 (two) clicks from AdWords for this AdGroup. Well, that’s not unexpected - users do frequently re-use searches. It is pretty disturbing though, and if I knew less about AdWords I might become concerned at this point about whether i was getting the clicks that I’d paid for.

So what is the time distribution of these 24 clicks - could they be the result of a few users clicking mostly on the same day, so that Google sees only a few sessions initiated by an AdWords click?

Hmm. Inconclusive.

Unassigned Impression Effects

At least none of these apparently non-search impressions has resulted in clicks. I measure a very low conversion rate for domain park and error page clicks across my other clients. But… hang on… this impression rate makes my CTR appallingly low (well, appalling compared with 80%). Instead of an 87.5% CTR, I have an 18.75% CTR.

What effect does that have? Well, in a different client account, I have one keyword that gets $0.02 when the CTR is 51% and $0.45 when the CTR is 49%. CTR can have a *huge* effect on my AvCPC, and can bump the account between a positive and a negative ROAS.

So… What does Google say about impressions and CTR and Quality Score.

There is a single sentence, uncovered by Rich Ball, that says that Broad Match impressions don’t affect the CTR history. Since the AdWords Learning Center and other resources don’t even acknowledge that non-keyword-search impressions form part of the Google Search Pages, there are no identifiable statements that describe the effect of non-search impressions on search CTR. So these unreported impressions could be non-search and have an unknown effect, or possibly broad match search queries that are unreported and would have no effect, or even exact or phrase match searches that are unreportes, that would negatively affect the QS. It’s a whole cloud of uncertainty.

Google guys, this is not right. You can’t expect to be treated as a respectable advertising channel when you conceal what you do, set expectations that you don’t deliver, and fail to explain why you consider what you are doing as fair and reasonable.

If you deliver impressions to sites that you are embarrassed to reveal - that sends its own message.

Following Up

This article was started in September 2007. Since then, I’ve done some more work deliberately setting up content match to address a specific type of site. I can manipulate Google’s content match so that 90%+ of impressions go to sites that Google won’t describe. I’m pretty sure, from the design of the content match AdGroup (keyword selection), where the impressions go. It is disturbing that I’m measuring the success of my campaign by my inability to see where the adverts are placed… In this case, the answer is fairly harmless - it’s a Google owned property. Looking at it shows the adverts, but this domain is never shown in placement reports. Being content match, it isn’t going to show up in any useful way in a Search Query report.

Summary

Mismatches between the impressions delivered and the placement of those impressions are disturbing. They make it look as though Google has something to hide.

Failing to report the search queries that were tested by Google is also disturbing. It is extremely useful data for advertisers to know what searches Google is showing their adverts on. When you combine this with the kerfuffle (in the same time period) about advertisers having their adverts shown against material (on MySpace and YouTube, IIRC) that they didn’t like - it is doubly disturbing. This makes it look as though advertisers are not given the right to determine that they don’t wish to appear on hate sites or on sites by other ethically challenging content publishers. If Google simply conceals where the adverts are shown, it raises concerns. Rather than achieve front page prominence for appearing to endorse race hate sites, or dubious ethical positions, businesses may prefer to just not use AdWords. Google needs to be a bit more open. Eric Schmidt is on record as saying that Google won’t be an obstacle to people finding out what Google knows about themselves. Why aren’t the businesses that fund Google’s growth, treated with the same respect? We need to know where our adverts are being shown, in order to control our PR exposure.

The concern about concealing search queries is mitigated, slightly, now that it is clear that only queries that exact or phrase match with keywords (however they are specified) affect the Quality Score. It took more than a year from the introduction of the Quality Score to discover this. That’s an unreasonably slow migration of information that affects AdWords users. That’s worrying. Again, there is some concern that the concealment of this information may give rise to PR problems. Assume that the search query “(insert your own deeply offensive phrase here)” causes your company advert to appear - are you happy about that? What if Google knew, but didn’t tell you? Would that feel pretty close to a betrayal? Are you sure it isn’t happening now? Is it happening right now, but concealed by glib reports with “1439 other unique search queries”?

So, some marks to Google for beginning to provide Search Query reports. Low marks because of the lack of respect to the people who pay you. Let’s see if you can do better in 2008.

Update

2008-01-28 In response to Gerda’s, comment below, better specified that some of the impressions could be from unembarassing sources including keyword search in a variety of match types - but that even this has an effect on the all-important (and still somewhat obfuscatory) AdWords Quality Score - by withholding collected and potentially reportable information from advertisers, Google scores no points for open communication and clarity. Gerda’s point is good, and I hope that my reply to that comment sheds more illumination.