I was wrong - Search History Permutation probably isn’t the sole cause of problems. Amongst my clients, I can see Search History Permutation (where the words in the current and previous search query are combined to generate new searches, and adverts are conscripted that the searcher wasn’t lookiing for). However, other clients are also having new problems, not obviously caused by Search History Permutation. I don’t, yet, understand what else is going on. But let’s at least get the Search History Permutation out of the way.
One client has had Google Analytics running for a few years - so I can do some basic time series analysis. If I do a cross correlation on impressions, click, visitors and conversion rates, they are consistent from Feb 2006 until approximately March 1st this year. Now, that’s *after* the first point at which I became aware of Search History Permutation. So there may be something else going on, too.
Search History Permutation Signals
If you have been hit by this, then Search Queries will start turning up in web analytics, that have one or two words that you’d normally expect, and some other odd words that you might not expect, or combinations of words that you would expect to see in different keywords. For example, if you are advertising using “free dentistry” and “dentist poughkeepsie”, you might see search queries for “free dentist”, “free poughkeepsie”, “dentist dentistry” and “dentistry poughkeepsie”, as well. Some of these are similar to the sorts of expansion that Google does already with Broad Match. If you used “free dentistry” and saw “free dentist”, that is exactly the sort of expansion that Broad Match is already capable of delivering. However, “free poughkeepsie” is *not* normal - well, not for a dentists keywords, anyway.
How frequently you get hit by Search History Permutation will depend on the precise mix of keywords that you have in the account. If you happen to have keywords where searchers will typically make consecutive searches that each include a word from your keywords, then you might get dragged into unexpected auctions to advertise for searches that you don’t expect to appear in.
The frequency appears to be highly variable - but only for certain types of account. What distinguishes those accounts? They are high bidders, with niche keywords. When searchers have a set of searches combined, then the rare niche keywords are triggered at an abnormally high frequency. Because the bids are high, these adverts are conscripted to take part in auctions where the real search is very different from the targeted keywords.
In accounts with low bids - perhaps because, like travel, the industry has a lot of browsers and relatively few bookers on any one day - the effects are less easy to see, because most of the combinations that will be generated, end up either with no advertiser (”cheap family” - taken from consecutive searches for “cheap holiday” and “family holiday”), or with plausible Broad Match relevance (”holiday holiday” would be a plausible synthesised search, which would trigger Phrase Matched and Broad Match holiday adverts).
So the signal will be, in many accounts, subtle and all but invisible. This means that regular activities, such as adjusting bids to keep inside ROI or profit targets, may conceal the effects. There will be a small proportion of advertisers for whom the new Search History Permuted Searches form a substantial fraction of their impressions. These are the advertisers who suffer - but they need those peculiar combination of search queries to trigger them.
Looking in detail at some mass market and lower bidding clients, suggests that the frequency of unusual search queries is about 1 in 30 to 1 in 60 (about 3% to 1.5%). This is hard to distinguish though, because I normally run some Broad Match keywords in these accounts anyway, and Google’s natural extension of Broad Match could have snared some of these searches. The frequency could plausibly be as low as 1 in 50, and Google may have turned up the dial on Broad Match to capture a wider range of queries (which, if you’ve read the previous articles, means that the auction will tend to generate more money, because each auction will usually include more bidders and that drives up the paid price).
New Signal?
Cross correlation is a statistical technique that asks “how similar are the variations in these two data sets”. If I have a daily record of impressions, for example, I can break it into two periods, and compare, as in thi case, Feb 2006 to Feb 2007, with the range Feb 2007 to Feb 2008. I get a strong correlation co-efficient, meaning that every year, the pattern is pretty similar. It is even better when I leave out the fixed holiday dates - Christmas and New Year fall on different days of the week.
Looking at my mass market clients, I can see that there may be a signal of broadening Broad Match, aside from the Search History Permutation. I have accounts where I have managed the daily spend to approximately the same budget spend for ages - given that there is a weekly profile of search, when daily variation is expected, so long as the weekly profile is maintained. These have suddenly escalated since March 1st. The bids were the same, the keywords and adverts the same - and thousands of active keywords are involved. Why would the paid price escalate, suddenly?
This may actually be other side of the Search History Permutation, though. Consecutive searches mostly include implausible searches that wouldn’t trigger anything, or plausible searches that weren’t part of the current keywords. So in a small number of occasions, additional advertisers are recruited. This pushes up the bids. Even an appearance as low as 1 in 40 (2.5%) causes a measurable price change in the auction, when you have a hundred thousand auctions a day.
Date Inconsistency
Now, I believe that I first saw “odd adverts”, which I didn’t immediately connect with Search History Permutation, some time in late January. A lot of AdWords advertisers make insufficient use of negative keywords, so I’m used to seeing inappropriate adverts showing up from time to time. However, I’m pretty sure that the effects were visible, at least sometimes, in January 2008.
So why do most of my accounts that show a change, show it from March 1st, rather than January the somethingth?
I have no idea. None whatsoever. I’m worrying at this discrepancy. What if the changes in prices and impressions that I’m seeing are not caused by Search History Permutation, but some other algorithmic tweak?
Well, I do see *some* visible effects - there are some odd searches in the logs - odder than I’d expect to come from Broad Match.
You know the biggest signal that I’ve found something? Visits from the part of the world that mostly houses Google, to the blog article about Search History Permutation. When the first article was published - either the Googleplex decided my recreational pharmaceutical mix was unusually strong and they couldn’t stop laughing, or I struck a nerve, because it is the highest daily readership from Palo Alto that I’ve seen - IP addresses in the Google ranges. (No, I don’t really do drugs stronger than coffee or capsaicin - sheesh, a joke).
There’s probably still a piece of this puzzle that I’m missing. Ah well, back to trying signal analysis techniques and staring at lists of keywords and search queries, bids and average paid prices. There’s some way to make this make more sense - I just to have find it.
Summary
Search History Permutation probably has an effect across AdWords on the order of about 1-2% of searches, though the effect may be more pronounced especially in niche markets.
The effect appears to be an increase in Average CPC of somewhere around 3-5%, but may be higher in niche markets. I haven’t sorted out the maths to justify this, just observations, so far.
For accounts with the right niche keywords and higher bids, the effect may be pronounced - sometimes drastically reducing conversion rates, possibly on the order of -90%, because so many of the wrong searches are brought in. For example, if you specialise in one named category of something, and bid high for it, then I think that you’ll be shown much more frequently for the mass market searches than the niche searches - and that will destroy CTR, conversion rates and profit margins.
