Anyway, returning to the main web site, at last, you can do a *limited* Google Search As If You Were In Another Country Or Language… I say “limited”, because at present we’ve only got the UK, US, Germany, France and Japan covered, in English, French, German and Japanese. If you have a language or country you need, drop us a line or add a comment on the blog and we’ll add it.

So, how do we know this tool works?

When we first developed the tool, we phoned search engine marketeers that we knew in the USA, South Africa, Japan, Australia and other parts of the world, to check what they saw when they did a search, and we compared it with what our tool was showing. They were the same, subject to the usual Google variations of ranking that different data centres deliver.

If you want to get some independent evidence, and you’re not in the UK, give us a call, and we’ll tell you what we see – so you can check whether it’s really useful :)

So, what’s top, right now, for the search for “SEO”, in each of the UK, US, Germany, France and Japan? Here’s the tool results:

As you should be able to see, we’re pulling up different adverts, and different organic searches. If you just change the country code – it’s not enough.

So… How can this tool be improved for you?

• Google Local – reviewed businesses typically rank higher
• Google Product Search – good reviews help more prominence in listings
• Prospects like reviews – next closest thing to word of mouth recommendation
• Reviews can be listed when people search for products and the business name

Unsatisfied buyers are typically around five times more likely to tell someone of a bad experience than satisfied buyers are to tell someone of a good experience.

As with anything, if you ask, you might get – don’t ask and you probably won’t get… Another great reason to communicate frequently and easily with your customers.

Review Sites

These sites are known to be sites that Google uses for reviews – it is not an exhaustive list:

• Qype UK UK places – pubs, restaurants, shops, businesses
• Reseller Ratings
• Yahoo! Merchant Reviews
• ShopZilla
• ViewPoints
• Epinions
• Google Checkout – part of the merchant interface for the Google payment service
• PriceGrabber (UK and US sites)
• DoYoo
• Ciao

I’m used to the phenomenon of users ReTweeting, in which the original author is given attribution. Are there really so many Twitter users out there, with nothing original to say for themselves, and who don’t have the generosity of spirit to apportion authorship, or is there a sponsored campaign going on to post substantially identical messages *except for* the shortened URL?

And there’s more…

As I say, I’m a relatively new Twitter user, so it is possible that this morally dubious behaviour is common. I hope that there some kind of community sanction that can be taken against users who ReTweet without attribution – but I don’t see anything obvious short of claiming them as spammers.

As Twitter increases in importance, this bad practice of plagiarism seems likely to increase, and starts to decrease the social networking value of Twitter.

It may also point to how corporations are starting to use Twitter to manipulate opinion; trying to force particular topics onto the “breaking list”. If I’m behind the curve on this, who is covering this kind of stuff about Twitter use and abuse and the likely future threats to Twitter’s value? I’m certainly going to be a lot more reluctant to consider the breaking meme listing, knowing that it may be being manipulated without apparent oversight or action.

Some of these are clearly spamming – the two different accounts at the break in the screenshots are obviously related and identical postings – operated by a common source, I suspect. Some appear to be fairly normal accounts, though I didn’t go looking for other plagiarism, it is possible that they may have been systematically ReTweeting without attribution.

Summary

It appears that Twitter is definitely reaching an importance level that means we can expect results to become affected. I know I’ve seen manipulation of trends, but I’d understood until very recently that this was mostly larking about. It could have been test runs to exercise a promotional network and demonstrate to likely buyers that the service would work.

Attribution, always an important topic for search engines, is even more important for social networking. Its part of trusting your news sources. Seeing this has damaged, for me, trust in the Twitter service.

If you tend to the dark side, along with spamming Google, spamming and plagiarism on Twitter probably are part of your mix, or will be.

As an ordinary user, I can’t see any sanction other than to report spamming; but it looked to me as if many of these people were otherwise just ordinary joes. Account closure for minor bouts of overenthusiasm seems harsh.

Updates

Danny Sullivan finally catches up with ReTweeting, misattributed and malattributred Twitter spam. (I’m joking – I got there earlier, he got there in more depth – good article).

City and Regional Targeting

In January 2007, I did some tests that used a rare keyword. Think “curmudgeon” and an adjective like “blue” – that wasn’t it, but the same sort of thing – low advertiser volume, but some real pages would be returned. I used 100 different IP Addresses that I’d collected, to identify locations. Some of the IP Addresses were known to Google AdWords as only being in the UK. Other IP addresses were more precise, down to a town in the UK.

I created campaigns that targeted towns, some that targeted regions, and some that defined rectilinear areas using latitude and longitude. Each had unique adverts, so I knew which Campaign and AdGroup I was seeing. And I had unique keywords, and shared keywords, so I could explore attempts to trigger specific AdGroups, or to see which AdGroup would win, if several were competing.

In Jan 2007, if I used a non-localisable IP Address (one of the 20% or so that have no location in the UK smaller than “being in the UK”), then I got different results from someone using a localisable IP Address, even when I used a place name in the search query.

Now, when I ask for a place name in the geotarget, I see the advert, even when I’m not in the geotarget. Except for one curious case… Internationally. I’ll come back to that in a moment.

If I’m advertising in the US, then, and I set up for Boston Harbour Crab And Tackle, I can geotarget Boston… and when people type “boston crab”, my advert will be shown, with the magic fifth line that indicates a geotargeted advert. My advert probably won’t be shown if people type “oakland crab”… This is a major advance over what I saw last year.

Talking to some Googlers, it sounds as though Google has added “geotargeted search query parsing“. In other words, they look at the query and try to infer whether you really want local results. This is likely to be complicated in the USA for several place names. If I recall correctly, every state has at least one Springfield, and most states have a Jackson County – working out which one you mean could be interesting, if you miss out contextual geotargeting information, such as the State.

International Searches And Geotargets

It’s pretty good that you can geotarget a region within a country, and get searches inside the country and not only inside the region, when the searchers are interested in the geographic area… But what happens to overseas searchers? If I want to take a ski holiday in the south of France, can I see adverts targeted to specific areas? It seems not.

As of the experiments conducted in early Dec 2007 and Jan 2008, I can’t find any evidence in the US or the UK (two large markets for Google) that a query for a location, coming from an IP address outside the country, will get regional or smaller advertising targets.

The impact that this has on the design of campaigns is that you should have campaigns for non-domestic countries, with at least two sets of AdGroups. One set includes the geographical place names and the other set of AdGroups is substantially the same, but the keywords don’t mention the location – they may have negative keywords for locations where you know you won’t convert.

Going Mobile?

I’d have thought that mobile phones were the gold spot for geotargeted adverts… but right now, you can’t geotarget adverts for mobile phones on Google. I suspect that this will evolve. I mean, if I’m looking for “restaurant” on my mobile, I probably want something within a few miles. Unless I type “restaurant boston” – when I want Boston restaurants.

I P Location

I haven’t seen any significant advances in technique, deployed in databases. Accuracy and precision are probably still dependent on the country, and still require pretty large targets for accuracy.

However, the increased access of non-local searchers, now allowed to see locally targeted adverts, may more than compensate for the low accuracy in some countries. There’s a mildly complex calculation to show whether geotargeting should be used, depending on the (per country) variable rates of searches using location names in search and the IP location accuracy.

Conclusions

Google continues to evolve geotargeting. It’s more useful this year than last year. I suspect that is a synergestic effort with the organic search results, which can also be improved by better location tuning.

This evolution is to be expected – economic analyses usually show that the bulk of people’s purchases are made within a few miles of the home location. So local targeting is likely to be important – a lesson that can be learned from the rising prominence of sites like eBay’s Gumtree and newer sites such as Qype.

Search query parsing is pretty tricky. I still have some experiments to do to determine the full impact of searches like “flight from london to new york” to see how effective the parsing is… Does the location of a searcher affect results? Does it figure out that “London” is the location for this search and suppress Manchester or Birmingham sources for flights?

This does complicate web analytics. AFAICS, previously I could be pretty certain that IP addresses for a geotargeted campaign were delivered from inside my desired target. This still seems to be true at country level. Below country level, if and only if the search has come from the same country, then search query parsing may allow non-local searchers to find small region geotargets.

Search Query Parsing may now allow geotargeted campaigns that were pretty marginal (because of the low accuracy of IP Location services) to be more cost effective, despite the additional management activity for the multiple campaigns.

Changing the geotarget can change your paid search performance quite markedly. However, commentators, and even Google, publish some wildly inaccurate information. I’ve read a few strange myths about geotargeting, in various forums. I’ve got some clients for whom geotargeting at country, and smaller areas, is very important, so I’ve done a bunch of experiments to find out what really happens. Here’s a brief account of some of the results.

What Is Geotargeting?

Geotargeting is one of the ways to address adverts to people who are more likely to buy. Other mechanisms within AdWords are the advertising system (keyword targeting, content match, site targeting), publishing system (web, mobile/cellular, print), languages, message type (text, graphic, video) and network (Google only, Search Partners, AdSense sites, etc).

Geotargeting relies on clues from users, most notably the IP address of the users’ web browser, to identify where the user is. Potentially useful additional clues are keywords or content that implies a geographically relevant targeting for the site. For example, “elephants denver” may imply that there is a geographical interest in Denver – though “John Denver” doesn’t imply geographical interest to the same extent. Intent is all important, and notoriously difficult to manage in search engines and in paid search.

Setting Geotargets

Geotargets are set at the campaign level of AdWords. Other advertising systems have different ways to select the geotarget. For example, Yahoo!Search Marketing selects the country geotarget at the account level, so a multi-country target audience needs multiple accounts (which can be tied together under one login). Some paid search systems do not have any way to select the geotarget at all, even at country level, or if they do offer a geotarget, may not enforce it effectively. We’ve measured some lower tier search engines that offer country specific targeting, with 55% of clicks out of the target area. Using third party (and cheap) geolocation services, suggests that Google is around 98% accurate at country level targeting – that’s within the measurement error I expect from the geolocation DB we’re using, so Google could be as much as 100% accurate or as low as 95% (1 in 20 adverts served to the wrong country – enough to run several large click fraud businesses; back of the envelope calculations for accuracy, with some hand waving and unmeasured assumptions, which would take an article longer than this to explore… another time, perhaps).

In AdWords, use the Campaign level “Edit Settings” link, and look at the lower part of the right hand column. You should see a language setting (surprisingly useful for some clients) and the geotargeting settings. This usually defaults to the country in which the business is registered. So a UK business address defaults to a UK (England, Wales, Scotland, Northern Ireland) target.

Measuring Geotarget Performance

There are two main reasons to look at geotargeting performance. Clicks from outside your geotarget should result in wasted spend – if clicks outside your geotarget result in sales, you should consider a larger geotarget. You may also want to specifically send different adverts to sub-regions in your main service area. For example, a national advertiser with local stores may want different adverts when the web user is within the local service area of a store – in this case, national adverts may focus on “buy online” and local adverts may carry specific local offers and “buy online or come to the store” messages.

Some web analytics packages report the country or smaller regions, if they have a relationship with a geolocation service. Google’s web analytics package, Google Analytics (Urchin, as was), now uses the same geolocation service as AdWords. Because the geolocation service is the same, using Google Analytics to help tune regional or country specific targeting is useful; when the web analytics package reports the same location data as is used for targeting, you can retarget and tune. You need to tune with respect to the geolocation service used by the paid search or advertising vehicle… because if their service area definitions differ from that of a third party analysis system, you’ll mistune the delivery.

However, there is a weakness to using and believing a geolocation service shared between the analytics and delivery systems. If the same location service is used for both targeting and measurement, then misdelivery can not be identified. How so?

Assume that you target only the USA. You set the campaign geotarget to the USA. You measure using Google Analytics to see if any adverts are clicked on from outside the USA. When the geolocation services used by delivery and measurement are the same, then errors in location will not be shown – because the assumed location in both cases will be the USA. So you’d be unable to see errors in geotargeting delivery, when the analytics system shares location data.

When checking the delivery effectiveness, you really need to use a different geolocation service. That is, if the AdWords geolocation service shows an address in Mexico as being in the USA, this could not be identified by Google Analytics… but a third party geolocation service could show the misdelivery.

Geotargeting Myths

There’s some strange stuff that people believe about geotargeting. Some of it is promoted by Google – probably not intentionally, but caused by some unstated assumptions that may not always be true.

IP Address Gives Away Too Much Personal Information

My domestic IP address is really sufficient to localise me to the UK only. Most geolocation services show that I’m 150 miles north, or about 60 miles south, of my real position. This is because internet connections don’t map to the physical world. Only in special cases can you use an IP address to accurately give location. There’s some newer techniques, none of which I’ve yet seen in wide usage, that can do things like measuring the response time, to narrow down the location of an IP address. But even those aren’t normally more effective than a few tens of miles (in the UK – could be closer in the USA, I think).

The way that most services seem to derive location is to use ISP’s optional information about the position of their routers, to give the end point location. In places like the UK, with a national telecoms structure, the router may be hundreds of miles from the end user. In places like the US, there is usually a good reason to believe that the end point is probably only tens of miles from the router, or not in Virginia. Why Virginia? Because that’s where AOL and Uunet and some other national ISPs located central services… and so many end users will be shown as coming from their web proxy/cache services, as the ISP tries to keep down their upstream bandwidth usage (that’s not a lot to do with Internet Marketing, but knowing this stuff helps to interpret why European advertising shows US clicks – much of it is US based web/cache proxies for European users).

Complicating all of this are “problematic” services. Some ISP’s use “dynamic IP Addresses”. This is very common with dialup services and also affects some ISPs offering ADSL. A dynamic IP address means that it could be used by people at opposite ends of the country in a relatively short period. Mobile data networks may give different IP addresses in a single session, as a result of many causes, from switching between bands, or changing base stations. There are also deliberate services such as anonymising proxy servers, that disguise the end point IP address, and may shift the address during the session, possibly around the world. Many ISPs use web cache/proxies, resulting in requests coming from the proxy server, not the user. Similarly, many large businesses will have a national gateway or even an international gateway between internal users and the internet – for example, I know of a business in the UK, with dozens of offices around the UK, with a gateway in Peterborough and I believe that some of the larger computer businesses with international networks have gateways in the San Jose/Mountain View area for all international offices.

While it is possible to get geolocation data, potentially very accurately (down to 10 metre/30 foot range, with the right technologies), for mobile/cellular phones, the cost has been high. The last project I was on that looked at using mobile phone position had a cost of around $0.20 per query (2002 price – cost is likely to have changed, probably downwards; see the comment by Josh, below). If your average advert cost is in that range, it would make geotargeted mobile advertising rather too expensive, though the location cost could be shared by all advertisers targeting that user/location.

The use of obfuscating technologies, such as dynamic IP addresses or anonymising proxies, whether intentional or not, decreases confidence that all IP addresses can be mapped to any location much more specific than the solar system… IIRC there have been some satellites and other space vehicles with IP addresses. :)

Static IP addresses, stable for long periods, offer the opportunity for the location to be more correctly determined, over time. But is there any sign that services from Google use this?

Google collects a lot of information from me, when I do local searches, or list my business, or register to do other physically addressed activities that would let them infer the location of this IP address. Despite having had the same IP address for years, and having supplied plenty of location information, Google Analytics clearly shows my IP address as being in the UK, and nothing more precise. This suggests that Google is not yet using all the sources of geolocation information that it could. Sample of one, though – not a lot of confidence in this answer!

Local Adverts Are Prioritised

Google staff often assert that local targeted adverts are shown in preference to adverts for a larger area.

This is an effect, not a policy.

If you run an advert for a local service, in a local area, you should expect that the CTR will be higher than a national advert, with a non-local message (or even with a local message). The higher CTR for a local advert will result in a higher QS for the advert, so the locally targeted advert will run in preference to the national advert.

However, there is nothing within Google that specifically prioritises local keyword search adverts. If you bid higher for national adverts than local adverts, you can get to a point where national adverts will be shown.

When testing messages, or if you have different offers that should be made locally and nationally, then you need to be aware not only of the CTR, but the bid, to ensure that national adverts are not sent locally.

You might think that you could use campaign level targeting to offer adverts targeted to a region, and another campaign that targets adverts elsewhere. This is an illusion. Because some IP addresses can not be given a location within the country, you need three campaigns – one for the target, one for other identified areas excluding the target and a lower QS (managed by CTR and bid) national advert that captures everyone without a specific location.

With an account cap on campaigns, you’ll rapidly run out of campaign settable targets with any large business of multiple locations. You’ll need multiple linked accounts to service more than a few tens of campaigns and make them sensibly sort into the types of target that they address.

85% Accurate, or 94%, or… What?

Google offers a variety of numbers for the accuracy of geotargeting. This appears to be because there are a lot of numbers needed to specify geotargeting – more than a single number. Representing the performance as a single number means that there are a lot of ways to choose that number and interpretations as to the effect. Worse, I believe that the numbers vary according to the country, because of the way that IP addresses and ISP’s work in different countries.

Given a geographically targeted advert, the following cases can happen – this is basically just a Venn diagram of the cases where a location is known:

* The advert is shown to an IP address identified as being inside the target; the user is physically located in the target; this is good
* The advert is not shown to an IP address that is physically located in the target area, because the IP address is not located there; this is bad
* The advert is shown to a user outside the geotarget, because the IP address is incorrectly assigned to the target; this is bad
* The advert is not shown to a user outside the geotarget; this is good
* The advert is shown, although the IP is outside the geotarget, because the keyword includes location specific names; arguably good.
* The advert is not shown, despite using location targeted names; this is arguably bad.

And there’s odd cases… like what should happen when you only know the location to within, say, a country:

* Advert is not shown, because the location isn’t known to better than a country; arguable
* Advert is shown, despite the location being imprecise; arguably wrong

To really understand geotargeting, Google should be publishing figures that represent the likelihood that an advert will be seen by a target audience, the likelihood that the advert will be seen by users outside the physical target, and the likelihood of missing audience that is in the target area. Why all those numbers? Because when those numbers dip below a certain value, you are better off not using geolocation services. I’ll probably write another article about this… it took me a long time to work out what the critical values are, and why, and it will bloat this article and distract from the key points, if included here!

Self Tuning

I’ve seen several commentators explicitly state that Google will tailor advert delivery to geographical regions. That is, if you use a nationally targeted campaign, but offer adverts for Denver and Chicago, that the system will select Chicago adverts for Chicago addresses and Denver adverts for Denver addresses.

I can’t find any evidence that this works.

One experiment that demonstrates this is to send adverts in multiple languages to areas with two significant populations speaking different languages. This should result in advert serving in the ratio of language use, and with analytics showing that the CTR is at the language-appropriate level, and the clicks come from the major population centres for the language. What actually happens is that the dominant language (highest CTR/highest revenue) advert is selected and served to the whole region. This appears to be true whether the region is within a country or crosses national borders.

You’d expect that if this effect worked, it would be most likely to work, say, when targeting adverts to Switzerland, with mixed populations of German, Italian and French speaking natives and quite a few migrants with other languages. Or when using French and Dutch, and targeting Belgium. Or using German and French and targeting Germany and France… Any way you slice and dice it, there is no way that Google is selecting a language specific advert for a geographically well defined area. I infer that no other location specific offers will be differentiating the delivery, either.

Use of locality keywords

In tests that I’ve done, when I use locality specific keywords, I don’t see locally targeted adverts – or rather I do, but they are targeted for where I am, not for where I’m interested in. This has an effect on, say, realtors/estate agents. If you are moving, you want a local realtor where you are moving to… that means you need to use national advertising with locality keywords.

The implications of Google’s documentation are that geotargeting solves a multiplicity of audience problems. I think it works with one model of geotargeting. Extending the use to infer that it solves other problems is overstating the capability.

Summary

Google should be providing better information, at country level and smaller regions, for the accuracy of geotargeting. This needs to express the likelihood of reaching a user in a target area, and the likelihood of reaching users outside the target area, as a minimum. Ideally, there’d also be explicit information about the likelihood of reaching a user in an area only by using national targeting (that is, the proportion of users in an area who have only national location or a different region for their geolocation).

Google should be considering geotargeting of mobile advertising. This is likely to be a growth area and mobile users will need local information – both local to where they are now, and local to where they will be, soon. Important for iPhone users, eh?

Advertisers should now be considering at least six campaigns for a local businesses that may be used by non-local searchers:

* the locality – keyword
* national – keyword
* the locality – keyword with mobile adverts only (though this is probably pretty pointless at present)
* national – keyword with mobile adverts only
* the locality – content
* national – content

Google should be offering some kind of measure of anonymising and proxy services that are being used. These disguise geolocation and can cause mis-delivery (non-US users being shown US advertising, when US proxies are used, for example). When obfuscated locations rise to a critical value, geolocation services cause more problems than they solve.

Well, I’m pressed for time… so I’ll leave this article for the moment… but there’s a lot more to look at with geolocation, click fraud, ROAS and conversion improvement techniques. For another article, eventually…

Updates

2007-10-26 Edits for clarity; corrected typos and some poor word choices. Also a pointer to a really pertinent comment by Josh, below.

2007-10-29 Added a little more detail on the complications of proxy servers and company gateways, triggered by CPCcurmudgeon’s comment, below. I believe that he’s a network systems programmer with experience working for a search engine – a credible commentator.

2008-02-26 Newer article published on AdWords geotargeting differences in 2008 to reflect more recent experiments and changes in Google’s behaviour.

There’s an increased interaction with AdWords data, with cost information being nicely embedded. I can see some stuff I’ll want to dig into. The Landing Page analysis looks really helpful, for example – possibly complementing or drawing on the experience from Google Website Optimiser?

I’ve been running silent recently, with a lot of work for clients and a series of experiments on geotargeting. Unlikely as it may seem at first glance, geotargeting and click fraud are related. Daniel Jupp’s article does bring this out, talking about armies of Indian and Chinese workers who click on adverts, and botnets that defraud advertisers.

So, lets look at Daniels’ article and find the gold and the dross and, more importantly, the unmentioned bits.

Defining Click Fraud

The article does highlight the basic issue – what does an advertiser, not a search engine company, identify as fraudulent activity? Unfortunately the article does focus on measures that the search engines find useful for their business purposes. I think that much of what I categorise as fraudulent, is activity that benefits the search engine, rather than the advertiser. There is a fundamentally different business goal for search engines and for advertisers.

The article hits at, but doesn’t state out straight, that search engines want to offer as many clicks as possible, but advertisers are (mostly) looking for the highest revenue potential. This means that search engines don’t mind passing clicks to advertisers that won’t convert, just so long as no-one can take them to court for it, and the dilution of lower quality clicks doesn’t provoke advertiser defection. This leaves a gap, where search engines are prepared to offer low quality clciks that cost advertisers money, from sites that advertisers wouldn’t want, if they could control the sources better. It may be that’s just my reading – the views represented here are, of course, mine, and not those of Daniel Jupp, I’m interpreting what he’s said into my words.

Daniels’ article also discusses the ways that click fraudsters can work, and takes a quick look at anti-competitive clicking.

So, all the right sorts of elements in the article, but I don’t think it pushes far enough or hard enough.

• Whose definition of click fraud? The loss to advertisers, or the loss to the search engine company?
• Lots of ways to get clicks that reduce the ROI – meaning subtle techniques may be needed to work it out
• Use of automation, and low cost human clickers

It should be noted though, that a badly set up paid search campaign also yields a low return on investment. That shouldn’t be called click fraud – it is incompetence by the advertiser, and can be solved by adding some training or expertise. I expect that, as the industry matures and the search engines disintermediate the experts at the advertising agencies, that user incompetence will play an increasing role in allegations of fraud. It simply isn’t search engine or click fraud if you use the wrong keywords and a bad landing page. Its solvable, but probably not historically – and those burned by the experience are going to take a while to return to the industry.

Alien Clickers

Quite a lot of the article is spent on addressing human clickers in China and India. This is, I feel, somewhat of a red herring. Geotargets are important. Some of the lower tier search engines don’t let you choose which countries you target, but that’s clearly not a rational or useful decision – a standard marketing acronym is PESTEL (Political, Economic, Social, Technological, Environmental and Legal), all of which pertains to a country and to even smaller units. Where you cross significant boundaries with a message, you risk a reduction in the value of that message.

For example, being British, I can go out for the evening, get pissed and have a good time. Those of you in the USA will wonder why I could have a good time when I’ve been annoyed. Language meanings are geographically relevant.

If you suffer from low converting traffic from India and China, just don’t target those areas, or pay for a lower position in those countries. With Google, MSN and Overture/Yahoo, that’s easy. It’s not so easy with some of the lower tier search engines, where you get whatever traffic they give, from wherever it comes. Your only recourse with the lower tiers is to examine your traffic and complain when you find odd patterns of use.

There are ways for non-local users to see your AdWords adverts, but it is a bit harder than just typing “google.co.jp” to see, for example, Japanese adverts, and both the IP address (via IP location services) and any referer_info will give clues that this has been done. I can think of ways to disguise even that, but then I can also think of ways to check on that… it’s an arms race.

I’m pretty sure that geographically isolatable click fraud can be controlled, by advertisers and agencies – so long as you don’t want to target those territories. If you have to target those (and other) territories, known for their higher than usual click volumes and lower than usual purchases, then you need to get subtle and use all the information that you can squeeze out of visitor behaviour.

The Subtle Knife

Despite the apparent intention to define click fraud, the article once again falls prey to allowing the search engines to define what it is. The article focuses on what the search engines report as identified click fraud. Frankly, it’s not the search engines that are failing, but the agencies and advertising industry.

Where is the external oversight and control over Google’s use of domain parks, as a result page for keyword search? A domain park is clearly not a search page. It doesn’t look even remotely like the process shown on the AdWords sign in page, which is what advertisers think they are buying.

Where is the external targeting control for keyword match? Google offers a very sophisticated system that will probe to get the highest CTR, but advertisers are not buying clicks – at least, the sophisticated ones aren’t. The smart advertiser is trying to buy interested visitors. If you just want clicks, there’s plenty of techniques to offer undifferentiated and largely unlikely to convert visitors.

That means that it is actually important that advertisers and their agencies complain when they find, for example, a keyword about financial services, with a financial services advert and a financial services landing page, being matched to a search for a “hair loss tonic” or a “weight loss clinic”. That’s clearly not an appropriate match, but very few agencies check for this. That sort of mismatch is unusual on Google, but we’ve found exactly this example on lower tier search engines.

How do I know that the agencies don’t check? Because they don’t use analytic systems that offer the information. You need either a well designed web analytics package (most aren’t able to do this properly) or to use web server log file analysis to compare the keyword with the actual search.

This verification of search match quality isn’t made easier by the search engines. While there are some pieces of information passed in the destination URL, it may miss out the actual search string. Kudos to Overture – their automated advert tagging offers both the user search and the keyword, so you can do a conceptual match. Google, however, persistently recommends users should look at the referer_info (sic – it really is called that in the standards documentation) field, which often isn’t passed. In one account I was looking at, more than a third of the dubious clicks had no referer_info. That’s partly because one of our signatures for a suspicious query is a missing referer_info field, especially on a non-AOL IP address.

There’s undoubtedly more that the search engines can and should do to demonstrate that they are trustworthy. After all, this year, I’m expecting that we’ll be handling well into seven figures of spend on behalf of clients. That’s a minute fraction of the revenue stream of the search engines… they are being trusted with a lot of cash, and very poor systems of control and oversight. And the advertising industry isn’t grasping that they need to be the ones to hold the search engines to account – because if they don’t, then advertisers may defect to other, perceived safer, media.

Where else can you get help? The web analytics vendors do not have reports that check on whether clicks came from the origins you bought. The search engines don’t pass completely harmless (to them, if they are behaving properly) information about the search, which would help advertisers. There are click fraud detection services, but every one that I’ve looked at (a handful – a small sample of the dozens that a casual search will show), does not configure the destination URL properly, to allow for disambiguation of double clicks; nor do they use Google’s own daily data on invalid click rates; nor do they recommend ways to configure the account so as to maximise data collection in such a way that future analyses could be easier.

There certainly are things that you can do to add more tracking data to destination URLs on Google. It is possible to develop software to do a conceptual match on actual searches versus the offered keyword, to see whether the search engines are extending to rational broad matches or just optimising revenue. It is very difficult, without more co-operation from search engines, to identify when they use, for example, content matching sites in nominally keyword search advertising.

Optimising Revenue

The intent of search engines is really at odds with those of advertisers. Search engines want to maximise their revenue, which means offering opportunities for adverts to be clicked. Advertisers want to find visitors who are interested in their products and may buy. Not a casual, passing interest, but a 1 in a 10, or even a 1 in 3 chance of actually buying something.

That the search engines have ther own interests at heart has been obvious. Look at Google’s “Budget Optimiser”. This is a tool designed to make sure that Google extracts every penny from the advertiser. Even if the account has AdWords Conversion Tracking, the optimisation is not for keywords that sell, but for keywords that generate clicks. Why would any sophisticated advertiser care about the volume of clicks? It’s the volume and value of conversions that counts. So even when Google have the data to optimise on behalf of a user, they choose to optimise their own revenue.

Good deeds go unpunished

I think it’s great that Daniel has spoken out about click fraud and the needs of the advertiser. I think it’s a shame that he didn’t push that one step further into probing whether the advertising industry should just be using the search engine’s definition of click fraud, and whether there is additional information that the search engines should publish, at least to the advertiser that has bought a click, to assure them that their money is being properly spent in the hopes and intention of delivering a valid interested visitor, rather than just optimising the revenue stream.

As the title of this article suggests, so long as the advertising industry accepts the search engine definitions of fraud, and fails to hold the search engines accountable for mismatching keywords, for geotargeting abuse, for using “keyword search” to host domain parks, and for “budget optimisation”, there will continue to be complaints that the medium can offer the opportunity for unexpectedly low returns – click fraud.

Although I’ve been too busy to be part of the debate, I do see that Rich@Apogee has found a client who gets a significant fraction of their clicks, on a supposedly search network, from content matched sites. This seems to me to be clearly beyond the line in the sand that search engines should not cross, without offering additional information and control to advertisers.

Advertisers have to wake up to the idea that just because these systems are technologically sophisticated, they can’t forgo the inspection that has been required in, for example, direct mail and magazine subscription circulation. These types of measure are currently lacking in the industry, and it will be up to advertisers to organise – because if they leave it to the search engines, the definitions of click fraud and acceptable behaviour will be left to organisations who fundamentally and legally owe a responsibility to shareholders, not to advertisers. That means making the maximum profit for the SE, not for the advertiser.

This mismatch of advertiser needs and search engine needs is, I think, at the heart of why you’ll hear complaints about click fraud. And it is why, when the search engines investigate, they’ll find no problem or rare problems. Nonetheless, as an agency working on behalf of advertisers, I can clearly identify some activities that search engines do, that I know will reduce my clients ROI. Is that click fraud? Arguably, I’d say it is – that’s the core of the issue for click fraud. Do you, can you, trust that every search engine, acts in the best interests of the advertiser? If they don’t, what mechanisms do they offer to assure advertisers that every effort is made to offer a high quality click stream?

Regrettably, the SE’s seem to have decided that the right course to handle this is a stubborn silence and an assurance that the SE’s protect themselves (not the advertiser) from their own definition of click fraud. It’s not enough. Not with a multi-billion dollar business.