• Google will struggle to retain revenues using a variety of techniques
• Searchers will spend more time browsing and convert after more clicks
• Online revenues will generally increase – but business margins will be squeezed
• Internet Theft Scandals – Click Fraud, Phishing and Account Theft

Details and the consequences? Read on…

Google Will Adjust To Preserve Revenues

The basic idea of paid search is simple. Searchers submit search queries, and advertisers pay to have their adverts shown to more or less interested searchers. This meshes with the purchasing process (buying model) in several ways. At the most basic, when someone searches for your unique trademarked business or product name, it is likely that this searcher is intending to buy something. More likely than if they’d typed a competitors name. When they search for something that describes the product category, they are less likely to buy – they are in an earlier stage of the buying process.

Auctions have some basic characteristics. As a broad generalisation, the more bidders there are in an auction, the higher the revenue for the sellers. So Google’s goals have to include increases in the number of bidders in an auction – Broad Match and adding ever more advertisers all help with this. Convincing advertisers to bid high – traffic scales heavily with position; Google needs to convince advertisers (or, rather, prevent them from becoming aware of the relationship) that position and click quality are not related, but that position and volume are related. Finally, Google needs more outlets, so that advertisers see more reasons (more impressions) to be part of the ecosystem; however, that largely means adding new, smaller volume, publishers – some will have niche specialist interests, but overall this encourages more Made For AdSense sites, which, IME, have an appallingly poor conversion rate and value (reducing click quality by adding more outlets).

You’d expect that in retaliation, advertisers would then want to stick to Exact Match and using only Google’s Search Pages. There’s other factors at play, though.

Users miskey – they can’t remember the name of the company properly. So someone looking for the travel company Thomas Cook might key “thomson cooke”, “tomas cuik’ and all sorts of other variations. This means that Exact Match isn’t enough to identify all people looking for the trademark or brand name. Broad match helps by allowing Google to send these near misses to the right advertiser. Google do a phenomenal job of matching; if an advertiser like Travelocity doesn’t use “cheap holidays” as a search term, then Google will match a high bidding Broad Match keyword to that search.

People will also type more specific searches, longer searches; most searches, by a long margin, have more than two words in the search query. Users will type “logitech mac support” or similar, to more rapidly jump to stuff they know exists somewhere in the vendors site. Some of these longer searches won’t lead to sales – as in this example, some searches are support inquiries. So Exact Match also fails as being too specific, because these search queries wouldn’t be matched. Phrase Match is useful to find unexpected variations and more specific searches – leading to the idea that you can help users to jump deeper into the site with the right combination of technology and advert. Again, typos and miscomprehension by users will mean that Broad Match captures additional searchers who intended to find the business, but would have failed with solely Exact or simple Phrase Matches in the campaign.

Google’s problem is that when someone has decided on “Honda”, or “iPhone”, the searcher is unlikely to be deflected by alternatives. The consequence is that there will be few competitors for a brand name. Brand names are usually the best return on investment – so the average costs per click will tend to decrease for brand names. This is part of what you’d expect from a basic analysis of the buying process – you generally type a specific company name after you have researched and typically just before buying. So there is an inbuilt pressure to decrease advertising on exact matched competitors name – resulting in lower revenues from trademark terms. That’s a problem for Google.

The earlier phase searches, which tend to be naturally higher in volume as people look for alternatives from which to choose, naturally imply a worse ROI – you need to pay for more clicks to support the research, and the conversion rate is lower because searchers are researching. This too tends to drive Average Cost Per Click down, in order to retain a positive return on investment.

Google needs to keep advertisers focused on Broad Match. This lets Google place unsold inventory, offer competitor adverts on trademark searches, and so on. The goal for Google is to increase the number of participants in the auction (helping increase average cost per click and hence inreased revenues), to find ways to increase advertising on unsold inventory and to affect the minimum bid.

Some techniques that are likely to be used by Google?

Expect to hear more about the difficulty of reaching interested searchers with organic search, now that personal search is becoming more deeply embedded. If advertisers think that they are missing audience because of personal search, that they can only reach with paid search, then there will be more advertisers and increased competition. This will be partially driven because of the effects of recession… in some market segments there will be reduced interest, and this will naturally translate as reduced search volumes, resulting in fears that personal search is sapping share of voice.

Expect to hear more about Quality Score changes, probably surrounding the calculation of the minimum cost to appear on a page. This is not the same as, but is confusingly similar in name to, the estimated cost to appear on the first page. The intention will probably be that if the CTR is lower than some target value derived from the rest of the network, that the minimum price is adjusted to meet some revenue goal – it won’t be written like that – I can’t guess how Google will describe the technique, other than that is likely to involve some description about “improving user search experience”. There will still have to be a way to allow $0.01/click – a feature that attracts many advertisers, few of whom have any real chance of achieving this… but it remains an important differentiator over Yahoo and the other competitors in this space. I think it is possible to offer both, because you only get $0.01 on high volume, high CTR keywords – IOW, not things that affect most advertisers.

Expect more dilution of “real search”. Google still wants additional outlets and will continue to re-present domain parks, “fake search”, selected content match outlets and other “opportunities” as if they were what naive advertisers expect from keyword search (that is, that the advert is shown directly in response to a real user search that leads to a directly relevant site – much as happens with organic search results). Google will want the increased impression volume and the *overlap* of keywords with different intent that allows a single outlet to have larger counts of competing advertisers.

Expect more messaging about the synergy of paid and organic search – that even when you have achieved page dominance for the targeted keyword, you should still be advertising as well topping organic rankings. This increases competition, ensures that there are at least ten plausible advertisers, etc. I expect the messaging around this to increase as advertising volume and value shrink.

Expect more overseas advertisers. Google doesn’t offer USD bidding to smaller overseas bidders. The exchange rate isn’t notified to overseas bidders. So, especially when currency exchange rates are in flux, Google can make margin on exchange rates. I expect that Google can easily and all-but-undetectably collect additional revenue on overseas bidders, by tweaking exchange rates – AFAICS, the auction is held in USD, so exchange rate changes are needed for anyone not using USD. This would be non-US income – no-one in the USA will blink at using exchange rates to improve revenues from non-US companies.

Radically – if Google were to remove Exact Match, they could make dramatic transformations of revenue expectation. Would they do this? I think they would, if they could claim that it improved the search users experience. If advertisers fail to appear on a substantial fraction of searches, especially when those clicks turn up in organic search clicks, it would allow Google to say that exact match was preventing searchers from seeing the results that they want to see. This would be a huge step for Google, and they’d probably need a lot of research. It’ll be piloted by some large accounts and evidence will probably be drawn from wide-category vendors, like eBay. It’ll turn out to be complete rubbish for narrowly focused vendors, but Google’s objectives are maintain share of search, and only secondarily to satisfy advertisers – the auction will take care of some that advertiser anxiety.

Increased Search Time & Reduced Conversion Rates

Most people are planning on reducing spend. They’ll do so by spending more carefully. They’ll look around more. They’ll be taking more personal recommendations. Trust in the stability of large organisations and well known brands will continue to be eroded – which means that the right businesses, with the right trust validations, can emerge from nowhere; just because your business has been running for 5, 10, 50 or 150 years is no guarantee that you haven’t recently based your business on false expectations of investments and earnings. A new business with the right accreditations can play on the same field as a 200 year old business, and can raise fear and doubt about the financial stability of well established players.

So, shoppers will be more wary. You’ll need to improve your sites to have online messages address current concerns. The fears, uncertainties and doubts that were addressed in marketing communications last year, won’t work so effectively this year; the concerns are different. Your customers will be concerned that even buying from a major brand will damage them, or is at least risky. Reassurance and validation will be important – especially with the media focusing on bad news. Bad news sells and media outlets will want more viewers. The at-risk media outlets, old broadcast media, seeing declining share of advertising budgets will want to stimulate sales through increasing interest in controversy, further weakening consumer confidence. Failure to address offline media reports online will impact conversion rates – though you will want to avoid directly feeding the controversy.

Minor brands should be able to gain, if they are cash flow positive or can find investors with imagination, foresight and cash. Guarantees and warranties, future-safe products and ways to reduce service costs for consumers; ways to manage domestic finances effectively; ways to cut back while still having luxuries.

The results of these will be to put some strange pressures on paid search. Companies that have previously survived on word of mouth and low or zero advertising, will need new customers. They’ll advertise. As new advertisers, they’ll make a lot of mistakes and won’t be willing to pay for expert assistance. That will increase claims of click fraud, and apply upward pressures on bids, sometimes from advertisers that shouldn’t be in the auction.

Existing advertisers will want to improve performance and will decrease budgets and target the spend on the most effective keywords and adverts. This will increase pressures on the paid search companies to dilute the inventory – the result is likely to be that overall clicks/conversion will continue to worsen, otherwise the paid search companies will report losses, and decreases in search volumes. It’s always bigger news if a big company loses, than a bunch of smaller businesses – so expect that larger businesses will use their asymmetric control of information to manage smaller customer businesses expectations and margins, through manipulating click quality. Expect the search engines to protect themselves at the expense of smaller advertisers.

ROI will generally worsen, but there will be islands where specific companies have struck the right messages that resonate with users. The result will be that paid search competition will continue to heat up. I expect that the average CPC will decline, the total value of paid search will decline, but some advertisers in various niches will have justifiably higher Average Cost Per Click and increased spending.

Affiliates

The affiliate industry will also become even more heated. Out of a job? Looking to make money with a low capital investment? Then you could become an affiliate…

However, who needs more novice affiliates? What does an influx of untrained, cash starved and at least initially ineffective affiliates do, when the real super affiliates (not the one man and a dog operations blogging from “super affiliates are us”, but the real, quiet, and highly effective super affiliates) already handle about 80% of affiliate traffic? The answer is that these new affiliates will provide free advertising for marginally effective businesses, by spending their own money essentially as an investment. Expect more companies to switch to affiliate advertising models to control their in-house marketing costs and to reduce spend on advertising agencies. Expect a lot more annoyed novice affiliates.

The affiliate industry will boom; but leave a lot of disillusioned “internet advertisers” in their wake. There will be some emergent new stars – not everyone who comes in will fail, and some existing leaders will cash out and move on. Expect some churn in the top affiliate products – but that’s pretty standard in the industry anyway.

I expect that the real super affiliates will be the same next year as this year – they have developed their techniques, and they are effective. They might retrench a little and change their focus on the businesses they are interested in, but they’ll survive and probably continue modest growth; their main obstacle to growth has historically been cash; 60 and 90 day payment cycles on sales conversions mean that working cash is tied up, effectively limiting them to “four to six inventory turns” a year. That reduces their rates of growth, and with banks not in lending mode, these guys are bottled up, at least in paid search.

Online Revenues Will Increase

Well, they will unless the internet theft scandals are outrageously large. In a search for the best value, comparison shopping online and internet supported purchases over the phone will become more important. Where businesses aren’t already selling online, there will be pressures now to do so, to reduce the costs of sales. The final result will be that more is sold online – even though total sales from all sources will decline. There may be some countries where there is a decline in online sales, but it will be less than the decline in total sales – the internet will be seen as one of the brighter spots. Just.

There’s only one reason for internet sales to increase. If executed effectively, you can reduce the costs of making the sale. That tends to be less and less true the lower the volume of sales; so expect continued outsourcing to places like Yahoo!Merchant Stores, where the infrastructure and development costs have already been put in place, and allows dropping the cost of sales. The interim step between often dreadful sites with no compelling call to action and a full online sales site, is a phone number; but placed on an inactive site that doesn’t reflect your best price and the unique value in buying from you rather than a competitor, this will have no effect. So expect the more savvy small business to look for CMS based websites where they can update their own content. I’m expecting that smarter SME’s will be investing more in templated CMS backed websites than the current static crop.

And metrics. Web analytics – free tools like Google Analytics – properly used, can reveal a lot. The problem for SMEs is setting up and understanding the data. There should be an increased demand for skills in setting up analytics, and interpreting what is happening.

Internet Theft Scandals

With money tight and some smart people out of a job… expect internet fraud to increase. With businesses looking for excuses to write off problem debts, expect more disclosure of problems, probably triggered by an inadvertently exposed online scam. This is obviously a tentative projection, as it depends on unforeseeable circumstances. However, this year is the first year since 1994 (when I started internet sales seriously), when I can see reasons to add excuses to the balance sheet, some understanding of the risks to businesses and that even corporations are affected by fraudsters, and a large value of online business that probably will increase (at least in relative terms), and an increase in technological capability by scammers and fraudsters. The combined pressures might make it attractive for the first time to attribute losses to technologically sophisticated thieves.

Online security for the average consumer has not improved in any seriously identifiable way since 1994, other than the progressive plugging of vulnerabilities in web browsers and servers. The introduction of Secure HTTP (https, or “secure server” technology) back then, provided users with a somewhat artificial degree of confidence. Many web sites offer access to financially significant resources with only an account name and password. Account names and passwords are insufficient for best security practices. Banks now often offer multiple levels of password and CAPTCHA, and may require authentication through encrypted PIN checkers – beginning to approach the holy trinity of security (“something that you are, something that you have and something that you know” – at least two of those are addressed by the better banks). However, there are still far too many ways to spend money online that are protected only by guessable account names and passwords, and by essentially unprotected mechanisms to send money to scammers.

There will be scandals about this. Probably soon. If there’s a sniff of a problem, then the offline media will pounce. These media need to pounce and will argue forcefully; their businesses are in decline, and they’ll need to savage the failures of online businesses to help protect their own. While this would be a significantly negative sum game for all online sales, the benefit of defection by an attacked business will be high; “We really made money, and we only show a loss because of internet theft we couldn’t control” will be an attractive excuse to some business, at some point. And then the skies will open and the extent of internet crime will be an issue.

That will damage sales. Hugely. Even if the effects are already part of current accounting systems, reported as part of business accounts and effectively managed by all online businesses, the *perception* of reductions in safety will be very damaging. I don’t see any sign of widespread adoption of even “best practice” account management – internet wide consistent login methods and messages, proper understanding of what a “secure server” means, and so on. While users are prepared to pay money to sites they can’t properly identify in return for promises to deliver products they don’t receive, across national borders (losing nationally accountable and interested police enforcement), this problem will continue to grow. The individual losses are small; the investigations complex. At some point, this will become a more serious problem to deal with than Enron and Madhoff. I think it’ll be this year.

Consequences

New advertisers will start by thinking about paid search – increasing pressures there. This will result in disappointment and claims of click fraud, especially if new advertisers discover that a click is not a click… different clicks have different values, even if Google conflates clicks from multiple sources. Expect a lot more noise and heat from new advertisers who feel they have been mislead.

Existing advertisers will mostly reduce and focus spend. AvCPC will have a downward pressure, counteracted by the SE’s including low value inventory – weakening conversion rates and hiding it in a generally slower market with inbuilt tendencies for longer latency and reduced likelihood of sales. ROI will get worse; the question is whether the business can take a few quarters of bad ROI in order to survive to an upturn. There should be increased attention to improving the web site, however, this will be diluted by the feelings of senior management that enough work has been done on the web site – “it’s a finished product” – and that there have been sales, so all that is needed is more visitors. IME, that’s usually wrong – there’s usually a lot of ways to make a site more effective in selling, even for long latency sales.

There will be a switch to increase SEO efforts. If purchasers will spend more time in research anyway, increasing the clicks and site visits per sale, then organic search results become more interesting. This will increase interest in spammy linking as people learn their way into search engine marketing. There will also be increased interest in affiliate advertising – if you aren’t skilled in internet advertising, then getting affiliates to advertise on your behalf will be interesting. However, this again should result in increased interest in web site design improvements; it won’t, because once a business have “put your brochure online” the naive perception is that there’ll be nothing else to do. There’s still far too many naive advertisers with ineffective web sites and a management team that focuses on volume of traffic, not the quality of that traffic and the quality of the response from the website.

There will be a decrease in search volumes for products. Despite the increased research for purchases, search volumes for commercial products will be negatively affected. That’s partially because the economy is slowing, but mostly because people are learning the online environment; they know where to go to get various things at decent prices, so they don’t have to do so much searching and researching – I haven’t heard anyone complain for the last 18 months that they are “useless at searching”. This year may be first year where the combined effects of knowing your internet neighbourhood and slowing commercial activity, actually dent the growth of both paid and organic search.

A significant internet theft scandal will cause a dramatic decrease in online consumer spending, if it bursts. Expect new web browser technology and new web site authentication services. At the moment, I’m pretty sure that the only way to tackle the identify theft problem is solutions that, as an industry, we’re nowhere near considering. The short term consequences will be to push the internet clock back towards use supporting research for purchases, not for actual online purchases.

CMS’s will become more important for SME’s. They’ll finally be able to build customised landing pages, important for improving the effectiveness of paid search.

Web analytics, neglected or misinterpreted, will increase in significance – and Google Analytics low price point makes it a very attractive platform to use, despite the poor general knowledge about how to best use it. The downside of “free” is that expectations of the value are low, and users are unwilling to pay a lot to understand the benefits; smarter businesses will invest in learning how to use GA effectively (IOW, not as it comes out of the box).

If this is what I think it is, then it this is another variety of click fraud – clicks paid from your money, in your account, that don’t benefit you but indirectly benefit the scammers and Google. Let’s have a look at what these scammers do, and see if they are detectable.

Cautionary Note

Be careful when you visit a site like this. A classic malware attack is to get people to visit a compromised site, that hosts malware that will enlist your machine in a botnet. If you go looking at stuff like this you need to be very careful that you don’t get compromised.

I think I’m pretty safe. I used an old email address that hasn’t been used for a Google Account previously. I used a unique password for that address – never previously used. I also changed everything immediately afterwards. I didn’t connect it with our MCC, and I used an account with no adverts and no funds from any source – so there’s no trace of my identity or connection with our business. I also used Flock for the first time – a social networking variant of Mozilla Firefox – to avoid any residual cookies and so on. I did most of the initial work in a virtual machine running Windows on my Mac – making it pretty difficult to penetrate the security – and when I saw no malware, switched back to Mac OS X for screenshots. I really wouldn’t advise looking at these criminal activities unless you take at least the steps I used. I expect that someone who has been involved in InfoSec more recently would suggest even more protective measures.

Domains

The domains used for this phishing attempt were source-adwords.com and ads-source.com. Like previous domains, these are registered to French mailing addresses. Not the same addresses as the previous round of messages – so it may be that they are abusing the identity of otherwise innocent parties. Since the scammers aren’t counting on the domain lasting for enough time to be fully registered, they don’t really need a real physical address that reaches them. I’m excluding legal requirements – these guys are criminals after all, so expecting them to obey any European laws about registering correct business addresses is excessively optimistic.

The name servers they use do seem to be consistent. This may suggest a relationship with some kind of hosting service. I must check that out and see whether these servers are all in the same facility.

What I Did

When I got this round of phishing emails, I checked the “whois” records, and captured info about the claimed domain owner. I then attempted to log in with fake password – looking like a typo of the real password. If it was a malware download, I figured they’d go for both valid and invalid logins. They don’t appear to be delivering a malware load, or at least not the range of sites that I’ve seen.

Another common Trojan technique is to put up a fake login page, and then issue an error message, even if the right data is submitted, redirecting to the right site – so at the point at which you become suspicious, you are now looking at the real site. When you rekey your details, they work. Most people assume that they miskeyed the blanked out password. The scammers meanwhile have collected your details and can now login safely.

With a twinge of doubt, I submitted a real account name and password – knowing that the account was pretty much vanilla, having been just set up and being completely unfunded. That let me in to their stumpy site. Half the links don’t go anywhere. What it did give me was this offer:

Google’s been offering SMS alerts for some time. I’ve signed up to them for many of my client accounts, and I know what the screen looks like. This isn’t it – and notice the wierd check box with pseudo-English offering “I agree with security types”?

This screen may give the scammers another revenue opportunity, if you give your cellphone number – but I don’t know much about mobile fraud mechanisms, yet. They obviously don’t care about that mechanism though, because they’ll gladly accept an empty phone number while giving a message that, yes, they’ll be giving me alerts. A note for non-US users – this page may strike you as odd, because it is clearly configured for a US phone number. Inside the US, of course, you won’t see the number format as jarring.

If you think you’ve seen this on your screen, you’re probably at risk.

The Evidence Trail

I briefly did some work in Information Security a few years ago, working with CAP Gemini’s InfoSec teams in the UK, and others. This data is not up to the standards of their digital forensics, but there are some interesting pieces of information we can pick out.

Oh ho – here’s the hot clue – MonetaAccount? Not something associated with anything I’ve been doing. Obviously, just as they use multiple peoples names for the Domains they use, this name may not be unique. I’d have to look at a few more phishing attempts before seeing the pattern here. Moneta does seem to be associated with mobile phone topups and instant charging. Perhaps this is way to send clicks to Moneta, or that Moneta is being used to extract funds (e.g. asking Google to close the account and send funds to Moneta?). Remember that Moneta may not be directly involved. If they run an affiliate program, this could be a, hrrm, “excessively enthusiastic” affiliate, using someone elses’ money.

These guys apparently aren’t using the AdWords API. If they were, there’d be a clue in the Access tab of my compromised account. It would probably also be easier for Google to detect and track them down.

However, the speed of checking the account name and password means that the Phishing Server is passing data back to the malicious software pretty quickly. There may be a signature that *Google* could recognise, of attempted access to an account from a know suspicious IP address. I’ve certainly had no warning that my account ID’s have had attempted use, and that I should check my account.

I can think of other techniques they might use, but I’m not sure that they are using them. I only like to document stuff that I’m pretty sure they’ve thought of already. They’ll be spending a lot longer thinking about and doing this activity than I can afford to spend pre-emptively working out what they do – an old dilemma for InfoSec. The baddies only have to break your site once to count a win and you have to defend against all the baddies, all the time, and can’t count coup on a successful defence.

What Is Google Doing?

They are clearly working with domain administrators. Nether of these latest sites are now working.

OTOH, Google sent a pretty bland and generic message when I told them that this account was compromised.

Their email also didn’t clearly explain how to create a new account ID and to change the password, though it said that you should. This sounds like classic advice from a technical wizard, who has no idea that ordinary users have problems translating the words into actions. I’ve been playing with some screen capture toys for the Mac, so I may make a video about adding accounts and removing access for the old account. It’s a good reason for all that playtime (well, there’s another reason, too… and you might find out about it!)

There appear to be at least two different levels of response that Google offer. If I have an ordinary account, not linked to my MCC, and I use the support contact information, I get a response offering the right suggestions, slightly more slowly than I’d expect for a security/financially related response, with information that is hard to parse for non-IT/InfoSec literate users. If I have a similar problem with an agency linked account, I can phone, and get specific immediate advice within about 10 minutes and an escalation to a security specialist.

What Should You Be Doing

Since I wrote this article, another of my clients has had suspicious activity. We’re looking in to it, but it currently appears that a secondary user with a unique user account, may have clicked on a phishing message, giving access to some third party who set up an AdSense account link.

So, what can you do to defend yourself?

• Don’t click on links in emails that lead to account name and password forms – type the name directly (PayPal, eBay or AdWords) or use a bookmark that you set up.
• Read the URL of the site carefully before you do anything involving secrets or money.
• Make sure the secondary account users know the hazards – and disable (remove) unused secondary accounts to reduce risk.
• Read your History log every so often – exclude bid changes, which are probably the most common activity and look for the wierd events, such as new ID’s being added, or new destination URLs being set for keywords and adverts, or new and unexpected campaigns.

Summary

At least one class of AdWords Phishing scam is gaining access to accounts. What they do to accounts with funds is not yet known – but I’ll guess that you find new keywords added, with a new destination URL or possibly even new adverts. You may find links to AdSense or new payment processors – possibly signalling funds being leached through fraudulent clicks or by shutting down the account and stealing residual funds.

Advertisers and agencies should always key in the name of the AdWords site or use a known good personal bookmark. Don’t use links in email.

Google could do more to authenticate their emails to users and establish that they have access to data that scammers would have to guess. EBay does this – using a personal name that I have registered so that my account email includes details that are only shared between me and eBay. This helps me to trust those emails more.

Google’s explanations are not yet clear enough for ordinary members of the public to manage a problem. A clearer, step by step description of the process to change account details would help.

Google could more positively warn users. I’m expecting that the scammers use a characteristic signature for account access – a server somewhere that is logging in. If they were really smart, they’d use a botnet and have a compromised home machine access the account. However, there is a pattern for legitimate users. Most AdWords users will use the same IP addresses or ranges. Google could establish the normal pattern and then send emails to warn of abnormal patterns, when those abnormal patterns have gained account access and performed a signature set of activities. The signature looks detectable…

I’d like to see some active blogging from Google about this threat to advertisers, and how Google is protecting us. After all, activity undetected by advertisers is cash in Google’s pocket – not fraud. It only becomes fraud when known. There is an incentive for Google to brush this under the carpet and recognise more spend as revenue and less as fraudulent activity. If Google want to be my friend, then they have to act like it, and not hide the evidence of third party malfeasance.

While the frequency of these scam messages have been increasing recently, I suspect that the volume of click fraud is low. So it is a low risk – but a likely high impact for each affected account.

I suspect that the main subjects of this scam will be small volume advertisers, who are not AdWords daily usage experts – they won’t know what normal Google messages are like or what the evolving UI now looks like.

Looks like Moneta should be involved – the AdWords History Tool shows the Moneta Account ID. Backtracing within Moneta should allow identifying the perpetrators.

Updates

2008-07-03 Added new details of a real attack. Language tidy up. New section on activity you can do for self protection. Clarity on investigative activity vs screenshots – sense s/be unchanged but now clearer why the screenshots are Mac based.

If Cade Metz is correct, then this software might be causing some interesting problems for Google and Google’s customers and possibly even for SEOs. Why? Because every time a customer of Version 8 of the AVG Technologies Linkscanner tool does a search, the tool checks all the links on the returned results page. It does so by emulating user behaviour. That means that web server log file based web analytics packages may have problems, because the interaction looks like users visit and mostly bounce immediately. The volume of apparent users would increase, and all the increase would be associated with increasing bounce rates. I can imagine spending a lot of time working on client web sites to try and solve that conversion decrease problem – only to discover that my time was wasted by this tool.

I can understand why AVG Technologies have gone this route – but I think there’s better ways to implement what is needed, than this. The current implementation apparently causes a lot of other problems. Let’s have a look at what the problems might be – admittedly this is speculative, until I actually test the wretched stuff… No, I don’t deeply trust reporters to get things completely accurate. I’ve been quoted in news myself, and so have some of my colleagues and clients; I know how the quest for a story can lead to implications that weren’t in the technical origins of the item. That link for Cade Metz above takes you to a page that says he’s pretty good and trustworthy, though.

Things That Jump Out As Problems

Apart from the suspected clicking on paid search adverts? Well, this software visits sites and tries to look like a real user. So if Google doesn’t have a threat signature for this activity, it’ll feed AdSense adverts to those pages. You won’t get clicks on those adverts – driving CTR down – but this will affect all advertisers, so you should not *relatively* suffer. But there will still be ugly questions asked of Google about decreasing CTR and to the web marketing team about why the latest innocuous changes cause response rates to collapse…

And, of course, the publishers of CPM adverts (e.g. placement targeted adverts) will pay for every thousand impressions – even if they are just bogus web page loads generated by software. So advertisers will have to expect an increased vigilance and the possibility that Google might be missing a chunk of fake clicks. Now, while Google could find a signature for this (ten or twenty requests for links off the same page), advertisers can’t – they just get an extra hit each time there’s a search results page on which they appear. Advertisers have no clue that every one else on that page also gets a hit… which means that Google could hide the problem and just reap higher impression rates and more clicks.

So the key advertiser problems are possibly clicking through on paid adverts and that might cause advertisers additional clicks that should be identifiable as invalid, and additional CPM payments for content match on clicked through adverts.

Tough On Threats. Easy On You. Looks Like Malware.

AVG has focused on servicing user needs. Normally that’s a good thing. Google also focuses on user needs. One of the golden goals of marketing is to satisfy an unstated need. “Needs” are crucial to effective marketing.

However, destroying the value of analytics systems, and causing additional advertiser costs and provoking web management teams worldwide to go into a tizzy over declining conversion rates and increased bounce rates… could be seen as less than helpful.

This smacks more of an unwillingness to look at the problem properly, or ignorance, than of malice. However, the effects on advertisers, web analytics providers, Google, hosting services and so on – well, that’s quite a substantial group of offended suppliers.

This section title, BTW, is derived from AVG’s corporate tagline… I thought it was mildly humorous, anyway, in a satirical way.

Better Ways To Build A Mousetrap

I’m a fan of Akismet. This is the tool that protects comments for this blog. Akismet has a central repository of known bad comments. When a comment is submitted to this blog, Akismet looks for the same comment in the repository. If found, then the comment is tagged as spam. Frequent and worthwhile commenters get passed by Akismet. However, I sometimes get new comments in my moderation queue from people that the software wants me to review. If I approve the comment, then it is added to the central repository as a positive for that commenter – but if it is marked as spam, then it joins the other spammy comments in the DB. That way the community gets to check and rate, and not everyone has to look at every comment – only those freshly exposed to new comments and commenters are asked to evaluate. It is an effective tool – though I can think of techniques that might undermine it. Of much more than 10,000 comments tagged as spammy, I’ve personally been asked to review less than a hundred. That’s an insignificant ratio and effective protection.

Using a similar technology might slow lookup for AVG (one query with ten items to a central DB, followed by some number of visits for sites with insufficient records; versus at least ten visits per search query) or speed it up – I haven’t done the math; it isn’t my product and I’m more concerned to make sure I have usable analytics. With my InfoSec-stuff hat on top of my AdWords and Internet Marketing hats, I’d prefer something that would allow a statistical sampling of a site by a range of browsers from different IP addresses, by means that reduce the load on web servers and minimise the perversion of web analytics.

Things That I Need To Check

Size of the problem… Cade Metz estimates up to 20 million users of AVG. AVG claim to be fourth in size on the AV market. In reasonably mature global markets like AntiVirus products, I usually assume that fourth typically means single figure percentage – somewhere in the 5-9% share range. That may not sound a lot, but 20 million seats is still a good sized business. If all 20 million used this link checker, then for search usages, it looks like 200 million or so users. That’s noticeable, especially given that for many businesses search (organic and paid) is a substantial fraction of all visitors. So this doesn’t look like a small problem – but I do want to invest a little time in confirming those numbers from some recent market share data and something that gives the market size. If it proves that AVG only have 1% of the market and less than 10% of all users use AV, the problem is a non-issue. :)

Cade Metz didn’t clearly state whether the AVG link checker executes JavaScript. If it is a browser plugin, I can imagine that it might hook to deep layers that allow it to look at the results of executing JavaScript. That would mean that it could also submit the image requests used by JS based web beacons/Page Bugs as part of its’ investigation to discover malware on the target site. So even Google Analytics, NedStat, CoreMetrics and Omniture would not be immune from perverted statistics. If the LinkChecker doesn’t check for images fed by JS based Page Bugs, then it misses a source of possibly compromised image files – so good InfoSec practice should be to check those servers, because if I were malicious, that’s where I’d hide a payload.

It isn’t clear from the article whether this LinkChecker does anything with Flash. Web Analytics and user tracking with Flash Cookies are increasingly popular – users mostly don’t know about them, and web browsers don’t have mechanisms to clear them, unlike ordinary cookies. If this malware checker is to be effective, it probably should be looking at Flash Cookies, as I can imagine that these might be used as an attack vector. So even Flash Cookie based web analytics could be affected.

It isn’t clear from the article whether the AVG LinkChecker *only* looks at organic search results or whether it also clicks through paid search links. If it does, then advertisers will see unusually high CTR from users with the LinkChecker installed, and will see conversion rates decrease and conversion costs increase. This is… undesirable… it’s a form of invalid click, normally a result of some type of malware!

However, if this tool is supposed to defend against malware, then the programmers that make malware can adapt by using low cost adverts – because if the LinkChecker *doesn’t* check adverts, that’s where anyone with the wit to write effective code will hide the payload. Duplicate an OK site under a new URL, submit 2 cent adverts and hope to pay 1 cent if the volume and CTR can be made high enough. Takes a bit of work, but I can imagine doing it.

That’s a bit of a nasty problem. If you don’t check the advertised target site, then you might offer malware loaded sites to users. If you do check, you increase advertisers costs and increase the accusations against Google that it is a rip off. Pragmatically – AVG is responsible for the implementation – if they click on my clients adverts with no intention of purchasing, and cause cash to flow to Google… if the pattern is visible to Google it should be an invalid click and no cost should be paid by my client. So AVG’s implementation has a cost implication to Google (tracking and denying this stuff costs), and an indirect cost resulting from further increased mismatch between Google Analytics and Google AdWords, thereby increasing fears of click fraud.

More subtly – what is it that triggers this code into doing a malware check? Is it the name of the site (so have AVG hardcoded all the Google domain variants?) and how do they recognise stuff like a Google Custom Search Engines? What about the Google Search Box on various sites – do they recognise those results pages? What about Yahoo and MSN Live? My guess is that if this has been implemented to work as widely as possible, then they’ll be looking at the URL parameters (tags) to see whether they look like Search Engine tags, and perhaps coupling it with some kind of wildcard (regex) matching for a built in list of major search engines. If they haven’t then they are missing at least 30% of search activity.

If they do check on signatures of search, then they may also catch some non-search sites – e.g. CMS and product catalogue sites and in-site searches, identifying links on those pages as being worth checking for malware. Oh dear.

We do have some tools for checking web server log files and we have clients with multi-GB of compressed log files per day… So I can check for the signature and get some clue as to the magnitude of the problem. However, chances are that AVG has focused on specific segments. If those segments don’t overlap with my clients segments, then I will underestimate the global impact. If the segments overlap significantly, then I’ll overestimate the effect.

Actions?

Apart from firing up Windows on Monday, I’m going to write to AVG. If their customer base is the size they claim (fourth largest AV solution) then this malware tool is likely to account for something in the range of 25% to 50% of search engine traffic from users with an active AV installed. That’s possibly quite a lot.

If The Register article is based on anything real, then this could have a significant adverse effect on metrics. Even worse, the effect will involve an escalating number of users. You can’t just apply a fixed offset (e.g. “subtract 5%”) that works for all time – it’ll need tweaking as the AVG customers upgrade. I need to check some log files on Monday and get some idea of the impact.

This may change some of the SEO and conversion improvement activities that I have ongoing. At least until we have retrospectively cleared recent months analysis of user behaviour. Some analytics packages can’t (won’t) do this. For example, a Google Analytics filter added today to remove the signature, takes effect from the time it was written, not retrospectively. So my client stats only work properly from today forwards, and until AVG change the signature.

Conclusions

The web is getting complex. Complex enough that efforts intended to achieve protection against malware can be interpreted by a different community also using the web, as malware.

More investigation is needed to see how this product handles JavaScript, Flash, other search engines and paid search adverts.

The original report suggests that this will make a difference to some advertising metrics – mostly making it look that there are more searches on at least organic rankings than before, and that a very low percentage of these will convert. This could mean increased advertising costs, until Google add an invalid click pattern. Google has to do this as individual advertisers will be unable to see the coordinated clicking on Google. It would be good to hear from Google (Analytics Blog, Inside AdWords blog, Ghosemajumder’s blog) how malware like this is handled.

If Google handle this properly, and the impact is as large as it seems, advertisers should be looking for an increased invalid click and impression rate – possibly both on content match and search.

The link checker could imply increased CTR and decreased ROI, by reducing the likelihood of sale in response to a visit.

I’m still thinking about this one. It could account for some stuff that I’ve seen on some client sites over the last month or so. I’ll just have to wait until Monday to get a good start on it. I’ll try to flag any updates…

Related Links

WebMasterWorld thread about signatures and webmaster responses to link scanners.

SEOmoz article about malware abuses of Web Analytics.

Updates

2008-06-15 – even before I publish… I’ve heard back from Pat Bitton, Head of Global Communications at AVG. I’m impressed that they are actively involved in the emerging news about this, rather than hiding till it blows over.

2008-06-16 – Fixed some typos. Edits for clarity. Heard back a second time from Pat Bitton – AVG have read this blog article. I’m beginning to think that Cade Metz should be on my watch list. Very interesting issue he dug up. AVG 8 Trial Version download now complete. Installation in progress.

2008-06-17 – Installed AVG 8 Trial. AVG 8 offers a toolbar with a Yahoo!Search default. Ran a few tests. LinkScanner shows site check mark in MSIE for Google, but doesn’t for Yahoo. Why would the preferred search engine not have link checks? Odd. Is this a way to penalise Google in some wierd way? Or an oversight to neglect checking Yahoo? Must do some more thinking about what is happening here, and some more experiments and reading of log files.