In the first part of this article series about Google AdWords Professional certification, I looked at some questions and answers that I considered to be of doubtful value. Here’s some more questions and answers from the exam that I think reflect badly on the value of a GAP certificate to an advertising professional and degrades the value to a client of choosing someone with a GAP, as a specialist paisd search marketeer.
Keeping It Real
I couldn’t see any way to offer a critique of this without the whole question. The correct answer crucially depends on this AdWords Learning Centre page about the criteria for invoiced AdWords accounts in the UK. That page has been updated since the exam. On the day of the exam, and for at least a few weeks afterwards, the criterion was a spend of at least £750 per month for the previous three months. That, by my arithmetic, makes both answers B and D correct.
£750 per month is a daily spend of approximately £25. Even now, with the page showing a revised monthly minimum spend of £1,500, that’s still only £50/day, so both answers B and D are still correct, subject to Google’s unannounced criteria for establishing a credit record.
Which would you have picked and which would you have marked as correct? Which GAP candidate would you prefer to have? The one who answered B or the one who answered D? Note that I have worked for businesses with no established credit record with Google and no monthly spend at all, who qualified because the parent company of the group to which they belong would have qualified. So, strictly speaking, even answer A is true, if you set the right assumptions in place. So, with the right assumptions, A, B or D is correct.
Which is the more important to a client? Knowing the rules or knowing the rules that aren’t actually implemented as written down? What would be marked as correct? Had I chosen “A” would I have been penalised? Was I penalized for choosing either of the two potentially correct answers? Who can tell… that’s not the basis of a useful test, is it?
System Updates Obsolete Questions - Again
The following question is again missing crucial assumptions. Is this Standard Edition or the more recent simplified Starter Edition? When the campaign was set up, were we assuming default configuration? There’s an even more complex scenario, in which we set up two campaigns and carefully orchestrated bids…
So what were the answers?
- Google Only - potentially true, if it was a non-default configuration and I’d disabled the Search Network
- The Search Network Only - potentially true, if I was running a second campaign for Google search (no Search network) and bidding highly, so that this campaign with both Google Search and Search Pages was only or predominantly seeing impressions from the Search Network.
- All properties where your advert appears - clearly wrong, as content network impressions are not accrued to individual keywords
- Google and the search and the content network - also clearly wrong for the same reason that impressions are not accrued to individual keywords for content network.
But, hang on… the right answer isn’t even listed… if we assume that the question is about a default Campaign configuration. The right answer is “A” and “B” - Google Search and Search Pages. There was no answer presented that allowed a correct response, without making an assumption that wasn’t clarified in the question.
I just hate questions where you don’t even get a choice of one correct answer. What do you think the value is to a client who has a GAP who gets this answer correct, in Google’s marking scheme? Answering this one and getting a mark would show the GAP candidate to be worse than a candidate who left all the boxes unchecked because the right answer wasn’t present. Want to bet that Google penalises an empty answer and rewards an incorrect answer? If the correct answer is to leave all the boxes unchecked - want to bet how many students check *any* box, in order to have answered the question? This is poor technique.
Obsoleted Questions
Another question that would prefer GAP candidates weren’t aware of the current state of the system. When this question was asked, back in September 2007 and up until July 2008, this question again had multiple correct answers.
Normally, you are charged on a CPM basis, normalised to a click - Cost Per Click. You can also set up Placement Targeted aka Site Targeted campaigns, where instead of keywords you define the sites on which you want your adverts to appear - and you pay per mille (CPM) on the impressions. So answer A is true.
However, you could set up a special (Beta) campaign for “Cost Per Action”, where you might set, ohh, £40 per action. If you got just one click and that consumer bought, then you’d pay £40 per click. If OTOH, you received 1,000 clicks and no sales… you’d pay nothing at all. So Answer B is also true. And there is no answer allowing clicks, impressions and conversions.
So the more knowledgeable GAP candidate was faced with a quandary - what is the right answer? Should one answer “A” - the answer that Google expects? Or fail to answer as the correct answer is not in the list? Want to bet who gets a point? The klutz who wasn’t even aware of the Beta CPA program?
So what is this test for? To find people who know how AdWords *used* to be, or how it works now? Given that the test is supposed be annual, you’d expect that this implies it should be testing current knowledge, not some aged and no-longer-available system. Apparently, you’ll score better if you don’t know how the system works now. Hardly satisfactory for clients and not much of an inducement to learn, if by learning new stuff you’d score worse…
More Imponderable Answers
Another question with multiple correct answers based on obsolete AdWords knowledge.
“B” was correct from the time of the exam to a couple of weeks ago when the Beta of the CPA campaign was withdrawn. Until then you could control the costs with a CPA bid.
“D” is also partially correct - MaxCPC lets you control the cost per click in keyword search campaigns.
However, there is also the MaxCPM for site targeted/placement campaigns; the Budget Optimiser which will gleefully take over control of the MaxCPC that is bid (you retain control over the maximum of the MaxCPC); and so on. I can think of at least two more ways to constrain the bidding system for the cost of an advert. There is no single answer that contains all of the right responses. Not all of the right responses are offered. This is really weak question, again testing for knowledge of a version of AdWords that doesn’t exist any more.
Confusion - Is It An Exam Or Marketing Technique?
This next one is pretty poisonous.
The only answer that comes close, forcing you to select it is:
However, after many years of using web analytics and AdWords, I am absolutely certain that SmartPricing has nothing to do with ROI. Why not? Well, we need to understand a bit about what SmartPricing *is*, before we can know what it *isn’t*. When you bid for keywords, you should find that your best performing adverts result in a bid that is higher than the paid price. This is because Google is, effectively, rewarding you for a high relevance result and encouraging you to continue striving for relevance. What is the equivalent for Content Network *publishers* - not the advert quality, but the visitor quality?
The Content Network - all those AdSense sites - well, some of them aren’t such high quality display sites as Google. The initial problem, really a hangover from the early introduction of AdWords was that a click is not a click. Despite Google’s oft-repeated mantra in training materials and even in the Starter Edition, that this is so. Clicks have different values. Part of the reason is that the Content Network is the main candidate for click fraud - so most of the time you want to pay less for a content network click than for a keyword cick (not always - there are circumstances where the reverse may be true). In the very early days of AdWords, there wasn’t an identifiable mechanism that allowed a different bid for Content Network advertising. Nowadays, there is, and you can bid more or less on the Content Network than you do for keyword search.
So, one way round these combined problems - lower value returns on clicks on the content network for many advertisers, and an early inability to separately adjust the MaxCPC for the Content Network - is for Google to introduce SmartPricing. This discounts sites where Google thinks the value is lower. I have no idea how they form the rules, but I know that ROI has nothing to do with it.
ROI can not have anything to do with it, because Google don’t require conversion tracking before they apply SmartPricing. That is, you can set up an AdWords campaign, fail to use AdWords Conversion Tracking and you will still find that different sites pay a different characteristic price. So your ROI has nothing to do with SmartPricing.
OK, so you may say that Google then averages data from other advertisers who have got conversion tracking? I can’t argue with that. Perhaps they do. But the performance of a different product is not a strong predictor of the performance of another clients product for whom there is an overlap in keywords and sites. Think “segmentation”, “targeted messaging”, “demographics”, “price”, etc. Just because Advertiser A achieves a positive ROI for a specific site, there is no implication that Advertiser B will achieve the same result.
Now, worse, what if we *do* have conversion tracking? Does SmartPricing mean that web sites that have delivered several conversions on a low click volume will see an ramped up price, relative to sites that have delivered thousands of clicks and no conversions? Well, from my client data - no. For example, one well known social networking site sent several thousand clicks at a high average cost per click, with no conversions. A lower volume and less well known local directory and classifieds service sent a few hundred clicks, with a conversion rate in the tens of clicks per conversion, at a lower “SmartPrice”. I was certainly convinced that SmartPrice has the square root of -1 to do with ROI (oh dear - a maths pun - it is an “imaginary number”).
If the obfuscatory SmartPricing has anything to do with ROI, it is to do with the ROI of unrelated businesses.
My guess is that SmartPricing has more to do with competition for the space on the publisher site than with the value to the advertiser. That is, I suspect that when many different adverts *could* be placed on the publisher site, then it has a higher value to Google - so Google sweeten the pot for publishers by offering them more advertiser funds. Nice, eh?
Whatever, putting this “educational” answer intended to make GAP candidates feel good about how wonderful Google is, is a misapplied piece of marketing flummery. It is not a valid, useful and truthful question that separates the professional AdWords expert from the ignorant or deluded.
I’d be quite interested to know how Google do SmartPricing - and I bet that a lot of affiliates would be too, because manipulating SmartPrice factors would help affiliates increase income… And as an agent for advertisers, I’d like to know what I should be looking out for on sites, too. Is it likely that Google will reveal the secrets? I suspect not. Whatever they do do, though, it *isn’t* any of the offered answers for this question. And that’s just shabby.
Incandescence Triggered By Fatuous Irrelevance
I was furious when I encountered this question in the exam.
The answer they are looking for is “AdGroup”, because they don’t include as an answer “Keyword”.
What professional search marketer will bid all keywords in an AdGroup at the same price, all the time?
I don’t think I’ve ever come across a client account with anything other an minute smatter of keywords with a similar conversion rate and cost, and they were in different AdGroups. Meaning that pretty much every keyword has a different price, as soon as it gets out of the testing regime. If you pick a GAP qualified individual who has answered “AdGroup”, then you probably have a search marketing problem. This is a very, very poor question to identify high performing paid search marketeers.
Once again, the serious SEM would be left either offering an answer they know to be wrong, to gratify Google, or leaving the question unanswered because the right answer isn’t present. And of course, it all gets more complex when we consider the various bidding optimisation tools now built in to AdWords.
Insufficiently Insightful Analysis Of Click Fraud and Web Analytics
Now, this question looks like another piece of marketing flummery, or demonstrates that Google has very low expectations for GAP certified individuals.
You have to pick three out of 4 answers. So, hmm, you have one chance in 4 of failing to select the right answers. Technologicaly, that’s the same probability as guessing a single correct answer… But what if there are more answers than are shown? What if there are ways to construe an answer that at first blush appears to be wrong, as right when you take into account another factor?
This question assumes that I am so grossly incompetent at examining click fraud that I have not enabled auto-tagging (setting up the extremely useful “gclid” tag) and/or that I have failed to set up useful tags to identify the Campaign, AdGroup, etc.
I wouldn’t honestly think of checking answer A, because when I’m testing for click fraud, I know when my clients adverts have been showing on properties other than Google. Once again, the GAP candidate with a clue would be marked down for answering according to best practice, instead of answering for weakest practice.
Intriguingly, the answers miss two of the most common causes for mismatches. You can see and excess of clicks from Google, and some (most) web analytics don’t show you all the clicks, even if you have had them.
If you have set up proper tracking, you most often find that you have an excess of clicks from Google - because Google presents *valid* clicks. That is, you may get a click that Google later decides is false, so you get visitors from a keyword that Google swears blind has no clicks. In the most confusing case that I encountered, I’ve seen a conversion from a keyword with no impressions. All the impressions and clicks went unbilled as invalid, but this “invalid visitor” went on to purchase something.
Web Analytics packages tend to use a “ABC(E)” session as the basic visitor unit. And they typically only count the referral that started the visit. The consequence is that if a visitor has used adverts twice or more to find the site within a session, then those clicks and searches tend to be suppressed. You then appear to have fewer visitors than you expect and no easy way to identify that visitors are using the adverts multiple times to find you. In extreme cases I’ve counted single customers using adverts costing approx $15 up to seven times in a half hour period, and converting - not click fraud, but a function of the way that some people think about using search.
Now, these deeper answers to click fraud mismatches aren’t represented. I’d want to know that my GAP certified agent wasn’t so confused about click fraud that they’d waste time and effort pursuing something with an easy and simple explanation. There are real cases that do demand effort, but if they get lost in the noise of self-inflicted wounds (failing to tag and failure to understand the web analytics reporting mechanisms), you have a less than fully effective paid search marketeer.
There are, of course, even deeper answers… But this is just to illustrate how offensively limited a GAP certification is. It is no criterion to choose someone who really knows what they are doing, over someone with a vague idea.
Yet More
Well, time to cut this article before it becomes much longer. I intend to cover another half dozen or so problematic questions and answers in the next part.
By now, you must have picked up the idea that there’s around 20 or so questions with dubious answers.
That’s interesting, because there is a pass mark you must achieve to get the certificate… And it falls perilously close to the count of the questions you must answer correctly without encountering one of these dodgy questions. In other words, almost everyone with a GAP and anyone with a high score has intentionally or unintentionally answered some questions with the wrong answer - wrong in terms of the value to their client.
Summary
Taking the GAP is a pretty good start - I wouldn’t trust someone that couldn’t take the GAP and score at least 60%. But I wouldn’t put much trust in them, because to score much more than 80% means that they were answering questions as correct that had no good answer. Not that scoring as low as 60% would mean that this was a more qualified candidate than one that scored 80%. The one that scored 80% may know enough to present the answer that Google wants, as well as knowing the proper answer.
A test that fails to differentiate between the standard user and the outstanding is of limited value to clients when selecting a paid search professional to work with.
Google must provide better questions in this test. Relying on candidates to knowingly answer badly phrased questions and poorly selected answers is not very confidence building. Including questions that offer marketing inspired puffery (the “SmartPricing” nonsense described above) is sheerly offensive use of a manufacturer sponsored test, and puts Google in a very poor light.
Actually, I’m sufficiently annoyed that I might put together my own test, or be willing to help someone else to develop an effective certification test. If you are interested in developing a test and certificate that separates the beginner from the expert, and that tests knowledge of the system as it is now rather than as it was a few years ago, drop me a line or append a comment! :)
