The standard problems are somewhat less illuminating – banned sites form the majority of these, and the problems are usually associated with link spamming. Our experience of banned sites is usually a call from the new buyer of an established domain, who suddenly finds his site has been banned, as the previous owner boosted the site’s rank prior to sale with mass purchases of undeclared paid backlinks. Dealing with this is usually a matter of removing undeclared backlinks where possible and reworking the site’s paths to avoid URLs referenced by the spam, while designing and developing white hat link building strategies to avoid future problems. That resolution can be more complicated if the site has a mix of high quality backlinks and spammy backlinks to the same page…

The major class of problems that delivers serious insights are the things that web server developers do, and that CMS designers do. Microsoft’s IIS has been an especially rich source of insight. Previous articles on this blog have looked at ranking problems with case folding (case insensitive path names) and with IIS’s cookieless mode user tracking. In both these cases, resolving the problems correctly showed Google’s characteristic times to correct the index after crawling.

More recently, we’ve been getting stuck into some problems with the ways that Content Management Systems handle paths and page contents. Again, much like the IIS technical exercises, these page-template and CMS-specific issues create signatures on the site, that when corrected, tend to be corrected across the entire site at the same time. With a reasonably popular site, it is then possible to predict Googlebot’s significant re-crawl of content, and the delay between that re-crawl and the increase ranking of existing content.

Of course, re-ranking pages allows another set of tests that one couldn’t normally do on a user site. Clients are usually focused on getting back into the listings. With suitable corrections, the site can not only get back, but start to rank for keywords that were previously untouched. This exposes a new audience segment to the sites’ content. If the content is not addressed during the re-ranking exercise, then we sometimes get to see what happens when poor page content is exposed to the firehose of a major volume keyword – allowing the investigation of page content on user behaviour and the subsequent inferred response by Google when the content is judged inadequate by user behaviour.

Finally, one can occasionally find an example of a single off-site link that materially changes how the site ranks. This is usually discovered on small business sites, where some combination of offline connections or a lucky piece of PR manages to snag a high quality backlink to an inner page, with keyword loaded anchor text. The business owner may be puzzled by why one page ranks highly on a specific keyword, and the rest of the site is barely present. Identifying the link, or handful of links, involved and showing the business owner how much effect a high quality backlink can have, is usually an enjoyable experience.

These are valid ways to conduct scientific testing of ranking factors. Although it is much easier, and more predictable and repeatable to conduct tests on low search-volume keywords, Google’s behaviour is necessarily different on low volume and high volume searches. Much of SEO lore depends on testing on low search volume keywords. This gives undue weight to factors that are insignificant, or of low impact, on higher volume searches. It’s only technical failures on high volume sites that can yield the discovery of the importance of ranking factors for competitive terms. And since you can’t, in good conscience, go to the owner of a high ranking site and ask to sacrifice traffic for the sake of an experiment, technical failures on these sites tend to be the best way to collect the data – albeit sporadically.

There is an entire sub-set of scientific method associated with looking at rare examples of events that illuminate topics that are very difficult to experiment on. For example, almost anything involving a rare species of organism is non-destructive and observational, only. Much of astronomy depends on a rare event being observed, such as the impact of Shoemaker-Levy 9 on Jupiter. Most science classes teach the basic methods of science, which are easiest to demonstrate in class with repeatable experiments. However, this overemphasises the importance of reproducibility – observation of rare events is still a crucial scientific technique, albeit harder to properly explore in a science lesson in school. This is probably, also, one of the reasons that evolution is regarded with suspicion – the main investigative technique for evolution, as for astronomy, is observation, not reproducible test; when your understanding of science is driven from being able to reproduce tests, then observational science is downrated in importance and treated with suspicion.

In the SEO world, a reliance on reproducible testing will, I think, tend to overemphasise techniques that work for low search volumes, at the expense of optimisation techniques that are more effective for highly competitive and higher search volume keywords.

AdWords Is An Auction

Basically, AdWords is an auction for space, on each keyword. The more money your advert returns to Google, the more highly the advert will be ranked. If your advert is sufficiently disappointing to users, or misleading, or violates one of a bunch of rules, no amount of money will keep the advert in play. What you pay is determined by the value of the advert below you. If your competitors aren’t willing to pay a lot, you don’t pay a lot. If your advert is outrageously good, then you may end up paying less per click to beat your competitors (you’re renting the space, not each advert).

One of the basic rules of auctions, especially an auction where your payment is determined by the bidders below you, is that the more bidders there are, the higher the value of the auction to the auctioneer. So, Broad Match, the default choice for AdWords, allows Google to recruit as many advertisers to the auction as will reasonably be satisfied. And Google effectively runs an auction for each space on the screen – so increasing the pool of bidders has a large impact on the whole page of results for a keyword.

Another key element for Google and advertisers is satisfaction. If advertisers get dragged into auctions for searches that they don’t want their advert to be shown on, they’ll get some clicks, and those’ll probably be low quality clicks – poor conversions. On the other hand, users mis-key, so a certain amount of Broad Matching activity is useful – it helps you find miskeyed stuff and search queries that you hadn’t considered. But, overall, if the quality of Broad Match is bad, then you’re likely to bid lower on Broad Match, or switch to an Exact Match strategy. So there’s a pressure on Google to keep Broad Match to some reasonable matches, rather than just anything, all the time (that doesn’t mean that a few wild excursions are impossible, just less likely).

A Beautiful Mind versus Broad Match

AdWords can control the use of Exact Match to evade using Broad Match, to some extent, by telling you that search volumes are too small to match your keyword. That’s kind of true – there is a computational cost to managing a huge number of exact matches with a low likelihood of being searched for. So there’s a way to make sure that some quantity of Broad Match stays in play, without everyone heading for Exact Match. The ratio is probably determined using something derived from the Nash Equilibrium – you’ve seen that film “A Beautiful Mind”? That’s the John Nash I’m talking about.

OK, so what’s match types and search queries and auctions got to do with the Google accreditation and Agencies? I think they are intimately bound together, but through an unexpected connection… small advertisers.

Small Advertisers – Welcome To Frustration City

Small advertisers tend to have low budgets. They can’t take part in all auctions for all relevant keywords. The smartest ones use exact match, and geotargeting and the other tools – but you need to have invested time and effort to learn those, which often has a significant cost for small businesses. There’s a lot of users that just don’t understand AdWords, and don’t have the time to invest in learning.

Looking at examples I’ve handled in the AdWords Help Forum… They bid globally for a business with a 20 mile service radius. Their adverts are rubbish – describing their place of business or their name, but not what they do. Their landing pages fail to have a call to action. These advertisers tend to have a bad time with AdWords. They may abandon accounts and start again – causing Google to suspend the account because it looks like a scammer. These small advertisers complain, in numbers. They do so on the AdWords Help Forum, in their thousands.

Abandoned AdWords Users

So far, Google hasn’t really addressed the needs for these users. There was an attempt to create a Starter Edition, but as a cure, it was worse than the disease, and pushed users in directions that were less likely to lead to success, not more. The Starter Edition is gone. It didn’t solve the problem. But there’s still lots of dissatisfied users, asking for help in public.

Marketeers will recognise a classic problem here. Small advertisers, low budgets, can’t afford the services of a big agency. Larger agencies like to have clients who spend upwards of $1000/month – it’s just not cost effective for either party (agency or client) to really consider the $100/month budget client. But these smaller advertisers are present in vast numbers.

AdWords Auctions and the Small Business

What do we know about the auction? The yield of the auction depends on the count of bidders, to a substantial degree. If we can find a way to prevent small advertisers from defecting, then their low value contribution should yield a higher value contribution from larger advertisers. By taking part in a fraction of the auctions (to fit the budget) this mass of small advertisers pushes up the Average Cost Per Click. It’s in Google’s interests to make sure these small advertisers participate, and get some value.

But they’re expensive to create, these small accounts, and low yield for an agency. The Google accredited AdWords consultants and agencies out there, well, they could service these guys, but one and two person agencies tend not be supported by software – effective AdWords management software is pretty expensive. So the very small advertiser tends to be unaddressed, and tends to be dissatisfied – but Google needs them, in order to maximise returns from the largest advertisers.

AdWords – Fixing The Small Advertiser Problem

We can square this circle and resolve the conundrum, I think. What about if Google creates a support team, in a low labour cost country? You pay a little more to have the account set up, and Google uses its’ massive automation to provide a moderately successful AdWords account (statistically, some will be rubbish and some will be awesome, but on the whole, they will barely return a profit for Google). This mass of small advertisers will create a pressure on the auction, pushing up the major advertisers a few cents – on billions of clicks. That’s where the value lies.

By providing more support for small advertisers, and reducing defection, Google makes only a small amount of profit. Google can point to this near-philanthropic effort and everyone will feel good about the multi-billion behemoth doing good. But it’s more clever than that. The support for the small advertisers creates an uplift in the auction, delivering millions of dollars to billions of dollars per year, from larger businesses.

AdWords Agency Impact

But if there’s a low cost way to get your account set up and managed, what happens to the small AdWords agency? Gradually deprived of small businesses with problems to solve, these guys will be pushed out of the market – squeezed between Google below, and larger agencies above.

The new Google Marketplace appears optimised to allow the largest agencies to appear highly ranked – so there’s a force for consolidation amongst small and large agencies.

And that would solve another problem. The deceptive agencies that phone up and say they have a special arrangement with Google that lets them place companies on page 1 of search results. The staff who make these calls have no idea whether they are selling SEO or PPC – they just know there’s a special relationship and you can buy your way to the top.

That’s a brand problem from Google. It causes a number of small businesses to believe that Google’s organic results are affected by payments – which is neither obviously true (indirectly it is true – but the money paid doesn’t go to Google) nor helpful (Google benefits from appearing to be uninfluenced by third party money – a fair and impartial judge). By and large, these less-than-entirely-ethical agencies will also be squeezed. That helps Google, and in turn, encourages more advertisers and discourages spend on SEO link building (much of which is wasted after about six weeks, anyway – and the cross contamination means that on informational searches, where there is no commercial intent, Google can start to deliver better informational and less commercial content).

When will this consolidation happen? I have no idea. I’m not well plugged in to Google AdWords insiders these days. If I were, I probably couldn’t write about this, as it’d probably be part of an NDA. But I used to be well connected, and I can guess from observing recruitment (I’ve seen Google advertising for people with knowledge of the Nash Equilibrium, for example), and from thinking about auctions, and marketing… I could be wrong, of course. I’m not always right, but then so few people are. :)

AdWords Agency Consolidation

I’ll guess it’ll happen in the next year. Recessions are a good time to think about and come up with new services to extract more money. We’ve already seen experiments in some countries where Google offered to set up small accounts – in the UK, for example.

It takes a large business, like Google, time to take on and train the staff, once they’ve done the test marketing… So, I’d expect something within the next eighteen months. That’s about enough time to hire an army of third world workers, train them, and develop the supporting websites and collateral, including a physical mail, TV and phone campaign… If you aren’t already planning how to grow or merge your small AdWords agency, this may be a good time to put that consideration back into play.

This would also be a strong play to counter the consolidation of Bing and Yahoo search results, due to come on the agenda, shortly…

That plant has about 300,000 employees. What’s the UK’s suicide rate? That UK government statistics site has the rate at about 12.5 per 100,000 – that’s not “suicide attempts”, that’s “suicides”. It doesn’t actually say “12.5″ in that nugget – it gives a headline rate for males and females – and women are less likely to die by suicide, by a lot. I’ve averaged the male rate and the female rate and waved my hands a bit, to get to 12.5 per hundred thousand.

Male suicide rates are about 3 to 4 times higher in the UK

So that Chinese factory, at 9 suicides for 300,000 staff, is actually running at around 1/4th the suicide rate in the UK… Or is it?

The government charts are normalised for the age; the chart represents how many die, over a lifetime, from suicide. We’re taking a small segment of people’s whole life, working for this business. So is it fair to equate a lifetime rate, with a few years history of that business?

Well, the UK stats show that here, the suicide rates are pretty much independent of age, and more related to the sex of the people; men are much more likely to die of suicide (it’s not clear from these stats whether equal numbers of men and women attempt suicide, and men are just better at it, or if fewer women attempt suicide – there are other sources for that question).

So, the rate of 12 per hundred thousand is roughly the same, for each age group when averaging the male/female rates, over the two and a bit decades of this report. That means we’d expect to see a total of about 30 to 40 suicides amongst 300,000 Chinese workers in the last year, if they had UK rates of suicide. If that inference is correct, this factory is doing something that we in the West should learn – they’ve crushed the suicide rate hugely, to about 25% of what should be expected, if it was in the West and the factory was entirely average, with a 50/50 male/female mix.

But of course, the Chinese are demons. We don’t like their human rights abuses, sweatshops and competitive rates, their intolerant politics and their manipulative economy, or the way they cut free access to western websites – and undercut Western company’s prices. So a story about how horrible they are, plays to the image we have. It’s easier to place a story that plays to preconceptions than a story that says “Shock probe into UK working practices suggests we should learn from the Chinese” – and I think you’d find few people in the West advocating 30p/hour and 12-15 hour days as a way to decrease suicide rates (hmm – make employees so busy and tired they don’t have the time or energy to commit suicide?)

Some western media are getting rational about this, and playing down the numbers – like The Times Online, which has gone one better than this article, and looked at Chinese suicide rates – and even points out the difference between urban and rural rates.

But the *low* rate of suicides at the factory rarely makes a play… maybe they have a lot more female workers than male? That’d probably account for the rate.

The other interesting factor in the stories is the emphasis on the perceived increasing rate of suicides. Is that a real increase in rate, though, or a statistical fluke? I don’t have enough of the data to begin to make a start, but infrequent events can cluster – epidemiologists have a hard time with data that points to clusters of diseases, when there is no real relationship. This partially stems from the astonishing and pioneering work in the UK in 1854, where John Snow linked Cholera to the use of particular water sources and cluster analysis showed that infection was related to the water source. Cluster analysis is an important tool – but later work has found misleading clusters, needing careful statistics work to find out what actually goes on, rather than discovering flukes.

Is there a real increase in suicide rate at the factory? Hard to say on the evidence given. It could just be a fluke – after all, you can toss a coin and get 8 heads – it’s exactly as likely as any other specific sequence (8 tails, or four heads then four tails, etc). The trick is to demonstrate that there is a difference from the rates you’d expect. And on the face of it, the only evidence is that this factory has a lower rate than expected.

Conclusions

Either some journalists are irrational and just looking to fill column inches with poorly understood and badly misrepresentative stories, or I’ve misunderstood the stats. I’ll be thinking about the stats a bit more. Maybe I’ve interpreted them badly – statistics are easy to misunderstand.

Well done The Times. Better balance and insight. That’s what we’re after. Naughty Independent – muck raking and subjecting readers to poorly evaluated reporting from organisations with an agenda? I don’t really know what’s really happening, but I’m not convinced by a lot of the coverage, and less convinced by The Independent than The Times.

Why hasn’t Apple played the card that says how their work requirements mean that the companies they work with have lower suicide rates? Why isn’t Dell on the firing line to the same extent for their use of this factory, or is this a way to knock Apple and try to dent sales and taint the iPad on the eve of the UK launch? Interesting, isn’t it, to think about the agenda that the media may be playing, whether consciously or driven by other players in the game.

Notice that recent articles are sometimes shown twice or more, with different URLs

Why are the articles shown twice or more? Because they have different URLs. They are the same page content, but on different paths. Duplicates. Notice that Matt’s blog is not penalised for these duplicates – because they aren’t what Google considers to be duplicate content. They are the same content, reached by different paths, and in this case the different paths are tracking parameters for Google Analytics. It’d be more than a little annoying if Google handled “duplicate” pages, caused by using their own tracking parameters, as if they were different resources!

Why aren’t other articles on the blog shown twice or more times? What magic makes it that two day old articles get three showings, but articles five days old and older show just once?

Interesting, isn’t it?

Now, instead of sorting by date order, sort by relevance. Let’s look for those duplications again:

The duplicate pages, with different URLs appear lowest for relevance.

So these duplicate pages of the last few days, appear as the lowest relevance pages in the last year of articles on Matt’s site. Yet they have the same text as the main article… or do they? Might there be something magic that the cache shows us?

The cached versions show us that they were taken up to a week previously – and reveal that these were found from Feedburner. Google is taking different URLs, determined by the tracking tags, and including them in results. The weight for the tagged pages is lower – Matt’s blog doesn’t refer to these URLs, and nor does anything else that Googlebot will crawl. So these additional pages rank lower, despite having the same content – the difference lies in the backlinks, and perhaps other factors.

But if there are no external backlinks to these articles, what makes them appear in results, at all, in any position? This, I think, is where we drag in another factor that seems to apply, especially to blogs. Trust. If a blog is well trusted by Google, then articles posted will tend to rank highly in search results, in as little as a few minutes for highly relevant searches (relevant, that is, to what Google thinks the article is about, which mostly means the title). Amongst the two hundred-or-so factors that Google is looking at, is whether they trust your results are likely to satisfy searchers. If your articles have enough hits from satisfied (long reading) users, you leap up the rankings within minutes of posting. Less trusted, lower weight postings won’t appear on page one, if they appear at all. That trust is extended to articles with no backlinks – these pages appear in results because they’re coming from a trusted site.

Why do these duplicate pages disappear, and when, and what does that tell us?

At some point in the last week, Google has removed the extra results for other page references with additional parameters. If we look again next week, we’ll see that the extra results for the current recent articles have also disappeared, but if Matt makes more posts, we may see those articles with extra tracking parameters for a brief period. Why do these “duplicate” pages disappear?

Well, one reason is that Google detests spam. A bad page of search results would be a page that contained different sites with the same article – because that wouldn’t reflect the diversity of opinion and solution on the web. Google would prefer to have ten different answers to the search query, than one answer on ten highly ranked sites. There’s a conscious effort at Google to compare articles.

How does Google identify that? Well, we can take some guesses by looking at the search results. Notice that bit that says “cached”? Taking several shots of a page (cached pages), over time, lets Google see that page content is evolving. User generated content accretes to a page – and that’s visible in successive cached snaphots.

Google can therefore see which parts of a page are static and which are likely to be UGC, even without any effort to understand the page structure. Matts’ articles get a lot of commentary, so it’d be pretty easy for Google to determine that a common core of the page is unchanging – and identical. How easy? Well, there have been tools to perform that kind of textual analysis with computers since at least the 1970’s to my certain knowledge – I used them back then. More recent techniques have used data compression techniques to compare compression rates of samples, using the highly optimised algorithms for good data compression – and Google has smart people who probably do more complex stuff than that.

Canonical Link References

One more datum to collect! Canonical Link References – does Matt’s blog use the canonical link ref to make sure that Google knows the best URL for this article?

The answer is very much affirmative – Matt so much likes the canonical link reference, it is added twice in the header! Hmm – some problem with conflicting plugins, perhaps? I don’t think this duplication of the canonical link reference is intentional. Fortunately they are the same, or we’d get into some discussions about whether Google believes the first or second canonical link reference!

What *should* the canonical link reference do for Google, when we see the variant forms with tracking tags? It should tell Google that the preferred form is the one without the tracking tags. We should end up with just the preferred form showing in search results. And that’s what we see – but the canonical link reference isn’t the only way that Google looks for probably duplicated data. You can tell, if you look for other blogs that use FeedBurner, but that don’t use canonical link references. Those blogs still get deduplicated listings – showing that other components to deduplicate are still working, but still take days to do so.

What does the transitory presence of these pages tell us?

Thinking solely about what Matt’s results are showing us – not taking any evidence from other experiments and tests into account…

I think the presence of these pages in the search results says that Google is reading FeedBurner feeds for Matt’s blog, and getting tagged data, tagged for Google Analytics. I’m guessing that the tags are not set as ignored in Matt’s Webmaster Console, so Google sees the tagged pages as different. Because users are not linking to them, and the blog itself doesn’t refer to them, the usage eventually dies out – where “eventually” means less than a week. And the canonical link reference probably also helps.

Note that the alternate pages *are* present. Not highly ranked, but present. So… the content is the same, the server is the same, but the links to these pages are different; the untagged page is referenced within the blog (by any category or blog tag and the archive), and after a day or so, there are probably some links to these articles, all probably pointing to the untagged page. So that tells us that backlinks are important – or there’d be no reason why these tagged pages shouldn’t rank as highly – the user experience is likely to be the same, the content was (at the time of first snapshot/cache) the same. So if backlinks within Matt’s blog weren’t important, then the pages should rank with equal weight… and they clearly don’t.

This observation also says something important to us – if we watch our web server log files carefully! You may have read some of my other articles here about web server log file analysis for search engine optimisation – I think the web server log files tell us important things about how Google perceives our sites. If we see Googlebot requesting a tagged resource, then that tells us something about what resources we have out there, our state of canonicalisation, and Google’s speed of change. We want to know the speed of change, because when we’re trying to improve performance, we’ll want to see when we might expect to see results, at earliest.

We don’t, unfortunately, have access to Matt’s web server log files… but you have access to yours… What do they tell you, under similar conditions? I may return to this topic, as I think the research is pretty interesting for what it tells us about how Google goes about making decisions.

Summary

Google sees the same blog article under different URLs over time. Google collapses the references to a single page URL – probably using intrinsic information (page content) and supplied information (canonical link references), and backlinks, and the Webmaster Tools mechanism to instruct Google to ignore certain parameters.

URLs for alternate page presentations get into the results for trusted sites, even if these URLs have no weighty backlinks. Not highly ranked, but they do appear. They appear quickly and disappear within a few days. This suggests that whatever it is at Google that evaluates, takes a few days to do so.

I don’t think this points to content being the key, but to content with backlinks and user-preferred results, as being the most important. And the presence of these duplicates is a hint that a site is trusted. Not a great hint, as if you don’t use FeedBurner or another similar tagging resource, you won’t see this effect.

I’ve not considered the backlinks to Matts blog in detail… there are a lot and the signal is messy. I’ve left that analysis out of this article. Besides, discovering the timeline of backlinks is itself a pretty tricky exercise, unless you, as the experimenter, control them; that’s definitely not the case for Matt’s blog!

What’s especially interesting is that an article, without significant backlinks to it yet, can appear high in search results, but that this tends to be for the canonical representation of the page, even though Google is probably learning of the page via FeedBurner/pingomatic notification (IOW, likely to be tagged). So the first mechanism that notifies Google of the URL, is probably *NOT* the canonical form – yet the canonical form is listed highest, within minutes. That suggests some fast processes at Google for identifying and evaluating a page’s relevance, and then some slower processes to determine whether the page can be justified for continuing presence.

And that behaviour of high ranking without external justification, in turn, has some implications for the weight that will flow from a blog article… It is initially likely to be low, and if the article “sticks” in the results because it helps users, then it gains some kind of value. Otherwise, the weight of the page will remain low – though, strictly, understanding that evolution of the weight of the article requires looking at the impact of links from articles in a blog. Another day, perhaps ;)

I hope you found this close look at search results amusing, if not educational.

I’ve not considered FaceBook advertising; it’s sufficiently similar to standard demographic and content targeted advertising that I don’t think this type of review would benefit from considering it. Doesn’t mean that I won’t consider writing about it!

I’ve also not considered how the organisations *could* engage with audiences – this is a review of how they *are or are not* engaging with the FaceBook audience, and overlaps with reputation management.

We looked at Amazon, John Lewis (for American readers, John Lewis is an employee-owned department store that offers the price promise “never knowingly undersold”), Dixons (a largely European group of high street stores), Tesco (major supermarket, with increasing global presence, with online store and electronics goods sold in store) and Comet (another high street electronic goods retailer). I should point out that we have a client (Car Phone Warehouse) in this space, but this work was not done for them. I have omitted them from results, however.

FaceBook

As one of the first generation of large scale social media platforms, I’m pretty sure that FaceBook in its current form will come to be regarded as primitive. I’m not going to focus on FaceBook itself, but on the ways that current businesses are using what’s present, and the likely near-term future – the next six to twelve months. Barring, of course, some emergent and disruptive event!

Something that I’m becoming aware of is that I increasingly find people, outside the business sphere, who do not use email. They message almost entirely within FaceBook and mobile phones. Emails sent to this group are unread, unless they are sent a reminder in a format they do use – text or phone! They are largely younger – teen’s and 20’s – and often have no personal use of a computer in the workplace. Nonetheless these are buyers, potential customers. Ignoring their preferred communication methods should be a mistake that needs to be addressed – if not now, then soon, as that audience seems set to increase.

Amazon and FaceBook

Amazon had no FaceBook fan page

Well, that was easy… No fan page. Why not? Guesses, of course, but Amazon is reasonably well represented in shopping presence on FaceBook through its’ affiliate marketers, and on-site customer feedback is a legendary strength of the brand. User reviews have been a central part of Amazon’s offering and early differentiation, and they have had a pretty relentless drive for customer satisfaction matched with on-site feedback, and post-delivery feedback.

For example, I fairly recently bought a £2.50 HDMI connector from Amazon, and have been sent two emails soliciting feedback on the purchase experience. The email solicits a response, just as with the purchase experience, on the Amazon site. This is a characteristic model that dominates current approaches; the expectation that the user will go to the web site to transact and interact. I expect that this pattern will change for many businesses, especially smaller ones, over the next few years.

My expectation is that the increasing use of social media by internet users will see a substantial shift in the way in which people purchase. Having become familiar with interfaces like FaceBook, and spending a large fraction of their time there, users will tend to find other user interface designs less familiar and more awkward. I’m expecting that purchasing through FaceBook applications will start to feel more natural, and that FaceBook will be a place where increasingly large numbers of users will look for purchase, feedback and service.

Amazon are no fools, and I’m expecting that we’ll see more attention focused on presenting offers and purchasing within FaceBook – and that the customer service experience may need to be presented on FaceBook, too. That’ll mean an official Fan page, I suspect, will appear, when Amazon has worked out how to make it count, and reconciled themselves to a reduced footfall on the Amazon site. That’ll be interesting, and I suspect will need significant messaging to industry analysts to warn that sales will continue despite falling in-store presence. In other words, I don’t expect a fast transition for Amazon, because they’ve done such a fantastic job using their own site, and there will be significant resistance to changing a working model.

FaceBook and John Lewis

Tolerably rich mix of interaction, generally positive.

John Lewis, with physical store presence, can and probably should have several FaceBook fan pages, because the online presence is substantially different in nature from that of Amazon. The relationship with the customer is also substantially different – many customers will have experienced in-store customer service and the well trained staff, who have a vested interest in seeing satisfied customers. I must admit that when I started this review, I was expecting John Lewis to make a strong showing, and probably show the best customer experience for a “clicks and mortar” store, possibly even beating Amazon.

The rich commentary in this FaceBook snapshot conveys positive customer interaction well, I think. The messages largely show happy fans, and an in-store promotion. The volume of traffic isn’t large, though. That short screen snapshot represents about six to eight weeks of activity, and misses out any announcements about the store’s usual annual sale period and doesn’t specifically mention any further special events. This looks like early efforts to engage and to develop an understanding of what people want from a FaceBook fan page, and interaction with what appears to be a good brand relationship with customers.

Contrast this slow messaging with some of the hyperactive viral games on FaceBook and their persistent invitations to join someone’s farm or gangster family. John Lewis is taking a very restrained and low key approach to messaging – but I suspect that this isn’t a considered strategy, more a matter of accident as to who is involved and how.

The fan page itself isn’t especially well developed – brand image of the store logo, but a single location shown, and the image could be more supportive of the brand. There are other things that can be done to and for a fan page, and this is not an outstanding example of how to use the space that FaceBook offers.

Having a single fan page is probably not the right approach for this chain. John Lewis has specific stores in high footfall locations. Long distance travel to the stores is unlikely, so a national fan page is probably useful as a central point for the group, but individual stores probably need their own presence, with their own special in-store events and interactions, supporting the more personal shopping experience.

Dixons and FaceBook

When the customer controls the message and the brand fails to respond, you get this.

The contrast between Amazon, John Lewis and Dixons presence is fascinating. Amazon has focused on online customer service, and handles it on their site. John Lewis has a high reputation for service, and shows some customer interaction on FaceBook. Dixons appears to be ignoring their online presence on FaceBook. The short screen snapshot shows fairly frequent annoyed customer postings, with no corporate response. This doesn’t show the brand in a positive way. When angry customers control the content of the FaceBook page, you probably have a pretty serious online reputation problem. I’ll be looking for supporting signs of that elsewhere, in other online aspects in other articles, to indicate where this brand is failing to engage, and what damage that might be causing.

Comet, Tesco and FaceBook

Comet had no discernable presence on FaceBook – given the nascent level of understanding about how to use FaceBook to interact with customers, that’d probably understandable. The lack of an easily identifiable fan page means that the negative perceptions developed by Dixons, aren’t present – but neither is there any opportunity to create a positive impact.

Tesco’s fan page is branded, a handful of reviews and a strange selection of photos. Presence, some fandom, but not a managed reputation resource for the business.

Summary

None of those reviewed appears to have really taken FaceBook on as a way to interact with their client base. Amazon probably has some strategic decisions to take, and a lot of tactical considerations about how to handle emerging social media. John Lewis, with their usual customer service focus, is leading the clicks and mortar group, but is still not well focused on the large number of FaceBook users.

Fan pages are not well developed in this sector, if present. Customer service connections appear to be weak, and even the best presented clicks and mortar stores are not creating store-specific experiences.

While some promotions are presented, there appears to be little engagement with what is physically happening in stores, and with off-line or even on-line advertising. None of this group, in the sample period, appeared to be offering any kind of competition or game to attract attention.

National Insurance is paid in two parts – by the employee and by the business. The employee has a cap on their contribution. The business does not. So National Insurance increases naturally tend to hit higher paid employees overheads more. Want to save on your National Insurance costs? Don’t increase payments to your higher paid employees so much (see this “FactCheck” on the Chancellor candidates debate and the “wealth gap”)

The changes proposed by the Labour Party are to increase the NI rate for higher paid employees. The Conservative claim is that this increase in National Insurance will increase the reluctance of employers to hire more staff, in a recession. I’m sure that for very large businesses, there may be some impact – but if the impact is to reduce hiring of lower paid staff, that’s pretty perverse. It’s not unheard of for laws to have unintended consequences, and I’ll gladly concede that it is possible that a tax change clearly intended to target higher paid employees, might unexpectedly affect lower paid staff recruitment. I’ve never run or been a senior executive staff member in an organisation that employed more than 20 staff, so I don’t have any real experience of making decisions on staff hiring where 1% NI changes would be worth a whole employee – I’ll have to go with these large business leaders saying that it will affect the number of staff they hire.

The real deception though, appears to be how equivalent savings in Government spending are to be made. The analogy that seems to be being made is that, like many businesses, the easiest way to save is to reduce employment. Cut your staff and you cut costs, and it can be done pretty quickly. I understand, from what I’ve heard in the debates and previously, that the main plank for Conservative spending reduction, is to delay filling jobs in Government, mostly by slowing hiring of contractors from the private sector. If you defer hiring a staff member by a year, you save a year of spending in wages. Seems plausible, on the surface.

In a recession, few businesses will be carrying underemployed staff who would turn into Government contract staff. People who aren’t placed on contract, and have no immediate prospect of being placed, will be let go. They will become unemployed. Or, putting it the other way round, when the Government wants a private sector employee, the contracting company hires someone off the lists of the unemployed.

This means that Governments are not like businesses in several ways. One of the ways in which the UK government is different from a business, is that the UK government pays unemployment benefit to the unemployed. So, when a Government job is not filled immediately, the consequence is that instead of the tax funds going to an private sector employee, some of the taxes go to a different part of Government spending – unemployment.

If that’s the right chain of reasoning, when David Cameron says that he will save Government spending by deferring recruitment instead of raising a tax, he’s also sustaining Government spending on unemployment benefit. In other words, the “savings” won’t be at the level that has been predicted, and the impact will be to increase unemployment in the private sector from the organisations that provide Government contract staff, relative to the staffing that would have been in place without the savings. Increased “savings” from deferring contract placements, result in increased unemployment figures with an approximate one for one correspondence of Government posts unfilled versus unemployed people.

Let’s put some numbers to this. I’ll look at £1B savings in Government spending. Before the saving on hiring the Government spends £1B on those staff including National Insurance (employers part) and takes back from those employed staff, the employees NI and taxes. So about 10% (assuming that these staff are at the lower end of the pay scale) of the payments return to the government. £1B saving means that we lose £100M of government revenue. So we’ve actually only saved £900M net from our £1B staff saving.

Worse, all those staff are now candidates to remain on unemployment benefit, instead of taking those people off the unemployment register. How much do we fail to reduce the unemployment benefit? Making perhaps egregious assumptions, an employee might expect to pick up more as an employee than as someone unemployed – let’s say that the benefit of being employed, for a low paid worker, is to be about 20% to 25% better off from having a job, than being unemployed. So £1B in spending on employees, translates to about £750M in unemployment benefit we don’t have to pay. Or put it another way – saving £1B in spending by deferring Government spending, results in keeping unemployment payments of £750M, plus foregoing £100M of taxes.

I don’t see a huge saving from deferring payments on staff like this. I do see a significant impact on unemployment spend – it won’t decrease as quickly as if the “savings” hadn’t been implemented. I think the Conservative approach to cutting government spending isn’t going to be more than about 20% effective. The result is that in order to save £6B (the target mentioned in interviews) would actually mean cutting Government hiring by £30B. And that has a very large impact on the count of people in unemployment and the private sector businesses that hire staff to the Government.

In the last few years, the Labour Government claims to have reduced public sector spend by £15B, and our public sector spend monitor, the National Audit Office says that some of that saving is, hmm, insufficiently clear that it is a saving. But the Conservatives are going to cut £6B in actual spend – which means a £30B cut in Government employment, without significantly reducing services? I just don’t believe it.

Technically, I suppose that this criticism also applied to the Labour Party and the Lib Dems – I’ve quickly skimmed their manifestoes and can’t see specific savings techniques for large sums. Lots of little bits, and claims of being completely costed. I couldn’t see where the complete costings (with first order repercussions) were in either Labour manifesto or the Conservative manifesto or the Liberal. I haven’t read the Green Party or other Manifestos, yet. I’m biased – I won’t be reading the BNP or UKIP manifestos; I regard both of these parties as substantially irrational and unpleasant and I’ve got better ways to spend my time. The Monster Raving Loony Party is a whole other game.

I’m not averse to reducing Government spending. I run a small business, and any serious way to cut my tax bill (personal or business) is welcome. But I just don’t see how trying to save £6B in real money, is possible by deferring hiring, without deferring £30B of hiring. Tell me that saving £30B of hiring savings can be done purely by efficiency, in a few years of UK Government budgets and I think I’m going to have to call you damnably suspicious…

I’d rather have my small business pay 1% more on National Insurance for higher paid staff, than to see £30B increase in the unemployed – the impact on unemployment in large businesses through that 1% tax, can’t possibly have the same level of misery for the rise in the count of unemployed, can it?

So, where’s the failure in logic and my understanding of the economy? We can’t have prime ministerial candidates misleading the public like this, can we? I must have misunderstood how the savings work. I just can’t see how. Help!

Apart from the validation of Matt Cutts statements, there’s one result that appears blindingly obvious. Malformed URLs don’t pass anchor text – and by implication, weight. In the context of the example in the article, adding a space to a URL in the anchor, destroys the value. Googlebot changes spaces (which aren’t valid characters in a URL) into “%20″ symbols. In Dave Naylor’s article, that means that the Googlebot will do a DNS lookup for a domain that doesn’t and can’t exist – spaces are not allowed in domain names. If the URL in the anchor’s href had been a fully pathed URL, then a space would be added to the end and converted to a “%20″.

That full URL, with an appended “%20″ won’t be found on the site. It should appear, at some point, in web server log files as a 404 for a Googlebot visit. 404’s don’t pass weight. So why the surprise that a malformed URL would fail?

I think the real point, not cleanly spelled out in the article, is that web browsers don’t parse pages the way that search engine spiders parse pages. A browser will cope with the embedded space. That ability of a browser to infer the useful thing to do, doesn’t make the space into a valid character in URLs – not without being escaped, anyway. And the consequence of appending the space, will be that a web spider makes a request for a resource that will usually be 404′ed, unless the administrator has used Apache mod_speling or an equivalent typo-correction tool (which should yield a 301 redirect to the correct resource).

Attempting to infer the SEO value of browser interpreted behaviour, without understanding Googlebot behaviour, will create confusing and misleading problems.

I’ve fairly recently had discussions with a few web site managers who’d made what appears to me to be the most peculiar decision – to block Googlebot because of the traffic impact. This resonated with a previous short article that I’d posted, about a problem identified by a Google staffer who was running his own blog. He’d seen his blog dropped from search results and was looking for why that might be happening.

There’s certainly a potential problem – low bandwidth sites may suffer if Googlebot consumes the available bandwidth. But if you don’t have Googlebot crawling, then how are you going to appear, anyway?

You could use the Webmaster Tools to request that Google slows the crawl for your site. This should still result in having the crawling and indexing, and minimal damage to the traffic. But just disabling the crawl, by using robots.txt to block all crawling, or to block crawling of large sections of the site that should have user interaction, is probably a mistake.

There is also the legitimate concern that Googlebot’s visits might be draining server resources at peak traffic periods. That’s moderately difficult for non-technical site owners to work out. Google Analytics (and the other JavaScript page bug based web analytics packages, such as CoreMetrics, Omniture, Webtrends, etc) measure user visits, not Googlebot and other bot visits. Verifying that Googlebot isn’t interfering with and slowing down visitors, is pretty much impossible to understand without going to web server log file analysis.

Web Server Log File Analysis

I like web server log files. There’s things I can find out from them, in a few hours, that I simply can’t find from Google Analytics, CoreMetrics and Omniture. Look at this graph, for example. I’ve taken web server log files from a UK-targeted business, and extracted Google-inspired visits and Googlebot visits, by hour.

The graph shows that Googlebot is busiest when users are less present. That is, when Google can see visitors coming to the site, the crawl volume is reduced.

This pattern of making Googlebot most active when the site visits are least active, seems to be the most common pattern that I can see in clients’ web server logfiles. It makes a lot of sense for Google, too:

• Continuing visits by Googlebot allow them to check that the site is still working (preventing Google from delivering users to a 404′ed page)
• Site performance under load can be monitored (helping Googlebot tune crawling rates, and verifying that users are getting responses from the site, mostly)

Summary

Googlebot seems to be quite smart about when it visits sites. The more users that are being sent to a site in a given hour, the relatively lower rate that it crawls. So Googlebot should never get in the way of visitors, under normal conditions.

Simply disabling Googlebot looks like a weak way to go.

Following suggestions from the Google Webmaster Blog, if you have areas of the website that change at different speeds, you might want to validate multiple webmaster consoles for different sections of the site. That would allow setting different crawl rates. I’ve not tried this, yet… I don’t have a client for whom I want to restrict crawling speeds!