Spam in Comments, Unattributed Content

This article was triggered by a possibly spammy comment, and lead to investigating the quality of the site that was offered in the link. Bizarrely, it leads to further reflections on Google, and the Content Network. I struggle to make the Content Network work for my clients, at a reasonable ROI, and Google itself turns out to be part of the problem.

I use Akismet for this blog, and it traps most of the spam, most of the time. Occassionally it traps a real comment but the most frequent problem it presents is a comment that is unique, but uninformative. Was the comment made just to gain rank for a less known blog? One of the clues that I use is the age of the posting. By and large, the older the posting, the more meaningful the comment has to be. So for a brand new post, I might, barely, let a “great article, you are always interesting” through. When the article is six months old, you need a substantive comment with some real content or a proper question.

Another clue that I use, but that is annoyingly hard to extract from FeedBurner and Akismet, is what brought that commenter to the site. I’ve just found a comment that fell into my “it’s an old article” and “uninformative comment” category - so I wanted to see if it was just linkspam. Looking at the webserver logfiles shows that the search this user made was ‘[SEO] “powered by wordpress” “leave a comment” “no comment”‘. So, it’s spam. Looking at the site was another clue.

Why do I care?

Well, neighbourhood, and signal to noise ratios. I really don’t like linking to Get Rich Quick sites - mostly splogs. Most of them are, IMO, a way for the owner to get rich and not for the visitors to get rich. I hope that I’m a responsible marketeer - so I want to lose the scammers and make them work harder for their money.

The other issue is signal to noise ratios. Blogs that repeat the same tired information just annoy me. Don’t they annoy you?

I was looking for a clients’ problem and the search showed the same original article information popping up in blog after blog. Very few of the repeats gave attribution, and none of those that I read expanded on the information in the original blog. This isn’t such a good example, but the original gave away too much information about my clients’ problem… but check out a search for “akismet-admin workaround“. See how the same info is repeated, sometimes without attribution, but exactly the same edit is proposed? Notice how Google suppresses duplicate information - it is indexed and ranked, but not shown unless you click the “Repeat the search with the omitted results included”.

I don’t really know how to solve the problem of other sites duplicating unattributed articles, at *this* end. That is, as a single blog, denying links to sites that exist mostly for self-promotion, any action is pretty ineffective. However, Google could apply harder filters. Even in the weak example search that I cite, blog entries like this image, are plentiful:

This is pretty typical of sites where I wouldn’t want my client adverts shown, and that offer little value to users (because the information is already available, better presented, part of an active community). Note just how irrelevant that advert is to the search and the article content. It’s still an impression, though. If you clicked on something that irrelevant, imagine how interested you are likely to be in the product… I suggest (and conversion rates support this) that you have no immediate interest - nothing that fits in a 30 day cookie window, at least.

The problem for Google is that sites like these generate revenue. While Google has been strict with preventing AdWords advertisers from affecting search results - there really does seem to be a strong wall that prevents AdWords from affecting organic results - Google probably should recognise that part of the reason for duplicating information is to gain rank, so that visitors will be exposed to Google’s own advertising system. AdWords isn’t directly affecting search results - but the publishing system is.

Conclusions

Personally, I’m going to continue rejecting lightweight comments that lead to splogs or splog-like sites. This is more work than I’d like, but until de-spamming tools can recognise and mark splogs in links, I’ll do it by hand. This slightly increases the value of this blog to real readers.

Since the point of splogs is to take popular (or possibly popular) articles and republish them, in the hopes that the search engines will reward them, it looks as though the main cause for their rise is search engine ranking systems. And that’ll be the reason for them to disappear again. So the main activity for splog reduction, which will in turn have an effect in reducing comments on real blogs that are merely part of a SEO strategy for sploggers, will have to come from the SE’s.

I will continue to eject splogs from my clients’ Content Networks, by using site exclusion. Conversion from splogs is the lowest that I have measured. (Yes, I have now seen a conversion from a domain park).

Google could reduce the frequency of duplicate content entries in listing by a filter that requires a proper reference to a site with similar content. This would make the “lowest possible effort” Get Rich Quick site builders to have to do some work. Since the original author should be the earliest article, this should end up that *mostly* the reward for duplicating articles will be removed. Will this be hard to do? Yes. But that’s what happens when you rely on a citation model for pagerank - Google have made the bed…

Google should demote the rank of blogs that promote AdSense content *visually* before the article. I’ve yet to come across a blog that is valuable to users, that devotes the top 600 pixels to advertising. If you can find me some counter-examples, I’d be delighted to see them! :) This will increase CTR in the Content Network. Evidence - none. But I do have experience of working with large clients and building websites; when people linger on a site rather than bouncing in a few seconds, there’s a higher chance that they’ll be engaged and will click on something else important to them.

I’d like to say that one of the clues to a good blog is whether the articles have substantive comments (not just pingbacks) - but if scammers knew this was important, then it’d be enough to generate loads of comments (automata for this would be *easy* to write and hard to detect). And “substantive” is terrifically difficult for software to detect.

Controlling the quality of content match sites will have a beneficial effect for Google, eventually. Right now, they are a junkie. They need a fix of advertising and Wall Street expects continual growth. However, adding new publishers who simply duplicate content, often without attribution, simply makes crawling the web harder and fills organic search results with useless listings. If Google is strict about reducing spammy publishers, then search results in Google will improve, and revenues will at least stay in the same range… and it is likely that overall CTR in the Content Network will rise as users spend more time on useful sites.

Perhaps Akismet could be improved for comment spam decision making if it included the first-seen referrer_info for the commenter. This would make it easier to work out why a commenter is submitting comments - I’ve no problem with marketeers promoting their sites here, if they make a substantive comment, based on solving real problems. I just don’t like degenerate freeloaders. Call me biased.

Update

2008-02-08: An older article has an interesting and recent comment by John Nagle, about a system to reduce exposure to spammy sites. Personally, I don’t think it will work for most advertisers. IME, the sheer quantity of low volume sites means that such a system will expend a lot of effort to deny sites that wouldn’t re-appear anyway. Put it like this… In a few tens of minutes I can sign up to new free domain host, and be publishing a spammy site, with an aged domain. However, getting traffic from each advertiser, sufficient to trigger a rejection, is a slow process - the site will make money. I believe that while making new, ranked, spam sites is faster than detecting and removing them (for all advertisers - not just the one) the problem will continue to drag down all content advertising.