I logged into our banks internet banking facility. I wanted to make a payment on Friday, 5th February 2010. I went through the identification of the recipient, and the payment and changed the offered date from Monday 8th, to Friday 5th. It refused the payment request, saying that I must choose a date between Monday the 8th and some date in June.

So I picked the 6th (Saturday). Still no go. Then the 7th (Sunday). Still wouldn’t accept it. Finally, I selected the 8th February, a Monday. Bingo! Payment will be processed.

It then processes the payment and tells me that the payment will be taken from the account on Friday the 5th February and deposited in the target account later that day. Which was exactly what I wanted in the first place, and it kept telling me it wouldn’t do.

Imagine my annoyance at having, for ten to fifteen minutes been told I couldn’t do what I wanted, and then being told that despite offering a different date for the payment, it would be paid earlier than I’d actually requested? What if I *had* wanted to pay on the Monday, 8th Feb? Then the funds would have been out of my account on the 5th, and not the 8th, as requested.

I’m disturbed by my Banks inability to work out when they are prepared to act, and the misrepresentation of the dates of actual activity, versus requested. It makes me less confident that they’ll do as I request. and I’m now more likely to pick up the phone or drop into the Bank. Or to change banks.

Purchase Processes - Especially For Special Offers

Our client had a similar issue - there was a particular offer they made, which couldn’t be fulfilled through the order interface, until after another action had been completed. But this wasn’t expressed clearly to users. They wanted the offer, signed up for what they were told to buy, and then couldn’t see the mechanism to access the offer - until they’d completed another stage in the the checkout. There were technical reasons why this order of actions was important - but the user could have been given a text notice that this is what the system would do. It’s no good when only *your* staff know how the system works - it is the customer that needs to be told, and told clearly, or conversion rates online suffer and the customer service phone lines heat up.

The only diagnostic that this company identified was an increased dropout in the funnel and decrease in conversion rates - when they were expecting total sales to increase.

This is a classic example of “you have to understand your business the way that a customer sees it, not how you know it works”.

This article is part of a series of short meditations on user interaction and the extension into social marketing, following on from a look at reducing the barriers to interaction, and some of the pros and cons of allowing user generated content, in “SEO: Remember Relevance?“

Have a look:

Dentist Lake Worth
http:// www. dentallakeworth. com | randomizewee@gmail.com | 122.53.68.2

Symptoms of swine flu are similar to most influenza infections: fever (100F or greater), cough, nasal secretions, fatigue, and headache, with fatigue being reported in most infected individuals. Some patients also get nausea, vomiting, and diarrhea. In Mexico, many of the patients are young adults, which made some investigators speculate that a strong immune response may cause some collateral tissue damage. Some patients develop severe respiratory symptoms and need respiratory support (such as a ventilator to breathe for the patient). Patients can get pneumonia (bacterial secondary infection) if the viral infection persists, and some can develop seizures. Death often occurs from secondary bacterial infection of the lungs; appropriate antibiotics need to be used in these patients. The usual mortality (death) rate for typical influenza A is about 0.1%, while the 1918 “Spanish flu” epidemic had an estimated mortality rate ranging from 2%-20%. Swine flu in Mexico (as of April 2009) has had about 160 deaths and about 2,500 confirmed cases, which would correspond to a mortality rate of about 6%, but these initial data have been revised and the mortality rate currently in Mexico is estimated to be much lower. By June 2009, the virus had reached 74 different countries on every continent except Antarctica, and by September 2009, the virus had been reported in most countries in the world. Fortunately, the mortality rate as of October 2009 has been low but higher than for the conventional flu (average conventional flu mortality rate is about 36,000 per year; projected novel H1N1 flu mortality rate is 90,000 per year in the U.S. as determined by the president’s advisory committee).

From Google h1n1 (Swine Flu) tracking in the UK, 2010/02/05 at 6:49 AM

So, what makes this spam, rather than a valued contribution?

• It is not from a person, but from a targeted keyword
• The supplied URL is appropriate to the keyword
• The topic is appropriate to an element of the article, but not the meaning
• The author is in the Philippines - not itself a definite marker, but contributes to the likelihood - but is wildly unlikely to be the targeted dentist
• The submitter has identified the article based on search results, not on the content - that is, a previous search lead to finding the article, but the article was not read and understood and an appropriate response composed. Instead a well crafted and rare piece of text was composed and submitted - but was irrelevant to the thread
• It has nothing to do with the relatively rare topic of dentistry; barely discussed online. Whereas Swine Flu gets a lot of high interest coverage… if you were targeting spam, you’d want high ranking, frequently searched topics.

Google’s search results show the signs of this specific craftwork in many blogs, where the indications of spamminess have been overlooked. Some of the comments left by this spammer are even on-topic and thread-relevant.

You could consider this the dark side of creating frictionless interfaces. OTOH, amongst the more sophisticated SEO spammers, there’s software that signs up for places like PC World (creates fake email address registrants, completes the forms with fictitious data, submits the forms, reaps the emails and authenticates, and then adds spammy comments). Creating friction to exclude spammers can decrease interaction with real users - there is a balance of evil to strike; how much near spam will you let through, in order to make sure that you can engage with the right audience?

This is a moderately clever piece of spam - it evaded Akismet, for example. If I’d been writing about the US situation, then I might even have let the spam through, as a useful contribution to the thread.

Is it entirely evil, like some of the spam submitted here? No. But this is almost good enough to pass as a valid commentary in a social forum. If you spam, then that’s the kind of area you need to be in, to make it through to trusted forums. Serious, unique content that contributes to the dialogue. Good effort, but only an ‘F’ - you were busted on relevance!

This article is part of series of short meditations on social marketing and customer interaction, following on from yesterdays “Google. Foot. Gun. Shoot.” and the previous article about soft paywalls “PC World - Online Subscriptions”

• Complaints that “Google won’t accept my Credit Card“
• Making promo vouchers work
• Suspension for various reasons, often involving guaranteed ways to make money from home, as an affiliate

By and large these users have been cut loose from Google Customer Service. The users have been sent through an automated path to the forum, rather than given human customer service. The causes of this bulk of visitors are, I think, complex, but Google appears at least partly complicit in its’ own generation of clients that it would prefer it didn’t have.

Look at this chain of events for example…

I spotted a mention of a previous Merjis article on another blog. The entire article had been scraped, along with many other sites’ content, to help that blog rank well on searches involving making money from home.

The site was monetised with AdSense adverts for a well known affiliate recruiter. So, first question: knowing that this advertiser will be promoting affiliates, very few of whom have a satisfactory experience to judge from the AdWords Help Forum, why is the advertising permitted either by AdWords or by AdSense?

Clicking on the AdSense advert takes the user to the affiliate recruitment site. They’ve avoided the landing page popup problem, by offering a JavaScript in-page popup to collect email addresses and names. A clever way to avoid the old problems of the classic affiliate squeeze page being given a low landing page quality score.

The popup is the main lead generator - because it makes a free offer that you receive in email. So the offer is invisible to Google bots - they’d have to complete a form, which, by convention, bots aren’t supposed to do (form filling could take actions on sites, like deleting postings, or ordering stuff, etc). And after that, the bot would have to read the email to understand the offer… which is usually delivered on another website.

So you arrive via the email cutout on a third site (the first is the blog-scraper with AdSense but no direct ties to AdWords, the second has no direct AdSense or AdWords connection, and the third site is the one with the sales proposition). The third site reveals that you make money by… signing up for AdWords. And Google will give you a promotional voucher for signing up for hosting via a third party. So, for that matter, will Yahoo, for this most recent “promotion”.

The series of cutouts in this chain makes it harder to detect. But the user still perceives the offer as validated by the major search marketing programs. The organic ranking, the promo vouchers, the AdSense adverts all combine to put the Google brand into the frame for the affiliate recruiter. Not quite as good as endorsement, but imagine how much harder it would be to convince people, if the AdSense adverts and Google comarketing weren’t available?

So Google indirectly pays the affiliate recruiter for running scraper sites, subsidising their costs of advertising, then offers a promo voucher via third party arrangement, which validates the shabby offering, that will recruit people who turn up in the AdWords Help Forum, desperate to make money from home with no knowledge. The volume of people turning up with problems is too high for Google to bother offering Customer Service… and it’s a lot of self inflicted woundings.

You could say that these users should be careful. That it is their responsibility to watch out for what they buy into. I could counter with the notion that providing payments and promotions to organisations that lead to a poor customer service experience appears to be something Google won’t tolerate in other advertisers… but it is OK for Google to do? Why?

Why am I writing about this? I used to enjoy working out small customer problems in the forum. It helped my troubleshooting skills. I could detect emerging problems more quickly. The current forum is useless for that. It’s just a place for people who should never have been recruited by Google, to get free customer service because Google isn’t willing to spend time on them. The mechanism of requiring a public posting decreases the volume of queries, as well as decreasing the value of the forum as a publicly accessible forum.

When we’re looking at social media, the misusage of the AdWords Help Forum as a public customer service forum suggests that there are some deep problems inside Google’s understanding of social media and the implicit and explicit contracts. But that’s another article for another day… In the interim, think of this as the other side of yesterday’s article about PC World and their soft paywall - where PC World made the barrier to interaction too high for me, Google has made almost frictionless something that probably shouldn’t be exposed in public.

Rant mode engaged…

PC World is using an analagous “grab everything early” model for it’s online forums. Instead of the relaxed model used by Google, where you can add increasingly large amounts of personal data to your online persona, PC World *requires* a form to be completed, and users to identify what email newsletters they’ll receive, in order to drop a comment on an article.

Making this even worse, is that the form user interface has a particularly stupid problem. When filling in details, you must provide email address, country of residence and… a US Zip Code. I live in the UK. Strangely, I don’t have a US Zip code for my UK address.

Annoyingly, every time I tried a new way to pass this obstacle, it would keep re-checking the “please bombard me with offers for stuff I don’t want” box. If I uncheck the box, have the courtesy to leave it unchecked. This forced re-checking may well increase sign up rates, but it doesn’t make PC World more loved and needed.

I poked, quickly, around the PC World site looking for a way to provide feedback, and couldn’t find an obvious way to suggest that the registration process was both onerous and broken. So that’s why i’m posting here… PC World has made it clear that the audience is US-only and Foreigners can fuck off aren’t important. And PC World don’t want to hear from me unless they can try and sell me something. I’m not interested in engaging with that kind of business. So, I won’t be part of PC World forums, even for a single comment.

Lessons

If you want to offer social media, it has to be, like a good sales process, easier to be engaged than not. Good marketing and purchase systems make you feel welcome and make the process so easy that it is almost easier to buy than walk away (and that *doesn’t* mean making walking away harder with mousetrapping and other technological tricks). Look at Twitter and FaceBook - easy to sign up and you get invited to be involved, not pressured.

Undoubtedly PC World would see the newsletter sign up rate decline if they changed their registration process. And the deliberate discarding of my preference to avoid unwanted email is just insulting. What, my preference to avoid newsletters I won’t read, is not important to you?

I know PC World wants money, and they want to act a gateway to valuable information. But guys, you aren’t the sole source of information. You need to (re-)read Clay Shirky on Paywalls. What you’re doing with onerous registration forms for commenters, is creating a soft paywall around interaction. That’s not a good idea, long term, IMO. Competitors who make it easier to engage, will probably start beating you - as there’ll be more comments from people less engaged tightly… and those people could fall into your influence if you handle them well.

PCWorld guys, stop thinking like an offline paper with an online presence. Start thinking about how to properly engage with an audience.

Rant over.

Summary

There’s lessons about handling user interactions here.

Users don’t mind engaging with you.

They don’t like being “sold at”.

They don’t like you requiring lots of data before they engage lightly.

Can I avoid PC World? Absolutely. It wouldn’t break my heart if I never saw another PC World article. If they do cover things that interest me, and I think either the author or a commenter has missed the point, I might comment. But now, I’d rather Twitter it than do so on the PC World site. And that makes Twitter richer and PC World poorer.

Think about “friction”.

How hard is it for your users to engage with you? How easily can they point out problems that might help you? Yes, they’ll post other stuff too, but you know what? I got serious, private, answers about the best way to use GitHub public and private repositories from Tekkub. Imagine how I feel about Tekkub and GitHub now. Compare that with how I feel about PC World. Losing GitHub would be a major problem for me. I already pay GitHub for a private Repository. I’d pay them more, to make sure they kept running. PC World? I genuinely don’t care.

Brands have to engage with and be useful to their clients, to be well regarded over long periods. I think PC World set up short sighted performance measurements surrounding registration and sign up, and lost the plot. Make sure you don’t do the same for your business.

A recent graphical rework and substantial posting by Kim Clinkunbroomer (MrsC) has rocketed the popularity outside the AdWords Help Forum.

So why doesn’t Google take the money?

Having seen several thousands of these questions, and looked at a handful of accounts with the problem, I have an answer - maybe not *the* answer, but *an* answer. In all the cases that I’ve seen, Google doesn’t want the account to be opened and to advertise.

Why not?

Because the site or the AdWords Account holder is in breach of, or is suspected to be in breach of, one or more of Google’s Terms and Conditions. It may even be offering a scam, such as those that Google is now actively trying to stop.

So far as I can see, the usual reasons for problems are:

• Opening multiple Google Accounts for the same offer (they may be different URLs, but the offer to the consumer is the same or substantially identical)
• Opening multiple Google Accounts to use promotional vouchers - the voucher program is intended to help businesses new to advertisers, not to provide free continuous advertising
• Offering a “dodgy deal” - Google lists a collection of types of offer that it is unlikely to accept, especially the “Google Money” and “Google data entry jobs”, or one that is becoming quite common in approaches to us to help, offering pirated data such as movies and music.

This third item - the “Google Money Tree”/”Google Treasure” type of offer, or knowingly offering material that could be in breach of copyrights, appears to be a common cause for rejection by Google. And the person suffering from the refusal to open an AdWords account is usually certain that the offering is valid - because they’ve bought an eBook at a price anywhere from $9.97 to $997.00 that assures them that other people have made money this way, and it is fool proof, and accepted by Google. It isn’t.

These scams are a pretty fool proof way for the original operator at the top of the pyramid to make money. But if you look carefully at the details, there are usually clues, visible to those who know something about AdWords. The major clues are that very often the scammers will imply that you get paid by Google. They’ll show a cheque *from Google*. If you get a cheque from Google, then that’s probably because you’re running AdSense, not AdWords. Easy for a novice advertiser to confuse those words, but they mean very different things.

AdWords is an advertising system. You pay Google to advertise offers. The offers have to comply with Google’s guidelines. Under AdWords, Google would usually only pay you a fraction of advertising costs, in the event of a few specific problems in the account - and that would usually be in the form of credits against future payments, in the account; not a check in the post, in other words.

AdSense is revenue generation for publishers. You need a lot of visitors, who click on adverts, to make a decent revenue from AdSense. AdSense will pay you, and if you’re good, it will pay you more than it costs to run your site.

If you see a “make money from home”, or a “work for Google” offer, that involves signing up for AdWords, look for signs that the offer is valid. Check the AdWords Editorial Guidelines before you sign up. Be harsh in your assessment of the offer. Don’t be taken in by “special price” offers, by people who show you an AdSense cheque, and especially ask yourself this question:

If this is such a great deal, why isn’t Google already flooded with advertisers who’ve been making money this way?

AdWords has been running in one form or another for about eight years. In that time, Google has seen hundreds of thousands of advertisers. There are people who’ve made money out of selling these pyramid systems. But this is a mature market - the opportunities to make lots of money have largely vanished.

The more advanced scammers know that you know that the easy ways to make money have gone. So they’ll now sell you “secrets and tricks of the AdWords Masters”, to target “massive, low spending keywords” with certainty of making money. Ask yourself why they wouldn’t be doing this for themselves. It isn’t because they are kind and generous. It’s because they make more money by duping the desperate.

Why won’t Google take the money anyway?

A common complaint from those denied the opportunity to advertise, is that Google is turning down their money, and that it’s not up to Google to decide what can and can’t be advertised.

This is a confusion based on the American “right of free speech” and the assumption that Google will find offering deceptive adverts to be in its’ best interests. Let’s look at those…

Deceptive advertising

Google has been successful because it offered high quality search engine results. Users trust the results, sometimes far in excess of the trust they should place. I’ve heard users complaining that a business *should* be selling a product or service, because the business has been accidentally highly ranked by Google for that product or service.

If Google were to destroy that trust, by offering adverts that mislead users, there is a risk that the business will see a downturn in search engine users. The reputation for delivering good results will be damaged, and overall advertising revenues will be affected. Google goes to a lot of effort to make sure that adverts are likely to be well regarded by search engine users.

Right of free speech

You have the right to say what you want in an open forum; within limits.

When you pay for the channel, then the owner of the channel can decide what they want to carry.

Google AdWords is at liberty to turn down your offer of money to carry a message that they don’t want to carry. Just as a newspaper doesn’t have to carry offensive material in an advert - the publisher has the right the choose the content. If you don’t like it, you have the right to advertise using your own system - it just won’t reach the audience that Google has built through its’ choice of what to carry.

Summary

If Google isn’t taking your money, stop worrying about that, and start worrying about the offer you are making. It may be part of some affiliate recruitment scam, or is an overexploited niche or is deceptive to users in some way. It may make the original designer of the scheme richer, but late joiners will suffer.

If you are running a business, make sure you aren’t offering duplicate adverts - trying to bring people to two sites with the same offer is an obvious trick to try, and Google has been handling that problems *for years*. It’s one of the fastest ways to a permanent ban, once they spot it. And it can happen if you’ve instructed an agency and then open your own account (e.g. if you’ve taken on an agency that offers to get you to number one in search results, without disclosing that they are using AdWords to do so).

If you have a real, legitimate business that doesn’t offer stolen copyrighted material or other offensive content, then pay attention to your site. You may be unexpectedly hosting malware, or using words that associate your product with standard online scams. If you can’t work out what the problems might be, then ask an AdWords expert.

Harsh criticism, you might think, but I’ve been part of Google’s attempt to de-staff and automate customer service, by relying on the willing help of volunteers. I used to be a Top Contributor in the AdWords Help Forum. I contributed a lot of posts, over the years. In the early days, it was a good place to learn AdWords problems, and to spot problems and policy changes before they were announced. Over the last few years, Google has taken to sending users with problems to the Forum first, rather than to customer service.

Now, I don’t mind answering, in public, questions about AdWords, if I have the time to do so. What I don’t like is answering the same question, repeatedly, when it it caused by a failure to understand information architecture and search engine optimisation and I especially detest it when the question is entirely within the province of Google. An example of that? A user saying that their credit card has not been accepted by Google. There is no reason for anyone outside Google to know that. There is nothing that anyone outside Google can do about that. It’s a problem caused by Google deciding to decline payment - so what can you, or I, or any one else outside Google, say to the n’00th person to claim their card payment has not been accepted? There’s an entire subsection of the AdWords Help Forum that consists of would-be AdWords users complaining that their cards have not been accepted.

It’s not a nice place for a volunteer to hang out. And the rewards are minimal… because the interesting problems are now very few and far between. Anyone wanting to ask a serious question about *how* to use AdWords effectively, is lost in the noise of people whose adverts aren’t running yet (another Google internal problem, where Google has decided to defer approval of the adverts, or keywords, and no-one outside Google knows the precise reason) and other repetitive self inflicted customer service wounds. I wince when I look at the forum - so my response to this miserable treatment of Google’s customers is that I’ve resigned as a Top Contributor.

Agency and GAP Treatment

Agencies can have a Google Account Representative assigned to them. Sufficiently large accounts may have a “vertical” strategic team associated with them. So at any one time I’ve had between one and three account reps to handle problems. I’m actually pretty pleased with the way that works. I can have a new MCC set up inside a day. I can get billing questions resolved, and sort out linking, and raise account limits and have adverts pushed through expedited reviews.

But if you are a mass market, self-signed up customer, with a low budget? Tough. You’re on the self-help program. And that can be a maze of twisty links, all leading inexorably to the AdWords Help Forum, for volunteers to solve problems that Google has created for you.

Google Staffers

Google has a team of staffers who respond to Google-oriented questions. These are the AdWords Pros (AWPs). Empathetic. Pleasant. Courteous. Access to a lot of areas inside Google.

They answer thousands of posts. By eyeball estimate, the most prolific answerer, AWP.Bindu, from India, answers somewhere in the range of 5,000 to 10,000 individual threads a year, having checked the account to find out what the status is. She’s unfailingly nice, and really does try to help.

The AWPs are also ready and willing to help the Top Contributors tackle problems. Top Contributors, or TCs, have to answer frequently, and correctly, and courteously. Otherwise they won’t be TCs. It takes time. Months of answering several questions, correctly, every day. TCs have their own private area in the forum, invisible to normal users.

Why Am I Spitting Rivets This Time?

Google isn’t merely annoying an army of small businesses who’ve had their AdWords accounts terminated for unstated and apparently irreparable reasons, but it isn’t even able to obey it’s own guidelines for making websites. And if it did, the help services it offers, supported by volunteers, would be a lot easier to find, slightly more useful, and less painful for the volunteers that help Google deliver customer service.

Look at this ageing article by Google’s own Matt Cutts, on the use of the Nofollow link attribute. Old news for SEO. The summary of that article is that “page rank sculpting with nofollow is dead, and we may penalise organisations that overuse nofollow”. And what does *that* mean? It means that the nofollow attribute, originally intended to help reduce spam in discussion forums and blogs, has been so widely adopted that the use of the attribute is destroying the way that the web passes link weight.

What does Google do, in its volunteer supported web forums? It nofollows each and every link in every article, even when posted by a Google staffer, much less the “trusted” Top Contributors. The consequence of that is postings with links that refer users to better postings with good information, are not ranked any higher as a consequence of the additional links.

What does Matt actually say?

I wouldn’t recommend closing comments in an attempt to “hoard” your PageRank. In the same way that Google trusts sites less when they link to spammy sites or bad neighborhoods, parts of our system encourage links to good sites.

In other words, when Google staffers have nofollow to pages inside Google, or to pages outside Google, they are doing the right thing. By nofollowing links within the help forum, useful pages inside Google aren’t ranked more highly, and external authoritative resources are also not helped. Ultimately that should lead to Google penalising itself for attempted pagerank sculpting. Amusing, eh?

Why Are Customer Problems Exposed To The World?

Google has taken a philosophy of user contributed content and applied it to ludicrous lengths. As I said earlier, there is no way that someone outside Google can usefully comment on why a specific advertisers adverts are not running. There’s lots of reasons. They usually boil down to “you’re in an account review or an advert review”. But that’s not the specific help that advertisers are looking for - like why their card has been refused.

It is reasonable for users to ask other users about the impact of geotargeting, or what kind of advert copy works best, and the reasonable fees for an agency. But asking other users about decisions that Google has made, for reasons kept private inside Google? It makes no sense to me, or to anyone else that I’ve spoken to.

The philosophy that users are frequently enough nice and helpful is fine. I am, even with a snitty posting like this, attempting to help other users on the internet. It is even, in a painful way, helpful to Google, by pointing out that the policies they have derived is one thing and reality another.

I like the idea of the forums. I use the forums myself. But to withdraw customer service for wounds that Google has inflicted, and only Google can comment upon, and to direct those damaged customers to volunteers for support, is just not sane. Or if it is sane, it is sane in ways that mean that Google can reduce staffing - it isn’t sane in terms of developing a reputation for good customer service.

And lo and bhold, what do we see when the Nexus One ships? The same philosophy that users will help. How can I, sitting outside Google, respond to a question about when someone’s phone will ship? Or issue an RMA for defective hardware? Or deal with provisioning questions? These all need levels of of authority that are really part of the social contract between user and service provider. A relationship in which random third parties should not be welcomed by either side.

Yes, do run an AdWords Help Forum. But regard it as a supplement to Customer Service, not a replacement for it. That’s why I resigned as a Top Contributor. Preventing users from advertising for internal reasons, then telling them to consult ignorant third parties, is demeaning for all involved parties.

And then nofollowing every useful link, makes the value of posting much lower than it should be, forcing the volunteers to keep answering the same questions, again and again. Some proper information architecture and linking would help a lot.

N.B. I’m looking at this again, after the recent reports of an SSL/TLS protocol vulnerability. I’ll want OpenSSL 0.9.8l or 0.9.8m. And I’ve had problems simply compiling later versions than 0.9c from source; if the library is installed as the default version, I get a protocol failure during authentication, for reasons I don’t currently understand. If you’re looking at upgrading, that library version change should be high in the list of priorities, and I have no currently usable solution.

I use a Mac, and tested that the latest revisions of Ruby 1.8.7 worked for my projects, using “rvm”, a magic little tool for managing ruby environments. That let me make sure that my apps worked on the newer revision.

Caveats: I’m fairly new to Capistrano - this is the fourth deployment of anything I’ve done with Capistrano. There may be better ways to do this. I couldn’t see anything directly relevant when I went looking. I’m half wondering whether there should be some version of gemcutter for Capistrano tasks - it’d have to be divided by the host operating system and revision, I think… So somewhat more complex than gemcutter!

Github and Debian Etch

I’ve used git-based plugins for some of the projects. To download that source, using Capistrano, you need a “git” capable of handling subprojects. The git-core with Debian Etch is 1.4, too early to handle subprojects. So the git version needs to be upgraded to something more recent. I’ve picked git 1.6.6.

If you don’t upgrade the got-core to something more recent, you can end up wasting quite a lot of time working out why your Mac downloads a project properly, and the server target doesn’t download the submodules…

Compiling git was problematic, until I worked out what appear to be the right configuration options.

It appears that you can’t have a newer git in /usr/local while keeping the old git-core lying around, unlike most other packages. A newer git in /usr/local seems to conflict with the older system version. So part of the preparation activity needs to be to remove any existing cognito or git-core packages.

The Capistrano process

To manage the build on the collection of machines, I downloaded the sources that I wanted, created a systemsadmin rails project, and capified it:

cd tools
rails systemadmin
cd systemadmin
capify .

Add the servers you want to upgrade to the list of application, db and web servers. You’ll need to make sure that you can connect to each server with “ssh”, and the best way to do that is to upload your public key - there’s plenty of articles about setting up ssh…

I’m assuming that your login identity on each server has the capability to manage files in /usr/local/src. If it doesn’t, then you’ll need an extra capistrano step to “{sudo} chown -R my-deploy-id /usr/local/src”, where “my-deployment-id” is the your deployment user id.

I added a new set of tasks to Capistrano’s config/deploy.rb :

namespace :ruby do
desc “Send current packages to each machine”
task :sendem do
upload( “downloads/git-1.6.6.tar.gz”, “/usr/local/src/downloads”, {:via => :scp})
upload( “downloads/ruby-1.8.7-p248.tar.gz”, “/usr/local/src/downloads”, {:via => :scp})
upload( “downloads/rubygems-1.3.5.tar.gz”, “/usr/local/src/downloads”, {:via => :scp})
end

desc “Prepare build environment”
task :installbuild do
run “apt-get update && apt-get -y upgrade”
run “apt-get -y install build-essential zlib1g-dev libxml2-dev libxslt-dev openssl-dev libiconv-ruby”
run “apt-get -y remove git-core cogito git-doc rcs”
end

desc “Untar the packages”
task :untar do
run “cd /usr/local/src; tar xzf downloads/ruby-1.8.7-p248.tar.gz”
run “cd /usr/local/src && tar xzf downloads/git-1.6.6.tar.gz”
run “cd /usr/local/src && tar xzf downloads/rubygems-1.3.5.tgz”
end

desc “Build basic Rails environment on all servers. Assumes that the rub-source.tgz file has been sent to the server, then builds all standard components and installs them”
task :compile do
run “cd /usr/local/src/git-1.6.6 && ./configure –prefix=/usr/local –with-openssl –without-curl –without-tcltk && make && {sudo} make install”
run “cd /usr/local/src/ruby-1.8.7-p248 && ./configure –prefix=/usr/local –with-readline –with-openssl –with-iconv –without-tk && make && make test && {sudo} make install”
run “cd /usr/local/src/rubygems-1.3.5 && {sudo} ruby ./setup.rb”
run “{sudo} gem install rails”
end

desc “Create various rc files”
task :rcfiles do
upload( “homedir/dot.gemrc”, “.gemrc”, {:via => :scp})
run “{sudo} gem sources -a http://www.gemcutter.org/”
run “{sudo} gem sources -a http://gems.github.com/”
end

desc “Setup mongrel_cluster and nginx configuration files”
task :server_setup do
run “{sudo} cp $(locate ‘resources/mongrel_cluster’ | head -1′ /etc/init.d/mongrel_cluster”
run “{sudo} mkdir -f /etc/mongrel_cluster”
run “{sudo} cp {path}/config/mongrel_cluster.yml /etc/mongrel_cluster/{application}.yml”
run “{sudo} mkdir -f /etc/nginx/vhosts”
run “{sudo} cp nginx/{application}.conf /etc/nginx/vhosts”
run “{sudo} update-rc.d mongrel_cluster defaults”
end

end

Using the series of “run” commands, means that the various steps will not move on, if any server fails to complete the preceding run. This means that you don’t get to install the latest rubygems, until you’ve completed the compilation for ruby and installed it.

This seems to have worked for me… though it is possible that an earlier required step has already been done on all machines. I did spend some time trying to work out if a newer readline, open-ssl or libiconv, etc, made any useful difference. I couldn’t see any huge improvement benefits, so I haven’t included those.

I had huge difficulty upgrading the open-ssl version to 0.9.8m, or 0.9.8l; whenever I did, whatever the combination of configuration options (’[no-]shared’, ‘[no-]threads’, ‘zlib’, ‘zlib-dynamic’), I couldn’t use ssh to connect after compilation and installation, even though openssl completed all tests (’make tests’). I’m not sure why at the moment, and it’s added some impetus to a slow migration from Debian Etch for these VMs.

I do use a Mac for my desktop, so this environment is intended to support the type of apps that we normally use in a production environment - MySQL or Postgres, rather than Sqlite3.

I toyed with the idea of making a stream of capistrano tasks for each individual piece of code, but the way I do things tends to batch-updating. And, if I’m honest, I’ll admit that I just comment out steps that have fully worked… so I can keep a full set of sources, and upgrade the pieces on a range of machines with a few comments inserted. Not perfect, by any means, but functional when there’s a small group of developers. For a larger team, I’d want something more robust and probably something that tests a little more between steps. OTOH, the deployment scripts should be part of a staged testing process… and that would weed out many of the problems.

If there’s a better way to do this - where “better” doesn’t involve learning to use the Debian packaging system - let me know. Why do I preclude Debian packaging? These activities would never be submitted upstream, and I’d have to learn Debian packaging if I wanted to maintain Debian. I don’t. So I’ll use common shell commands that will work on pretty much any Linux host; in the commands above, the main Debian specific commands are the “apt-get” commands to install or remove components, and the ‘update-rc.d’ script to manage ‘mongrel_cluster’ and ‘nginx’ configuration files. Replace those commands with a Fedora equivalent, and this should be usable there.

mongrel_cluster

I believe that one edit is required for “mongrel_cluster”. The resources/mongrel_cluster file appears to have a typo. There’s a line to change ownership of the PID files that reads “$USER.$USER” and should be, I think, “$USER.$GROUP”. I’ll submit that suggestion via GitHub/Lighthouse.

nginx

I’ve previously set up nginx on these machines. The capistrano tasks here don’t include and document the nginx configuration changes. I don’t use the default nginx configuration file; specific hosts have been removed from the default configuration file and I have a “vhosts” directory with one configuration file per vhost, which includes the local cluster configuration.

monit

On the Etch based servers, I’m running “monit”. I haven’t included their configuration and setup here.

• Google Local - reviewed businesses typically rank higher
• Google Product Search - good reviews help more prominence in listings
• Prospects like reviews - next closest thing to word of mouth recommendation
• Reviews can be listed when people search for products and the business name

Unsatisfied buyers are typically around five times more likely to tell someone of a bad experience than satisfied buyers are to tell someone of a good experience.

As with anything, if you ask, you might get - don’t ask and you probably won’t get… Another great reason to communicate frequently and easily with your customers.

Review Sites

These sites are known to be sites that Google uses for reviews - it is not an exhaustive list:

• Qype UK UK places - pubs, restaurants, shops, businesses
• Reseller Ratings
• Yahoo! Merchant Reviews
• ShopZilla
• ViewPoints
• Epinions
• Google Checkout - part of the merchant interface for the Google payment service
• PriceGrabber (UK and US sites)
• DoYoo
• Ciao

i-Lab 'Web Presence' slides

I realise that a naked PowerPoint isn’t much use… But until I get around to shortening what ended up as a nearly two hour talk, with Q&A… here’s the slide set. It’s a QuickTime movie, click to advance, or use left and right cursor keys to move backwards and forwards.

As well as this, I promised the group that I’d publish some resources. Those’ll come in some further postings.