I loved “Read This Before Installing Rails 3.1.3” – near perfect. It even includes a link for what to do if you have XCode 4.2. Here’s the extra bits you might stumble over:

Install Ruby 1.9.3

Instead of the installation of Ruby step in the “Read This” article, do this:

rvm pkg install iconv
rvm pkg install readline
rvm remove 1.9.3
rvm install 1.9.3 –with-iconv-dir=$rvm_path/usr –with-readline-dir=$rvm_path/usr

I found no other problems – there are, though, things that need to be done when upgrading an application from Rails 3 to Rails 3.1.

Updates

After nanofunk suggested that the “–with-conv-dir” and “–with-readline-dir” weren’t needed, I tested with “–gcc=clang”, as he suggested and I had compilation failures. I upgraded to rvm 1.10.0 and tried that – it also fails.

Using rvm 1.10.0 also results in the compilation commands that I give above, also failing. So, as of rvm 1.10.0, I don’t know how to build a working Ruby 1.9.3 on Mac OS X Lion with Xcode 4.1. I may have to upgrade the Xcode version?

WIth rvm 1.10.0, using “–with-gcc=clang” or without, and no additional command line options, I get an error during compilation of the intrinsic readline.c file. So I suspect that I need to do something about the additional library… Just can’t work out what, right now.

When I work it out, I’ll add the details.

I poked around for a while and found a few examples that did part of what I was trying to achieve – but nothing that converted a UTF-16LE TSV into a UTF-8 CSV (strict). I believe, on the basis that it works on the programs that I’ve been having difficulty with, that the following program stub example (fixed input and output file names) is capable of being tweaked to do the right thing for you.

#!/usr/bin/env ruby -KU

outfile.flush
outfile.close


I hope you find this useful. If you’ve suggestions to improve the core function, please tell me. This is used inside a project that I have – so I’m not fussed about making this a fully competent command line program. I’m most concerned about the horrendously ugly and clumsy “split(2,100000000)”.

N.B. I’m looking at this again, after the recent reports of an SSL/TLS protocol vulnerability. I’ll want OpenSSL 0.9.8l or 0.9.8m. And I’ve had problems simply compiling later versions than 0.9c from source; if the library is installed as the default version, I get a protocol failure during authentication, for reasons I don’t currently understand. If you’re looking at upgrading, that library version change should be high in the list of priorities, and I have no currently usable solution.

I use a Mac, and tested that the latest revisions of Ruby 1.8.7 worked for my projects, using “rvm”, a magic little tool for managing ruby environments. That let me make sure that my apps worked on the newer revision.

Caveats: I’m fairly new to Capistrano – this is the fourth deployment of anything I’ve done with Capistrano. There may be better ways to do this. I couldn’t see anything directly relevant when I went looking. I’m half wondering whether there should be some version of gemcutter for Capistrano tasks – it’d have to be divided by the host operating system and revision, I think… So somewhat more complex than gemcutter!

Github and Debian Etch

I’ve used git-based plugins for some of the projects. To download that source, using Capistrano, you need a “git” capable of handling subprojects. The git-core with Debian Etch is 1.4, too early to handle subprojects. So the git version needs to be upgraded to something more recent. I’ve picked git 1.6.6.

If you don’t upgrade the got-core to something more recent, you can end up wasting quite a lot of time working out why your Mac downloads a project properly, and the server target doesn’t download the submodules…

Compiling git was problematic, until I worked out what appear to be the right configuration options.

It appears that you can’t have a newer git in /usr/local while keeping the old git-core lying around, unlike most other packages. A newer git in /usr/local seems to conflict with the older system version. So part of the preparation activity needs to be to remove any existing cognito or git-core packages.

The Capistrano process

To manage the build on the collection of machines, I downloaded the sources that I wanted, created a systemsadmin rails project, and capified it:

cd tools
rails systemadmin
cd systemadmin
capify .

Add the servers you want to upgrade to the list of application, db and web servers. You’ll need to make sure that you can connect to each server with “ssh”, and the best way to do that is to upload your public key – there’s plenty of articles about setting up ssh…

I’m assuming that your login identity on each server has the capability to manage files in /usr/local/src. If it doesn’t, then you’ll need an extra capistrano step to “{sudo} chown -R my-deploy-id /usr/local/src”, where “my-deployment-id” is the your deployment user id.

I added a new set of tasks to Capistrano’s config/deploy.rb :

namespace :ruby do
desc “Send current packages to each machine”
task :sendem do
upload( “downloads/git-1.6.6.tar.gz”, “/usr/local/src/downloads”, {:via => :scp})
upload( “downloads/ruby-1.8.7-p248.tar.gz”, “/usr/local/src/downloads”, {:via => :scp})
upload( “downloads/rubygems-1.3.5.tar.gz”, “/usr/local/src/downloads”, {:via => :scp})
end

desc “Prepare build environment”
task :installbuild do
run “apt-get update && apt-get -y upgrade”
run “apt-get -y install build-essential zlib1g-dev libxml2-dev libxslt-dev openssl-dev libiconv-ruby”
run “apt-get -y remove git-core cogito git-doc rcs”
end

desc “Untar the packages”
task :untar do
run “cd /usr/local/src; tar xzf downloads/ruby-1.8.7-p248.tar.gz”
run “cd /usr/local/src && tar xzf downloads/git-1.6.6.tar.gz”
run “cd /usr/local/src && tar xzf downloads/rubygems-1.3.5.tgz”
end

desc “Build basic Rails environment on all servers. Assumes that the rub-source.tgz file has been sent to the server, then builds all standard components and installs them”
task :compile do
run “cd /usr/local/src/git-1.6.6 && ./configure –prefix=/usr/local –with-openssl –without-curl –without-tcltk && make && {sudo} make install”
run “cd /usr/local/src/ruby-1.8.7-p248 && ./configure –prefix=/usr/local –with-readline –with-openssl –with-iconv –without-tk && make && make test && {sudo} make install”
run “cd /usr/local/src/rubygems-1.3.5 && {sudo} ruby ./setup.rb”
run “{sudo} gem install rails”
end

desc “Create various rc files”
task :rcfiles do
upload( “homedir/dot.gemrc”, “.gemrc”, {:via => :scp})
run “{sudo} gem sources -a http://www.gemcutter.org/”
run “{sudo} gem sources -a http://gems.github.com/”
end

desc “Setup mongrel_cluster and nginx configuration files”
task :server_setup do
run “{sudo} cp $(locate ‘resources/mongrel_cluster’ | head -1′ /etc/init.d/mongrel_cluster”
run “{sudo} mkdir -f /etc/mongrel_cluster”
run “{sudo} cp {path}/config/mongrel_cluster.yml /etc/mongrel_cluster/{application}.yml”
run “{sudo} mkdir -f /etc/nginx/vhosts”
run “{sudo} cp nginx/{application}.conf /etc/nginx/vhosts”
run “{sudo} update-rc.d mongrel_cluster defaults”
end

end

Using the series of “run” commands, means that the various steps will not move on, if any server fails to complete the preceding run. This means that you don’t get to install the latest rubygems, until you’ve completed the compilation for ruby and installed it.

This seems to have worked for me… though it is possible that an earlier required step has already been done on all machines. I did spend some time trying to work out if a newer readline, open-ssl or libiconv, etc, made any useful difference. I couldn’t see any huge improvement benefits, so I haven’t included those.

I had huge difficulty upgrading the open-ssl version to 0.9.8m, or 0.9.8l; whenever I did, whatever the combination of configuration options (‘[no-]shared’, ‘[no-]threads’, ‘zlib’, ‘zlib-dynamic’), I couldn’t use ssh to connect after compilation and installation, even though openssl completed all tests (‘make tests’). I’m not sure why at the moment, and it’s added some impetus to a slow migration from Debian Etch for these VMs.

I do use a Mac for my desktop, so this environment is intended to support the type of apps that we normally use in a production environment – MySQL or Postgres, rather than Sqlite3.

I toyed with the idea of making a stream of capistrano tasks for each individual piece of code, but the way I do things tends to batch-updating. And, if I’m honest, I’ll admit that I just comment out steps that have fully worked… so I can keep a full set of sources, and upgrade the pieces on a range of machines with a few comments inserted. Not perfect, by any means, but functional when there’s a small group of developers. For a larger team, I’d want something more robust and probably something that tests a little more between steps. OTOH, the deployment scripts should be part of a staged testing process… and that would weed out many of the problems.

If there’s a better way to do this – where “better” doesn’t involve learning to use the Debian packaging system – let me know. Why do I preclude Debian packaging? These activities would never be submitted upstream, and I’d have to learn Debian packaging if I wanted to maintain Debian. I don’t. So I’ll use common shell commands that will work on pretty much any Linux host; in the commands above, the main Debian specific commands are the “apt-get” commands to install or remove components, and the ‘update-rc.d’ script to manage ‘mongrel_cluster’ and ‘nginx’ configuration files. Replace those commands with a Fedora equivalent, and this should be usable there.

mongrel_cluster

I believe that one edit is required for “mongrel_cluster”. The resources/mongrel_cluster file appears to have a typo. There’s a line to change ownership of the PID files that reads “$USER.$USER” and should be, I think, “$USER.$GROUP”. I’ll submit that suggestion via GitHub/Lighthouse.

nginx

I’ve previously set up nginx on these machines. The capistrano tasks here don’t include and document the nginx configuration changes. I don’t use the default nginx configuration file; specific hosts have been removed from the default configuration file and I have a “vhosts” directory with one configuration file per vhost, which includes the local cluster configuration.

monit

On the Etch based servers, I’m running “monit”. I haven’t included their configuration and setup here.

Both OS X and Debian have rails 2.1.0, ruby 1.8.5 (Debian) and 1.8.6 (Mac OS X), sqlite3 3.3.0 (Debian) and 3.4.0 (Mac OS X).

Use the following commands on each:

rails test
cd test
script/generate scaffold testaroo qing:text jing:string
rake db:migrate


Works just fine on OS X. On Debian I get:


~/src/test$ rake db:migrate
(in /home/jdc/src/test)
rake aborted!
Could not find table 'testaroo'


The config/databas.yml file is identical on OS X and Debian - I've extracted just the development section:


# SQLite version 3.x
# gem install sqlite3-ruby (not necessary on OS X Leopard)
development:
adapter: sqlite3
database: db/development.sqlite3
timeout: 5000


And the db directory on both systems is the same, with a "development.sqlite3" file and the migrate directory. The schema file... that's where the problem is. The OS X version of this file includes the following lines:


create_table "testarees", :force => true do |t|
t.text "ming"
t.string "qing"
t.datetime "created_at"
t.datetime "updated_at"
end


Those lines are missing from the Debian version. I've seen quite a few questions about problems with the db:migrate showing missing tables, and none of them that I've read, have pointed at the schema as being the problem.

At a wild guess, the downrev Ruby causes some kind of problem that prevents addng the pseudo-SQL code. Tracking into this level has already taken hours. If anyone has a clue, or can nominate a high S/N forum to post, that'd be nice.

As it is, I'll develop on Mac, and see if I can deploy on Debian - but I might force an upgrade to Ruby 1.8.6, just in case that's it.

]]>