Using the rails installed by apt-get was a bit confusing because the rails project being generated was not the rails version that I expected. I had installed the rails gem v2.2.2 but the config/environment.rb file showed rails v2.1.

I fixed this issue by removing the default rails.

sudo apt-get remove rails

And then using an alias:

export GEMDIR=`gem environment gemdir`
export RAILS_VERSION=`gem list rails | grep "rails " | cut -b8- | awk '{sub(/\)/, "");print}'`
alias rails='ruby $GEMDIR/gems/rails-$RAILS_VERSION/bin/rails'

Now when I do

rails dool

The project uses the right version of rails!