Announcing Capistrano Server Extensions

Maybe you’ve heard of Capistrano a tool written by Jamis Buck in Ruby that helps you automate deployment to multiple servers.

Capistrano & Deprec

This ‘gem’ of a gem is huge time saver. Believe me, anyone who has used it has surely reaped the benefits of


$ cap --apply-to .
$ cap setup
$ cap deploy
$ cap rollback
$ cap update_current
$ cap watch_load
$ cap uptime
$ cap disable/enable_web
$ cap some_long_complex_task_that_would_normally_take_more_than_a_couple_of_hours_if_done_manually

etc....

This beauty is such a versatile beast (pun intended) that you can easily automate tasks for a range of use cases (app deployment, system administration, backups, profiling, benchmarking etc etc)

And many people have gone ahead and extended capistrano to do their bidding with evermore complex tasks.

One such person is Mike Bailey who was obviously really pissed off at having to setup the same old software on every new VPS he was configuring for his clients. (Please note that this is pure speculation and the real reason may in fact be very different). In any case, he decided to remedy the situation and wrote the deprec gem.

What does deprec do?

It basically augments capistrano to allow you to do things like:


$ cap setup_ssh_keys   # copy out your public keys
$ cap install_rails_stack # install apache, ruby & rubygems, mongrel, rails on the hosts defined in your capistrano recipe etc.
$ deprec --apply-to . --name bailey --domain bailey.net.au
$ cap deprec_setup #setup your application to allow easy server setups
$ cap deploy_with_migrations #deploy your application
$ cap restart_mongrel_cluster #start app server
$ cap restart_apache #start web server

i.e with this gem you can very easily setup a brand new VPS with a full rails stack including the MySQL database and Apache 2 web server.

Customizing Deprec

When I first discovered deprec, I immediately realised that this was going to be saving me a lot of time when setting up our own clients VPS’, but I wasn’t really enamoured of the choices of web server or database. Furthermore I wanted to be able to select the right tools for the job. For one client I may use MySQL but for another I’d use postgres (Well actually I prefer using postgres always but maybe you’d like to do that :).

My current preference for a rails stack is the following very lightweight setup:

Nginx => Mongrel => Rails => Postgres

So as we had a couple of small new clients whose servers needed setting up I spent a couple of days and wrote my own extensions to deprec that added support for nginx (see this for details) and PostgreSQL. I also added a few other bits and pieces that allowed me to do things that I like to do with every new VPS I setup.

Exploring Capistrano Server Extensions

So first off we install the Capistrano Server Extensions gem (very lame name I know) via:


sudo gem install -y capserverext

That’ll bring down all required dependencies including capistrano and deprec if you don’t already have them.

Side Note

I’m unashamedly assuming I’m talking to a Ruby crowd and that you have at least ruby 1.8.4 installed but let me say one thing to the other crowds out there:

“Get ruby installed and try this out”

It doesn’t matter whether your setting up LAMP servers, or j2ee servers, this stuff kicks real ass and will save the day for you on many occasions, seriously. (I’m a converted java programmer myself so I know this would come in really handily for setting up something like the jdk, Tomcat/Jetty/Resin, JBoss/Geronimo)

Heck we could even spiff it up to install Oracle. Now that’s a great idea. Imagine being able to preselect a type of Oracle setup and just typing: $ cap install oracle_cluster and go away for a cuppa.

Anyway I’m diverging a lot I know…sorry.

So once capserverext has been installed your first action is to


$ cap show_tasks

Which will show you a bewildering list of possible tasks to perform.

From this list you can pick and choose, mix and match different tasks for every occasion.

But let’s run through what you’d need to do to get a rails stack setup with nginx and postgres (currently only on a Ubuntu server but I’m working on supporting debian and Gentoo).

First let’s ask capistrano to tell us what we should do to use this thing. Type:


$ cap capserverext_usage
 * executing task capserverext_usage
      * rails myproject
      * cd /path/to/myproject
      * deprec --apply-to . --name user --domain myprojectsdomain.com
      * Add following entries to deploy.rb:
          require 'capserverext/recipes'
          require 'capistrano/ext/monitor'

      * Edit deploy.rb appropriately:
        * set deprec_combination to either (:nginx_postgres, :nginx_mysql, :apache_mysql, :apache_postgres)
          for a combination other than the default: ':nginx_postgres'
        * set ssh_security_port for an ssh port other than the default 22 (Defaults to: '8888')
        * set mongrel options (best to accept the defaults)
        * set repository
        * any other custom options

      * run: "cap prepare_ssh" 
      * set ssh_options[:port] to the value of ssh_security_port in deploy.rb
      * run: "cap prepare_host" 

Using Capistrano Server Extensions

Thanks, that was handy. Now let me type the following:

On my local machine:


$ cd ~/dev/freelance
$ rails amicmuseuprecolombi.org   #A domain for one of my apps
$ cd amicsmuseuprecolombi.org
$ deprec --apply-to . --name saimon --domain amicsmuseuprecolombi.org

This’ll create a deploy.rb capistrano recipe for us in ./config/deploy.rb.

Quickly edit it to add:


  require 'capserverext/recipes'
  set :deprec_combination, :nginx_postgres
  set :repository, "http://www.webtypes.com/svn/projects/#{application}/trunk" 

  set :mongrel_servers, nginx_proxy_servers
  set :mongrel_port, nginx_proxy_port
  set :mongrel_address, nginx_proxy_address

Now type:


  $ cap prepare_ssh

And set the secure_ssh_port in deploy.rb to 8888 like so:


set ssh_options[:port], 8888

Now finally type:


  $ cap prepare_host

When that’s done type:


  $ cap deploy_with_migrations
  $ cap restart_mongrel_cluster
  $ cap restart_nginx

That takes about a total of 10 minutes from start to finish!

And visit http://amicsmuseuprecolombi.org (Our latest creation based on a globalized version of Mephisto) to see the result of having typed in those few commands. Well we also wrote the code for the site but you get the gist :)

Yep that’s all it takes to get a lean rails stack setup on your server .

Variations on the same theme

“I haven’t heard of nginx before. Can’t I use Apache?” someone says.

No problem. Set:


  set :deprec_combination, :apache_postgres

  $ cap prepare_host

“I prefer MySQL.” another says.

No problem. Set:


  set :deprec_combination, :nginx_mysql

  $ cap prepare_host

“No I meant with Apache.” they say again.

No problem. Set:


  set :deprec_combination, :apache_mysql

  $ cap prepare_host

Under the cover

Some of you will be asking what do these commands do exactly and I’ll point those of you to the very readable source code of the capserverext gem and the deprec gem. (Capistrano code is also very readable but you don’t need to go that far.)

So what will capserverext do remotely:

The prepare_ssh task will:

  • Connect via root and set root’s password
  • Change ssh port for security reasons
  • Create a deployment user, add it to the ‘deploy’ group
The prepare_host task will:
  • Connect with the deployment user and copy over your local machines public ssh keys.
  • Reconnect without asking for passwords anymore (apart from the first time it uses sudo)
  • Give the deployment user sudo privileges.
  • enable the ubuntu ‘universe’ repository
  • Install requisites allowing compilation of other packages, ruby, subversion, openssl
  • Install the PostgreSQL database server (8.1)
  • Install rubygems
  • Install rails, mongrel, mongrel_cluster and builder
  • Install nginx
  • setup generic deployment paths
  • setup app specific deployment paths
  • Configure nginx for production
  • Configure postgres for production (Creates a new database according to the name given for the production environment in database.yml, a database user and gives access privileges to that user)

That’s it. And you can do this again and again, over and over, on any number of hosts.

The beauty of this is that many tasks are composed of other smaller tasks so you can execute a specific task without having to run the whole lot.

Say you want to just install and configure nginx:


$ cap install_nginx
$ cap setup_web_for_nginx

or just apache


$ cap install_apache
$ cap setup_web  (deprec task)

Conclusion

Let me conclude by saying that I hope this article has shown you the flexibility and usefullness that Capistrano can be put to and how easily you can now setup your VPS servers for most combinations of setup.

Of course there’ll be a lot of you that need to tweak things for their own purposes, just create a ~/.caprc file and customize away (That’s how I did it)

I’m going to continue working on this to add support for more linux distributions. Mac server users will have to wait until I decide to make the jump (It’ll inevitably come but I’m not sure when yet).

If you found this handy or useful please provide feedback and also let Mike Bailey know and then we can perhaps integrate this stuff into deprec (so you don’t need to be installing two separate gems).

Thanks for reading,

Saimon Moore

17 comentarios sobre “Announcing Capistrano Server Extensions”

  1. Cliff

    Running cap capserverext_usage results in an error on my machine after installing. Capserverext installed versions 1.4.1 of both deprec and capistrano, but I don’t seem to have access to any capserverext syntax. Any ideas about what I might be doing wrong?

  2. Saimon

    Hi Cliff,

    Could you post a bug report here
    with the error trace you’re getting?

    I’ll take a look and see what’s up.

    Thanks,

    Saimon

  3. Cliff

    Done, thanks Saimon. :)

  4. Matthew King

    To get this working with Ubuntu 6.10, I had to add a task that makes sure the deployment user’s shell is bash (as 6.10 and up now default to dash).

    Details in this blog post

  5. Saimon Moore

    Hi Mathew,

    Thanks for bringing this to my attention.

    I’ve been investigating and it seems that ubuntu has changed /bin/sh to link to /bin/dash rather than /bin/bash. However, after further investigation it still appears that the default login shell for normal users (and root) is /bin/bash so I’m still confused as to why your deployment user’s shell was set as dash.

    In any case, i like you’re solution and will incorporate it into the next release of capserverext.

    Regards,

    Saimon

  6. Ari

    I am having issues with the gem myself as well…

    I am consistently getting: command “sudo chmod 766 /usr/local/apache2/conf/httpd.conf” failed on 000.000.000.000

    regardless if I am not installing apache

  7. Valery

    Hi Saimon,

    I am going for your extension, because deprec is postgresql-ignorant.

    in Section “Exploring Capistrano Server Extensions”

    stated:

    sudo gem install -y capserverext

    ... then later:

    cap show_tasks

    but under many tasks listed i see no word postgresql according to the sweaty link (http://pastie.caboo.se/47330) you gave.

    Perhaps some special “required”s needed in deploy.rb ?

    I am not giving up am still fighting :)

    thanks, Valery

  8. Valery

    ... and the same trouble like Cliff has:

    cap capserverext_usage /usr/lib/ruby/gems/1.8/gems/capistrano-1.4.1/lib/capistrano/actor.rb:562:in `method_missing’: undefined method `capserverext_usage’ for #<capistrano::actor:0xb79b22bc> (NoMethodError) from /usr/lib/ruby/gems/1.8/gems/capistrano-1.4.1/lib/capistrano/cli.rb:268:in `send’ from /usr/lib/ruby/gems/1.8/gems/capistrano-1.4.1/lib/capistrano/cli.rb:268:in `execute_recipes!’ from /usr/lib/ruby/gems/1.8/gems/capistrano-1.4.1/lib/capistrano/cli.rb:268:in `each’ from /usr/lib/ruby/gems/1.8/gems/capistrano-1.4.1/lib/capistrano/cli.rb:268:in `execute_recipes!’ from /usr/lib/ruby/gems/1.8/gems/capistrano-1.4.1/lib/capistrano/cli.rb:239:in `execute!’ from /usr/lib/ruby/gems/1.8/gems/capistrano-1.4.1/lib/capistrano/cli.rb:12:in `execute!’ from /usr/lib/ruby/gems/1.8/gems/capistrano-1.4.1/bin/cap:11 from /usr/bin/cap:16:in `load’ from /usr/bin/cap:16

    thanks, Valery.

  9. Valery

    well, i give up. i can’t bring this to work under ubuntu feisty. It’s a pity, because it sounds really great, it is exactly what i need over deprec :(

  10. Saimon Moore

    Hi Valery,

    I’m very sorry for not picking up on this. Are you still having trouble with this?

    Please contact me directly (You can leave a message via my contact form on http://saimonmoore.net and then I’ll email you directly) and I’ll try and help you sort out your problems.

    Regards,

    Saimon

  11. Robert Berger

    Will this work with capistrano 2.0 beta? ie v1.99.3? It didn’t seem to off hand when I run it on my system I get:

    % cap capserverext_usage the task `capserverext_usage’ does not exist Exit 1

  12. Matthew King

    I’m hoping this information will only be useful for a short time.

    nginx 0.6.6 fails at the ‘make install’ part because of a bug in the install script.

    There’s a patch in the message at: http://article.gmane.org/gmane.comp.web.nginx.english/1294/match=sysconfdir

    When you grab the patch from the web, however, you have to fix the @@ part, as it has gotten munged to <at><at>.

  13. Matthew King

    So I forgot to state clearly in my last comment that the need for a patch sorta defeats using capserverext until the patch gets rolled into an nginx release.

    I’m writing up a workaround on my weblog: http://www.automatthew.com

  14. Saimon Moore

    Hi guys,

    I’m about to integrate capservext with deprec 2 which will support capistrano 2. Until then hang in there.

    Matthew, thanks for the low down….

    Regards,

    Saimon

  15. Dash

    Thanks for a cool gem … when can we expect the next version of this gem?

  16. Mick

    Solution for “method_missing” problem

    I hit the same problem as others when trying to run

    cap capserverext_usage

    I debugged it using NetBeans. One solution is to add the following to your ~/.caprc file:

    require 'capserverext/recipes'

    The instructions in the original blog post say to put that line in deploy.rb, but that didn’t work for me as it seems that capsitrano (1.4.1) trys to interpret the “capserverext_usage” command line argument before it actually looks at deploy.rb.

    Hope this helps.

    cheers, mick

  17. mick

    A solution for Ari’s reported problem:

    Ari wrote: > I am having issues with the gem myself as well… > > I am consistently getting: command “sudo chmod 766 > /usr/local/apache2/conf/httpd.conf” failed … > > regardless if I am not installing apache

    I suspect the problem is cause by using an incompatible version of deprec. I am using deprec 1.9.0 and I hit this problem. The fix, if you want to continue with deprec 1.9.0, is to define the following:

    task :after_setup, :except => { :no_release => true } do
      # setup_servers
    end

    The above overrides the definition of after_setup defined in deprec to stop it from invoking setup_servers which is the command that is trying to setup apache.

    A possibly better alternative would be to change capserverext so that it defines setup_servers to setup nginx, rather than apache.

Deja tu comentario