Announcing Capistrano Server Extensions
Maybe you’ve heard of Capistrano a tool written by Jamis Buck in Ruby that helps you automate deployment to multiple servers.
Capistrano & Deprec
This ‘gem’ of a gem is huge time saver. Believe me, anyone who has used it has surely reaped the benefits of
$ cap --apply-to .
$ cap setup
$ cap deploy
$ cap rollback
$ cap update_current
$ cap watch_load
$ cap uptime
$ cap disable/enable_web
$ cap some_long_complex_task_that_would_normally_take_more_than_a_couple_of_hours_if_done_manually
etc....
This beauty is such a versatile beast (pun intended) that you can easily automate tasks for a range of use cases (app deployment, system administration, backups, profiling, benchmarking etc etc)
And many people have gone ahead and extended capistrano to do their bidding with evermore complex tasks.
One such person is Mike Bailey who was obviously really pissed off at having to setup the same old software on every new VPS he was configuring for his clients. (Please note that this is pure speculation and the real reason may in fact be very different). In any case, he decided to remedy the situation and wrote the deprec gem.
What does deprec do?
It basically augments capistrano to allow you to do things like:
$ cap setup_ssh_keys # copy out your public keys
$ cap install_rails_stack # install apache, ruby & rubygems, mongrel, rails on the hosts defined in your capistrano recipe etc.
$ deprec --apply-to . --name bailey --domain bailey.net.au
$ cap deprec_setup #setup your application to allow easy server setups
$ cap deploy_with_migrations #deploy your application
$ cap restart_mongrel_cluster #start app server
$ cap restart_apache #start web server
i.e with this gem you can very easily setup a brand new VPS with a full rails stack including the MySQL database and Apache 2 web server.
Customizing Deprec
When I first discovered deprec, I immediately realised that this was going to be saving me a lot of time when setting up our own clients VPS’, but I wasn’t really enamoured of the choices of web server or database. Furthermore I wanted to be able to select the right tools for the job. For one client I may use MySQL but for another I’d use postgres (Well actually I prefer using postgres always but maybe you’d like to do that :).
My current preference for a rails stack is the following very lightweight setup:
Nginx => Mongrel => Rails => Postgres
So as we had a couple of small new clients whose servers needed setting up I spent a couple of days and wrote my own extensions to deprec that added support for nginx (see this for details) and PostgreSQL. I also added a few other bits and pieces that allowed me to do things that I like to do with every new VPS I setup.
Exploring Capistrano Server Extensions
So first off we install the Capistrano Server Extensions gem (very lame name I know) via:
sudo gem install -y capserverext
That’ll bring down all required dependencies including capistrano and deprec if you don’t already have them.
I’m unashamedly assuming I’m talking to a Ruby crowd and that you have at least ruby 1.8.4 installed but let me say one thing to the other crowds out there:
“Get ruby installed and try this out”
It doesn’t matter whether your setting up LAMP servers, or j2ee servers, this stuff kicks real ass and will save the day for you on many occasions, seriously. (I’m a converted java programmer myself so I know this would come in really handily for setting up something like the jdk, Tomcat/Jetty/Resin, JBoss/Geronimo)
Heck we could even spiff it up to install Oracle. Now that’s a great idea. Imagine being able to preselect a type of Oracle setup and just typing: $ cap install oracle_cluster and go away for a cuppa.
Anyway I’m diverging a lot I know…sorry.
So once capserverext has been installed your first action is to
$ cap show_tasks
Which will show you a bewildering list of possible tasks to perform.
From this list you can pick and choose, mix and match different tasks for every occasion.
But let’s run through what you’d need to do to get a rails stack setup with nginx and postgres (currently only on a Ubuntu server but I’m working on supporting debian and Gentoo).
First let’s ask capistrano to tell us what we should do to use this thing. Type:
$ cap capserverext_usage
* executing task capserverext_usage
* rails myproject
* cd /path/to/myproject
* deprec --apply-to . --name user --domain myprojectsdomain.com
* Add following entries to deploy.rb:
require 'capserverext/recipes'
require 'capistrano/ext/monitor'
* Edit deploy.rb appropriately:
* set deprec_combination to either (:nginx_postgres, :nginx_mysql, :apache_mysql, :apache_postgres)
for a combination other than the default: ':nginx_postgres'
* set ssh_security_port for an ssh port other than the default 22 (Defaults to: '8888')
* set mongrel options (best to accept the defaults)
* set repository
* any other custom options
* run: "cap prepare_ssh"
* set ssh_options[:port] to the value of ssh_security_port in deploy.rb
* run: "cap prepare_host"
Using Capistrano Server Extensions
Thanks, that was handy. Now let me type the following:
On my local machine:
$ cd ~/dev/freelance
$ rails amicmuseuprecolombi.org #A domain for one of my apps
$ cd amicsmuseuprecolombi.org
$ deprec --apply-to . --name saimon --domain amicsmuseuprecolombi.org
This’ll create a deploy.rb capistrano recipe for us in ./config/deploy.rb.
Quickly edit it to add:
require 'capserverext/recipes'
set :deprec_combination, :nginx_postgres
set :repository, "http://www.webtypes.com/svn/projects/#{application}/trunk"
set :mongrel_servers, nginx_proxy_servers
set :mongrel_port, nginx_proxy_port
set :mongrel_address, nginx_proxy_address
Now type:
$ cap prepare_ssh
And set the secure_ssh_port in deploy.rb to 8888 like so:
set ssh_options[:port], 8888
Now finally type:
$ cap prepare_host
When that’s done type:
$ cap deploy_with_migrations
$ cap restart_mongrel_cluster
$ cap restart_nginx
That takes about a total of 10 minutes from start to finish!
And visit http://amicsmuseuprecolombi.org (Our latest creation based on a globalized version of Mephisto) to see the result of having typed in those few commands. Well we also wrote the code for the site but you get the gist :)
Yep that’s all it takes to get a lean rails stack setup on your server .
Variations on the same theme
“I haven’t heard of nginx before. Can’t I use Apache?” someone says.
No problem. Set:
set :deprec_combination, :apache_postgres
$ cap prepare_host
“I prefer MySQL.” another says.
No problem. Set:
set :deprec_combination, :nginx_mysql
$ cap prepare_host
“No I meant with Apache.” they say again.
No problem. Set:
set :deprec_combination, :apache_mysql
$ cap prepare_host
Under the cover
Some of you will be asking what do these commands do exactly and I’ll point those of you to the very readable source code of the capserverext gem and the deprec gem. (Capistrano code is also very readable but you don’t need to go that far.)
So what will capserverext do remotely:
The prepare_ssh task will:
- Connect via root and set root’s password
- Change ssh port for security reasons
- Create a deployment user, add it to the ‘deploy’ group
- Connect with the deployment user and copy over your local machines public ssh keys.
- Reconnect without asking for passwords anymore (apart from the first time it uses sudo)
- Give the deployment user sudo privileges.
- enable the ubuntu ‘universe’ repository
- Install requisites allowing compilation of other packages, ruby, subversion, openssl
- Install the PostgreSQL database server (8.1)
- Install rubygems
- Install rails, mongrel, mongrel_cluster and builder
- Install nginx
- setup generic deployment paths
- setup app specific deployment paths
- Configure nginx for production
- Configure postgres for production (Creates a new database according to the name given for the production environment in database.yml, a database user and gives access privileges to that user)
That’s it. And you can do this again and again, over and over, on any number of hosts.
The beauty of this is that many tasks are composed of other smaller tasks so you can execute a specific task without having to run the whole lot.
Say you want to just install and configure nginx:
$ cap install_nginx
$ cap setup_web_for_nginx
or just apache
$ cap install_apache
$ cap setup_web (deprec task)
Conclusion
Let me conclude by saying that I hope this article has shown you the flexibility and usefullness that Capistrano can be put to and how easily you can now setup your VPS servers for most combinations of setup.
Of course there’ll be a lot of you that need to tweak things for their own purposes, just create a ~/.caprc file and customize away (That’s how I did it)
I’m going to continue working on this to add support for more linux distributions. Mac server users will have to wait until I decide to make the jump (It’ll inevitably come but I’m not sure when yet).
If you found this handy or useful please provide feedback and also let Mike Bailey know and then we can perhaps integrate this stuff into deprec (so you don’t need to be installing two separate gems).
Thanks for reading,
Personas felices
Proyecto real
Web abierta
Running cap capserverext_usage results in an error on my machine after installing. Capserverext installed versions 1.4.1 of both deprec and capistrano, but I don’t seem to have access to any capserverext syntax. Any ideas about what I might be doing wrong?
Hi Cliff,
Could you post a bug report here
with the error trace you’re getting?
I’ll take a look and see what’s up.
Thanks,
Saimon
Done, thanks Saimon. :)
To get this working with Ubuntu 6.10, I had to add a task that makes sure the deployment user’s shell is bash (as 6.10 and up now default to dash).
Details in this blog post
Hi Mathew,
Thanks for bringing this to my attention.
I’ve been investigating and it seems that ubuntu has changed /bin/sh to link to /bin/dash rather than /bin/bash. However, after further investigation it still appears that the default login shell for normal users (and root) is /bin/bash so I’m still confused as to why your deployment user’s shell was set as dash.
In any case, i like you’re solution and will incorporate it into the next release of capserverext.
Regards,
Saimon
I am having issues with the gem myself as well…
I am consistently getting: command “sudo chmod 766 /usr/local/apache2/conf/httpd.conf” failed on 000.000.000.000
regardless if I am not installing apache
Hi Saimon,
I am going for your extension, because deprec is postgresql-ignorant.
in Section “Exploring Capistrano Server Extensions”
stated:
... then later:
but under many tasks listed i see no word postgresql according to the sweaty link (http://pastie.caboo.se/47330) you gave.
Perhaps some special “required”s needed in deploy.rb ?
I am not giving up am still fighting :)
thanks, Valery
... and the same trouble like Cliff has:
cap capserverext_usage /usr/lib/ruby/gems/1.8/gems/capistrano-1.4.1/lib/capistrano/actor.rb:562:in `method_missing’: undefined method `capserverext_usage’ for #<capistrano::actor:0xb79b22bc> (NoMethodError) from /usr/lib/ruby/gems/1.8/gems/capistrano-1.4.1/lib/capistrano/cli.rb:268:in `send’ from /usr/lib/ruby/gems/1.8/gems/capistrano-1.4.1/lib/capistrano/cli.rb:268:in `execute_recipes!’ from /usr/lib/ruby/gems/1.8/gems/capistrano-1.4.1/lib/capistrano/cli.rb:268:in `each’ from /usr/lib/ruby/gems/1.8/gems/capistrano-1.4.1/lib/capistrano/cli.rb:268:in `execute_recipes!’ from /usr/lib/ruby/gems/1.8/gems/capistrano-1.4.1/lib/capistrano/cli.rb:239:in `execute!’ from /usr/lib/ruby/gems/1.8/gems/capistrano-1.4.1/lib/capistrano/cli.rb:12:in `execute!’ from /usr/lib/ruby/gems/1.8/gems/capistrano-1.4.1/bin/cap:11 from /usr/bin/cap:16:in `load’ from /usr/bin/cap:16
thanks, Valery.
well, i give up. i can’t bring this to work under ubuntu feisty. It’s a pity, because it sounds really great, it is exactly what i need over deprec :(
Hi Valery,
I’m very sorry for not picking up on this. Are you still having trouble with this?
Please contact me directly (You can leave a message via my contact form on http://saimonmoore.net and then I’ll email you directly) and I’ll try and help you sort out your problems.
Regards,
Saimon
Will this work with capistrano 2.0 beta? ie v1.99.3? It didn’t seem to off hand when I run it on my system I get:
% cap capserverext_usage the task `capserverext_usage’ does not exist Exit 1
I’m hoping this information will only be useful for a short time.
nginx 0.6.6 fails at the ‘make install’ part because of a bug in the install script.
There’s a patch in the message at: http://article.gmane.org/gmane.comp.web.nginx.english/1294/match=sysconfdir
When you grab the patch from the web, however, you have to fix the @@ part, as it has gotten munged to <at><at>.
So I forgot to state clearly in my last comment that the need for a patch sorta defeats using capserverext until the patch gets rolled into an nginx release.
I’m writing up a workaround on my weblog: http://www.automatthew.com
Hi guys,
I’m about to integrate capservext with deprec 2 which will support capistrano 2. Until then hang in there.
Matthew, thanks for the low down….
Regards,
Saimon
Thanks for a cool gem … when can we expect the next version of this gem?
Solution for “method_missing” problem
I hit the same problem as others when trying to run
I debugged it using NetBeans. One solution is to add the following to your ~/.caprc file:
The instructions in the original blog post say to put that line in deploy.rb, but that didn’t work for me as it seems that capsitrano (1.4.1) trys to interpret the “capserverext_usage” command line argument before it actually looks at deploy.rb.
Hope this helps.
cheers, mick
A solution for Ari’s reported problem:
Ari wrote: > I am having issues with the gem myself as well… > > I am consistently getting: command “sudo chmod 766 > /usr/local/apache2/conf/httpd.conf” failed … > > regardless if I am not installing apache
I suspect the problem is cause by using an incompatible version of deprec. I am using deprec 1.9.0 and I hit this problem. The fix, if you want to continue with deprec 1.9.0, is to define the following:
The above overrides the definition of after_setup defined in deprec to stop it from invoking setup_servers which is the command that is trying to setup apache.
A possibly better alternative would be to change capserverext so that it defines setup_servers to setup nginx, rather than apache.
thanks for your server extensions
Thanks for this article and most generaly for this very amazing website :) this information most important for me.
Web Tasarımı, Really nice article imho. It’s very enlightening :)
I always follow your site thank you wish you continued success. Thank you.
super seite . schrägaufzug deinimmo hauslifte
danke für die info