11 Jun 2011
Deploying With Bundler Notes
I gave a talk at RailsConf 2011 about deploying with Bundler. These are my notes for the talk. I’ve also posted my slides for the talk. Combined, they probably serve as a somewhat reasonable replacement for having seen the talk itself.
Bundler exists to make booting your application consistent and repeatable, guaranteed. It does this by having you write a Gemfile listing of all the gems that your application needs to boot. When it installs those gems, it records the exact version of every gem into the Gemfile.lock file. Other developers are guaranteed to have the same gems you do. Your production servers are guaranteed to have the same gems you do.
This is a big deal.
It seems kind of easy and obvious now in retrospect, but in the bad old days, it could take anywhere from hours to days to get a new developer or a new server up and running your app with all the right versions of the gems that you needed. That entire process has been boiled down to running bundle install and waiting for a minute or two.
So! You start developing a beautiful new application. At some point, someone else starts working on it with you, and they only have to run bundle install and then they can start developing, too. Later, you realize that you want to deploy this application to a server in production so that the entire world can see it. Fantastic.
Now that Ruby application servers all use Rack to communicate with your underlying application, Rails, Sinatra, and every other web framework can be treated identically for deployment. Just put the library you use into your Gemfile, and your app will have all the gems that it needs to run on the server.
I’m going to cover the various deployment options that are available to apps that use Bundler, starting with the simplest and easiest, and expanding to include the more unusual and more complicated scenarios at the end.
The most drop-dead simple thing that you can possibly do to get your bundled application up and running on a server on the public internet is to have someone else do it. If you use Heroku, you can git push heroku master, and if you use Engine Yard, you can run ey deploy. In either case, their infrastructure will take care of everything. They check and see that you have a Gemfile, they run Bundler with settings tuned for their specific environments, and then they boot your application and you can start using it right away.
The next level of complexity is running your own server, installing Phusion Passenger, and having Passenger manage each of your app server instances. It’s not that much more complicated, but this is the point at which you run into the first potential Bundler deployment issue: installing system gems requires root permissions.
You might be tempted to add sudo bundle install to your deployment script and call it a day, but that turns out to be a terrible idea. The crux of the problem is that bundle install saves configuration information that is specific to this application – what groups of gems to ignore (like :development or :test), where the gems are installed to, all of those sorts of things. If you run bundle install as root, that file is created and owned by root. That means the Passenger app server (which runs as the nobody user by default) can’t read or write to the bundle configuration.
Bundler contains an “emergency workaround” for this situation where Bundler will invoke sudo itself to install the gems. It’s a hacky workaround at best, though, because entering your sudo password isn’t scriptable. Even more awkwardly, many system administrators don’t give regular users the ability to sudo at all. This means installing to system gems is ruled out completely. The much more effective solution is to install the application’s gems into a path owned by the web server user. This is simple to do with the command bundle install --path vendor/bundle.
Okay, so now we have our gems installed on the server into an application-specific path. There is one potential issue remaining: what happens if a developer changes the Gemfile, forgets to run bundle install, and then tries to deploy? The deploy script will install the bundle on the production server, but the server will be running against gems that have never been tested at all. You can avoid the potential disaster by using bundle install --frozen. Frozen mode means that Bundler checks the Gemfile against the Gemfile.lock file, and refuses to install if they don’t match. The error message instructs you to run bundle install on your development machine, make sure everything works, and then check in a new Gemfile.lock with the versions that you are now sure work with your application.
Now your application works in production, you can deploy without superuser permissions, and you will never run with gems in production that haven’t been tested yet! Since this is the most common case, Bundler wraps all of this up into a single option that you can pass to the install command: bundle install --deployment.
But wait, you say, what if I use Unicorn or Rainbows or Mongrel or something like that? Well, it’s pretty simple as well. Just add the server to your Gemfile. When you want to run your server, use e.g. bundle exec unicorn. This will make sure that the exact version in your Gemfile will be started, regardless of other versions on your system.
The final complication that we sometimes see while deploying to production environments is servers that have been heavily firewalled, and cannot connect to rubygems.org to download gems for installation at deploy-time. Sometimes it’s possible to use a proxy server, but when even that isn’t an option, Bundler has your back. Run the command bundle pack to download the .gem file for every gem that your application needs into the vendor/cache directory. When you run bundle install later, Bundler is smart enough to automatically look in vendor/cache. It will install your bundle using those gems as a source, and completely avoid contacting rubygems.org, allowing you to deploy into the most restrictive server setup.
At this point, you’re probably thinking that this sounds like a lot to remember, and a lot of effort to get set up. Fortunately, Bundler even goes beyond providing the –deployment flag. It integrates directly with Capistrano and Vlad, the two most popular deployment systems. Adding Bundler to a working deploy setup is incredibly easy. With Capistrano, just add require 'bundler/capistrano' to your deploy.rb, and everything will work. With Vlad, just require 'bundler/vlad' in your deploy.rb, and make sure that your deploy task also runs the task vlad:bundle:install. That’s it !If you need to adjust Bundler options, like the location the gems are installed to, or the list of groups that shouldn’t be installed, you can. Just set some variables in your deploy script before Bundler runs. The details of how to configure everything are available at gembundler.com/deploying.
The next feature release of Bundler, version 1.1, will include several features that make deploying easier in specific scenarios. The biggest change, which everyone will be able to use, is that Bundler no longer downloads the entire Rubygems source index before installing. Nick Quaranto has added a new API to rubygems.org that allows Bundler to download only the gem specs that it needs to install your Gemfile. This is a great improvement. In my tests, it cut down the time to install a small Gemfile from 30 seconds down to under 10 seconds. Another new feature that can help with deployments is –standalone mode. When you install your bundle with standalone mode enabled, Bundler is no longer needed at the system level. Your application becomes self-contained, and is able to load its installed gems itself. If you want to try out these features now, and help us finish testing them, you can get a prerelease of Bundler 1.1 today by running gem install Bundler --pre.
So that’s how to deploy your application and all its dependencies reliably and repeatably using Bundler.