Setting Up A Rails Development Environment Using Docker
11 Sep 2016
A couple of years ago I wrote about how to setup a local development environment using Vagrant. While Vagrant has worked out great, containerization with Docker has become all the rage. So I decided to take it for a spin to see what all the fuss is all about. I’m going to use an existing Rails app as my test subject.
Requirements
So before we get started, here’s what you’ll need to install:
Folder structure
Here’s an overview of the folder structure we’ll be creating. We’ll store our bundle, Postgres data and the web app itself on our host. We’ll also symlink the keys folder to our SSH keys: ln -s ~/.ssh/ keys
I’ll get more into the advantages of doing so further along in this post.
myapp-dev-box/
├── Dockerfile
├── docker-compose.yml
├── docker-sync.yml
├── myapp/
├── bundle/
├── pgdata/
└── keys/ -> ~/.ssh/
Dockerfile
Base
First thing we’ll do is create a Dockerfile which will be our starting point inside an empty dir called myapp-dev-box. Since this is for a Rails project, we’ll base this container off the official Ruby image on Docker Hub which you can think of as GitHub for containers.
FROM ruby:2.1.3
Dependencies
Next we’ll install some dependencies that we need for the web app. Notice we will run this as a single command. Reason being, Docker caches the state of the container after each command into an intermediate container to speed things up. You can read a bit more about how this works here.
So if we need to add or remove a package, it’s recommended to re-run the entire step including apt-get update to make sure the entire dependency tree is updated.
RUN apt-get update && apt-get install -y \
build-essential \
wget \
git-core \
libxml2 \
libxml2-dev \
libxslt1-dev \
nodejs \
imagemagick \
libmagickcore-dev \
libmagickwand-dev \
libpq-dev \
ffmpegthumbnailer \
&& rm -rf /var/lib/apt/lists/*
SSH Keys
Now let’s create a folder for our app and a folder to store the SSH keys. The SSH keys are needed to checkout private repositories as part of the bundle install step inside the container.
Alternatively, you can make use of a build flow tool like Habitus to securely share a common set of keys and destroy them later in the build process. You can read more about it here. It supports many different complex build flows making it ideal for production use. Although it adds more complexity than we need just for a development environment so I’ve dediced against using it here.
You can also always create a separate set of SSH keys (without password) and place them in the same folder as the Dockerfile to be used within the container. Although this approach is a lot less secure as those keys would essentially become part of the container cache and could be exploited if someone gets hold of the image history. I wouldn’t recommend it.
Feel free to skip this step altogether if your Gemfile doesn’t reference any private repositories.
We’ll also add Github and BitBucket domains to the known hosts file to avoid first connection host confirmation during the build process.
RUN mkdir /myapp
RUN mkdir -p /root/.ssh/
WORKDIR /myapp
RUN ssh-keyscan -H github.com >> ~/.ssh/known_hosts
RUN ssh-keyscan -H bitbucket.org >> ~/.ssh/known_hosts
Bundle
Generally, we would copy Gemfile.* and run bundle install as a separate step into our Dockerfile so it can be run once and cached during subsequent runs.
Although that has some downsides especially for a development environment. First and foremost, the bundle would have to be rebuilt from scratch every time the Gemfile is changed which could get frustrating if it’s changed frequently.
Since I was primarily focused on a development environment and wanted to make this setup process as frictionless as possible for new devs, I decided to set it up in a way where the state of our bundle can be persisted after running it once even after we shut down the container and start it back up, just like it would on a VM or a local machine. And it would utilize the same SSH keys that are already present on the developer’s machine.
To make that happen, we will go ahead and point the GEM_HOME to a root folder called bundle which will be synced from the host. We’ll also update the bundle configuration to point to that path.
ENV GEM_HOME /bundle
ENV PATH $GEM_HOME/bin:$PATH
RUN gem install mailcatcher
RUN gem install bundler -v '1.10.6' \
&& bundle config --global path "$GEM_HOME" \
&& bundle config --global bin "$GEM_HOME/bin"
docker-compose.yml
Now that we have setup our base app container, it’s time to build and link a couple of supporting containers to run our app. Docker Toolbox includes a great tool called docker-compose (previously known as fig) to help us do just that.
Let’s start by defining which services we want to run, we’ll split our app into a db, redis, web and job services. For db and redis services, we’ll use the official Postgres and Redis images provided by Docker Hub without any custom changes. For job and web services, we’ll instruct it to build the image from the Dockerfile which we just created in the current directory in the previous section.
version: '2'
services:
db:
image: postgres
volumes:
- ./pgdata:/pgdata
environment:
POSTGRES_DB: myapp_development
POSTGRES_USER: postgres
POSTGRES_PASSWORD: password4postgres
PGDATA: /pgdata
redis:
image: redis
web:
build: .
command: bundle exec rails server --port 3000 --binding 0.0.0.0
volumes_from:
- container:myapp-web-sync:rw
- container:myapp-bundle-sync:rw
volumes:
- ./keys:/root/.ssh/
ports:
- "3000:3000"
environment:
REDIS_URL: redis://redis:6379
links:
- db
- redis
job:
build: .
command: bundle exec sidekiq
volumes_from:
- container:myapp-web-sync:rw
- container:myapp-bundle-sync:rw
volumes:
- ./keys:/root/.ssh/
ports:
- "6379:6379"
environment:
REDIS_URL: redis://redis:6379
links:
- db
- redis
volumes:
myapp-web-sync:
external: true
myapp-bundle-sync:
external: true
Most of the configuration under each service instance is pretty self-explanatory but there are a couple of interesting things to note. First, under the web and db instance, you’ll notice we have a links section which tells docker compose that these two services depend on the db and redis services. So docker compose will be sure to start the linked services before it starts web or db.
Second, you’ll notice we have a volumes and volumes_from sections. volumes is the native docker syntax for mounting a directory from the host as a data volume. While this is super useful, it tends to be very slow! So for now, we are mostly going to limit using it for sharing the database and the SSH keys. For things that are most disk I/O intensive, we’ll define external volumes in volumes_from section which will utilize a gem by Eugen Mayer called docker-sync. It will give us the ability to use rsync or unison which should significantly boost performance.
For that, we’ll need to define yet another configuration file in which we will define what myapp-web-sync and myapp-bundle-sync volumes will do. As their name suggest, we’ll use each of them to sync our web project files and the bundle respectively. Note, each sync project will have to have it’s own unique port.
docker-sync.yml
syncs:
myapp-web-sync:
src: './myapp'
dest: '/myapp'
sync_host_ip: 'localhost'
sync_host_port: 10872
myapp-bundle-sync:
src: './bundle'
dest: '/bundle'
sync_host_ip: 'localhost'
sync_host_port: 10873
Showtime
Assuming you have your web project cloned into myapp-dev-box/myapp let’s go through the following steps:
Install the bundle
host $ docker-compose run web bundle install
Note we will only have to do this once, since we have the bundle state shared between our app and job services using a docker-sync volume. We only need to re-run this if/when the Gemfile changes.
Run the migrations
host $ docker-compose run web rake db:migrate
Again like the previous command, this will only need to be run initially and when there are changes thereafter since the state of the database is persisted on the host as well.
Start
This command is a helper which basically starts the sync service like docker-sync start and then starts your compose stack like docker-compose up in one single step.
host $ docker-sync-stack start
If everything was configured correctly, you should now be able to access your app on http://localhost:3000
Stop
Once you are done, you can call another helper which basically stops the sync-service like docker-sync clean and also remove the application stack like docker-compose down
host $ docker-sync-stack clean
That’s a wrap. Let me know if you run into any unexpected issues.