As developers, we work in a world of abstractions. Each piece of technology that we use is built upon layers and layers of other systems. This allows us to build software more efficiently. Why recreate what has already been created? Frameworks such as Ruby on Rails, Django, and Node.js were created to abstract away the painful parts of web development. They set up standards and best practices to build web applications. We have all of these amazing tools to build applications, but for a long time, we still struggled to easily deploy and scale them.
Then came Heroku. Heroku is the Platform as a Service that changed how we deploy web applications. Heroku automates the pain points of deploying code and has established best practices to build applications that need to scale. We no longer need to deal with the pain of setting up load balancers, patching servers, or scrambling to scale up our infrastructure in response to high traffic.
Heroku is easy to get started with and use. For developers who are new to the Heroku way, some of the conventions might go against the ones you use to deploy code. Building scalable and highly performant web applications requires us to think about the design of our applications differently.
In this book, we will learn how to set up our applications for success on Heroku. We'll learn about what is happening behind the scenes and use this knowledge to make our applications fast and reliable from the very beginning.
In this chapter, we will cover the very basics of deploying to Heroku. We will practice deploying three different open source projects, each using a different language and framework. Through this practice, we'll learn the essentials for deploying any application to Heroku and become more confident when it is time for us to deploy our own code.
Heroku applications are created and administered from the command line. To get started with Heroku, we need to install the Heroku Toolbelt. It contains everything we need to create and deploy new applications.
The toolbelt is an installer for three command-line tools:
In this recipe, we will install the Heroku Toolbelt, making sure our machine is set up to use the Heroku CLI. We'll also be briefly introduced to the Heroku CLI. We'll learn about Git and Foreman later in the chapter.
Note
If you already have the Heroku Toolbelt, it might be beneficial to go through the following steps again to ensure that the latest version is installed.
First, we need to create a Heroku account with the following steps:
Let's go to www.heroku.com and create an account if we do not already have one.
Next, let's install the Heroku Toolbelt. Specific download and installation instructions are available at https://toolbelt.heroku.com/ for Mac, Windows, and Linux.
Once the Heroku Toolbelt is installed, we can verify that everything is working by opening up a terminal and running the following command:
$ heroku --version heroku-toolbelt/3.11.1 (x86_64-darwin10.8.0) ruby/1.9.3
We should see the version of the Heroku Toolbelt we are using printed to the console.
Now that we have the Heroku Toolbelt installed, let's log in to our account via the CLI and authorize our computer by uploading our public key using the following steps:
Let's log in by opening up a terminal and running the following command:
$ heroku login Username: youremail@example.com Password (typing will be hidden): Could not find an existing public key.Would you like to generate one? [Yn]Generating new SSH public key.Uploading ssh public key /Users/mc/.ssh/id_rsa.pub
If we do not have an existing public key, the Heroku CLI will provide us with instructions on how to create one here. This key will be uploaded to Heroku's servers and used for authentication whenever we push new code to our applications.
We can ensure that we are authenticated with the
auth:whoami
command. If logged in successfully, it will print our e-mail address:$ heroku auth:whoami youremail@example.com
Finally, we should go to the Heroku dashboard and verify our account by adding a credit card. Having a verified account will allow us to scale our applications and install add-ons (https://dashboard.heroku.com/account).
The Heroku Toolbelt installs all the necessary tools to create and administer our Heroku applications. It's essential for us to become comfortable with Heroku's command-line tools. Even though many tasks can be completed on Heroku's website, not everything is available through the dashboard. For full control over our applications, we have to use the CLI.
Ever wondered how Heroku keeps us logged in to the CLI? During the login process, Heroku stores an API key for our account in our .netrc
file. The .netrc
file is a dotfile that lives in our home directory. It's a common file that applications use to store credentials to log in to remote hosts. The API key stored in this file is used for subsequent logins and keeps us logged in to our Heroku account. If we open our .netrc
file, we'll see an entry for api.heroku.com. If we ever run the auth:logout
command, it deletes the entry from our .netrc
file, thus logging us out.
Interested in seeing the source code for the Heroku CLI? It's open source; take a look at https://github.com/heroku/heroku.
Git is one of the most popular version-control systems used in software development. It allows teams of developers to work on the same code base without overwriting each other's changes. Git is a core piece of the Heroku platform, and having a basic understanding of how it works is a prerequisite to deploy code to Heroku.
In this recipe, we'll learn enough about Git to deploy code to Heroku.
Tip
Unfamiliar with the command line? There is a great resource to quickly get up to speed on the basics at http://cli.learncodethehardway.org/book/.
Git allows us to track every change to our source code. This makes it simple to go back in time and revert changes as well as view the history of our code. Let's open up a terminal to get started by performing the following steps:
If we've never used Git before, we'll want to tell it our name and e-mail. These will be used to identify us as the author in all our commits:
$ git config --global user.name 'First and Last name here' $ git config --global user.email 'yourname@example.com'
Now, let's create a new directory to practice with:
$ mkdir LearningGit $ cd LearningGit
In our new directory, we'll need to initialize a new Git repository:
$ git init Initialized empty Git repository in /home/mc/LearningGit/.git/
Now, let's create a new file in our project using
touch
:$ touch new_file.txt
We can use the
status
command to check whether this file is currently untracked by Git:$ git status On branch master Initial commit Untracked files: (use "git add <file>..." to include in what will be committed) new_file.txt nothing added to commit but untracked files present (use "git add" to track)
We need to explicitly tell Git to track the file using the
add
command:$ git add new_file.txt
Our file is now created and being tracked by Git, but it is not yet committed to our repository's history. Let's commit it now:
$ git commit -m "Adding new_file to Git" [master (root-commit) b79ca0b] Adding new_file to Git 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 LearningGit/new_file.txt
To view the history of our Git repository, we can use the
log
command. Use the q key to escape from viewing logs. Use the arrow keys to scroll:$ git log commit b79ca0b7c7671789cc8359fe43e1144af835c2d1 Author: Mike Coutermarsh <coutermarsh.mike@gmail.com> Date: Mon Mar 3 20:31:47 2014 -0500 Adding new_file to Git
We now know the essential commands to use Git from the command line. We're able to create a new repository, add and track the files, as well as view our repositories' status and history. These are just a limited set of Git's commands, but they are enough for us to get started with Heroku. We'll be using Git throughout this book to track and push our code. We'll build on the skills we learned here in the later recipes.
An easy way to understand Git commits is to think of each commit as a photograph of our project files. Each time we make a commit, Git takes a photo of our files. We are then able to view each commit to see exactly what changed in our project. If we need to go back in time, we can just revert to the state our code was in before the commit was made.
Git is a distributed version-control system. This means that there is no need for us to be online or connected to a server to use it. In Git, there is the concept of remotes. These are other instances of the Git repository on other servers or machines. If we have a Git repository on GitHub, then this is known as a remote. Having remotes is useful because they act as a central place where all the members of a team can push their changes and retrieve the changes made by others. Having the full Git repository on multiple machines and servers also makes it much more fault tolerant. If anything were to happen to our remote repository, each team member would still have a local copy that can be used to create a new remote elsewhere.
Remotes are an important concept for us to understand because they are the basis to deploy to Heroku. Each Heroku deploy is simply the push of a local Git repository to a remote provided by Heroku. It's amazingly simple; we'll learn about it in detail later in this chapter.
Using and learning about Git from the command line can be challenging. Luckily, there are a few easy-to-use desktop apps that make using Git really simple:
GitHub for Mac available at http://mac.github.com/
GitHub for Windows available at http://windows.github.com/
SourceTree (Mac or Windows) available at http://www.sourcetreeapp.com/
Tower (Mac) available at http://www.git-tower.com/
SmartGit (Linux) available at http://www.syntevo.com/smartgithg/
Grab the Git cheatsheet at http://cheat.errtheblog.com/s/git
For an interactive tutorial on using Git, check out TryGit at https://try.github.io
Packt has a great beginner's guide to using Git, Git: Version Control for Everyone, Ravishankar Somasundaram
It's time for us to deploy our first application to Heroku. If you've deployed applications to Heroku before, this will be a good review. If this is your first time, you'll be learning the common steps taken to deploy any application to Heroku.
The creators of Heroku have experience in deploying and scaling countless web applications. They've seen it all. From their experiences, they have created a methodology known as the Twelve-Factor app. The Twelve-Factor app is a set of 12 rules that will guide us to build an application that is easy to deploy, easy to maintain, and, most importantly, easy to scale on a cloud platform. No matter what language or framework we are using to build our application, these twelve rules will apply.
Note
Visit http://12factor.net/ to learn more about the Twelve-Factor app.
Ruby on Rails follows most of the twelve rules out of the box. This makes it a good place to start when learning how to deploy to Heroku, because it requires minor configuration changes. In this recipe, we will be deploying Refinery, a popular open source Ruby on Rails Content Management System (CMS).
To run this application locally, we need to have Ruby Version 2.1.3 installed by performing the following steps:
One of the easiest ways to install Ruby is to use Ruby Version Manager (RVM). We can find the latest installation instructions for RVM at http://rvm.io/rvm/install.
Once RVM is installed, we can run the following command in a terminal to install Ruby 2.1.3:
$ rvm install 2.1.3
We'll use Bundler to manage and install our applications' dependencies. Let's make sure we have the latest version installed by running the following command:
$ gem install bundler
This application also uses a Postgres database. We'll be using Postgres frequently throughout the book; if we do not have it installed on our machine, now is a good time to get it set up:
For OS X, the easiest way to install Postgres is via the Postgres app available at http://postgresapp.com/
For Windows and Linux, see the Postgres download page at http://www.postgresql.org/download/
We'll set up and deploy our application from the command line. Let's open a terminal to get started using the following steps:
First, we need to download the source code for our sample app from GitHub. We can do this using
git clone
:$ git clone https://github.com/mscoutermarsh/refinery_heroku.git
Now, let's navigate to our new directory and create a new Heroku app. Creating a new app will also add a new
heroku
remote to our Git repository. This remote is where we will be soon pushing our code for deployment:$ cd refinery_heroku $ heroku apps:create Creating cryptic-chamber-6830... done, stack is cedar http://cryptic-chamber-6830.herokuapp.com/ | git@heroku.com:cryptic-chamber-6830.git Git remote heroku added
We will tell Heroku how to run our app with a Procfile. In the root directory of our new app, we'll create a new Procfile to tell Heroku how to start up our web service. Let's create the file using the
touch
command:$ touch Procfile
Now, let's open our new Procfile and add the following line. This will tell Heroku how to start our web server process:
web: bundle exec unicorn -p $PORT -c ./config/unicorn.rb
We can now commit these changes to Git:
$ git add Procfile $ git commit -m 'Adding Procfile for Heroku'
Next, let's add the Twelve-Factor app gem to our application. It will automatically configure our application's logging and assets to work correctly with Heroku. Let's open our application's Gemfile and add the following line:
gem 'rails_12factor', group: :production
As we've added a new gem, we'll want to run
bundle install
to update our application's dependencies:$ bundle install
Note
To learn more about Bundler, take a look at http://bundler.io/.
We'll need to make another commit with our latest changes:
$ git commit -am 'Adding 12 factor gem'
This application uses a Postgres database. We'll need to add Postgres to our Heroku application. Let's do this now:
$ heroku addons:add heroku-postgresql:dev ----> Adding heroku-postgresql:dev to cryptic-chamber-6830... done, v3 (free) Attached as HEROKU_POSTGRESQL_GOLD_URL Database has been created and is available ! This database is empty. If upgrading, you can transfer ! data from another database with pgbackups:restore.
Note
The Heroku CLI knows which application to add the database to, because our current Git repository has a
heroku
remote that points to this Heroku application. If we wanted to run the command for a different application, we could append--app application_name
to the end of the command. This will be very useful once we have multiple applications deployed to Heroku.Ruby on Rails uses an environment variable to connect to the database. We can set this now using the
promote
command. This will assign our new database's credentials to theDATABASE_URL
environment variable.We'll use the database name given to us in the previous command as the argument in this command:
$ heroku pg:promote HEROKU_POSTGRESQL_GOLD -----> Promoting HEROKU_POSTGRESQL_GOLD to DATABASE_URL... done
We're now ready to push our code to Heroku. We'll do this using Git's
push
command. We'll need to specify theheroku
remote and our master Git branch:$ git push heroku master Initializing repository, done. Counting objects: 92, done. Delta compression using up to 4 threads. Compressing objects: 100% (79/79), done. Writing objects: 100% (92/92), 35.83 KiB | 0 bytes/s, done. Total 92 (delta 11), reused 0 (delta 0) … -----> Discovering process types Procfile declares types -> web Default types for Ruby -> console, rake, worker -----> Compressing... done, 37.1MB -----> Launching... done, v9 http://cryptic-chamber-6830.herokuapp.com/ deployed to Heroku To git@heroku.com:cryptic-chamber-6830.git 46345bc..583680c master -> master
Now that our application's code is on Heroku, we need to completely set up our database by running migrations and seeding it with some data:
$ heroku run rake db:migrate $ heroku run rake db:seed
Our app is now ready to use! We can quickly launch a browser and view it with the
open
command:$ heroku open
Once our application is open, let's go to Refinery in the browser to register a user and start using the Refinery CMS.
In deploying this Rails application, we were introduced to a couple of Heroku concepts that we will be using when deploying any application to Heroku. Let's dig into them a little deeper now.
Each Heroku application should have a special file in its root directory that defines each of the processes required to run the application. This file is known as a Procfile. If we forget to include a Procfile, Heroku will try to guess what process we want to run. It's better for us if we're explicit about exactly what Heroku should do.
In this recipe, we created a Procfile that told Heroku what command to run to start our web server. The Procfile can be used for more than just web processes. In applications that also have processes running in the background, the Procfile is where we'd define how to start them. On Heroku, we can only have one web process. This is the only process that Heroku will direct web traffic to. Other processes will not be able to receive web traffic. If we find a use case where we need more than one type of web process running, this is a good indicator that we should have multiple Heroku applications.
When we ran the db:promote
command, we added an environment variable to our application to store our database's credentials. This is good practice and follows the conventions of the Twelve-Factor app. We should never store credentials for any service in our Git repository. It makes our credentials less secure, because they are then accessible to anyone who works on our code. It also makes them more difficult to change, because any change will require another deploy. Credentials tend to be very environment specific; having them as part of a Heroku application rather than our code base makes our application more portable. With all this being said, the key is to remember that when building any application for deployment on Heroku, we should build the ability to load credentials from an environment variable into our code.
When we pushed our Git repository to Heroku, the slug compilation process began. Heroku takes our Git repository, detects the language and the framework used, and begins to build a slug in our application. A Heroku slug is a copy of our application that is ready to be deployed on Heroku's servers at a moment's notice. For a Rails application, this means that all of the application's Gems have been installed, and its assets have been compiled. Heroku also removes any unnecessary files from our Git repository to make the slug as lightweight as possible. We can think of it as a snapshot of our production-ready application. Heroku hangs on to each slug it creates, making it easy for us to roll back to a previous slug if needed.
Find out more about the Refinery CMS at http://refinerycms.com/
Find out more about Foreman and the Procfile at http://ddollar.github.io/foreman/
To learn more about deploying Ruby applications on Heroku, take a look at Chapter 7, Optimizing Ruby Server Performance on Heroku
Heroku is a polyglot platform that can host applications built in many different languages and frameworks. In this recipe, we will learn how to deploy Ghost, a popular open source blogging platform built on Node.js.
We'll build on what we learned in the previous recipe, Deploying a Rails application to Heroku. Here, we'll see that there are a lot of similarities between deploying the two different applications. The process to deploy any application to Heroku is very similar to the previous recipe, irrespective of the language or framework in which it is written.
We'll be setting up and deploying Ghost from the command line. Let's open up a terminal to begin with by performing the following steps:
First, we'll need to download the Ghost source code from GitHub. We'll clone an existing Ghost Git repository that's been set up to run on Heroku:
$ git clone https://github.com/mscoutermarsh/ghost_heroku.git Cloning into 'ghost_heroku'... remote: Counting objects: 16411, done. remote: Compressing objects: 100% (7480/7480), done. remote: Total 16411 (delta 8481), reused 16381 (delta 8455) Receiving objects: 100% (16411/16411), 8.55 MiB | 1.75 MiB/s, done. Resolving deltas: 100% (8481/8481), done. Checking connectivity... done
Let's navigate to the new
ghost_heroku
directory and create a new Heroku application:$ cd ghost_heroku $ heroku apps:create Creating fast-coast-3773... done, region is us http://fast-coast-3773.herokuapp.com/ | git@heroku.com:fast-coast-3773.git Git remote heroku added
The configuration for our application is in the
config.js
file. Let's open the file now and update the default production URL to reflect our new Heroku application's URL (given to us from Heroku in the previous step):production: { url: 'http://my-ghost-blog.com', mail: {},
Commit the changes to Git:
$ git commit -am 'Updating production URL config' [master cd0ec0e] Updating production URL config 1 file changed, 1 insertion(+), 1 deletion(-)
We'll use Postgres as our database for Ghost. We can create a new Postgres database now:
$ heroku addons:add heroku-postgresql:dev Adding heroku-postgresql:dev on fast-coast-3773... done, v3 (free) Attached as HEROKU_POSTGRESQL_SILVER_URL Database has been created and is available ! This database is empty. If upgrading, you can transfer ! data from another database with pgbackups:restore. Use `heroku addons:docs heroku-postgresql:dev` to view documentation.
We'll need to set up our new database as the primary one for our application by promoting it. In the previous command, Heroku gave us a unique database name. It follows the format of
HEROKU_POSTGRESQL_COLOR_URL
. We'll use that name as the argument for the next command:$ heroku pg:promote HEROKU_POSTGRESQL_SILVER_URL Promoting HEROKU_POSTGRESQL_SILVER_URL to DATABASE_URL... done
Next, we'll need to set a configuration variable to let Node know which environment it's running on. Let's set it to
production
:$ heroku config:set NODE_ENV=production
Our application now has everything it needs to be deployed. Let's push our repository to Heroku to deploy our code:
$ git push heroku master
Once the build process is complete, our blog will be up and running. We can now launch a browser from the command line to see the following screen:
$ heroku open
To access Ghost's admin panel, add
/ghost
at the end of the URL. We can then create an account and start playing with our new blog.
Heroku uses a unique term for its web servers; it calls them dynos. A dyno starts out as a plain Ubuntu Linux web server. It's during the initial push and slug-compilation process that Heroku auto detects the type of application we are trying to deploy and installs the software necessary for it to run.
Heroku uses an ephemeral filesystem. This means that any files written to disk after the creation of the slug will not be persisted beyond the life of the dyno. All Heroku dynos are cycled every 24 hours. There is a good reason for this restriction: it allows our application to scale. If we were to allow file storage on Heroku dynos, we'd have to replicate the file across every dyno.
When writing blog posts with Ghost, we'll see that we are able to upload images. The problem with this feature is that Ghost currently stores these images on the web server. This won't work for us on Heroku; we'll have to use a file store outside Heroku, such as Amazon S3 or Dropbox.
See ghost.org to learn more about Ghost
Check out the Ghost project on GitHub at https://github.com/tryghost/Ghost
Heroku's killer feature has always been its ability to easily scale up and scale out our applications as our user base grows. This frees us from the pains of setting up and managing load balancers and additional servers on our own. In this recipe, we will be introduced to Heroku's dynos and workers as well as learn how to scale them both up and out as our applications grow.
Dyno is the term Heroku uses for its web servers. A dyno is simply a virtual private server that runs our application and responds to web requests.
Heroku has an additional class of servers known as workers. These are identical to dynos, with the exception that they do not serve web requests.
Both dynos and workers are available in three different sizes: 1X, 2X, and PX. The default size is 1X; this is a small virtual server with 512 MB of RAM. These are large enough to run most web applications. However, if we find that our application is constrained by the limited memory or CPU size, we can scale up our dynos up to 2X, which provides 1024 MB of RAM and twice as much computing power.
Note
If our application has only a single 1X dyno running, it will shut down after an hour of inactivity. To avoid this, we need to have at least two dynos running or use a single 2X dyno.
The largest process size is the PX or performance dyno. These are dedicated virtual servers that do not share resources with any other Heroku customers. They have 6 GB of RAM and 40 times the compute resources of the standard 1X-sized dyno. Performance dynos should only be considered for applications that have high memory and CPU requirements.
We'll use the Heroku CLI for this recipe. Let's open up a terminal and navigate to a directory with one of our existing Heroku applications and perform the following steps:
To view our currently running processes, we can use the
ps
command. It will show the type, the size, and exactly what's running:$ heroku ps === web (1X): `bundle exec unicorn -p $PORT -c ./config/unicorn.rb` web.1: up 2014/03/15 19:41:27 (~ 8s ago)
We currently have only one dyno running for this application. Let's scale it up to two; this will effectively double our application's capacity. Scaling processes are done with the
ps:scale
command:$ heroku ps:scale web=2 Scaling dynos... done, now running web at 2:1X.
The
scale
command is very flexible. If we want, we can scale both dynos and workers at the same time:$ heroku ps:scale web=2 worker=1 Scaling dynos... done, now running worker at 1:1X, web at 2:1X.
We can change the size of our dynos using
ps:resize
. Let's scale up our web dynos to 2X:$ heroku ps:resize web=2x Resizing and restarting the specified dynos... done web dynos now 2X ($0.10/dyno-hour)
We can also scale and change the size in the same command. Let's dial our dynos back down to one and adjust the size to 1X:
$ heroku ps:scale web=1:1x Scaling dynos... done, now running web at 1:1X.
To finish up, we can scale our workers back down to zero:
$ heroku ps:scale worker=0 Scaling dynos... done, now running worker at 0:1X.
Now that we have learned how to scale our applications, let's go a little more in depth to learn about the different types of Heroku dynos.
A dyno is simply a web server. When we create our application's Procfile, the web process that we define is what runs on our dynos. When a user visits our web application, their requests get sent to our dynos via Heroku's routing layer. The routing layer acts like a load balancer. It distributes our users' requests and monitors the health of our dynos. To handle more users, we can scale out our application by increasing the number of running dynos. This allows us to serve requests from more concurrent users. If we are currently running one dyno and adding another, we have theoretically doubled the amount of web requests that our application can respond do.
In our Procfile, any process other than web
will run on a worker. Workers are used to process background tasks such as sending out e-mails or generating PDFs. Any task that a user should not have to wait for is a good candidate that will run on a worker. For a Rails application, any background job (such as Resque or Sidekiq) will need to be run on a worker dyno. Workers can be scaled in exactly the same way as dynos. If our application has a large backlog of tasks that need to be completed, we can add additional workers to increase the number of tasks we can complete simultaneously.
To learn more about scaling, take a look at Chapter 6, Load Testing a Heroku Application