“Web servers are written in C, and if they’re not, they’re written in Java or C++, which are C derivatives, or Python or Ruby, which are implemented in C.”

– Rob Pike, co-creator of Go

The web server is a key component of any modern-day web framework. Expanding on the point made in the preceding quote by Rob Pike, the Cowboy web server, written in Erlang, is also in a way implemented in C. Cowboy is the default web server used by Phoenix, the ubiquitous web framework in Elixir.

In this chapter, we will not be learning C, unfortunately, but we will take a closer look at how a web server is designed. We will provide some background on how a web server is built and set up to communicate with a client using HyperText Markup Language (HTML).

We will also learn the fundamentals of how HTTP requests and responses work, including their anatomy. We will then learn how to construct an HTTP response and send it using a web server. Moreover, we will learn the fundamentals of web server architecture by examining the components of Cowboy. Lastly, we will learn ways to test a web server and measure its performance. Doing this will put us in a better position to build our own web server in the next chapter.

The following are the topics we will cover in this chapter:

• What is a web server?
• Fundamentals of client-server architecture
• Fundamentals of HTTP
• How an HTTP server works
• Using Cowboy to build a web server
• Using dynamic routes with Cowboy
• Serving HTML
• Testing the web server

Going through these topics and looking at Cowboy will allow us to build our own HTTP server in Chapter 2.

The best way to work through this chapter is by following along with the code on your computer. So, having a computer with Elixir and Erlang ready to go would be ideal. I recommend using a version manager such as asdf to install Elixir 1.11.x and Erlang 23.2.x, to get similar results as the code written in the book. We will also be using an HTTP client such as cURL or Wget to make HTTP requests to our server, and a web browser to render HTML responses.

Although most of the code in this chapter is relatively simple, basic knowledge of Elixir and/or Erlang would also come in handy. It will allow you to get more out of this chapter while setting the foundation for other chapters.

Since most of this chapter isn’t coding, you can also choose to read without coding, but the same doesn’t apply to other chapters.

The code examples for this chapter can be found at https://github.com/PacktPublishing/Build-Your-Own-Web-Framework-in-Elixir/tree/main/chapter_01

A web server is an entity that delivers the content of a site to the end user. A web server is typically a long-running process, listening for requests on a port, and upon receiving a request, the web server responds with a document. This way of communication is standardized by the Transmission Control Protocol/Internet Protocol (TCP/IP) model, which provides a set of communication protocols used by any communication network. There are other layers of standardization, such as the HyperText Transfer Protocol (HTTP) and File Transfer Protocol (FTP), which dictate standards of communication at the application layer based on different applications such as web applications in the case of HTTP, and file transfer applications in the case of FTP, while still using TCP/IP at the network layer. In this book, we will be primarily focusing on a web server using HTTP at the application layer.

Example HTTP server

If you have Python 3 installed on your machine (you likely do), you can quickly spin up a web server that serves a static HTML document by creating an index.html file in a new directory and running a simple HTTP Python server. Here are the commands:

$ mkdir test-server && cd test-server && touch index.html

$ echo "<h1>Hello World</h1>" > index.html

$ python -m http.server 8080

Serving HTTP on 0.0.0.0 port 8080 (http://0.0.0.0:8080/) . . .

If you are on Python 2, replace http.server with SimpleHTTPServer.

Now, once you navigate to http://localhost:8080/ on your web browser, you should see "Hello World" as the response. You should also be able to see the server logs when you navigate back to the terminal.

To stop the HTTP server, press Ctrl + C.

The primary goal of web servers is to respond to a client’s request with documents in the form of HTML or JSON. These days, however, web servers do much more than that. Some have analytical features, such as an Admin UI, and some have the ability to generate dynamic documents. For example, Phoenix’s web server has both of those features. Now that we know what a web server is, let’s learn about how it is used with the client-server architecture.

In the context of HTTP servers, clients generally mean the web browsers that enable end users to read the information being served, whereas servers mean long-running processes that serve information in the form of documents to those clients. These documents are most commonly written in HTML and are used as a means of communication between the client and the server. Clients are responsible for enabling the end user to send a request to the server and display the response from the server. Browsers allow the users to retrieve and display information without requiring any knowledge of HTML or web servers, by just providing an address (the URL).

At a given time, many clients can access a server’s information. This puts the burden of scaling on the servers as they need to be designed with the ability to respond to multiple requests within an acceptable period of time. Now that we understand a web server’s primary goal, let’s move on to the protocol that enables communication between web servers: HTTP.

HTTP is an application layer protocol that provides communication standards between clients (such as web browsers) and web servers. This standardization helps browsers and servers talk to each other as long as the request and the response follow a specific format.

An HTTP request is a text document with four elements:

• Request line: This line contains the HTTP method, the resource requested (URI), and the HTTP version being used for the request. The HTTP method generally symbolizes the intended action being performed on the requested resource. For example, GET is used to retrieve resource information, whereas POST is used to send new resource information as a form.
• Request headers: The next group of lines contains headers, which contain information about the request and the client. These headers are usually used for authorization, determining the type of request or resource, storing web browser information, and so on.
• Line break: This indicates the end of the request headers.
• Request body (optional): If present, this information contains data to be passed to the server. This is generally used to submit domain-specific forms.

Here’s an example of an HTTP request document:

GET / HTTP/1.1
Host: localhost:8080
User-Agent: curl/7.75.0
Accept: */*
Body of the request

As you can see, the preceding request was made with the GET method to localhost:8080 with the body, Body of the request.

Similarly, an HTTP response contains four elements:

• Response status line: This line consists of the HTTP version, a status code, and a reason phrase, which generally corresponds to the status code. Status codes are three-digit integers that help us categorize responses. For example, 2XX status codes are used for a successful response, whereas 4XX status codes are used for errors due to the request.
• Response headers: Just like request headers, this group of lines contains information about the response and the server’s state. These headers are usually used to show the size of the response body, server type, date/time of the response, and so on.
• Line break: This indicates the end of response headers.
• Response body (optional): If present, this section contains the information being relayed to the client.

The following is an example of an HTTP response document:

HTTP/1.1 404 Not Found
content-length: 13
content-type: text/html
server: Cowboy
404 Not found

The preceding response is an example of a 404 (Not found) response. Notice that content-length shows the number of characters present in the response body.

Now that we know how HTTP facilitates client-server communication, it is time to build a web server using Cowboy.

Cowboy is a minimal and fast HTTP web server written in Erlang. It supports several modern standards, such as HTTP/2, HTTP/1.1, and WebSocket, for example. On top of that, it also has several introspective capabilities, thus enabling easier development and debugging. Cowboy has a very well-written and well-documented code base, with a highly extendable design, which is why it is the default web server for the Phoenix framework.

Cowboy uses Ranch, a TCP socket accepter, to create a new TCP connection, on top of which it uses its router to match a request to a handler. Routers and handlers are middleware that are part of Cowboy. Upon receiving a request, Cowboy creates a stream, which is further handled by a stream handler. Cowboy has a built-in configuration that handles a stream of requests using :cowboy_stream_h. This module spins up a new Erlang process for every request that is made to the router.

Cowboy also sets up one process per TCP connection. This also allows Cowboy to be compliant with HTTP/2, which requires concurrent requests. Once a request is served, the Erlang process is killed without any need for cleanup.

The following figure shows the Cowboy request/response cycle:

Figure 1.1 – Cowboy request/response cycle

As you can see in Figure 1.1, when a client makes a request, Ranch first converts it into a stream, which is further handled by the router and handler middleware in Cowboy. Traditionally, a response is sent either by the router or the handler. For example, a handler could handle a request and send a response or, if no handler is present for a route, the router could also send a 404 response.

Cowboy also generates a few response headers, as we will see in the next section, where we build and test a Cowboy-powered web application.

In this section, we will take a look at some of the individual components of the Cowboy web server and use them to build a simple web application in Elixir.

Creating a new Mix project

Let’s start by creating a new Mix project by entering the following in your terminal:

$ mix new cowboy_example --sup

What is Mix?

Mix is a build tool written in Elixir. Its purpose is to bundle all the dependencies required by a project and provide an interface to run tasks that rely on the application environment. If you’re familiar with the Ruby world, you can think of Mix as a combination of Rake and Bundler.

Passing the --sup option to the mix new command allows us to create a Mix project with a supervision tree. A supervision tree (or a supervisor) is a process that simply monitors other processes and is responsible for automatically restarting any process within the tree if it fails or crashes. We will be using the supervision tree in this application to start and supervise our web server process to make sure it is started when the application is started and to ensure that it keeps running.

Now, we will add Cowboy as a dependency to our project by adding it to the mix.exs file’s dependencies:

mix.exs

defmodule CowboyExample.MixProject do
  # ...
  defp deps do
    [
      {:cowboy, "~> 2.8"}
    ]
  end
end

Follow it up by running mix deps.get from the project’s root directory, which should fetch Cowboy as a dependency.

Adding a handler and router to Cowboy

Now that we have added Cowboy, it is time to configure it to listen on a port. We will be using two functions to accomplish that:

• :cowboy_router.compile/1: This function is responsible for defining a set of requested hosts, paths, and parameters to a dedicated request handler. This function also generates a set of rules, known as dispatch rules, to use those handlers.
• :cowboy.start_clear/3: This function is responsible for starting a listener process on a TCP channel. It takes a listener name (an atom), transport options such as the TCP port, and protocol options such as the dispatch rules generated using the :cowboy_router.compile/1 function.

Now, let us use these functions to write an HTTP server. We can start by creating a new module to house our new HTTP server:

lib/cowboy_example/server.ex

defmodule CowboyExample.Server do
  @moduledoc """
  This module defines a cowboy HTTP server and starts it
  on a port
  """
  @doc """
  This function starts a Cowboy server on the given port.
  Routes for the server are defined in CowboyExample.Router
  """
  def start(port) do
    routes = CowboyExample.Router.routes()
    dispatch_rules =
      :cowboy_router.compile(routes)
    {:ok, _pid} =
      :cowboy.start_clear(
        :listener,
        [{:port, port}],
        %{env: %{dispatch: dispatch_rules}}
      )
  end
end

The preceding function starts a Cowboy server that listens to HTTP requests at the given port. By pattern matching on {:ok, _pid}, we’re making sure that :cowboy.start_clear/3 doesn’t fail silently.

We can try to start our web server by calling the CowboyExample.Server.start/1 function with a port. But, as you can see, we will also need to define the CowboyExample.Router router module. This module’s responsibility is to define routes that can be used to generate dispatch rules for our HTTP server. This can be done by storing all the routes, parameters, and handler tuples in the router module and passing them to the :cowboy_router.compile/1 call.

Let’s define the router module with the route for the root URL of the host (/):

lib/cowboy_example/router.ex

defmodule CowboyExample.Router do
  @moduledoc """
  This module defines all the routes, params and handlers.
  This module is also the handler module for the root
  route.
  """
  require Logger
  @doc """
  Returns the list of routes configured by this web server
  """
  def routes do
    [
      # For now, this module itself will handle root
      # requests
      {:_, [{"/", __MODULE__, []}]}
    ]
  end
end

We will also be using CowboyExample.Router itself as the handler for that route, which means we have to define the init/2 function, which takes the request and its initial state.

So, let us define the init/2 function:

lib/cowboy_example/router.ex

defmodule CowboyExample.Router do
  # ..
  @doc """
  This function handles the root route, logs the requests
  and responds with Hello World as the body
  """
  def init(req0, state) do
    Logger.info("Received request: #{inspect req0}")
    req1 =
      :cowboy_req.reply(
        200,
        %{"content-type" => "text/html"},
        "Hello World",
        req0
      )
    {:ok, req1, state}
  end
end

As you can see in the preceding code, we have defined the routes/0 function, which returns the dispatch rules routes for our web application. For the handler module, we’re currently using the CowboyExample.Router module itself by defining the init/2 function, which responds with "Hello World" and a status of 200, whenever invoked. We have also added a call to the Logger module to log all requests to the handler. This will increase visibility while running it in the development environment.

In order for our web server to start up when the app is started, we need to add it to our application’s supervision tree.

Now that we have added a router and a handler to our web server, we can add it as a child to our supervision tree by updating the list of children in our application module. For now, I will use a hardcoded TCP port of 4040 for our server, but we will use application-level configurations to set it later in this chapter:

lib/cowboy_example/application.ex

defmodule CowboyExample.Application do
  @moduledoc false
  use Application
  @impl true
  def start(_type, _args) do
    children = [
      # Add this line
      {Task, fn -> CowboyExample.Server.start(4040) end}
    ]
    opts = [
      strategy: :one_on_one,
      name: CowboyExample.Supervisor
    ]
    Supervisor.start_link(children, opts)
  end
end

In the preceding code, we’re adding to the supervised children a Task with the function to start the Cowboy listener as an argument that eventually gets passed to Task.start_link/1. This makes sure that our web server process is part of the application’s supervision tree.

Now, we can run our web application by running the mix project with the --no-halt option:

$ mix run --no-halt

Note

Passing the --no-halt option to the mix run command makes sure that the application, along with the supervision tree, is still running even after the command has returned. This is generally used for long-running processes such as web servers.

Without stopping the previous command, in a separate terminal session, we can make a request to our web server using the cURL command-line utility with the –v option to get a verbose description of our requests and responses:

$ curl –v http://localhost:4040/
*   Trying ::1:4040...
* connect to ::1 port 4040 failed: Connection refused
*   Trying 127.0.0.1:4040...
* Connected to localhost (127.0.0.1) port 4040 (#0)
> GET / HTTP/1.1
> Host: localhost:4040
> User-Agent: curl/7.75.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< content-length: 11
< content-type: text/html
< server: Cowboy
<
* Connection #0 to host localhost left intact
Hello world%

As we can see in the preceding code, we get the expected "Hello World" response along with the expected status code of 200. As mentioned in the previous section, Cowboy adds custom response headers to give us more information about how it was processed. We can also see headers for the type of server (Cowboy), content length, and content type.

We should also see an application-level log corresponding to the request in the terminal session running the mix project. The logs should look somewhat like this:

$ mix run --no-halt
20:39:43.061 [info]  Received request: %{
  bindings: %{},
  body_length: 0,
  cert: :undefined,
  has_body: false,
  headers: %{
    "accept" => "*/*",
    "host" => "localhost:4040",
    "user-agent" => "curl/7.75.0"
  },
  host: "localhost",
  host_info: :undefined,
  method: "GET",
  path: "/",
  path_info: :undefined,
  peer: {{127, 0, 0, 1}, 35260},
  pid: #PID<0.271.0>,
  port: 4040,
  qs: "",
  ref: :listener,
  scheme: "http",
  sock: {{127, 0, 0, 1}, 4040},
  streamid: 1,
  version: :"HTTP/1.1"
}

We can see that we’re logging all the details corresponding to the request including headers, the host, the URI, and the process ID of the process processing the request.

Congratulations, you have now successfully built a Hello World web server using Cowboy. Now, it’s time to add more routes to our web server.

Most web applications support the ability to serve not only a static route but also dynamic routes with a specific pattern. It’s time to see how we can leverage Cowboy to add dynamic routes to our router.

Say we want to add a new route to our application that responds with a custom greeting for a person whose name is dynamic. Let’s update our router to define a handler for a new dynamic route. We can also use this opportunity to move our Root handler (the init/2 function) to a different module. This makes our code more compliant with the single-responsibility principle, making it easier to follow:

lib/cowboy_example/router.exdefmodule

CowboyExample.Router do
  @moduledoc """
  This module defines all the routes, params and handlers.
  """
  alias CowboyExample.Router.Handlers.{Root, Greet}
  @doc """
  Returns the list of routes configured by this web server
  """
  def routes do
    [
      {:_, [
        {"/", Root, []},
        # Add this line
        {"/greet/:who", [who: :nonempty], Greet, []}
      ]}
    ]
  end
end

In the preceding code, we have added a new route that expects a non-empty value for the :who variable binding. This variable gets bound to a request based on the URL. For example, for a request with the URL "/greet/Luffy", the variable bound to :who will be "Luffy", and for a request with the URL "/greet/Zoro", it will be "Zoro".

Now, let’s define the Root handler and move the init/2 function from our router to the new handler module. This separates the concerns of defining routes and handling requests:

lib/cowboy_example/router/handlers/root.ex

defmodule CowboyExample.Router.Handlers.Root do
  @moduledoc """
  This module defines the handler for the root route.
  """
  require Logger
  @doc """
  This function handles the root route, logs the requests
  and responds with Hello World as the body
  """
  def init(req0, state) do
    Logger.info("Received request: #{inspect req0}")
    req1 =
      :cowboy_req.reply(
        200,
        %{"content-type" => "text/html"},
        "Hello World",
        req0
      )
    {:ok, req1, state}
  end
end

Similarly, let’s define the Greet handler for our new dynamic route. We know that the request has a variable binding corresponding to the:who key by the time it gets to this handler. Therefore, we can use the :cowboy_req.binding/2 function to access the value of :who bound to the request:

lib/cowboy_example/router/handlers/greet.ex

defmodule CowboyExample.Router.Handlers.Greet do
  @moduledoc """
  This module defines the handler for "/greet/:who" route.
  """
  require Logger
  @doc """
  This function handles the "/greet/:who", logs the
  requests and responds with Hello `who` as the body
  """
  def init(req0, state) do
    Logger.info("Received request: #{inspect req0}")
    who = :cowboy_req.binding(:who, req0)
    req1 =
      :cowboy_req.reply(
        200,
        %{"content-type" => "text/html"},
        "Hello #{who}",
        req0
      )
    {:ok, req1, state}
  end
end

In the preceding code snippet, we get the value bound to :who for the request and use it with string interpolation to call "Hello :who". Now, we have two valid routes for our web server: the root and the dynamic greet route.

We can test our updates by restarting the Mix application. That can be done by stopping the HTTP server using Ctrl + C, followed by running mix run --no-halt again. Now, let’s make a request to test the new route with Elixir as :who:

$ curl http://localhost:4040/greet/Elixir
Hello Elixir%

Cowboy offers another way to add dynamic behavior to our routes, and that is by passing query parameters to our URL. Query parameters can be captured by using the :cowboy_req.parse_qs/1 function. This function takes a binding name (:who in this case) and the request itself. Let’s update our greet handler to now take a custom query parameter for greeting that overrides the default "Hello" greeting, which we can put in a module attribute for better code organization:

lib/cowboy_example/router/handlers/greet.ex

defmodule CowboyExample.Router.Handlers.Greet do
  # ..
  @default_greeting "Hello"
  # ..
  def init(req0, state) do
    greeting =
    # ..
      req0
      |> :cowboy_req.parse_qs()
      |> Enum.into(%{})
      |> Map.get("greeting", @default_greeting)
    req1 =
      :cowboy_req.reply(
        200,
        %{"content-type" => "text/html"},
        "#{greeting} #{who}",
        req0
      )
    {:ok, req1, state}
  end
end

We have now updated our greet handler to use :cowboy.parse_qs/1 to fetch query parameters from the request. We then put those matched parameters into a map and get the value in the map corresponding to the "greeting" key, with a default of "Hello". Now, the greet route should take a “greeting” query parameter to update the greeting used to greet someone in the response. We can test our updates again by restarting the application and making a request to test the route with a custom greeting parameter:

$ curl http://localhost:4040/greet/Elixir\?greeting=Hola
Hola Elixir%

We now have a web server with a fully functional dynamic route. You might have noticed that we didn’t specify any HTTP method while defining the routes. Let us see what happens when we try to make a request to the root with the POST method:

$ curl -X POST http://localhost:4040/
Hello World%

As you can see in the example, our web server still responded to the POST request in the same manner as GET. We don’t want that behavior so, in the next section, we will see how to validate the HTTP method of a request and restrict the root of our application to only respond to GET requests.

Most modern web applications have a way of restricting requests to a route based on the HTTP method. In this section, we will see how to restrict our handlers to work with a specific HTTP method in a Cowboy-based web application. The simplest way of accomplishing that in a Cowboy handler is by pattern matching on the first argument of the init/2 function, which is the request. A Cowboy request contains a lot of information, including the HTTP method used to make the request. So, by pattern matching on the request with a specific HTTP method, we can filter requests based on HTTP methods. However, we will also be needing a general clause for the init/2 function, which responds with a 404 error.

In the Greet handler, let us update init/2 to match only on requests with the GET method, along with another clause that responds with a 404 (Not Found) error:

lib/cowboy_example/router/handlers/greet.ex

defmodule CowboyExample.Router.Handlers.Greet do
  # ..
  def init(%{method: "GET"} = req0, state) do
  # ..
  end
  # General clause for init/2 which responds with 404
  def init(req0, state) do
    Logger.info("Received request: #{inspect req0}")
    req1 =
      :cowboy_req.reply(
        404,
        %{"content-type" => "text/html"},
        "404 Not found",
        req0
      )
    {:ok, req1, state}
  end
end

Now, let’s make sure only GET requests are accepted by our server for the route. Let’s first make sure GET requests are working:

$ curl http://localhost:4040/greet/Elixir\?greeting=Hola
Hola Elixir%

It’s time to check that a POST request for the greet route returns a 404 error:

$ curl http://localhost:4040/greet/Elixir\?greeting=Hola -X POST
404 Not found%

This ensures that our route works only for GET requests. Another way of validating HTTP methods of our requests would be by using Cowboy middleware, but we will not be covering that in this chapter.

Cowboy middleware

In Cowboy, middleware is a way to process an incoming request. Every request has to go through two types of middleware (the router and the handler), but Cowboy allows us to define our own custom middleware module, which gets executed in the given order. A custom middleware module just needs to implement the execute/2 callback defined in the cowboy_middleware behavior.

Great! We have successfully enforced a method type for a route. Next, we will learn how to serve HTML files instead of raw strings.

Generally, when we write web servers, we do not write our HTML as strings in handlers. We write our HTML in separate files that are served by our server. We will use our application’s priv directory to store these static files. So, let’s create a priv/static folder in the root of our project and add an index.html file in that folder. To add some HTML, we can use this command:

$ echo "<h1>Hello World</h1>" > priv/static/index.html

The priv directory in OTP

In OTP (Open Telecom Platform or Erlang) and Elixir, the priv directory is a directory specific to an application that is intended to store files needed by the application when it is running. Phoenix, for example, uses the priv/static directory to store processed JavaScript and CSS assets for runtime usage.

Let’s add an endpoint to our server that returns a static HTML file:

lib/cowboy_example/router.ex

defmodule CowboyExample.Router do
  @moduledoc """
  This module defines routes and handlers for the web
  server
  """
  alias CowboyExample.Router.Handlers.{Root, Greet, Static}
  @doc """
  Returns the list of routes configured by this web server
  """
   def routes do
    [
      {:_, [
        {"/", Root, []},
        {"/greet/:who", [who: :nonempty], Greet, []},
        # Add this line
        {"/static/:page", [page: :nonempty], Static, []}
      ]}
    ]
  end
end

Now, we need a static handler module, which will look for and respond with the given page in the /priv/static folder and, if not found, will return a 404 error:

lib/cowboy_example/router/handlers/static.ex

defmodule CowboyExample.Router.Handlers.Static do
  @moduledoc """
  This module defines the handler for "/static/:page"
  route.
  """
  require Logger
  @doc """
  This handles "/static/:page" route, logs the requests and
  responds with the requested static HTML page.
  Responds with 404 if the page isn't found in the
  priv/static folder.
  """
  def init(req0, state) do
    Logger.info("Received request: #{inspect req0}")
    page = :cowboy_req.binding(:page, req0)
    req1 =
      case html_for(page) do
        {:ok, static_html} ->
          :cowboy_req.reply(
            200,
            %{"content-type" => "text/html"},
            static_html,
            req0
          )
        _ ->
          :cowboy_req.reply(
            404,
            %{"content-type" => "text/html"},
            "404 Not found",
            req0
          )
             end
    {:ok, req1, state}
  end
  defp html_for(page) do
    priv_dir =
      :cowboy_example
      |> :code.priv_dir()
      |> to_string()
    page_path = priv_dir <> "/static/#{page}"
    File.read(page_path)
  end
end

In the preceding module, the html_for/1 function is responsible for fetching the HTML files from our application’s priv directory, for a given path. If the file is present, the function returns {:ok, <file_contents>}, >}; otherwise, it returns an error, upon which we will respond with a 404 message.

We can test the preceding route by restarting our server again and making a request to the /static/index.html path. But this time, let us use the web browser in order to render the HTML contents properly. Here’s what you should see:

Figure 1.2 – Successful HTML response

Also, to make sure our 404 handler is working correctly, let’s make a browser request to /static/bad.html, a file not present in our application’s priv directory. You should see a 404 message:

Figure 1.3 – Failed HTML response

Now, we have a web server that can respond with static HTML files. It’s time to see how we can go about testing it.

Automated testing is a key part of any software, especially in a dynamic-typed language such as Elixir. It is one of the catalysts for writing deterministic software while documenting the expected behaviors of its components. Due to this reason, we will be making an effort to test everything we build in this book, including the Cowboy-powered web application we have built in this chapter.

In order to test our web application, we first need to be able to run our application on a different port in the test environment. This is to ensure that other /static/bad.html environments do not interfere with our tests. We also can use an application-level configuration to set a port on which the Cowboy server listens to all the requests. This will allow us to separate the test port from the development port.

So, let’s update our application to use the configured port or default it to 4040 using an @port module attribute:

lib/cowboy_example/application.ex

defmodule CowboyExample.Application do
  @moduledoc false
  use Application
  @port Application.compile_env(
          :cowboy_example,
          :port,
          4040
        )
  @impl true
  def start(_type, _args) do
    children = [
      # Add this line
      {Task, fn -> CowboyExample.Server.start(@port) end}
    ]
    opts = [
      strategy: :one_for_one,
      name: CowboyExample.Supervisor
    ]
    Supervisor.start_link(children, opts)
  end
end

We can make sure that the application configuration is different for different Mix environments by adding the config/config.exs file, and setting a different port in our config for the test environment. We will also be updating the logger to not log warnings. So, let’s add a config file with the following contents:

config/config.exs

import Config
if Mix.env() == :test do
  config :cowboy_example,
    port: 4041
  config :logger, warn: false
end

Note

Mix.Config has been deprecated in newer versions of Elixir. You might have to use the Config module instead.

Now, let’s add tests for our server endpoints. In order to test our web server, we need to make HTTP requests to it and test the responses. To make HTTP requests in Elixir, we will be using Finch, a lightweight and high-performance HTTP client written in Elixir.

So, let’s add Finch to our list of dependencies:

mix.exs

defmodule CowboyExample.MixProject do
  # ...
  defp deps do
    [
      {:cowboy, "~> 2.8"},
      {:finch, "~> 0.6"}
    ]
  end
end

Running mix deps.get will fetch Finch and all its dependencies.

Now, let’s add a test file to test our server. In the test file, we will be setting up Finch to make HTTP calls to our server. In this section, we will only be testing the happy paths (200 responses) of our root and greet endpoints:

test/cowboy_example/server_test.exs

defmodule CowboyExample.ServerTest do
  use ExUnit.Case
  setup_all do
    Finch.start_link(name: CowboyExample.Finch)
    :ok
  end
  describe "GET /" do
    test "returns Hello World with 200" do
      {:ok, response} =
        :get
        |> Finch.build("http://localhost:4041")
        |> Finch.request(CowboyExample.Finch)
      assert response.body == "Hello World"
      assert response.status == 200
      assert {"content-type", "text/html"} in response.headers
    end
  end
  describe "GET /greeting/:who" do
    test "returns Hello `:who` with 200" do
      {:ok, response} =
        :get
        |> Finch.build("http://localhost:4041/greet/Elixir")
        |> Finch.request(CowboyExample.Finch)
      assert response.body == "Hello Elixir"
      assert response.status == 200
      assert {"content-type", "text/html"} in response.headers
    end
    test "returns `greeting` `:who` with 200" do
      {:ok, response} =
        :get
        |> Finch.build("http://localhost:4041/greet/
                        Elixir?greeting=Hola")
        |> Finch.request(CowboyExample.Finch)
      assert response.body == "Hola Elixir"
      assert response.status == 200
      assert {"content-type", "text/html"} in response.headers
    end
  end
end

As you can see in the preceding module, we have added tests for the two endpoints using Finch. We make calls to our server, running on port 4041 in the test environment, with different request paths and parameters. We then test the response’s body, status, and headers.

This should give you a good idea of how to go about testing a web server. Over the next few chapters, we will be building on top of this foundation and coming up with better ways of testing our web server.

When I first learned about how a web server works, I was overwhelmed with the number of things that go into building one. Then I decided to look at the code of Puma, a web server written in Ruby, which is also used by Rails. I was surprised by how much more I learned by just looking into Puma than by reading articles about web servers. It is due to that reason that we are kicking off this book by looking at Cowboy. I believe that learning about the basics of Cowboy will better position us to build our own web server in the next few chapters.

In this chapter, we first learned the basics of a web server along with the client-server architecture. We also looked at the high-level architecture of Cowboy and learned about how some of its components such as the router and handlers work. We also added dynamic behavior to our routes by using path variables and query parameters, followed by serving static HTML files. We finished by learning how to test our routes using an HTTP client. In the next chapter, we will use what we learned in this chapter to build our own HTTP server from scratch.

Some of you might have noticed that we haven’t tested a few aspects of our web server. Using what you have learned in this chapter, complete these exercises:

• Write test cases for our web server that would lead to 404 responses
• Write tests for the static route that respond with HTML files

There are other (better) ways of testing an HTML response, which we haven’t covered in this chapter. We will dig deeper into those testing methods later in this book.