Home Programming Mastering Elixir

Mastering Elixir

By André Albuquerque , Daniel Caixinha
books-svg-icon Book
eBook $39.99 $27.98
Print $48.99
Subscription $15.99 $10 p/m for three months
$10 p/m for first 3 months. $15.99 p/m after that. Cancel Anytime!
What do you get with a Packt Subscription?
This book & 7000+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook + Subscription?
Download this book in EPUB and PDF formats, plus a monthly download credit
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook?
Download this book in EPUB and PDF formats
Access this title in our online reader
DRM FREE - Read whenever, wherever and however you want
Online reader with customised display settings for better reading experience
What do you get with video?
Download this video in MP4 format
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with video?
Stream this video
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with Audiobook?
Download a zip folder consisting of audio files (in MP3 Format) along with supplementary PDF
What do you get with Exam Trainer?
Flashcards, Mock exams, Exam Tips, Practice Questions
Access these resources with our interactive certification platform
Mobile compatible-Practice whenever, wherever, however you want
BUY NOW $10 p/m for first 3 months. $15.99 p/m after that. Cancel Anytime!
eBook $39.99 $27.98
Print $48.99
Subscription $15.99 $10 p/m for three months
What do you get with a Packt Subscription?
This book & 7000+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook + Subscription?
Download this book in EPUB and PDF formats, plus a monthly download credit
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook?
Download this book in EPUB and PDF formats
Access this title in our online reader
DRM FREE - Read whenever, wherever and however you want
Online reader with customised display settings for better reading experience
What do you get with video?
Download this video in MP4 format
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with video?
Stream this video
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with Audiobook?
Download a zip folder consisting of audio files (in MP3 Format) along with supplementary PDF
What do you get with Exam Trainer?
Flashcards, Mock exams, Exam Tips, Practice Questions
Access these resources with our interactive certification platform
Mobile compatible-Practice whenever, wherever, however you want
  1. Free Chapter
    Preparing for the Journey Ahead
About this book
Running concurrent, fault-tolerant applications that scale is a very demanding responsibility. After learning the abstractions that Elixir gives us, developers are able to build such applications with inconceivable low effort. There is a big gap between playing around with Elixir and running it in production, serving live requests. This book will help you fll this gap by going into detail on several aspects of how Elixir works and showing concrete examples of how to apply the concepts learned to a fully ?edged application. In this book, you will learn how to build a rock-solid application, beginning by using Mix to create a new project. Then you will learn how the use of Erlang's OTP, along with the Elixir abstractions that run on top of it (such as GenServer and GenStage), that allow you to build applications that are easy to parallelize and distribute. You will also master supervisors (and supervision trees), and comprehend how they are the basis for building fault-tolerant applications. Then you will use Phoenix to create a web interface for your application. Upon fnishing implementation, you will learn how to take your application to the cloud, using Kubernetes to automatically deploy, scale, and manage it. Last, but not least, you will keep your peace of mind by learning how to thoroughly test and then monitor your application.
Publication date:
July 2018
Publisher
Packt
Pages
574
ISBN
9781788472678

 

Chapter 1. Preparing for the Journey Ahead

Welcome to this incredible journey! This is the beginning of an odyssey that will take you through the many features of Elixir, and how to use them to build, test, deploy, and maintain applications. This journey may require some shifting of your mindset and how you think about programming (and problem-solving in general) if you're not already familiar with functional programming.

Before diving into this book, we want to point out that this introductory chapter is deliberately short. Although we'll be introducing the language, its tooling, and its ecosystem, this won't be a complete reference guide. Elixir treats documentation as a first-class citizen, and this is shown in the incredible documentation of its modules and functions. Hence, we decided to teach you how to search for what you need, and target on the core concept of book–building an Elixir application. We think that this pragmatic approach is the one that delivers the most value to you, as you'll be able to follow an application right from its inception and into its production, including its deployment and monitoring, which are sometimes overlooked.

In this chapter, we will cover the following topics:

  • Data types in Elixir
  • Working with pattern matching and how to apply it to various types
  • Working with functions and bundling them in modules
  • Working with collections
  • Using classic control-flow constructs (such as case)
  • Using typespecs on your functions
  • Creating behaviours, and adopting them in other modules
  • Using protocols to make our functions polymorphic
  • Some of the most useful tools that ship with Elixir
  • Calling Erlang libraries directly
  • Interacting with operating system processes through ports
 

Why functional programming?


Coming from an object-oriented background ourselves, we can't emphasize enough how valuable this shift is. Adopting the functional programming paradigm brings along some benefits, such as the following:

  • The data-transformation flow becomes more evident. This is in contrast with what usually happens in object-oriented programming, where objects strive to encapsulate the data; in functional programming, we mutate data using functions. This makes the transformations that are applied to the data explicit, which in turn makes the applications written this way easier to understand.
  • Functions are (mostly) side-effect-free. With immutable data, you can be sure that the value of a certain variable you hold a reference to will remain the same, since if another function or process wants to change it, it has to create a copy first—and operate on that copy. This makes code much easier to analyze and comprehend, since you can rest assured that your variables will remain as you expect. However, note that since Elixir isn't a pure functional language, your code can still have side effects. For instance, if a function writes to a file, it creates a side effect.
  • Programs can be parallelized easily. Coming out as an added bonus of immutability, it's usually very simple to parallelize this type of program, since there is no shared state.

One possible disadvantage of having immutable data is that you can incur a performance penalty, as each time you need to change something, you must make a copy of it. This greatly depends on the implementation, and while this concern is generally valid, Elixir employs clever techniques when compiling your code to minimize this effect. For instance, in certain conditions, Elixir can just point to existing variables when creating new ones, as it knows all variables are immutable and will never change.

If you're overwhelmed with all this functional programming jargon, lie down and relax, as we will explore these concepts in greater detail throughout this book.

 

Elixir and Erlang


Elixir, created by José Valim, runs on the Erlang VM (also known as BEAM). Erlang, developed at Ericsson more than 30 years ago, was created to improve the reliability and scalability of the company's telecom systems. Nowadays, it is used in a number of different settings, from database systems to chat applications. Erlang has fault-tolerance and distribution baked into its design, and is famous for running systems with nine nines of reliability.

Erlang's runtime is natively distributed, given that it was designed to be deployed on multiple telecom switches simultaneously. Programs running on the Erlang VM can take advantage of this by easily distributing an application across multiple nodes, but also across multiple CPUs—since multiple cores is just a specific case of a distributed system. This is an incredible selling point of the Erlang VM (and thus of Elixir), since in today's setting CPUs are not getting much faster, and we're instead seeing CPUs with an increasing number of cores coming out.

In this opening chapter, we will be introducing Elixir. Beginning with its data types, we will also look at pattern matching, anonymous and named functions, modules, and some control-flow constructs. Then, we will see how to work with collections, and then we will briefly touch on behaviours and protocols. The chapter will end with an overview of the incredible tooling Elixir provides, along with some ways to exploit the existing interoperability between Elixir and Erlang.

 

Elixir's data types


We will now describe Elixir's data types, which extend upon Erlang's data types. Elixir is a dynamic programming language. Consequently, you don't declare the type of each variable—it depends on the value it holds at each moment.

To improve the learning experience, we'll be providing some examples along the way. For now, we'll just use Elixir's REPL, IEx (short for Interactive Elixir). To start an IEx session, you must have Elixir installed on your machine. Elixir has an official page with instructions on how to do this if you don't have it installed,  whether using package managers, the precompiled version, or compiling from the source yourself:

http://elixir-lang.github.io/install.html 

Note

We will not dive into the memory usage of each type in Elixir. If you're curious about this, the official Erlang documentation contains detailed information: http://erlang.org/doc/efficiency_guide/advanced.html.

Provided that you have Elixir already installed on your machine, type iex on your terminal to start a new IEx session. With this, you can run the examples present in this chapter in your machine. Note that your default iex  prompt contains a number in between parenthesis, which represents the number of expressions you've entered in the current session, such as iex>(1). To declutter the output, in our examples, we've removed this number.

We'll be exploring IEx in greater detail toward the end of this chapter, in the Tooling and ecosystems section. Throughout the following subsections, we'll be mentioning some built-in modules in Elixir. We'll explore what modules are in the Functions and modules section—for now, it's enough to know that a module is a collection of functions.

Note

Whenever you're done with your IEx session, you can either press Ctrl C twice or Ctrl \. This will kill the operating system process that's running the Erlang runtime (along with all background jobs). Alternatively, you can stop the system in a more polite way by entering System.halt in the shell.

Integers

This type contains, as you would expect, numbers that can be written without a fractional component. The size of integers adjusts dynamically according to its magnitude—you don't have to worry about this: an integer will simply occupy more words in memory as it grows. Here's some basic arithmetic with integers:

iex> 25 + 8
33

To improve the readability of the code, you can also use underscores in between the digits of an integer, as shown here:

iex> 1_000_000 - 500_000
500000

Besides decimal, Elixir also supports integers written in binary, octal, and hexadecimal (using 0b, 0o, and 0x, respectively):

iex> 0b10001
17
iex> 0o21
17
iex> 0x11
17

Floats

In Elixir, floats are written with a decimal point, with digits before and after it, meaning that .1 is not a valid float in Elixir (as it is, for instance, in JavaScript). In Elixir, you have to be explicit and write the leading 0—so in this case, you'd write 0.1. Here's an example of the multiplication of two floats:

iex> 0.1 * 0.5
0.05

You can also write floats using the exponent notation, as shown:

iex> 0.1e3
100.0

Floats are represented in IEEE 754 double precision, which yields between 15 to 17 significant decimal digits. As usual, you should take care when comparing floats for equality.

Note

Beware that the division operator (/) always returns a float, even if the result of the division could be an integer:iex> 4/22.0 If you want to circumvent this behavior, use the auto-imported div function from the Kernel module. Also, if you want to get the remainder of a division, use the rem function.

 

Atoms

Atoms are a constant, whose value is its own name. They are always prefixed with a leading colon (:), followed by alphanumeric characters (and possibly _ or @). They may terminate with an exclamation or a question mark. Atoms are similar to enumerations in C and symbols in Ruby. Here are some examples of atoms:

iex> :ok
:ok
iex> :error
:error
iex> :some_descriptive_name!
:some_descriptive_name!
iex> :value@start
:value@start

You can create atoms with arbitrary characters with the following syntax:

iex> :"Atom name with arbitrary characters#$%^"
:"Atom name with arbitrary characters#$%^"

As with all data structures in Elixir, atoms can't be modified after their creation. Furthermore, they are not garbage-collected. Atoms are kept in the atom table, and upon compilation, their value is replaced by a reference to their entry on this table. This makes comparing atoms very efficient. As you'll learn throughout this book, this is one of the major use cases for atoms in Elixir, as we are constantly matching the return of a function against a certain expected atom.

Note

Since atoms are not garbage collected, don't create atoms dynamically from sources you can't control, as you can very easily use up all of the space allocated for the atom table. For instance, if you're parsing a JSON response and creating a map out of it, don't use atoms for its keys—use strings instead (both of these types, maps and strings, will be described later in this chapter).

Booleans

Elixir has three values related to Boolean operations: true, false, and nil (where nil represents the absence of value—similar to null in most other languages). However, those are just some syntatic sugar, as internally they are represented as atoms of the same name, as you can see in the following example:

iex> true == :true
true
iex> false == :false
true
iex> nil == :nil
true

You have the common Boolean operators, orand, and not:

iex> true or false
true
iex> true and false
false
iex> not false
true

However, these operators are type-strict in their first argument: they only accept true or false. If you pass anything else as an argument, you'll get BadBooleanError.

This is where the concept of truthiness and falseness enters. Similar to what happens in Ruby or C, false and nil are treated as falsey values, and everything else is considered to be truthy. The operators that work with falsey and truthy values are && (and), || (or), and ! (not):

iex> "a value" || false
"a value"
iex> "a value" && false
false
iex> nil && "a value"
nil
iex> !"a value"
false

Notice how these operators short circuit depending on the arguments. With ||, it returns the first value that's truthy, whereas with &&, it returns the first falsey value (in both cases, in the event those conditions never happen, they return the last value).

You also have the other normal comparison operators, such as greater than (>) and inequality (!=)—you can find the full list at https://hexdocs.pm/elixir/operators.html. The one that's worth pointing out is the strict equality operator, which, besides comparing values, compares types:

iex> 3 == 3.0
true
iex> 3 === 3.0
false

Tuples

Tuples are used to group a fixed number of elements together. They can hold any value—even other tuples. They are stored contiguously in memory, which provides constant access time to elements inside a tuple. You create a tuple surrounding the elements with curly brackes ({ and }), and separate the elements with commas:

iex> {:ok, 3.14}
{:ok, 3.14}

A common usage of tuples in Elixir is to pattern-match on the result of a function to ensure its success (usually with an :ok atom) or deal with an error. We will be looking to pattern matching and functions later in this chapter.

To access an element inside a tuple, we use the elem function (from the Kernel module), providing the tuple and a zero-based index:

iex> result = {:ok, 3.14}
{:ok, 3.14}
iex> elem(result, 1)
3.14

Note

Functions from the Kernel module are auto-imported. Thus, we don't need to prefix them with the module name.

To change the elements on a tuple, you can use the put_elem function. The arguments are similar to the elem function, but you also provide the new value for that position of the tuple:

iex> put_elem(result, 1, 1.61)
{:ok, 1.61}
iex> result
{:ok, 3.14}

Notice how the result variable hasn't changed. As we discussed in the beginning of this chapter, data in Elixir is immutable. As such, although we've updated the tuple with a new value, the original tuple hasn't changed—Elixir updated the value on a copy of the original tuple. This way our code is side-effect free, and any other function holding a reference to the result variable won't have any surprises.

The general recommendation in Elixir is that tuples should hold up to four elements—anything more than that and you probably should be using another type.

Lists

Lists are created by wrapping the elements we want inside it with square brackets ([ and ]), separating the values with commas. Internally, lists are implemented as singly linked lists, meaning that accessing the elements of a list is a O(n) operation. Lists aren't stored contiguously in memory as arrays in other languages. As with tuples, list elements can be of any type:

iex> [1, :an_atom, 0.5]
[1, :an_atom, 0.5]

We have the ++ and -- operators that are exclusive to lists, and serve to concatenate and subtract lists, respectively:

iex> [0, 1, 1] ++ [2, 3, 5]
[0, 1, 1, 2, 3, 5]
iex> [0, 1, 1] -- [1, 2, 3]
[0, 1]

To check whether a certain element is present in a list, you can use the in operator:

iex> 1 in [0, 1, 1, 2, 3, 5]
true
iex> 99 in [0, 1, 1, 2, 3, 5]
false

To get the head of a list, we use the hd function, whereas to get the tail of a list, we use the tl function:

iex> hd([0, 1, 1, 2, 3, 5])
0
iex> tl([0, 1, 1, 2, 3, 5])
[1, 1, 2, 3, 5]

Notice that the semantic of tail here is the list without its head (which is also a list), and not the last element of a list. We'll be exploring this concept in more depth, along with some more examples on how to work with lists, in the Working with collections section. For reference, you can find a detailed list of operations you can make on lists at https://hexdocs.pm/elixir/List.html.

Note

Appending to a list is a O(n) operation, as we need to traverse the whole list. Prepending to a list is O(1). To prepend an element to a list, you can use the following syntax: [new_element | list].

Maps

Maps are key-value data structures, where both the key and the value can be of any type. They're similar to hashes in Ruby and dictionaries in Python. To create a map, you enclose your key-value pairs in %{}, and put a => between the key and the value, as we can see in the following snippet:

iex> %{:name => "Gabriel", :age => 1}
%{age: 1, name: "Gabriel"}

In this case, the keys are both of the same type, but this isn't required. If your keys are atoms, you can use the following syntax to make the map declaration simpler:

iex> %{name: "Gabriel", age: 1}
%{age: 1, name: "Gabriel"}

To access the value associated with a certain key, put the key inside square brackets in front of the map:

iex> map = %{name: "Gabriel", age: 1}
%{age: 1, name: "Gabriel"}
iex> map[:name]
"Gabriel"

As with the map declaration, when the key is an atom, we have some syntatic sugar on top of it:

iex> map.name
"Gabriel"

Note

When you try to fetch a key that doesn't exist in the map, a KeyError error will be raised when using the map.key syntax–unlike the map[key] syntax, which will return nil.

To update a key in a map, you can use %{map | key => new_value}. If the key is an atom, we can use the same notation described previously:

iex> %{map | age: 2}
%{age: 2, name: "Gabriel"}

Note

If you're coming from an object-oriented programming background, you may instinctively use the following syntax to change the value of a key: map[key] = new_value. Remember that in Elixir all types are immutable and you never operate on the data structure itself but always on a copy of it.

This will only work for keys that already exist in the map—this constraint allows Elixir to optimize and reuse the fields list when updating a map. If you want to insert a new key, use the put function from the Map module:

iex> Map.put(map, :gender, "Male")
%{age: 1, gender: "Male", name: "Gabriel"}

As with all other types, in the official documentation, at https://hexdocs.pm/elixir/Map.html, you can find a pretty detailed reference on what you can do with maps.

Binaries

A binary is group of consecutive bytes. You create them by surrounding the byte sequence with << and >>. Here we are creating a two-byte binary:

iex> <<5, 10>>
<<5, 10>>

In the decimal base, a byte can only contain values up to 255 (otherwise it overflows). If we want to store values greater that 255, we need to tell the runtime to use more space to store this binary:

iex> <<5, 256>>
<<5, 0>>
iex> <<5, 256::16>>
<<5, 1, 0>>

As you can see, when we specify the size (16 bits in this case) we can see that the output as an extra byte and the overflow didn't occur. The size doesn't have to be a multiple of 8. In that case, a binary is usually called a bitstring.

Most programmers will not handle data at such a low level, so your use of binaries may not be that frequent. However, they're extremely useful in certain scenarios, such as processing the header of a file to find a magic number and identify the file type, or even when dealing with network packets by hand.

Strings and charlists

Strings are binaries with UTF-8 codepoints in them. You create a string with the usual double-quote syntax:

iex> "hey, a string"
"hey, a string"

Charlists are, as the name implies, lists of character codes. You create them using the single-quote syntax:

iex> 'hey, a charlist'
'hey, a charlist'

Since this is just a list, you can use the hd function to get the code for the first character:

iex> hd('hey, a charlist')
104

Note

You can find out the code of a certain character with the ? operator. For instance, to find out the character code of a lowercase d, you'd use ?d.

Both representations support string interpolation:

iex> "two plus two is: #{2+2}"
"two plus two is: 4"
iex> 'four minus one is: #{4-1}'
'four minus one is: 3'

Both representations also support the heredoc notation, which is most commonly used to write documentation. To create it, use three single or double quotes:

iex> """
...> a string with heredoc notation
...> """
"a string with heredoc notation\n"
iex> '''
...> a charlist with heredoc notation
...> '''
'a charlist with heredoc notation\n'

Note

The closing delimiter of a heredoc string/charlist must be on its own line.

Elixir provides sigils as another syntax to declare strings or charlists, which can be handy if you want to include quotes inside your string. You can use ~s to create a string and ~c to create a charlist (their uppercase versions, ~S and ~C, are similar but don't interpolate or escape characters):

iex> ~s(a string created by a sigil)
"a string created by a sigil"
iex> ~c(a charlist created by a sigil)
'a charlist created by a sigil'

There's another sigil that's worth mentioning: ~r, which is used for regular expressions. In the next snippet, we're using the run function from the Regex module to exemplify the usage of the ~r sigil:

iex> Regex.run(~r{str}, "a string")
["str"]
iex> Regex.run(~r{123}, "a string")
nil

You can find the list of supported sigils (and also how to create your own!) at http://elixir-lang.github.io/getting-started/sigils.html.

The convention in the Elixir community is to only use the term string when referring to the double-quote format. This distinction is important, since their implementation is very different. Functions from the String module will only work on the double-quote format. You should always use the double-quote format, unless you're required to use a charlist—which is the case, for instance, when you're using Erlang libraries. You can use the following functions to convert between the two formats:

iex> String.to_charlist("converting to charlist")
'converting to charlist'
iex> List.to_string('converting to string')
"converting to string"

Other types

We'll now succinctly describe some other types. We'll begin with the types that build upon some types we've already described: keyword lists, ranges, mapsets, and IO lists.

Keyword lists

A keyword list is a list in which its elements have a specific format: they are tuples where the first element is an atom (the second element can be of any type), as demonstrated in the following example:

iex> [name: "Gabriel", age: 1] = [{:name, "Gabriel"}, {:age, 1}]
[name: "Gabriel", age: 1]

We can create keyword lists using the following syntax:

iex> keyword_list = [name: "Gabriel", age: 1]
[name: "Gabriel", age: 1]
iex> keyword_list[:name]
"Gabriel"

As you can see from the previous snippet, a keyword list is indeed a list of tuples, with an atom; you can access values in a keyword list using the same syntax as you would in maps. As an alternative, you can use the get function from the Keyword module. Note that this way of declaring a keyword list is just syntatic sugar, as internally this still is a list of tuples–which means that searching for an item in a keyword list is O(n), and not O(1) as in maps.

In a keyword list, contrary to what happens in maps, you can have more than one value for a given key. Also, you can control the order of its elements. Usually, keyword lists are used to allow functions to receive an arbitrary number of optional arguments. We'll be showing an example of this when we look at named functions later in this chapter. You can find all the operations you can do on a keyword list at https://hexdocs.pm/elixir/Keyword.html.

Ranges

Ranges, again, similar to what happens in Ruby, represent an interval between two integers. To create a range, we use this:

iex> 17..21
17..21
iex> 19 in 17..21
true

Similar to what we do with a list, we can use the in operator to check whether a number is between the start and the end of a range.

MapSets

If you're looking for an implementation of a set in Elixir, you're looking for MapSet. You create and manipulate them with the functions from the MapSet module. Here are some examples:

iex> set = MapSet.new
#MapSet<[]>
iex> set = MapSet.put(set, 1)
#MapSet<[1]>
iex> set = MapSet.put(set, 2)
#MapSet<[1, 2]>
iex> set = MapSet.put(set, 1)
#MapSet<[1, 2]>

Sets, by definition, can't contain duplicates. So, inserting a value that's already there has no effect. You can find the documentation for the MapSet module at https://hexdocs.pm/elixir/MapSet.html.

There are three types, related to the underlying Erlang VM, that we have to mention before closing this section. They are as following:

  • Reference: A reference is a type created by the Kernel.make_ref function. This functions creates an almost-unique reference, which gets repeated around every 282 calls. We will not use references in this book.
  • Port: A port is a reference to a resource. The Erlang VM uses it to interact with external resources, such as an operating system process. We will talk a bit more about ports later in this chapter, when we discuss the interoperability between Elixir and Erlang.
  • PID: A PID is the type used to identify processes in the Erlang VM. You'll see PIDs in action later in this book, when we start working with Erlang VM processes.

To complete this section, there's a function that we want to highlight: the i function, which is auto-imported from the Kernel module. You can use it to find out more information about a data type. It will print information about the data type of the term you pass as an argument. Here is an example with a string:

iex> i("a string")
Term
  "a string"
Data type
  BitString
Byte size
  8
Description
  This is a string: a UTF-8 encoded binary. It's printed surrounded by
  "double quotes" because all UTF-8 encoded codepoints in it are printable.
Raw representation
  <<97, 32, 115, 116, 114, 105, 110, 103>>
Reference modules
  String, :binary
Implemented protocols
  IEx.Info, Collectable, List.Chars, String.Chars, Inspect

 

And, with this, we've finished our tour of the data types in Elixir! We didn't go into much detail, but with the links we left throughout this section, you'll see how incredible the documentation in Elixir is, and how easy it is to figure out what a certain function does or the purpose of a certain argument.

We will now jump into one of the most prominent features of Elixir (and also one that will definitely change how you write programs): pattern matching.

 

Pattern matching


When we were describing Elixir's data types, we used the = operator to bind a value of a certain type to a variable. We didn't stop there to explain what was actually going on, as the syntax is very similar to most dynamic programming languages. In fact, it is so similar that, at first glance, we assume it works the same way.

If you don't have a functional programming background, your first instinct would be to call the = operator, the assignment operator. However, in Elixir, it is called the match operator. Let's see the reason for this difference in nomenclature with some examples:

iex> x = 3
3
iex> x * 3
9
iex> 3 = x
3
iex> 2 = x
** (MatchError) no match of right hand side value: 3

Note

We've seen our first exception being raised on our IEx session. We'll discuss them later in this chapter, when we talk about control flow.

The first two statements are analogous to what you'd see with an assignment operator—we just set the x variable to 3, and then multiply that variable by 3, giving us the expected result of 9. Now, notice what happens on the following lines. We have an integer literal on the left-hand side of the = operator, and that is a valid expression, returning a value of 3. You're seeing the match operator in action.

Similar to what you do with equations in algebra, this operator tries to match the pattern on the left-hand side to the term on the right-hand side. On the first line of the preceding snippet, this means matching the x variable on the left to the 3 term on the right.

Elixir can make this match succeed by binding the x variable to the 3 term. On the third line, we're again matching the left and right sides, and it succeeds because x has the 3 value, so both sides are equal. On the next line, we're trying to match the 2 literal to the x variable. Elixir can't find a way to make this match work, which results in an error being raised. It's important to point out that binding to variables only happens when they are on the left-hand side of the = operator—when they're at the right-hand side, the variable is simply replaced by its value.

From this snippet, we can also notice that we always have a return value—even when doing a pattern match. This is the expected behavior, since everything in Elixir is an expression–there are no statements. Every operation you can do will always return a value, whether you're printing something to the console or making an HTTP request. While you can achieve the same things with expressions and statements, always having a return value is very useful because you can chain functions together and define the program flow according to the values being returned. In the case of pattern matching, when it is successful, we always get back the term that was matched on the right-hand side.

The match operator is not confined to bind variables to simple values–it's actually a very powerful operator that is able to destructure complex data types (and make your code ridiculously simple to read). We will now show how you can use this operator on several of the types we've presented in the previous section, while also demonstrating other aspects of the pattern matching process.

Pattern matching on tuples

The following snippet shows how pattern matching can be done on tuples:

iex> {number, representation} = {3.1415, "π"}
{3.1415, "π"}
iex> number
3.1415
iex> representation
"π"

The process here is the same as we have described in the preceding snippet. By setting the {number, description} pattern on the left-hand side, we're stating that we expect a tuple with two values—again, if that's not the case, a MatchError will be raised. In this case, the match succeeds, and we can see that the variables number and representation are bound to the expected values.

Note

Unlike Erlang, Elixir allows you to rebind a variable, which is why the following works:iex> a = 11iex> a = 77 However, a variable can only bind once per match:iex> {a, a} = {3, 3}{3, 3}iex> {a, a} = {2, 3}** (MatchError) no match of right hand side value: {2, 3} On the first line, the match succeeds because each a is binding to the same value, 3. On the second line, we get a MatchError because we're binding a to two different values on the same match. Later in this chapter, we'll see how we can make Elixir behave like Erlang in this regard, by using the pin operator.

We can set our expectations even further, using literals on the left-hand side:

iex> {3.1415, representation} = {3.1415, "π"}
{3.1415, "π"}
iex> representation
"π"

Now our expectation is a tuple with two elements, where the first one is the 3.1415 float literal. We can use this on other Elixir types as well, such as lists or maps. This technique becomes even more fruitful when we apply it to functions, as we will see in the next section.

Pattern matching on lists

Matching on lists is akin to matching on tuples. Here's a simple example:

iex> [first, second, third] = ["α", "β", "γ"]
["α", "β", "γ"]
iex> first
"α"
iex> second
"β"
iex> third
"γ"

There's nothing new here. What if, for instance, we don't care about the second element of the list? That's where the  _ (underscore) anonymousvariable is convenient:

iex> [first, _, third] = ["δ", "ε", "ζ"]
["δ", "ε", "ζ"]
iex> first
"δ"
iex> third
"ζ"

We're again matching a list with three elements, but now bind the second element to the _ variable, which means that we accept anything in that position and we won't use its value.

 

 

Note

The _ variable can never be read from–if you do so, you will get a CompileError:iex> _** (CompileError) iex:83: unbound variable _ This way, Elixir is protecting you from inadvertently reading from this variable, which could easily cause unexpected behaviors in your application.

As we've mentioned on the data types section, you can use the hd and tl functions from the Kernel module to get the head and the tail of a list. You can do the same with pattern matching:

iex> [first | rest_of_list] = ["α", "β", "γ"]
["α", "β", "γ"]
iex> first
"α"
iex> rest_of_list
["β", "γ"]

While in this contrived example, this approach yields no benefit, this technique is a fundamental piece to operate on a list using recursion. We'll look at this in greater detail in the Working with collections section.

 

Pattern matching on maps

To use pattern matching on a map, we set our pattern with the key-value pairs we want to match on, as you can see in the following example:

iex> %{"name" => name, "age" => age} = %{"name" => "Gabriel", "age" => 1}
%{"age" => 1, "name" => "Gabriel"}
iex> name
"Gabriel"
iex> age
1

Note that in this case we're matching on all keys of the map, but this isn't necessary–we could just match on age, for instance. However, your pattern may only contain keys that exist on the map that's being matched on, otherwise MatchError will be raised.

Sometimes, you may want to match on the value of a variable, instead of rebinding it to a new value. To this end, you can use the pin operator, represented by the ^ character:

iex> name = "Gabriel"
"Gabriel"
iex> %{name: ^name, age: age} = %{name: "Gabriel", age: 1}
%{age: 1, name: "Gabriel"}
iex> %{name: ^name, age: age} = %{name: "Jose", age: 1}
** (MatchError) no match of right hand side value: %{age: 1, name: "Jose"}

As we can see in the preceding snippet, we have the name variable bound to "Gabriel". We then match a map as we did previously in this section, this time using the contents of the name variable. This is equivalent to using the "Gabriel" literal on the left-hand side. When we're trying to match against a map that has a value different than that of the pinned variable, we get a MatchError, as expected.

Note

When working with the pin operator, the variable you're using must already be bound, as it will not bind the variable in case it doesn't exist. If you use the pin operator on a non-existent variable, you'll get a CompileError stating that the variable you're trying to use is unbound.

Pattern matching on binaries and strings

The following example shows how you can use pattern matching on binaries:

iex> <<first_byte, second_byte>> = <<100, 200>>
<<100, 200>>
iex> first_byte
100
iex> second_byte
200

As previously stated when describing binaries, this can be incredibly helpful when you're dealing with bytes directly, such as parsing a packet from a given network protocol. By applying pattern matching to binaries, you can extract bits or bytes as necessary, while your code remains extremely expressive.

Since strings are just binaries underneath, we can use the same strategy as we did in the preceding snippet:

iex> <<first_byte, second_byte>> = "YZ"
"YZ"
iex> first_byte
89
iex> second_byte
90

However, this isn't very helpful when dealing with strings, as you're getting the decimal code of the characters in UTF-8 (also, as UTF-8 is a variable width encoding, a code point may take more than one byte). To match on strings, the best approach is to use the functions from the String module, such as starts_with?, ends_with?, or contains?.

Note

As we've explained in the beginning of this section, everything in Elixir is an expression, and in pattern matching, when the match succeeds, the right-hand side of the expression is returned. Due to this behavior and taking into account that Elixir rebinds the variables on the left-hand side, we can write expressions such as the following one, binding multiple variables to the same value:iex> x = y = 100100iex> x100iex> y100

 

Functions and Modules


Despite not being mentioned in the data types section, functions in Elixir are a type as well–in fact, they are a first-class citizen, as they can be assigned to a variable and passed as arguments to other functions.

As with most functional programming languages, functions are an important type, hence they justify having their own section, away from other built-in types.

We will start by exploring anonymous functions, followed by an explanation of modules and named functions, and then we'll end this section with a quick tour of module attributes and directives.

Anonymous functions

Anonymous functions, usually called lambdas, are created with the fn keyword, as we can see in the following example:

iex> plus_one = fn (x) -> x + 1 end
#Function<6.99386804/1 in :erl_eval.expr/5>
iex> plus_one.(10)
11

Here, we are defining a function that takes one argument, which we've named x, and simply adds one to the provided argument. We then bind this anonymous function to a variable named plus_one, and execute it with 10 as the argument, using the syntax we can see in the preceding snippet. As expected, we get 11 back.

Note

There is no return keyword in Elixir–the return value of a function is the value returned by its last expression.

An anonymous function can also have multiple implementations, depending on the value and/or type of the arguments provided. Let's see this in action with an example:

iex> division = fn
...>   (_dividend, 0)      -> :infinity
...>   (dividend, divisor) -> dividend / divisor
...> end
#Function<12.99386804/2 in :erl_eval.expr/5>
iex> division.(10, 2)
5.0
iex> division.(10, 0)
:infinity

Imagine that we want a special division function, that, instead of raising ArithmeticError when dividing by 0, would just return the :infinity atom. This is what the anonymous function we see here achieves. Using pattern matching, we say that when the second argument (the divisor) is 0, we simply return :infinity. Otherwise, we just use the / arithmetic operator to perform a normal division.

Aside from the lambda with multiple bodies, notice that we prefix the unused variable with an underscore (_), as in _dividend. Besides increasing the readability of your code, following this practice will make the Elixir compiler warn you when you use a supposedly unused variable. Conversely, if you don't use a certain variable but don't prefix it with an underscore, the compiler will also warn you.

Note

Parentheses around arguments of an anonymous function are optional. You could write the plus_one function we introduced earlier as fn x -> x + 1 end.

Beyond accepting arguments, anonymous functions can also access variables from the outer scope:

iex> x = 3
3
iex> some_fun = fn -> "variable x is #{x}" end
#Function<20.99386804/0 in :erl_eval.expr/5>
iex> some_fun.()
"variable x is 3"
iex> x = 5
5
iex> some_fun.()
"variable x is 3"

As you can see, our anonymous function can access variables from the outer scope. Furthermore, the variable can be bound to another value, and our function will still hold a reference to the value that the variable had when the anonymous function was defined. This is usually called a closure: the function captures the memory locations of all variables used within it. Since every type in Elixir is immutable, that value residing on each memory reference will not change. However, this also means that these memory locations can't be immediately garbage-collected, as the lambda is still holding references to them.

We'll end this section on anonymous functions by introducing a new operator–the capture operator (represented by &).

This operator allows you to define lambdas in a more compact way:

iex> plus_one = &(&1 + 1)
#Function<6.99386804/1 in :erl_eval.expr/5>
iex> plus_one.(10)
11

This syntax is equivalent to the one presented before for the plus_one function. &1 represents the first argument of this lambda function—and, more generally, &n will represent the nth argument of the function. Similar to what happens in the fn notation, the parentheses are optional. However, it's better to use them, as in a real-world application, these lambda functions become hard to read without them.

Note

Besides providing a shorter way to define lambda functions, the capture operator can also be used with named functions. We'll explore this further in the next section.

Modules and Named Functions

In Elixir, modules group functions together, much like a namespace. Usually, functions that reside in the same module are related to one another. You create a module using the defmodule construct:

iex> defmodule StringHelper do
...>   def palindrome?(term) do
...>     String.reverse(term) == term
...>   end
...> end
{:module, StringHelper,
 <<70, 79, 82, 49, 0, 0, 4, 0, 66, 69, 65, 77, 65, 116, 85, 56, 0, 0, 0, 119, 0,
   0, 0, 11, 19, 69, 108, 105, 120, 105, 114, 46, 83, 116, 114, 105, 110, 103,
   72, 101, 108, 112, 101, 114, 8, 95, 95, ...>>, {:palindrome?, 1}}
iex> StringHelper.palindrome?("abcd")
false
iex> StringHelper.palindrome?("abba")
true

In the preceding example we're also creating a function inside the StringHelper module, using the def construct, that checks whether a given string is a palindrome. This is a named function, and contrary to the anonymous functions, must be created inside a module.

Note

Function names, like variable names, start with a lowercase letter, and if they contain more than one word, they are separated by underscore(s). They may end in ! and ?. The convention in the Elixir community is that function names ending in ! denote that the function may raise an error, whereas function names ending in ? indicate that that function either returns true or false–which is the case of our palindrome? function.

Note that unlike anonymous functions, we don't need to put a dot between the function name and the parenthesis when calling named functions. This is deliberate and serves to explicitly differentiate calls to anonymous and named functions.

Note

As the implementation of our palindrome? function is very small, we can inline it with the following syntax:def palindrome?(term), do: String.reverse(term) == term This works with other constructs that use the do ... end syntax, such as defmodule or if. We will explore if (and other classical control flow mechanisms) in the Control-flow section.

Before we go any further, as our examples are getting bigger, we must discuss how you can write Elixir code in files. As you can see in the previous example, you can define modules on an IEx session–however, any typo while writing them results in having to start from the beginning.

Put the contents of the last example in a file–let's call it "string_helper.ex" (we usually name the file with the name of the module we're defining in it). Elixir source code files may have two extensions: .ex or .exs. The difference between them is that the former is compiled to disk (creating .beam files), while the latter is compiled only in memory. We mostly use the .ex extension when working on a real application, except for the test files that use the .exs extension (as there's no point in compiling them to disk).

Having your file created, you can use the elixirc command in your terminal to compile it, passing the name of the file whose contents you want compiled. More interestingly, you can pass the filename to the iex command (iex string_helper.ex in our case). This will make Elixir compile your file, which will make our StringHelper module (and its functions) available in the IEx session. If you're already inside the IEx session and want to compile a new file, you can use the c command, passing the filename as a string:

iex> c("examples/string_helper.ex")
[StringHelper]

You can also nest modules:

$ cat examples/nesting_modules.ex
defmodule Helpers do
  defmodule StringHelper do
    # StringHelper code goes here
  end
end

Note

In the preceding example, the line starting with # is commented. That's the syntax to comment lines in Elixir. There's no syntax for multi-line comments–if you want to comment a block of code, prepend each line of that block with #.

However, during compilation, Elixir will prepend the outer module name to the inner module name, and separate them with a dot. This is just an amenity, as there is no relationship between these two modules. This syntax is equivalent to the following one, which is used much more in Elixir applications:

$ cat examples/nesting_modules_inline.ex
defmodule Helpers.StringHelper do
  # StringHelper code goes here
end

We'll now explain the concept of arity, with our palindrome? function as an example. Named functions in Elixir are identified by their module name, the function's own name, and their arity. The arity of a function is the number of arguments it receives. Taking this into account, our palindrome? function is identified as Helpers.StringHelper.palindrome?/1, where /1 represents the arity of the function. You'll be seeing this notation a lot when browsing through Elixir documentation.

This concept is important because functions with the same name but different arities are, in effect, two different functions. However, for a human, it wouldn't make much sense that two functions with the same name (but different arities) are unrelated. As such, only use the same name for different functions when they are related to one another.

The common pattern in Elixir is to have lower-arity functions being implemented as calls to functions of the same name but with a higher arity. Let's extend our module with an emphasize function:

$ cat examples/string_helper_emphasize.ex
defmodule StringHelper do
  def palindrome?(term) do
    String.reverse(term) == term
  end

  def emphasize(phrase) do
    emphasize(phrase, 3)
  end

  def emphasize(phrase, number_of_marks) do
    upcased_phrase = String.upcase(phrase)
    exclamation_marks = String.duplicate("!", number_of_marks)
    "#{upcased_phrase}#{exclamation_marks}"
  end
end

Here, we can observe it in action:

iex> StringHelper.emphasize("wow")
"WOW!!!"
iex> StringHelper.emphasize("wow", 1)
"WOW!"

Note

We've used the def construct to create functions. By using it, our functions are exported and can be called in other modules. If you want to change this behavior, and make a function only available within the module where it's defined, use the defp construct.

The function with an arity of 1 is implemented by simply calling emphasize/2. This is useful when you want to offer a broad interface on your module, which allows you to have some clients that simply want to call emphasize/1 and not have to specify the number of exclamation marks, but also have some other clients that want to call emphasize/2 and specify the number of exclamation marks.

When the code is as simple as in this example, this multitude of functions is not necessary, as you can achieve the same end result using default arguments. We do that by using the \\ operator in front of the argument name, and then the default value it should have:

$ cat examples/string_helper_emphasize_with_default_args.ex
def emphasize(phrase, number_of_marks \\ 3) do
  upcased_phrase = String.upcase(phrase)
  exclamation_marks = String.duplicate("!", number_of_marks)
  "#{upcased_phrase}#{exclamation_marks}"
end

This will generate two functions with the same name and different arities, as in the last snippet. If your function has multiple bodies, as in the next example, you must define a function header with the default arguments defined there:

$ cat examples/string_helper_emphasize_with_function_header.ex
def emphasize(phrase, number_of_marks \\ 3)
def emphasize(_phrase, 0) do
  "This isn't the module you're looking for"
end
def emphasize(phrase, number_of_marks) do
  upcased_phrase = String.upcase(phrase)
  exclamation_marks = String.duplicate("!", number_of_marks)
  "#{upcased_phrase}#{exclamation_marks}"
end

In this example, we're also seeing an example of how we can use pattern matching in named functions. Note that the order in which we define our functions matters. Elixir will search from top to bottom for a clause that matches. If we had put the clause where we're matching against 0 on the second argument at the end, that definition of the emphasize function would become unreachable, as the other definition is more general and always matches. Elixir will help you avoid these situations, as it will emit a warning during compilation, alerting you of this situation.

Apart from using pattern matching (as we've seen in this example and on anonymous functions), on named functions we can use guard clauses, which extend on the pattern matching mechanism and allow us to set broader expectations on our functions. To use a guard clause on a function, we use the when clause after the list of arguments.

To see an example of this, we will use a guard clause on our palindrome? function. Up to this point, we were accepting an argument of any type. If we passed an integer to this function, an error would be raised, as we would be trying to call String.reverse on an integer. Let's change that:

$ cat examples/string_helper_palindrome_with_guard_clause.ex
def palindrome?(term) when is_bitstring(term) do
  String.reverse(term) == term
end
def palindrome?(_term), do: {:error, :unsupported_type}

We now state that we're expecting bitstring as an argument. We've also created a new definition of our function, which runs when the match doesn't occur on the first definition. Here it is in action:

iex> StringHelper.palindrome?("abba")
true
iex> StringHelper.palindrome?(123)
{:error, :unsupported_type}

Using guard clauses in our functions can lead to a lot of duplication, since we may be repeating the same clause over and over again. To combat this, Elixir 1.6 introduced the defguard construct, which allows us to define clauses that can be reused.

 

Moreover, using this construct may improve the readability of your code, since we can extract complex guard clauses and give them descriptive names. Let's see the previous example implemented using defguard:

$ cat examples/string_helper_palindrome_with_defguard.ex
defguard is_string(term) when is_bitstring(term)

def palindrome?(term) when is_string(term) do
  String.reverse(term) == term
end
def palindrome?(_term), do: {:error, :unsupported_type}

In this simple example, there's no clear advantage to using this construct. However, as your modules, along with your guard clauses, grow more complex, this technique becomes incredibly useful. Note that you can use defguardp to define a guard clause that is not exported, and can only be used within the module where it's defined.

Note

You can use other type-checking functions in guard clauses, as well as comparison operators, and also some other functions. You can find the full list at https://hexdocs.pm/elixir/guards.html.

To end this section, we will now showcase one of the most eminent features of the language: the pipe (|>) operator. This operator allows you to chain function calls, making the flow of your functions easy to read and comprehend. This operator takes the term that's at its left, and injects it as the first argument on the function at its right. This seemingly insipid feature increases the readability of your code, which is amazing since code is read many more times than it is written. To see this operator in action, let's add some more logic to our palindrome? function: We will now remove leading or trailing whitespaces from the term we're checking, and we'll also make our comparisons case-insensitive. This is the result:

$ cat examples/string_helper_palindrome_with_pipe_operator.ex
def palindrome?(term) do
  formatted_term = term
    |> String.trim()
    |> String.downcase()

  formatted_term |> String.reverse() == formatted_term
end

While the impact may seem negligible in this simple example, you'll see the expressiveness this operator brings as we build our application throughout the book.

Module attributes, directives, and uses

Modules in Elixir may contain attributes. They're normally used where you'd use constants in other languages. You define a module attribute with the following syntax:

$ cat examples/string_helper_with_module_attribute.ex
defmodule StringHelper do
  @default_mark "!"

  # rest of the StringHelper code
end

Then, we can use the @default_mark module attribute inside the functions of this module. This attribute only exists at compile time, as it's replaced by its value during this process.

There are some other use cases for module attributes: you can register them, which makes them accessible at runtime. For instance, Elixir registers the @moduledoc and @doc attributes, which can be used to provide documentation for modules and functions, respectively. This documentation can then be accessed at runtime by other Elixir tools, as we'll explore in the Tooling and ecosystems section.

Note

We're now mentioning macros for the first time. Macros are Elixir's mechanism to do meta-programming–generating code that writes code. We will not touch macros in this introductory chapter, as they will be properly examined in Chapter 6, Metaprogramming – Code that Writes Itself.

Elixir provides three lexically scoped directives to manage modules, plus a macro called use. We'll describe them now:

  • alias is used to create aliases for other modules. You use it as alias Helpers.StringHelper, as: StrHlp, and you can then refer to that module as StrHlp. The as: portion is optional, and if you don't provide it, the alias will be set to the last part of the module name.
  • We use require when we want to invoke what's defined as macros in a given module. As stated in the official documentation, is_odd/1 from the Integer module is defined as a macro. To use it in another module, you have to require it: require Integer.
  • When we want to access functions from other modules without having to use the fully-qualified name, we use import. When we import a given module, we're also automatically requiring it. If we're constantly using String.reverse/1 for instance, we can import it: import String, only: [reverse: 1]. Now we can just use reverse directly in our module. Apart from only:, you can also use except: to import all but a given number of functions from a module. Besides function names, only: and except: also accept :modules and :functions (which are self explanatory). You can also just use import without any option, but this isn't recommended, as it pollutes the scope of your module–always try to pass the only: option when using import.
  • Last, but not least, we have use, which is a macro. This is commonly used to bring extra functionality to our modules. Beneath the surface, use calls the require directive and then calls the __using__/1 callback, which allows the module being used to inject code into our context.

For now, you don't need to know how all of this works. It is enough to know that you have these constructs to deal with modules. When we dive into macros later in this book, all of this will become much clearer.

 

Working with collections


Contrary to the most common programming languages, Elixir doesn't have while or do ... while constructs, which makes sense, given all data types are immutable. The way to iterate in Elixir is by using recursion, through functions that call themselves. Most of your needs when working with collections are covered by the high-level abstractions Elixir provides, meaning that you may barely use recursion when writing your Elixir applications.

Nevertheless, we'll begin this section by briefly describing recursion, and show an example of a recursive function in Elixir. Then, we'll see how we can process a collection using the Enum module, and finish the section by talking about the benefits of processing a collection lazily, and how to do it using the Stream module.

 

Looping through recursion

We'll show you how to create a recursive functions through two simple examples: doubling each element on a list, and multiplying consecutive elements of a list. As mentioned earlier, although you probably won't be using this in your day-to-day coding, it's still very important to understand how this work. This way, if the abstractions Elixir provides aren't enough for your use case, you can just create your own recursive functions to accomplish what you need.

Before jumping into the examples, let's explain generally how recursive functions work. In Elixir, they're usually implemented as a multi-clause function, using pattern matching to control its flow of execution. The first clause sets the condition that will stop the recursion, and is followed by other broader clauses that apply the recursion.

In the first example, we want to take a list of integers as input, and return a new list where each element is multiplied by two. Let's see the code for such a function:

$ cat examples/double.ex
defmodule Recursion do
 def double([]), do: []

 def double([head | tail]) do
 [head * 2 | double(tail)]
 end
end

Here are its results:

iex> Recursion.double([2, 4, 6])
[4, 8, 12]

Besides using multi-clause functions, we're also using pattern matching in two ways: to know when we've reached the end (and the empty list) and treat it accordingly; and to extract the head and the tail of a list, similar to what we've shown in the pattern matching section. The recursion happens when we call double(tail). As we're only passing the tail to the recursive call, we're essentially iterating through the list. When we reach an empty list, the first clause matches, we return an empty list, and all of the intermediate calls will unfold and create our new list.

What if, instead of returning a new list, we want to return a single value? We'll exemplify this by multiplying consecutive elements of a list. Here's the code to do it:

$ cat examples/multiply.ex
defmodule Recursion do
  def multiply([]), do: 1

  def multiply([head | tail]) do
    head * multiply(tail)
  end
end

Here's its use on an IEx session:

iex> Recursion.multiply([1, 2, 3])
6

The strategy is similar to the one shown in the previous example, except, instead of adding an element to a list at each step, we're now using our head as an accumulator. Also, it's important to note that, since we're doing a multiplication, our stopping condition must return 1 (the neutral element of this operation). The definition of the stopping condition varies between different problems, and is, arguably, one of the most important steps of defining a function recursively.

A common concern when dealing with recursive functions is its memory usage, as we have multiple function calls that will get into the stack. The Erlang runtime employs tail-call optimization whenever it can, which means that a recursive call won't generate a new stack push. For the runtime to do this optimization, you have to ensure that the last thing our function does is call another function (including itself)–or, in other words, make a tail call. Here's our multiply function updated to make tail calls:

$ cat examples/multiply_with_tail_recursion.ex
def multiply(list, accum \\ 1)
def multiply([], accum), do: accum
def multiply([head | tail], accum) do
 multiply(tail, head * accum)
end

The usual strategy is to pass an accumulator around, which enables us to use the tail-call optimization. Note that there's a trade-off here: On one hand, this optimization is important when dealing with large collections (since function calls don't consume additional memory); on the other hand, code that doesn't use this optimization is usually easier to read and comprehend, as it's usually more concise. When doing recursion, consider the advantages and disadvantages of each solution.

Eager processing with the Enum module

Having seen how recursion works in Elixir, we'll now show some examples of the abstractions that are built on top of it. We'll explore the Enum module, which contains a set of functions to work on collections. We've already seen some examples of collections in the Elixir's data types section, such as lists or maps. More generally, we can use the Enum module on collections that implement the Enumerable protocol.

Note

We haven't yet covered protocols. We will do so in the Protocols section.

Taking the two examples from our Recursion section, let's see how they become incredibly simple to implement using the Enum module:

iex> Enum.map([2, 4, 6], &(&1 * 2))
[4, 8, 12]
iex> Enum.reduce([1, 2, 3], 1, &(&1 * &2))
6

The map function receives a collection and a lambda, and returns a new list where the lambda is applied to each element of the collection.

The reduce function receives a collection, an accumulator, and a lambda. The lambda receives the current element of the collection and the accumulator, and the result of this lambda is the accumulator for the following iteration. At the end of the iteration, reduce returns the final accumulator value.

Note

We're using the capture operator to define a lambda. As we've previously hinted, you can also use it to capture named functions. In the following example, we're using the Integer.is_even/1 function to check which numbers are even in a collection:iex> require IntegerIntegeriex> Enum.map([1, 2, 3], &Integer.is_even/1)[false, true, false]

You'll see the Enum module being used in the application that we'll build throughout the book. For further usage of the Enum module, check its documentation at https://hexdocs.pm/elixir/Enum.html.

Comprehensions

Elixir provides another construct to iterate collections: comprehensions. As with the functions from the Enum module, comprehensions work on anything that implements the Enumerable protocol. Let's see a simple example:

iex> for x <- [2, 4, 6], do: x * 2
[4, 8, 12]

While, in this simple example, it is similar to Enum.map/2, comprehensions bring some other interesting features. You can, for instance, iterate over multiple collections and also apply filters. Let's see these two being applied in the following example:

iex> for x <- [1, 2, 3], y <- [4, 5, 6], Integer.is_odd(x), do: x * y
[4, 5, 6, 12, 15, 18]

Here we're doing a nested iteration–for each element of the first enumerable (which is represented by x), we will iterate through all elements of the second enumerable (represented by y). Also, we're applying a filter, and the body of our comprehension only gets executed when x is odd.

We won't be using comprehensions in the application we'll build throughout this book. However, it's important to mention them, as there are cases where using a comprehension instead of functions from the Enum module renders more elegant and expressive code

Note

In our example, all comprehensions are returning a list, which is the default behavior. We can change that by passing the into: option, as you can see in this example:iex> for x <- [1, 2, 3], into: %{}, do: {x, x + 1}%{1 => 2, 2 => 3, 3 => 4} As you can see, now we're getting a map back. The into: option takes a collection that will receive the results of the comprehension. This collection must implement the Collectable protocol. This protocol can be seen as the opposite of the Enumerable protocol, and is used to create a new structure from the values of an existing collection. This also has usage outside of comprehensions–the Enum.into/2 function uses this protocol to create a new collection based on an enumerable.

Lazy processing with the stream module

We will now talk about a different way of processing collections, which, as functional programming, may require a shift in your mindset. Before talking about lazy processing, let's enumerate some of the shortcomings of working with the Enum module. The Enum module is referred to as being eager. This means that when processing a collection, this module will load the entire collection into memory. Furthermore, if you have a chain of functions you want to apply to a collection, the Enum module will iterate through your collection as many times as the functions are applying to it. Let's examine this further with an example:

iex> [1, 2, 3, 4, 5] \
...> |> Enum.map(&(&1 + 10)) \
...> |> Enum.zip(["a", "b", "c", "d", "e"])
[{11, "a"}, {12, "b"}, {13, "c"}, {14, "d"}, {15, "e"}]

Note

The \ on the end of the first two lines is to stop our Elixir console from evaluating this line right away, and wait for a new line instead. This way, we can write these operations with the pipe operator on multiple lines, which makes them more readable.

We take our initial collection and iterate it to add 10 to each element inside it. This generates a new list, which is passed to our next function. This function will zip the two lists together, which will produce a new list, which is returned to us. In this simple example, we need to traverse our list twice to build the desired result.

This is where the Stream module, and lazy processing, becomes advantageous. When working with lazy enumerables, the entire collection never gets loaded into memory, and contrary to what we're accustomed to, the computations aren't made right away. The results are produced as they are needed. Let's see this same example with the Stream module:

iex> [1, 2, 3, 4, 5] \
...> |> Stream.map(&(&1 + 1)) \
...> |> Stream.zip(["a", "b", "c", "d", "e"])
#Function<66.40091930/2 in Stream.zip/1>

As you can see, we're not getting our final list back. When we feed our list to Stream.map, the list is not iterated. Instead, the functions that will be applied on it are saved into a structure (along with the collection we're working on). We can then pass this structure into the next function, which will further save a new function to be applied to our list. This is really cool! But how do we make it return the result we're expecting? Just treat it as a regular (eager) enumerable, by applying a function from the Enum module, and it will start to produce results.

To exemplify this, we'll use the Enum.take/2 function, which allows us to take a given number of items from an enumerable:

iex> [1, 2, 3, 4, 5] \
...> |> Stream.map(&(&1 + 10)) \
...> |> Stream.zip(["a", "b", "c", "d", "e"]) \
...> |> Enum.take(1)
[{11, "a"}]

As you can see, we're now getting the expected result back. Note that this is not a result of applying our computation to all the list and then just taking the first element. We've essentially only computed results for the first element, as that's all that was necessary. If you wanted to have the full list in the end, you could use the Enum.to_list/1 function.

Streams are a really nimble way to process large, or even infinite, collections. Imagine that you're parsing values from a huge CSV file, and then running some functions on them. If you're running your application on the cloud, as most of us are these days, you probably have a short amount of memory. Using lazy processing, you can avoid having to load the whole file, processing it line by line. If you're processing an infinite collection, such as an RSS feed, lazy processing is also a great solution, as you can process each element of the collection incrementally, as they arrive.

Note that while the Stream module is amazing, it will not replace your usage of the Enum module. It's certainly great for very large collections, or even if you have a big chain of functions being applied to a collection and only want to traverse it once. However, for small or even medium collections, the Stream module will perform worse, as you're adding a lot of overhead, for instance, by having to save the functions you'll apply instead of applying them right away. Always analyze your situation carefully and take this into account when choosing to use the Enum or the Stream module for a given task.

We'll be using functions from the Stream module in the application we'll build in this book. You'll learn more about the Stream module in Chapter 4, Powered by Erlang/OTP.

Note

Elixir provides some functions that wrap most of the complex parts of building streams. If you want to build your own lazy stream, check out these functions from the Stream module: cycle, repeatedly, iterate, unfold, and resource. The full documentation for the Stream can be found at https://hexdocs.pm/elixir/Stream.html.

 

Control flow


We're now introducing control-flow constructs. In Elixir, they aren't used as often as in traditional imperative languages, because we can fulfill our control-flow needs, using a mix of pattern matching, multi-clause functions, and guard clauses. Whenever you're about to use one of the constructs we're presenting in this section, stop and check whether it's possible to employ a more functional approach. Code without these traditional control-flow constructs is usually easier to understand and test. If you get to a point where you have nested conditionals, it's almost guaranteed you can simplify it by using one of the approaches I mentioned earlier. Either way, you'll occasionally use these constructs, so it's important to know they exist.

if and unless

These two constructs can be used with the following syntax:

if <expression> do
  # expression was truthy
else
  # expression was falsy
end

unless <expression> do
  # expression was falsy
else
  # expression was truthy
end

As with the def construct, they can be inlined. For if, you'd do this:

if <expression>, do: # expression was truthy, else: # expression was falsy 

For both constructs, the else clause is optional. They will return nil if the main clause doesn't match and no else clause was provided.

cond

cond can be seen as a multi-way if statement, where the first truthy condition will run its associated code. This may substitute chains of if ... else if blocks. Let's see this with an example on IEx:

iex> x = 5
5
iex> cond do
...>   x * x == 9 -> "x was 3"
...>   x * x == 16 -> "x was 4"
...>   x * x == 25 -> "x was 5"
...>   true -> "none of the above matched"
...> end
"x was 5"

true in a condition will serve as a default condition, which will run when no other clause matches.

case

case accepts an expression, and one or more patterns, which will match against the return value of the expression. These patterns may include guard clauses. These patterns are matched (from top to bottom), and will run the code associated with the first expression that matches. Here is a simple example:

iex> case Enum.random(1..10) do
...>   2 -> "The lucky ball was 2"
...>   7 -> "The lucky ball was 7"
...>   _ -> "The lucky ball was not 2 nor 7"
...> end
"The lucky ball was not 2 nor 7"

Note that your output may differ when running this example, as we're matching against Enum.random/1. In here, the default condition is represented by using _ in the pattern, which will match anything. Although a bit more condensed, the case construct is similar to a multi-clause function.

with

This control-flow construct, introduced in Elixir 1.2, accepts one or more expressions, a do block, and optionally an else block. It allows you to use pattern matching on the return value of each expression, running the do block if every pattern matches. If one of the patterns doesn't match, two things may happen: If provided, the else block will be executed; otherwise, it will return the value that didn't match the expression. In practice, with allows you to replace a chain of nested instances of case or a group of multi-clause functions.

To demonstrate the usefulness of with, let's see an example:

iex> options = [x: [y: [z: "the value we're after!"]]]               
[x: [y: [z: "the value we're after!"]]]
iex> case Keyword.fetch(options, :x) do                              
...>   {:ok, value} -> case Keyword.fetch(value, :y) do              
...>     {:ok, inner_value} -> case Keyword.fetch(inner_value, :z) do
...>       {:ok, inner_inner_value} -> inner_inner_value             
...>       _ -> "non-existing key"
...>     end
...>     _ -> "non-existing key"
...>   end
...>   _ -> "non-existing key"
...> end
"the value we're after!"

We're using the Keyword.fetch/2 function to get the value of a key from a keyword list. This function returns {:ok, value} when the key exists, and :error otherwise. We want to retrieve the value that's nested on three keyword lists. However, let's say that if we try to fetch a key that doesn't exist on the keyword list, we have to return "non-existing key". Let's achieve the same behavior using with, operating on the same options list as the preceding example:

iex> with {:ok, value} <- Keyword.fetch(options, :x),                
...>      {:ok, inner_value} <- Keyword.fetch(value, :y),            
...>      {:ok, inner_inner_value} <- Keyword.fetch(inner_value, :z),
...>      do: inner_inner_value                                      
"the value we're after!"

Note that, since our expression is really small, we're using the shorthand do: syntax (but we can also use a regular do ... end block). As you can see, we're getting the same result back. Let's try to fetch a key that doesn't exist:

iex> with {:ok, value} <- Keyword.fetch(options, :missing_key),      
...>      {:ok, inner_value} <- Keyword.fetch(value, :y),            
...>      {:ok, inner_inner_value} <- Keyword.fetch(inner_value, :z),
...>      do: inner_inner_value                                      
:error

Since we didn't provide an else block, we're getting back the value that didn't match, which is the return value of Keyword.fetch/2 when a key doesn't exist in the keyword list provided. Let's do the same, but by providing an else block:

iex> with {:ok, value} <- Keyword.fetch(options, :missing_key), 
...> {:ok, inner_value} <- Keyword.fetch(value, :y), 
...> {:ok, inner_inner_value} <- Keyword.fetch(inner_value, :z) do
...> inner_inner_value
...> else
...> :error -> "non-existing key"
...> _ -> "some other error"
...> end
"non-existing key"

Since we're now providing an else block, we can now handle error cases accordingly. As you can see, else takes a list of patterns to match on. As you do with case, you can use _ as a default clause, which would run when the patterns above (if any) didn't match.

As you can see, with is a very helpful construct, which allows us to create very expressive code that is concise and easy to read. Moreover, you can control how to handle each error separately, using pattern matching inside the else block.

Note

The first expression provided to with has to be on the same line of with itself, you'll get a SyntaxError otherwise. If you do want to have with on its own line, wrap the expressions provided to it in parentheses.

Exceptions

Much like if and else, exceptions in Elixir aren't used as much as in other popular imperative languages. Exceptions aren't used for control flow, and are left for when truly exceptional things occur. When they do, your process is usually running under a supervision tree, and upon crashing, the supervisor of your process will be notified and (possibly, depending on the strategy) restart it. Then, upon being restarted, you're back to a known and stable state, and the effects of the exceptional event are no longer present. In the Elixir and Erlang communities, this is usually referred to as the "Let it crash!" philosophy. We'll be examining this in greater detail in Chapter 3, ProcessesThe Bedrock for Concurrency and Fault Tolerance, when we talk about processes, supervisors, and supervision trees.

For now, I'll list the traditional error-handling constructs. You can raise an error with the raise construct, which takes one or two arguments. If you provide only one argument, it will raise a RuntimeError, with the argument as the message. If you provide two arguments, the first argument is the type of error, while the second is a keyword list of attributes for that error (all errors must at least accept the message: attribute). Let's see this in action:

iex> raise "Something very strange occurred"
** (RuntimeError) Something very strange occurred
iex> raise ArithmeticError, message: "Some weird math going on here"
** (ArithmeticError) Some weird math going on here

You can rescue an error by using the rescue construct (you can rescue from a try block or from a whole function, pairing it with def). You define patterns on the rescue clause. You can use _ to match on anything. If none of the patterns match, the error will not be rescued and the program will behave as if no rescue clause was present:

iex> try do
...> 5 / 0
...> rescue
...> e in ArithmeticError -> "Tried to divide by 0."
...> _ -> "None of the above matched"
...> end
"Tried to divide by 0."

Since we're not doing anything with the error, and just returning a string, we could just use ArithmeticError in the pattern. Only use this syntax if you want to capture the error itself. When none of the patterns match, we get the error back in our console:

iex> try do
...>   5 / 0
...> rescue
...>   ArgumentError -> "ArgumentError was raised."
...> end
** (ArithmeticError) bad argument in arithmetic expression

Furthermore, you can also pass an else and/or an after block to the try/rescue block. The else block will match on the results of the try body when it finishes without raising any error. As for the after construct, it will always get executed, regardless of the errors that were raised. This is commonly used to clean up some resources (closing a file descriptor, for instance).

Note

We've mentioned in this section that we don't use exceptions to control the flow of our programs, but in fact there's a special construct in Elixir for this. The syntax is similar to the one shown earlier, but you use throw instead of raise, and catch instead of rescue. As mentioned in the official Getting Started guide (http://elixir-lang.github.io/getting-started/try-catch-and-rescue.html) this should be used in situations where it is not possible to retrieve a value unless by using throw and catch. It's also mentioned that those situations are quite uncommon in practice.

 

Typespecs and behaviours


As already mentioned in the beginning of this chapter, Elixir is a dynamic programming language. As such, we don't declare the type of each variable, as it depends on the value each variable is bound to at each moment.

Usually dynamic programming languages yield higher productivity, as programmers don't need to declare types and can focus on developing the logic of an application. However, this comes at a cost: Certain errors, which in statically-typed languages would be caught at compile-time, may only be caught at runtime in dynamic languages. The time saved by using a dynamic language is then used (often in excess) on debugging in production.

We're not advocating for statically-typed languages–this book is about Elixir, after all. But what if you could have the best of both worlds?

It turns out you can! Type specifications, or typespecs, are a mechanism to annotate function signatures with the respective types (arguments and return values). Typespecs are also used to create custom data types. Let's explore them in more detail, before jumping into behaviours.

Typespecs

Typespecs are written using the @spec module directive, followed by function_name(argument_type) :: return_type. This module directive is placed right before the definition of the function we're annotating.

To demonstrate how to apply typespecs to a function, let's bring back our palindrome? function:

$ cat examples/string_helper.ex
defmodule StringHelper do
  def palindrome?(term) do
    String.reverse(term) == term
  end
end

Given that the module name is StringHelper, and that it's using functions from the String module, we can see that this function receives a string. As it uses the == operator, and also hinted at by the trailing ? on the function name, we know this function returns a Boolean. With this information, writing the typespec for this function is straightforward:

$ cat examples/string_helper_palindrome_with_typespec.ex
defmodule StringHelper do
  @spec palindrome?(String.t) :: boolean
  def palindrome?(term) do
    String.reverse(term) == term
  end
end

You can also define your own types to use in typespecs, by using the @type directive within a module (usually the module where that type is used the most). Let's say that you frequently use a tuple in your module that contains the name of a country on the first element, and its population on the second, as in {"Portugal, 10_309_573}. You can create a type for this data structure with this:

@type country_with_population :: {String.t, integer}

Then, you could use country_with_population in typespecs as you'd use any other type.

Defining a type in a module will export it, making it available to all modules. If you want your type to be private to the module where it's defined, use the @typep directive. Similar to the @moduledoc and @doc directives (to document modules and functions, respectively), you can use the @typedoc directive to provide documentation for a certain type.

Note

In typespecs, the string type is usually represented by String.t. You can also use string but the Elixir community discourages this, with the goal of avoiding confusion with the charlist type, which represents strings in Erlang. If you use string in typespecs, the compiler will emit a warning.

Typespecs also provide great documentation, as we can quickly grasp what types this function accepts and returns. This example only shows a subset of the types you can use–please refer to the official documentation to get a full list, at

https://hexdocs.pm/elixir/typespecs.html.

Dialyzer

Dialyzer (http://erlang.org/doc/man/dialyzer.html) is a tool that ships with Erlang and performs static analysis of code. It analyses compiled .beam files, making it available for all programming languages that run on the Erlang VM (such as Elixir!). While Dialyzer can be helpful on projects that don't have typespecs (as it can, for instance, find redundant code), its power is maximized on projects that have their functions annotated with typespecs. This way, Dialyzer is able to report on typing errors, which brings you closer to the security you can get on a statically-typed language.

Although we won't be exploring Dialyzer in this book, we highly recommend its usage, as it can be very helpful. Particularly, we feel that the Dialyxir library (https://github.com/jeremyjh/dialyxir), is a great way to integrate Dialyzer into Elixir projects, as it abstracts away part of the complexity of dealing with Dialyzer directly.

Behaviours

Behaviours provide a way to describe a set of functions that have to be implemented by a module, while also ensuring that the module implements the functions in that set. If you come from an object-oriented programming background, you can think of behaviours as abstract base classes that define interfaces. After declaring the behaviour, we can then create other modules that adopt this behaviour. A behaviour creates an explicit contract, which states what the modules that adopt the behaviour need to implement. This way, we can have our business logic tied to abstractions (the behaviour) instead of concrete implementations. We can swap two implementations of the same behaviour with very little and localized change in our application–the place where we define the implementation of the behaviour we're going to use.

Let's demonstrate this concept with an example. We'll define a behaviour, called Presenter, that has only one function: present. We'll then define a module that adopts this behaviour, called CLIPresenter. We could also have other modules that would adopt this behaviour, such as a GraphicalPresenter. A behaviour is created using the @callback directive inside a module, providing a typespec for each function this behaviour contains. Thus, to define our Presenter behaviour, we use the following:

$ cat examples/presenter.ex
defmodule Presenter do
  @callback present(String.t) :: atom
end

And now we define the module that will adopt this behaviour:

$ cat examples/cli_presenter.ex
defmodule CLIPresenter do
 @behaviour Presenter

 @impl true
 def present(text) do
 IO.puts(text)
 end
end

We can see this module working in the next snippet:

iex> CLIPresenter.present("printing text to the command line")
printing text to the command line
:ok

From Elixir 1.5 onwards, we may use the @impl directive to mark the functions that are implemented as callbacks for a behaviour. In our case, we'd put the @impl directive on top of the present function:

$ cat examples/cli_presenter_with_impl_annotation.ex
defmodule CLIPresenter do
  @behaviour Presenter

  @impl true
  def present(text) do
    IO.puts(text)
  end
end

We can be even more specific and use @impl Presenter, to state that this function is a callback from the Presenter behaviour. This brings two major advantages:

  • Increased readability, as it's now explicit which functions make up part of our API and which functions are callback implementations.
  • Greater consistency, as the Elixir compiler will check whether the functions you're marking with @impl are part of a behaviour your module is adopting.

Note that when you set @impl on a module, you must set it on all callback functions on that module, otherwise a warning will be issued. Furthermore, you can only use @impl on functions that are callbacks, otherwise a compilation warning will be issued as well.

Note

You can use @optional_callbacks to mark one or more functions as optional when adopting a certain behaviour. You need to provide a keyword list, with function names as keys, and arity as the value. If we wanted our present/1 function to be optional in the Presenter behaviour, we would use:$ cat examples/presenter_with_optional_callbacks.exdefmodule Presenter do  @callback present(String.t) :: atom  @optional_callbacks present: 1end

If we didn't implement all of the functions declared in the Presenter behaviour (by commenting the CLIPresenter.present/1 function, for instance), the Elixir compiler would emit the following warning:

warning: undefined behaviour function present/1 (for behaviour Presenter)

Behaviours help us follow the open/closed principle, since we can extend our system without modifying what we already have. If, in the future, we would need a new type of presenter, we'd just have to create a new module that adopts the Presenter behaviour, without having to change the behaviour or its current implementations. We'll be using behaviours in the application we'll build in this book, and you'll see them in action again in the next chapter.

 

Protocols


Throughout this introductory chapter, we've mentioned a couple of times that Elixir has protocols, with Enumerable being one of the examples. In this section, we'll dive into protocols and even define our own!

Protocols, like the behaviours we've seen in the last section, define a set of functions that have to be implemented. In that sense, both constructs serve as a way to achieve polymorphism in Elixir–being able to display multiple forms of behavior, but all linked to a single interface. While behaviours define a set of functions that a module needs to implement, and are thus tied to a module, protocols define a set of functions that a data type must implement. This means that, with protocols, we have data type polymorphism, and we're able to write functions that behave differently depending on the type of their arguments.

Let's now see how we can create a new protocol. We'll pick up, and extend, the example present in the official Getting Started guide (at http://elixir-lang.github.io/getting-started/protocols.html). We will define a Size protocol, which will be implemented by each data type. To define a new protocol, we use the defprotocol construct:

$ cat examples/size.ex
defprotocol Size do
  @doc "Calculates the size of a data structure"
  def size(data)
end

We're stating that our Size protocol expects the data types that will implement it must define a size/1 function, where the argument is the data structure we want to know the size of.

You can use the @doc directive to add documentation to this function, as you normally do with named functions inside modules. We can now define the implementation of this protocol for the data types we're interested in, using the defimpl construct:

$ cat examples/size_implementations_basic_types.ex
defimpl Size, for: BitString do
  def size(string), do: byte_size(string)
end

defimpl Size, for: Map do
  def size(map), do: map_size(map)
end

defimpl Size, for: Tuple do
  def size(tuple), do: tuple_size(tuple)
end

Note

We didn't define an implementation for the lists, as in Elixir, size is usually used for data structures that have their size precomputed. For types where we have to compute this on demand, such as lists, the length term is used instead of size. This is further observable by looking at the name of the function used to get the dimension of a list: Kernel.length/1.

With this defined, we can see our protocol in action:

iex> Size.size("a string")
8
iex> Size.size(%{a: "b", c: "d"})
2
iex> Size.size({1, 2, 3})
3

If we try to use our protocol on a type that doesn't have an implementation defined, an error is raised:

iex> Size.size([1, 2, 3, 4])
** (Protocol.UndefinedError) protocol Size not implemented for [1, 2, 3, 4]

Note

You can define an implementation for a protocol on all Elixir data types: Atom, BitString, Float, Function, Integer, Tuple, List, Map, PID, Port, and Reference. Note that BitString is used for the binary type as well.

 

Having to implement a protocol for all types may quickly become monotonous and exhausting. You can define a fallback behavior for types that don't implement your protocol by implementing the protocol for Any. Let's do this for our Size protocol:

$ cat examples/size_implementation_any.ex
defimpl Size, for: Any do
  def size(_), do: 0
end

You have to define the desired behavior when a type doesn't implement your protocol. In this case, we're saying that it has a size of 0 (which might not make sense, since the data type may have a size different than 0, but let's ignore that detail).

We now have two options for this implementation to be used: Either mark the modules where we want this fallback behavior with @derive [Size] (the List module, for instance), or use @fallback_to_any true in the definition of our Size protocol. The former is more laborious as you have to annotate each module that you want to assume the behavior for Any, while the latter is simpler since you make it work on all data types just by changing the definition of your protocol. In the Elixir community, explicitness is usually preferred, and, as such, you're more likely to see the @derive approach in Elixir projects.

While implementing protocols for Elixir's data types already opens a world of possibilities, we can only fully utilize Elixir's extensibility when we mix them with structs. We haven't yet talked about structs, so we'll introduce them in the next section.

Structs

Structs are an abstraction built on top of maps. We define a struct inside a module, with the defstruct construct. The struct's name is the name of the module it's being defined in (which means you can only define one struct per module). To defstruct, we pass a keyword list, which contains the key-value pairs that define the fields that struct has, along with their default values. Let's define a Folder struct:

$ cat examples/folder.ex
defmodule Folder do
  defstruct name: "new folder", files_info: [], path: nil
end

We can now use it in our IEx session:

iex> %Folder{}
%Folder{files_info: [], name: "new folder", path: nil}
iex> %Folder{}.name
"new folder"
iex> %Folder{}.files_info
[]

Elixir already has a File module, which provides several functions to deal with files. One of them is the File.stat/2, which returns a %File.Stat{} struct with information about the provided path. The files_info field in our %Folder{} struct is a list, which will contain %File.Stat{} structs as elements. Let's initialize a folder with one file:

iex> folder = %Folder{files_info: [File.stat!("string_helper.ex")]}
%Folder{files_info: [%File.Stat{access: :read_write,
   atime: {{2017, 12, 31}, {16, 58, 56}}, ctime: {{2017, 12, 30}, {3, 40, 29}},
   gid: 100, inode: 3290229, links: 1, major_device: 65024, minor_device: 0,
   mode: 33188, mtime: {{2017, 12, 30}, {3, 40, 29}}, size: 509, type: :regular,
   uid: 1000}], name: "new folder", path: nil}

Note that this example assumes you have a "string_helper.ex" file in the directory where you started iex. Also note that we're using File.stat!, which works similarly to File.stat, but, instead of returning a {:ok, result} tuple, it returns the result itself.

We now have our %Folder{} struct with one file. We can now show you the syntax to update a struct, which is similar to the one used in maps (or you can use the functions from the Map module). Assuming you also have a "recursion.ex" file on your current working directory, you can use this syntax to update the struct:

iex> folder = %Folder{ folder | files_info: [File.stat!("recursion.ex") | folder.files_info]}
%Folder{files_info: [%File.Stat{access: :read_write,
   atime: {{2017, 12, 30}, {20, 8, 29}}, ctime: {{2017, 12, 30}, {20, 8, 25}},
   gid: 100, inode: 3278529, links: 1, major_device: 65024, minor_device: 0,
   mode: 33188, mtime: {{2017, 12, 30}, {20, 8, 25}}, size: 270, type: :regular,
   uid: 1000},
  %File.Stat{access: :read_write, atime: {{2017, 12, 31}, {16, 58, 56}},
   ctime: {{2017, 12, 30}, {3, 40, 29}}, gid: 100, inode: 3290229, links: 1,
   major_device: 65024, minor_device: 0, mode: 33188,
   mtime: {{2017, 12, 30}, {3, 40, 29}}, size: 509, type: :regular, uid: 1000}],
 name: "new folder", path: nil}
iex> folder.files_info
[%File.Stat{access: :read_write, atime: {{2017, 12, 30}, {20, 8, 29}},
  ctime: {{2017, 12, 30}, {20, 8, 25}}, gid: 100, inode: 3278529, links: 1,
  major_device: 65024, minor_device: 0, mode: 33188,
  mtime: {{2017, 12, 30}, {20, 8, 25}}, size: 270, type: :regular, uid: 1000},
 %File.Stat{access: :read_write, atime: {{2017, 12, 31}, {16, 58, 56}},
  ctime: {{2017, 12, 30}, {3, 40, 29}}, gid: 100, inode: 3290229, links: 1,
  major_device: 65024, minor_device: 0, mode: 33188,
  mtime: {{2017, 12, 30}, {3, 40, 29}}, size: 509, type: :regular, uid: 1000}]

As you can see, we now have two files in our %Folder{} struct.

Note

Although structs are implemented on top of maps, they do not share protocol implementations with the Map module. This means that you can't, out of the box, iterate on a struct, as it doesn't implement the Enumerable protocol.

We'll end our little tour of structs with two more bits of information. First, if you don't provide a default value when defining the fields of a struct, nil will be assumed as its default value. Second, you can enforce that certain fields are required when creating your struct. You do that with the @enforce_keys module attribute. If we wanted to make sure path was provided when creating our %Folder{} struct, we would define it as following:

$ cat examples/folder_with_enforce_keys.ex
defmodule Folder do
  @enforce_keys :path

  defstruct name: "new folder", files_info: [], path: nil
end

If you don't provide path when creating this struct, ArgumentError will be raised:

iex> %Folder{}
** (ArgumentError) the following keys must also be given when building struct Folder: [:path]
    expanding struct: Folder.__struct__/1
    iex:46: (file)
iex> %Folder{path: "/a/b/c/"}
%Folder{files_info: [], name: "new folder", path: "/a/b/c/"}

Bringing structs and protocols together

Now that we have the %Folder{} struct defined, we can define its implementation for the Size protocol.

We'll first define the implementation for the %File.Stat{} struct, as we can then use this to implement the protocol for %Folder{}. Here's the implementation for %File.Stat{}:

$ cat examples/size_implementations_file_stat_and_folder.ex
defimpl Size, for: File.Stat do
  def size(file_stat), do: file_stat.size
end

# ...

With this in place, our implementation for our %Folder{} struct is as follows:

$ cat examples/size_implementations_file_stat_and_folder.ex
# ...

defimpl Size, for: Folder do
  def size(folder) do
    folder.files_info
    |> Enum.map(&Size.size(&1))
    |> Enum.sum()
  end
end

To find out the size of a folder, we sum the size of each file it contains. As such, this implementation iterates through our files_info list, using the Size implementation for %File.Stat{} to get the size of each file, summing all the sizes in the end. In the following snippet, we can see this implementation being used on the folder variable we just defined:

iex> Size.size(folder)
779

With this, we can see the full power of mixing structs and protocols, which lets us have polymorphic functions based on the data type of their arguments. We now have a common interface, Size.size(data), that allows us to find out the size of pretty much anything we want, provided that we implement the Size protocol for the data type we're interested in.

 

Tooling and ecosystems


The last section of this chapter is dedicated to one of the most renowned characteristics of Elixir: its incredible tooling. Also, since we can directly use Erlang libraries, we can take advantage of a very mature ecosystem that has been around for decades. First, let's begin with Elixir's interactive shell, IEx.

IEx

In the beginning of this chapter, we provided some instructions on how to start and stop an Elixir shell. You've seen it in use throughout this chapter. We'll now show you some other interesting things you can do inside IEx.

In this chapter, we've been giving you links to the official documentation of several Elixir modules. You can access this documentation right from IEx, using the h command. Let's say you want to know more about the Enum module:

iex> h Enum
Enum
#...,

The preceding code provides a set of algorithms that enumerate over enumerables according to the Enumerable protocol.

Note that, for brevity, the output of this command was cropped.

You can even pass a fully qualified function name, and get information about it. Let's say, for example, you don't remember the order of the arguments for the Enum.map/2 function:

iex> h Enum.map/2
def map(enumerable, fun)
@spec map(t(), (element() -> any())) :: list()

Returns a list where each item is the result of invoking fun on each
corresponding item of enumerable.
For maps, the function expects a key-value tuple.
## Examples

 iex> Enum.map([1, 2, 3], fn(x) -> x * 2 end)
 [2, 4, 6]

 iex> Enum.map([a: 1, b: 2], fn({k, v}) -> {k, -v} end)
 [a: -1, b: -2]

Another very interesting feature, which ships from Elixir 1.5 onwards, is the ability to set breakpoints from IEx. To showcase this, let's create a breakpoint on the StringHelper.palindrome?/1 function we've defined in the Functions and modules section, present in the "string_helper.ex" file:

iex> break! StringHelper.palindrome?/1
1
iex> StringHelper.palindrome?("abba")
Break reached: StringHelper.palindrome?/1 (string_helper.ex:2)

    1: defmodule StringHelper do
    2:   def palindrome?(term) when is_bitstring(term) do
    3:     String.reverse(term) == term
    4:   end

pry(1)> term
"abba"
pry(2)> whereami
Location: string_helper.ex:2

    1: defmodule StringHelper do
    2:   def palindrome?(term) when is_bitstring(term) do
    3:     String.reverse(term) == term
    4:   end

As you can see, you could access the argument passed to the function, term, and also use whereami to show the location where the breakpoint stopped the execution. To resume the execution, you use the continue command to go to the next breakpoint, or respawn to exit and start a new shell.

From Elixir 1.6 onward, you can also use pattern matching and guard clauses when setting a breakpoint. The breakpoint we defined earlier could also be set as break! StringHelper.palindrome?("abba"), which would make our breakpoint work when that function is called with "abba" as the argument.

Note

You can also use IEx.pry to set a breakpoint on the source-code file, instead of doing it via IEx. Use the method that's most appealing to you.

To finish this section, we'll show you a handy feature of IEx: the ability to access values from the history. Sometimes, you call a certain function and realize afterward that you wanted to bind the result of that function call to a variable. If this happens, you can use the v/1 function. You pass it a number, which represents the position of the expression from which you want to retrieve the value (starting at 1). You can also pass a negative number, which makes the position relative to the current expression in the shell. You can call this function without providing an argument, which is the same as calling v(-1)–which means we're getting the value from the last expression. Let's see an example:

iex> 7 * 3
21
iex> a = v
21
iex> a * 2
42
iex> v(-3)
21

There are more things you can do with IEx, such as configuring it or connecting to remote shells. Please refer to the official documentation (at https://hexdocs.pm/iex/IEx.html) for further usage examples of IEx.

Mix

Mix is the Swiss-army knife of the Elixir tools. It's used to compile and run your application, manage your dependencies, run tests, and even for profiling your code. You can see which Mix tasks are available by typing mix help in your terminal. We won't explain its usage here, since the next chapter's purpose is to show you how you can use Mix to create and maintain an Elixir project.

ExUnit

Elixir comes with a fully fledged unit test framework: ExUnit. We'll use it to write tests for the application built in the course of this book. Chapter 9, Finding Zen Through Testing, is dedicated to testing, where we'll demonstrate how you can write tests for your application that let you sleep at night.

Erlang interoperability

As we stated at the beginning of this chapter, Elixir targets the Erlang runtime. Elixir is compiled to byte-code that can run on an Erlang VM (or BEAM), and Erlang libraries can be used in your Elixir projects (and vice versa). The philosophy in the Elixir community is to not reinvent the wheel and directly use Erlang libraries when appropriate. The creation of Elixir libraries that simply wrap an underlying Erlang library is discouraged, as you can directly call an Erlang library from your Elixir code.

We can take advantage not only of Erlang libraries, but also of their tooling. For instance, Erlang ships with a tool called Observer, which allows you to monitor your server and provides you with tons of useful information about the Erlang VM, such as the running processes or dynamic charts of load. We'll explore this tool in greater detail in Chapter 11Keeping an Eye on Your Processes, when we talk about monitoring.

To utilize an Erlang library, you write its name as an atom and then call functions on it. Elixir doesn't have a Math module, so when more advanced mathematical operators are needed, it's common to call the math Erlang library. Let's use this library to calculate the natural logarithm of 10:

iex> :math.log(10)
2.302585092994046 

Note

You can check out Erlang's standard libraries at http://erlang.org/doc/apps/stdlib/index.html.

When we introduced the data types in Elixir, we mentioned the Port type, which is a reference to a resource used by the Erlang VM to interact with external resources. Let's now see how we can use them to interact with an operating system process. To interact with ports, we use functions from the Port module. In the following example, we'll use the whoami UNIX command, which prints the username associated with the current user. To do that, we open a port and provide the name of the executable we want to run:

iex> port = Port.open({:spawn, "whoami"}, [:binary])
#Port<0.3730>

We passed the [:binary] option so that we get our result as a binary instead of a list of bytes. We now use the IEx flush() helper to print the messages received by the port:

iex> flush()
{#Port<0.3731>, {:data, "dcaixinha\n"}}
:ok

As our operating system process died after returning its result, the port is also closed. If this was not the case, we could use the Port.close/1 function to explicitly close the port. Also note that this was a very simple example. You can have more complex interactions by using the Kernel.send/2 function to dynamically send messages (that contain commands) to your port. In the official documentation for ports (which is available at https://hexdocs.pm/elixir/Port.html), you can see how this can be achieved.

If all we're interested in is running an operating-system command and getting its result back, we can use the System.cmd/3 function, which is an abstraction on top of ports that allows us to achieve this effortlessly.

 

 

Summary


We've now reached the end of the first chapter, which contains a condensed introduction to Elixir. We didn't visit every aspect of the language, but provided several links that are worth exploring. Our goal is to provide the necessary knowledge for you to follow the next chapters, where we'll build a complex application. A lot of ground was covered in this chapter, so it's normal if you don't remember every concept we covered. Let's recap the most important points:

  • Elixir is a dynamic language, and the type of a variable is determined by the value it holds.
  • Every data type is immutable, which means that you never actually change the contents of a variable, you operate on copies of it. You can, however, rebind a variable, which will make it point to a new memory location, leaving its old contents untouched.
  • Elixir code is organized in modules, which contain a set of functions.
  • Functions are first-class citizens, as you can assign them to variables and pass them as arguments to other functions.
  • Iteration in Elixir is always made through recursion. There are no while or do ... while constructs. Elixir provides a set of modules, such as Enum and Stream, that abstract the recursion and let you work with collections efficiently.
  • The usual control-flow constructs, such as if and else statements, are less common in Elixir. You still use them occasionally, but you normally use a combination of pattern matching and multi-clause functions to control the flow of your programs.
  • Exceptions aren't used for control-flow. Instead, they are used for truly exceptional events. We rely on supervision trees (which we'll discuss in Chapter 3, ProcessesThe Bedrock for Concurrency and Fault Tolerance) to recover from exceptions.
  • You can annotate your functions with typespecs, or type specifications, bringing some of the safety of static-type languages into a dynamic language. These annotations also serve as a type of documentation for your functions.
  • Elixir provides great mechanisms to have extensibility in your code, such as Behaviours and Protocols.
  • Elixir comes bundled with amazing tooling, and you can also take advantage of all the libraries in the Erlang ecosystem.

 

In the next chapter, we'll learn how to use Mix to create a new project, while also describing the application we'll build throughout this book. This was the only chapter that contained ad hoc examples, and in the following chapters, we'll always use our application to exemplify the concepts that we want to explain.

About the Authors
  • André Albuquerque

    Andr Albuquerque is a software engineer at Onfido, after working in the banking industry for seven years. He has a master's degree from Instituto Superior Tcnico in distributed systems and software engineering, and, during his banking detour, he obtained a master's degree in economics. He is currently developing Onfido's microservices using Elixir and Ruby, learning every day about how applications can score and scale if we apply the correct tools and sound coding practices from the get-go. In his time off, he loves to build his own keyboards, play basketball, and spend time with his wife and son.

    Browse publications by this author
  • Daniel Caixinha

    Daniel Caixinha is a software engineer at Onfido, where he is using Elixir to build resilient systems that can also handle the high growth of the business. After graduating from Instituto Superior Tcnico, he joined the startup world, mainly using Ruby, but also got the chance to play around with Elixir. Upon joining Onfido, he got the chance to take Elixir more seriously, which made him fall in love with functional programming in general, and Elixir in particular. Besides building Elixir applications, he is fostering the use of Elixir, being also a member of the Lisbon Elixir meetup.

    Browse publications by this author
Mastering Elixir
Unlock this book and the full library FREE for 7 days
Start now