Clojure for Finance

By Timothy Washington
  • Instant online access to over 7,500+ books and videos
  • Constantly updated with 100+ new titles each month
  • Breadth and depth in over 1,000+ technologies
  1. Orientation – Addressing the Questions Clojure Answers

About this book

Clojure is a dynamic programming language with an emphasis on functional programming. Clojure is well suited to financial modeling as it is a functional programming language. Such languages help developers work with high-level mathematical abstractions without having to implement low-level code that handles the arithmetic operations.

Starting with the importance of representing data and calculations effectively, this book will take you all the way to being competent in financial analytics and building financial applications.

First, we introduce the notions of computation and finance, which will help you understand Clojure's utility to solve real-world problems in many domains, especially finance. Next, we will show you how to develop the simple-moving-average function by using the more advanced partition Clojure data transformation function. This function, along with others, will be used to calculate and manipulate data.

You will then learn to implement slightly more complicated equations, how to traverse data, and deal with branching and conditional dispatch. Then, the concept of side-effecting and its various approaches are introduced, along with the strategy of how to use data as the interface to other systems. Finally, you will discover how to build algorithms while manipulating and composing functions.

Publication date:
January 2016
Publisher
Packt
Pages
188
ISBN
9781785289286

 

Chapter 1. Orientation – Addressing the Questions Clojure Answers

Hello. In this book, I'll try to demonstrate Clojure's utility in processing and analyzing financial data. Whether it is core banking, risk assessment, compliance, and so on, this book's audience mostly comprises finance professionals who use Clojure to improve their Excel spreadsheets or existing toolsets. As such, this will be a beginners programming book, but it won't be light on substance. I want to assert a broader philosophical point: better tools make your professional life easier. By better, I mean tools that offers greater expressive power, which is a measure of a tool's capacity to let you do and see more with the same or less amount of effort as compared to other tools.

To justify replacing or augmenting your existing tools, I'll try to point out the benefits of using functional programming (Clojure, in this case) in an organization in order to fully represent and manage capital. You might want to know how you can quickly and effectively analyze financial conditions and decide when and how to invest and maximize capital growth. So, I'll begin by weaving together the two concepts of capital flows and state-of-the-art computation. You will see how concepts, such as functional programming (FP), part of computer science's cutting edge, enable a high fidelity of data representation and transformation and information extraction.

The goals of this book are as follows:

  • Highlighting the importance of quickly and effectively representing data and its calculations

  • Showing how Clojure can easily represent data and quickly perform calculations

  • Giving you confidence to deal with basic data types and manipulations

  • Pointing you to more advanced approaches to calculate and transform data as well as build algorithms

There are an unlimited number of financial equations we can study (such as the cost of capital, rate calculation, fee calculation, and so on). So, I think it will be more useful to focus on one particular application. As such, this book's pedagogy will be to walk you through slowly building a set of lagging price indicators that follow a moving stock price time series. I'll also introduce you to topics, such as I/O, concurrency, macros, types, and type theory. However, as these topics are out of scope for this book, we won't go into very much detail about them. These are tools that are used in building software libraries and systems. There are several places that you can refer to for more advanced information.

 

Notions of computation


I'll state up front my position that computer programming languages are simply tools that let us communicate with other programmers and give human control of the machine. But, what does it mean to compute? Alan Turing was an English mathematician who invented the disciplines of computer science and artificial intelligence. You can read more about him at http://en.wikipedia.org/wiki/Alan_Turing. He believed that computation can be thought of as an actualization or concrete version of a mathematical function. His ideas of computation can be found in the contributions he made to the concept of an algorithm and his description of a Turing Machine. He perceived an algorithm to be a step-by-step procedure used for calculations. These steps are expressed as a finite list of well-defined instructions to calculate a function. Further, Turing postulated a theoretical computer (the Turing Machine) that could calculate anything that is computable, in spite of the complexity involved.

Others, such as Alonzo Church, approached the notion of computation by trying to solve the Entscheidungsproblem, which is German for a decision problem (http://en.wikipedia.org/wiki/Entscheidungsproblem). The decision problem asks for an algorithm, where the input is a statement of first-order logic and answers yes or no according to whether the statement is universally valid. Alonzo Church proposed the lambda calculus as a formal system in mathematical logic that satisfied this decision problem. The lambda calculus expresses computation based on function abstraction and application using variable binding and substitution (http://en.wikipedia.org/wiki/Lambda_calculus).

Further still, MIT professor, Gerald Sussman, says, "We (as humans) really don't know how to compute" (you can read more at http://www.infoq.com/presentations/We-Really-Dont-Know-How-To-Compute). That is in comparison with data and computation that we see in the natural world. Plants, for example, can harvest up to 95% of energy from the light that they absorb. They transform sunlight into carbohydrates in one million billionths of a second, preventing much of this energy from dissipating as heat. This near instantaneous process uses the basic principle of quantum computing—the exploration of a multiplicity of different answers at the same time—to achieve near perfect efficiency. The protein structure of a plant, which can be derived using this quantum effect, somehow allows the transfer of energy but not heat. According to David Biello, "Inside every spring leaf is a system capable of performing a speedy and efficient quantum computation" (you can read more about it at http://www.scientificamerican.com/article/when-it-comes-to-photosynthesis-plants-perform-quantum-computation/). We as humans just don't understand how it happens. Gerald Jay Sussman compares our computational skills with a genome, concluding that we are way behind in creating complex systems such as living organisms.

Even if our tools and approaches are primitive, we can still grasp the importance of such capabilities. The structures of the modern world dictate tools that let humans quickly calculate, abstract, and derive algorithms (ways of representing abstract data, credit in this case). The computation tools we use simply give us fine-grained control of these problems. Without them, the modern world as it is today would not be possible. Strong computation has applications in every real-world discipline [engineering, science, business (credit and financial systems), and so on], or at least any discipline where data can be represented digitally.

 

Notions of finance


It's useful to think of capital as a store of time and labor. Let's consider money outside our current banking system. In The Ascent Of Money: A Financial History of the World by the Penguin Group, Niall Ferguson discusses the history of money and credit in human civilization (you can find it at http://www.amazon.com/The-Ascent-Money-Financial-History/dp/0143116177). This approach lets us consider how societies use capital to distribute wealth. Among other things, he argues that the evolution of credit and debt is as important as any technological innovation in the rise of a civilization. I've mentioned this to highlight the purpose of finance and the fact that money and finance have existed across many banking systems, including before our current one. It's important that this approach not be ideological. I want to frame the core purpose and function of a bank and flesh out the mechanics of money flows.

When programming (computers), the first task is to build an understanding of the systems we're trying to model and the problems that we are trying to solve. As such, let's distill some central notions of money and how it fits into the banking function. This is written in the spirit of banks simply becoming information processes and looking more like software companies. Technology produces alternative mediums of capital, such as cryptocurrencies, for example, bitcoin (you can find more about this at http://en.wikipedia.org/wiki/Cryptocurrency). But this concept is outside the scope of the book. I make the point to emphasize the seismic effect that software is having on finance and all modern professions. I will try to describe the context and constraints within which such banking software must operate.

Money is a medium of exchange, unit of account (divisible, fungible, or of a specific measure or size), store of value, and a standard of deferred payment. In economics, money creation is the process by which the money supply of a country or a monetary region (for example, the EU) is increased. Changes in the quantity of money may be caused due to the actions of a central bank, depository institutions (principally commercial banks), or the public. However, the major control rests with a central bank. The actual process of money creation takes place primarily in banks (refer to https://archive.org/details/ModernMoneyMechanics and http://upload.wikimedia.org/wikipedia/commons/4/4a/Modern_Money_Mechanics.pdf). In the U.S., the money creation process centers around the US Federal Reserve. So, a central bank may introduce new money into an economy by purchasing financial assets or lending money to financial institutions or governments. The majority of money in our modern economy is created by commercial banks who give loans or demand deposits.

There's much more to take into account. However, this context allows us to begin considering how to faithfully represent credit levels and flows within and between bank and bank-like entities. We can also start thinking about a commercial bank's core functions and constraints, and the systems it should implement to perform all of these.

 

Concrete types of computation


Most to all computer languages are Turing Complete or can compute every Turing-computable function. This refers to a language that has conditional branching and allows an arbitrary number of variables. If this is the case, then how do different language categories address the notion of computation?

  • Imperative or procedural programming (Assembly, C, Fortran, et al. ): This generally describes computation in terms of statements that change a program state.

  • Object-Oriented programming (Simula, Smalltalk, Java, C++, et al.): This grew out of a need to build larger and more complex systems. Its computation model focuses on data encapsulation and object interaction. While procedural programming treats computation in terms of statements that change a program state, the object-oriented (OO) paradigm encapsulates program states in objects and procedures in methods.

  • Functional programming (Lisp , R, Haskell, APL, et al.): This treats computation as the evaluation of mathematical functions and avoids state and mutable data. In contrast, procedural and object-oriented programming emphasize changes in state, while Functional Programming FP emphasizes the application of functions, very often with immutable data. FP has its roots in the lambda calculus, and many functional programming languages can be viewed as elaborations of lambda calculus.

There are other categories and techniques being researched, such as logic programming, type theory, and so on. We'll focus mainly on functional programming and Clojure, as they provide sufficient and powerful expressivity needed to model our problem domains. These provide a high fidelity of information, faithfully representing credit levels and credit flows within and between bank and bank-like entities. For example, the notion of a function used in imperative programming is that it can have side effects that may change the value of the program state. Functions lack referential transparency, where the same language expression can result in different values at different times depending on the executing program state. This has negative implications when making financial calculations or representing credit properly in your systems. Conversely, in FP, the output value of a function depends only on the arguments that are input to the function, which is much closer to a mathematical function. Eliminating these side effects make it much easier to understand and predict the behavior of a program. This and FP's close proximity to mathematical functions are some of the key motivations for the development of functional programming.

 

Tooling


Let's begin by setting up the tools we'll need to run the book's code. You can author the code for this book in any text editor you prefer. I'll be using Emacs. Leiningen will be the build tool that we'll use to compile and run our Clojure code. It lets us leverage Clojure's many key features shown as follows:

  • Functional programming or a first class function evaluation model (a la lambda calculus).

  • Immutability is a feature where data that is created is never changed. It is only transformed via the functions you apply.

  • Laziness is the feature of not evaluating any code until it is absolutely required. This saves computing resources.

  • Homoiconic, which refers to the programs you write, is actual data and vice versa.

  • A well-designed, which syntax makes your code easy to read and reason out.

  • A Read-Eval-Print-Loop (REPL) lets us quickly evaluate our code and make changes in real time.

The following steps will guide you to create your first Clojure project:

  1. The first requirement is a Java runtime, if you don't already have one. You can test this out by running java -version in the shell or command prompt provided by your system. If you don't have Java, you can download it from https://java.com/en/download/. You can also consult your OS' software or package management tool for the best way to install Java.

  2. The next step is to install Leiningen (you can do this from http://www.leiningen.org). Follow Leiningen's installation instructions by visiting http://leiningen.org/#install. You can test your installation by executing lein to see the help menu that is produced.

  3. Now we're ready to create a Clojure project and perform some simple operations.

  4. In your shell, within the directory of your choice, run lein new edgar. There should be a new project directory there named edgar/.

  5. Next, change into the edgar/ directory and run lein repl. After a few seconds, you will be brought into an REPL.

  6. From here, you can begin to evaluate primitive expressions, such as numbers and strings. In your REPL, type 1 [Enter], and then "foobar" [Enter].

  7. Clojure is also known as a list processing language or a LISP. Now we're going to evaluate a list with a function as the first argument, which is (+ 1 1).

  8. Clojure functions are evaluated in the first position of a list with all function arguments, including subexpressions, placed afterwards.

  9. Try evaluating (+ 5 (* 2 4 6)). Here, the * function is applied to 2 4 6, which produces 48. After this, the REPL applies + to 5 and 48, yielding 53.

Clojure has some special reserved characters that it needs to use exclusively. As such, we won't be able to use them when naming our program elements. The Macro characters section of Clojure's Reader page at http://clojure.org/reader, itemizes them.

 

A first look at Clojure's core functions


There are many places you can refer to for Clojure's core functions such as the official Clojure documentation (http://clojure.org/documentation). Additionally, this site provides a handy cheatsheet that gives a bird's-eye view of core functions and their categories (http://clojure.org/cheatsheet).

These are a few community maintained sites that help you get an overview of Clojure's core functions:

Let's take a look at the Grimoire Community Clojure documentation. If you navigate to http://conj.io, you'll see a good mix of the official cheatsheet's overview, with a dynamic ability to filter on function names that we type into a search field. The functions are also grouped into overriding concerns. We'll only touch on a few functions from each category. The following section will provide you with a Clojure environment where you can explore Clojure's core functions.

Note

Note that the text following ;; refers to comments and is not evaluated by Clojure.

Primitives

Let's have a look at the following list of primitives:

  • Nil: This simply represents the absence of a value. It is the lowest in any sort order (for example, when appearing in a sorted set) and equates to a logical false (for example, when appearing in an if condition):

    nil  ;; nil
    (type nil)  ;; nil
  • Booleans: Clojure's primitive boolean types are true or false. These are java java.lang.Boolean classes under the hood:

    true  ;; true
    (type false)  ;; java.lang.Boolean
  • Numbers: These are functions to help you test, generate, and manipulate numbers. Here is an example of how to apply the most basic number functions:

    (+ 9 5)  ;; 14
    (- 400 18.75)  ;; 381.25
    (* 6 10)  ;; 60
    (/ 50 5)  ;; 10
    (quot 50 5)  ;; 10 - the quotient (same as division `/`) of the numerator and denominator, respectively
    (rem 26 12)  ;; 2 - the remainder of the numerator and denominator, respectively
    (mod 26 12)  ;; 2 - the modulus (same as remainder `rem`) of the numerator and denominator, respectively
    (inc 67)  ;; 68 - increment the argument (a number) by 1
    (dec 100)  ;; 99 - decrement the argument (a number) by 1
    (max 250 45)  ;; 250 - the maximum number between the first and second arguments
    (min 250 45)  ;; 45 - the minimum number between the first and second arguments
  • Characters: Characters in Clojure are either single nonnumeric (*, +, !) or alphanumeric (a, s, d, f) things. Characters are prefixed with a backslash \h, \e, \l, \l, \o. We can also represent special characters, such as a \newline, \space, \return, and so on.

  • Strings: Strings are a simple sequence of characters. They are surrounded by double quotes, such as "hello". As you'll see in the next section, this is very close to the definition of a collection. So, some collection functions work on strings as well. Any function that takes a collection and doesn't require it to be a persistent collection can often operate on a string:

    (reverse "hello")  ;; (\o \l \l \e \h)
    (count "hello")  ;; 5
    (first "hello")  ;; \h

    This is a subtle distinction. Sometimes, you'll need to pull in string-specific functions that do what you mean. The clojure.string/reverse function versus the clojure.core/reverse function is such an example:

    (require '[clojure.string :as s])
    
    (s/reverse "hello")  ;; "olleh"
    (reverse "hello")  ;; (\o \l \l \e \h)
    (s/join ", " "hello")  ;; "h, e, l, l, o"

    We can also apply functions to patterns of strings. These patterns are described with regular expressions, which are written as strings (double-quotes) and prefixed with a hash. Regular expressions follow Java's Regular Expression system, which is specified in the java.util.regex.Pattern class, available at http://java.sun.com/j2se/1.5.0/docs/api/java/util/regex/Pattern.html. They can often be a more expressive way of targeting a string pattern in a large block of text. For example, we can target only continuous blocks of numbers. Or, we can target a specific word or character sequence, as shown here:

    (re-find #"\d+" "hello1234")  ;; "1234"
    (s/replace "Hello World" #"World" "Clojure")  ;; "Hello Clojure"
  • Symbols: Symbols in Clojure point to things in a system runtime. For example, the core functions we have looked at so far are actually represented by symbols that point to a function. So, if we try to evaluate inc or first outside a list, the runtime system will return a gnarly looking function representation. We can take this result function and use it somewhere else. Symbols can contain any alphanumeric character, must begin with a nonnumeric character, and can also contain *, +, !, -, _, and ?. We can create our own symbols or use a function, such as def, to assign values to symbols:

    (symbol 'hello)  ;; hello
    (symbol "hello")  ;; hello
    (symbol "clojure.core" "hello")  ;; clojure.core/hello
    (def hello "world")  ;; #'user/hello
    hello ;; "world"
  • Keywords: Keywords are like symbols, except they are self-referencing. If you pass a keyword to a Clojure environment, you'll get back the same :hello ;; :hello value. This makes them handy as keys or identifiers to other associations. For example, a popular Clojure pattern is to use keywords as keys in maps (discussed shortly) or other associative datatypes. This is because keywords act as self-referring functions. Being able to act like functions means that we can use keywords as functions of other data structures. So taking the previous example of the map, we can look up values on a map, such as (:foo {:foo "bar" :hello "world"}), and get the result, which is "bar":

    :hello  ;; :hello
    (:foo {:foo "bar" :hello "world"})  ;; "bar"

Collections

All of these Clojure collection types are what is known as immutable. This property means that once a collection has been created, it cannot change. If we want to operate on a collection, we can call a function over it. A new collection is returned and it shares the original's structure plus any modifications resulting from the function. This, as we'll explore later on, is a central part of Clojure's computation model. And, it turns out that it's an efficient use of a computer's resources. Let's take a look at the following Collection types:

  • Lists: Lisps, including Clojure, have a syntax that is represented directly in data structures. This is what's known as homoiconic—where the code is the data. We'll explore this concept later on. However, homoiconicity offers a lot of benefits, including the ability to dynamically change your running program or even produce new programs if you want. This is an advanced concept that we'll explore a little more in later chapters. Lists in Clojure (or any Lisp) are its most rudimentary data structure. They are a linear sequence of items. The first item points to the next and so on until the end. They are also the primary way of invoking functions. Evaluating this following form will apply the max function to its two arguments (max 4 90). In this format, the function always goes at the beginning of the list. All arguments to this function come after. An unevaluated list is only a data structure. As such, we can put anything at the beginning of a list (not just a function). However, we must prefix lists with an apostrophe (called quoting), for example, '(4 90 "hello"). Another important fact to note is that the efficient point of access for a list is its head. Randomly accessing a list, say, in its middle, is relatively inefficient:

    (conj '("a" "s" "d" "f") "z")  ;; ("z" "a" "s" "d" "f")

    Here, "z" gets "conjoined" to the front of the ("z" "a" "s" "d" "f") list, due to list's head first efficiency attribute

    (count '("a" "s" "d" "f"))  ;; 4
    (empty '("a" "s" "d" "f"))  ;; ()
    (empty? '("a" "s" "d" "f"))  ;; false
  • Vectors: Vectors are also a linear sequence of items from the beginning to the end, as shown in the following code. They are denoted by opposing square brackets, such as [75 6 452 40]. The difference, as compared to a list, is that they give efficient random access. They are also meant to provide a visual break from lists. This makes them comparatively easier than other Lisps for the purpose of reading Clojure source code. A vector is closer to an array as opposed to a list, which is more like a linked list:

    (conj ["a" "s" "d" "f"] "z")  ;; ["a" "s" "d" "f" "z"]

    Here, "z" gets conjoined to the end of the vector as vectors offer efficient random access

    (mapv inc [1 2 3 4])  ;; [2 3 4 5]

    This applies (or maps) the inc function over each of the vector's elements

  • Sets: Sets are a collection of unique items. This is to say that we can only put one of each item into a set. They are denoted by curly braces, prefixed with a hash, for example, #{1 2 3 4}. Your Clojure environment will give you an error if you try to create a literal set with a duplicate item, which is #{1 2 3 3}. There's also a neat trick to note. Sets (along with maps and keywords) can act as functions. In this case, a set can be placed in the first position of a list and then applied to an argument. What's happening here is that the set is acting as the "does this exist" function of its items. So, (#{"a" "s" "d" "f"} 2) will return nil, because 2 is not in the set. However, (#{"a" "s" "d" "f"} "d") will return the d item as it is in the set:

    (#{"a" "s" "d" "f"} 2)  ;; nil
    (#{"a" "s" "d" "f"} "d")  ;; "d"
  • Maps: Maps are Clojure's associative data structure. They associate keys to values. Keys and values can include any Clojure data structure. But, very often, you'll see Clojure code using keywords as keys. Maps are denoted by opposing curly braces, for example, {:hello "world" :foo "bar"}. Similar to sets, maps act as functions of their elements—in the form of an associative look up in this case. So, if I evaluate ({:hello "world" :foo "bar"} :foo) in a Clojure runtime environment, I'll get back bar:

    ({:hello "world" :foo "bar"} :foo)  ;; "bar"
    (assoc {} :a "b")  ;; {:a "b"}
    (zipmap [:hello :foo] ["world" "bar"])  ;; {:foo "bar", :hello "world"}

There are many more core Clojure functions. Some of the categories you should be seeing in Grimoire are mentioned as follows. Further in the book, we'll explore the concepts of vars, macros, first-class functions, sequences, and so on. But at least for now, you have a place that you can go to look up functions that pertain to these topics:

  • Sequences: Sequences are a central part of Clojure's computation model. They are a logical sequential representations over a more concrete form, such as a vector, list, set, and so on. Most of Clojure's core functions (map, reduce, and so on) operate on and produce sequences.

  • Functions: Functions in Clojure form the core of how to do things. As in other functional programming languages, functions are first class program elements that can be passed around and transformed just like data. As such, Clojure has functions that can change functions to delay evaluation, change their argument order, and so on.

  • Macros: Any computer language's syntax gets translated to an internally abstract tree in order to be run. Since Clojure code is already data (the property of being homoiconic), macros provide a way of changing an actual program.

  • Reader Macros: These are preexisting macros that are available in the language's runtime. They perform useful tasks such as converting a string into an internal date representation.

  • Metadata: This contains data about our functions or program elements. Metadata is attached directly to the thing it describes. However, it does not factor in the actual value of its subject. It simply provides useful information, such author information, documentation, and so on.

There are many more categories of functions to explore. Having a look at the various functions in Grimoire and the official documentation, will begin to give you a feel for each's category, and, thus, utility.

 

Summary


In this chapter, we were introduced to the notions of computation and finance. This gave us the background required to understand Clojure's utility to solve real-world problems in many domains and finance in particular. It also gave us an orientation to help us understand Clojure's approach to computation. Clojure fits under the banner of functional programming, a different approach from imperative or object-oriented programming. We took a first look at its core Functions that act on primitives (numbers, characters, strings, symbols, and keywords) and Collections (lists, vectors, sets, and maps). Many other core functions and their categories (sequences, functions, and others) will be addressed further on in this book. Finally, we installed the Leiningen build and run tool, which will help us build and run our functions throughout the book.

In the next chapter, we'll take a look at Clojure's principles and core artifacts. We'll begin to apply these to our financial data structures and functions.

About the Author

  • Timothy Washington

    Timothy Washington is a senior software developer with over 15 years of experience in designing and building enterprise web applications from end to end. His experience includes delivering stable, robust software architectures to organizations ranging from start-ups to Fortune 500 companies. His skills include managing agile projects, systems analysis and design, functional programming, DSL and language design, and object-oriented design, with contributions to the open source community.

    Browse publications by this author

Recommended For You

Clojure Programming Cookbook

Handle every problem you come across in the world of Clojure programming with this expert collection of recipes

By Makoto Hashimoto and 1 more