Chapter 2. Metaprogramming

In this chapter, we will cover the following recipes:

• Representation of a Julia program

• Programs for metaprogramming

• Expressions and functions for metaprogramming

• Macros

• Advanced concepts in macros

• Function and code generation

Introduction

Metaprogramming is a concept where by a language can express its own code as a data structure of itself. For example, Lisp expresses code in the form of Lisp arrays, which are data structures in Lisp itself. Similarly, even Julia can express its code as data structures.

This makes it possible for Julia to generate and transform code through a Julia program. Julia has really nice reflection properties. So, the property of metaprogramming makes it easy to handle repetitive programming and function execution in data science and, especially, while handling big data in the Map Reduce framework.

Representation of a Julia program

In this section, you will study the life of a Julia program and how it is actually represented and interpreted by Julia. You will also learn what is meant by "a language expressing its own code as a data structure of itself."

This section will act as a foundation for learning about the concept of metaprogramming and how Julia uses it for generating code.

Getting ready

To get started with this section, you must simply have your Julia REPL up-and-running.

How to do it...

Firstly, it is very important to know that every Julia program starts out as a string. Let's consider a short program for adding two variables as our Julia code and use it to learn how Julia interprets programs:

code = "a + b"

It would look like this:

Now, if you parse the preceding string code, it would return an object of type Expression. Let's check it by actually parsing an example Julia program and checking for its type:

check = parse(code)

The output would look like this:

You will learn...

Symbols and expressions

In this section, you will learn about symbols and expressions in detail. They have a syntactic importance in the metaprogramming concepts of Julia. So, this section would explain them in detail, so as to appreciate the concepts covered so far and those to follow.

Symbols

Symbols are the basic blocks of expressions. They are used for concatenating two strings together. They are also used as interned strings while building expressions.

Getting ready

There aren't any major requirements for this chapter. The only requirement is that your Julia REPL should be up and running.

How to do it...

Symbols can take in some arguments and then return the concatenated string of the string representations of those arguments. This is an example of how you can do it in the REPL:

• symbol("FirstName", "LastName")

  The output of the preceding command would look like this:

  

• symbol("FirstName", 45)

  The output of the preceding command would look like this:

  

• symbol("Foo", :Bar, 86)

  The output of the preceding command would look like:

  

Symbols create interned strings that are used for building expressions. An interned string is an immutable string that is used during string processing for optimizing time and space. The character : is used to create symbols. So, a symbol always...

Quoting

The usage of a semicolon to represent expressions is known as quoting. The characters inside the parentheses after the semicolon constitute an Expression object.

How to do it...

To check this behavior, let's check for the type of a similar statement that has an object inside the parentheses after a semicolon. This can be done in the REPL as follows:

typeof(:((a + b) * c) / 6))

The preceding command gives the following output:

Multiple expressions can be represented as a block by quoting them. The syntax would be as follows:

exp = quote
              some code
              some more code
              more code
              a little more
              ...
           end

An example with some code inside the code block would look like this:

Now, let's verify the type of the exp variable with the typeof() function.

So, the the code block enclosed inside quote and end is indeed an expression.

How it works...

Quoting is the concept of creating expression objects using the : character...

Interpolation

Sometimes, construction on Expression objects is difficult, especially when you have multiple objects and/or variables. This is used for easy and readable expression construction.

So, interpolation is a way to deal with this. Such objects can be interpolated into the expression construction through a $ prefix. This process is also called splicing expressions, variables, or literals into quoted expressions.

How to do it...

Suppose there is a literal p, which has to be interpolated for constructing an expression with other literals; this is how it would be done:

p = 6;
exp = :(20 + $p)

This is how it would look:

For nested quoting, each symbol must be quoted separately, along with splicing the overall parentheses of the nested expression:

p = 6;
q = 7;
:(:p in $(  :(:p * :q ) ) )

This is how it would look in the REPL:

Even data structures can be spliced into an expression construction. Now, let's consider the tuple data structure for splicing into an expression builder:

p = 6...

The Eval function

The eval() function is simply used for executing or evaluating an Expression object. The evaluations and executions are done in a global scope.

Getting ready

To get started with this section, you must simply have your Julia REPL up and running.

How to do it...

Let's work on some examples to understand the eval() function better.

Construct an expression for adding two variables:

1. First define the variable:

   p = 2
   q = 3
   

   The output would look like this:

   

2. Now, construct the expression:

   exp = :(p + q)
   

   

3. Now, check the value of the expression with the eval() function:

   eval(exp)
   

   

Now, let's look at functions that take in one or more Expression objects as input arguments and return another Expression object as the output. Let's understand this better through an example:

1. The following code creates a function that we discussed in the preceding example, one which takes in expressions as inputs and also return expressions as outputs:

   function example_exp(op, var1, var2)
   exp = Expr(:call, op...

Macros

In this section, you will be introduced to macros, which are used to insert generated code into the programs. So, a macro is simply a block of code that can be compiled directly rather than the conventional method of constructing expression statements and using the eval() function. The advantage of using macros is that a block of code that has to be hardcoded multiple times can be generated on-the-fly by creating macros for it.

Getting ready

To get started with this section, you must simply have your Julia REPL up and running.

How to do it...

1. Let's create a macro named welcome to print Welcome to Julia:

   macro welcome()
   return :(println("Welcome to Julia"))
   end
   

   This is how it would look when done in the REPL:

   

2. Now, let's check the macro you have created in the preceding step. Macros are represented by an @ before their name. So, your macro would be represented by @welcome(). It can be checked as follows:

   @welcome()
   

   This is how it would look when printed in the REPL:

   

3. Now, let's include...

Metaprogramming with DataFrames

In this section, you will learn about implementing the concept of metaprogramming to dataframes. Dataframes are data structures used for expressing data efficiently. So, using metaprogramming techniques helps speed up the process of dealing with data frames, by automated generation of repetitive tasks and easy syntax.

Getting ready

To get started with this section, you must install the DataArrays, DataFrames, and DataFramesMeta packages of Julia. They can be installed using the Pkg.add() function. Check for successful installation by executing the following in the REPL:

using DataFrames
using DataArrays
using DataFramesMeta

How to do it...

Let's start with the @with macro. It is used to express the columns of DataFrames as symbols. Let's verify this and play with the macro. Before that you need to define a DataFrame. Here is how you do it:

df = DataFrame(a = [1,2,3], b = [4,5,6])
@with(df, :b + 1)

This would add +1 to every value in the y column of the data frame...