Mastering Julia: Develop your analytical and programming skills further in Julia to solve complex data processing problems

Julia was designed with scientific computing in mind. The developers all tell us that they came with a wide array of programming skills—Lisp, Python, Ruby, R, and MATLAB. Some like myself even claim to originate as Perl hackers. However, all need a fast compiled language in their armory such as C or Fortran as the current languages listed previously are pitifully slow.

So, to quote the development team:

With the introduction of the Low-Level Virtual Machine (LLVM) compilation, it has become possible to achieve this goal and to design a language from the outset, which makes the two-language approach largely redundant.

Julia was designed as a language similar to other scripting languages and so should be easy to learn for anyone familiar to Python, R, and MATLAB. It is syntactically closest to MATLAB, but it is important to note that it is not a drop-in clone. There are many important differences, which we will look at later.

It is important not to be too overwhelmed by considering Julia as a challenger to Python and R. In fact, we will illustrate instances where the languages are used to complement each other. Certainly, Julia was not conceived as such, and there are certain things that Julia does which makes it ideal for use in the scientific community.

Julia was initially designed with scientific computing in mind. Although the term "data science" was coined as early as the 1970s, it was only given prominence in 2001, in an article by William S. Cleveland, Data Science: An Action Plan for Expanding the Technical Areas of the Field of Statistics. Almost in parallel with the development of Julia has been the growth in data science and the demand for data science practitioners.

What is data science?

The following might be one definition:

Data science is the study of the generalizable extraction of knowledge from data. It incorporates varying elements and builds on techniques and theories from many fields, including signal processing, mathematics, probability models, machine learning, statistical learning, computer programming, data engineering, pattern recognition, learning, visualization, uncertainty modeling, data warehousing, and high-performance computing with the goal of extracting meaning from data and creating data products.

If this sounds familiar, then it should be. These were the precise goals laid out at the onset of the design of Julia. To fill the void, most data scientists have turned to Python and to a lesser extent, to R. One principal cause in the growth of the popularity of Python and R can be traced directly to the interest in data science.

So, what we set out to achieve in this book is to show you as a budding data scientist, why you should consider using Julia, and if convinced, then how to do it.

Along with data science, the other "new kids on the block" are big data and the cloud. Big data was originally the realm of Java largely because of the uptake of the Hadoop/HDFS framework, which, being written in Java, made it convenient to program MapReduce algorithms in it or any language, which runs on the JVM. This leads to an obscene amount of bloated boilerplate coding.

However, here, with the introduction of YARN and Hadoop stream processing, the paradigm of processing big data is opened up to a wider variety of approaches. Python is beginning to be considered an alternative to Java, but upon inspection, Julia makes an excellent candidate in this category too.

Julia has the reputation for speed. The home page of the main Julia website, as of July 2014, includes references to benchmarks. The following table shows benchmark times relative to C (smaller is better, C performance = 1.0):

Benchmarks can be notoriously misleading; indeed, to paraphrase the common saying: there are lies, damned lies, and benchmarks.

The Julia site does its best to lay down the parameters for these tests by providing details of the workstation used—processor type, CPU clock speed, amount of RAM, and so on—and the operating system deployed. For each test, the version of the software is provided plus any external packages or libraries; for example, for the rand_mat test, Python uses NumPy, and C, Fortran, and Julia use OpenBLAS.

Julia provides a website for checking its performance: http://speed.julialang.org.

The source code for all the tests is available on GitHub. This is not just the Julia code but also that used in C, MATLAB, Python, and so on. Indeed, extra language examples are being added, and you will find benchmarks to try in Scala and Lua too:

https://Github.com/JuliaLang/julia/tree/master/test/perf/micro.

This table is useful in another respect too, as it lists all the major comparative languages of Julia. No real surprises here, except perhaps the range of execution times.

So, Julia would seem to be an ideal language for tackling data science problems. It's important to recognize that many of the built-in functions in R and Python are not implemented natively but are written in C. Julia performs roughly as well as C, so Julia won't do any better than R or Python if most of the work you do in R or Python calls built-in functions without performing any explicit iteration or recursion.

However, when you start doing custom work, Julia will come into its own. It is the perfect language for advanced users of R or Python, who are trying to build advanced tools inside of these languages. The alternative to Julia is typically resorting to C; R offers this through Rcpp, and Python offers it through Cython.

There is a possibility of more cooperation between Julia with R and/or Python than competition, although this is not the common view.

The Julia programming language is free and open source (MIT licensed), and the source is available on GitHub.

To the veteran programmer, it has looks and feels similar to MATLAB. Blocks created by the for, while, and if statements are all terminated by end rather than by endfor, endwhile, and endif or by using the familiar {} style syntax. However, it is not a MATLAB clone, and sources written for MATLAB will not run on Julia.

The following are some of Julia's features:

Julia's core is implemented in C and C++, and its parser in Scheme; the LLVM compiler framework is used for the JIT generation of machine code.

The standard library is written in Julia itself by using Node.js's libuv library for efficient, cross-platform I/O.

Julia has a rich language of types for constructing and describing objects that can also optionally be used to make type declarations. It has the ability to define function behavior across many combinations of argument types via a multiple dispatch, which is the key cornerstone of language design.

Julia can utilize code in other programming languages by directly calling routines written in C or Fortran and stored in shared libraries or DLLs. This is a feature of the language syntax and will be discussed in detail later.

In addition, it is possible to interact with Python via PyCall and this is used in the implementation of the IJulia programming environment.

At present, we will be looking at the download link. This provides links to 32-bit and 64-bit distros for Windows, Mac OS X, CentOS, and Ubuntu; both the stable release and the nightly development snapshot. So, a majority of the users getting started require nothing more than a download and a standard installation procedure.

For Windows, this is by running the downloaded .exe file, which will extract Julia into a folder. Inside this folder is a batch file julia.bat, which can be used to start the Julia console.

For Mac OS X, the users need to click on the downloaded .dmg file to run the disk image and drag the app icon into the Applications folder. On Mac OS X, you will be prompted to continue as the source has been downloaded from the Internet and so is not considered secure.

Similarly, uninstallation is a simple process. In Windows, delete the julia folder, and in Mac OS X, delete Julia.app. To do a "clean" uninstall, it is also necessary to tidy up a few hidden files/folders, and we will consider this after talking about the package system.

For Ubuntu (Linux), it's a little bit more involved as you need to add a reference to Personal Package Archive (PPA) to your system. You will have to have the root privilege for this to execute the following commands:

sudo apt-get add-repository ppa:staticfloat/juliareleases
sudo add-apt-repository ppa:staticfloat/julia-deps
sudo apt-get update
sudo apt-get install julia

The releases are provided by Elliot Saba, and there is a separate PPA for the nightly snapshots: ppa:staticfloat/julianightlies.

It is only necessary to add PPA once, so for updates, all you need to do is execute the following command:

sudo apt-get update

Ubuntu is part of the Debian family of Linux distributions, others being Debian itself, Linux Mint Debian Edition (LMDE), and Knoppix. All can install DEB packages and use the previous apt-get command.

The other major Linux family is based on the Red Hat distribution: Red Hat Enterprise, CentOS, Fedora, Scientific Linux, and so on. These use a different package management mechanism based on RPM files. There are also distros based on SUSE, Mandriva, and Slackware.

For a comprehensive list, look at the Wikipedia page: http://en.wikipedia.org/wiki/List_of_Linux_distributions.

The link is again available from the julialang.org downloads page. Julia uses GitHub as a repository for its source distribution as well as for various Julia packages. We will look at installing on CentOS, which is the community edition of Red Hat and is widely used.

su –
(type root password)
yum update
yum install git

git clone git://Github.com/JuliaLang/julia.git

git clone git://Github.com/JuliaLang/julia.git
Cloning into 'julia'...
remote: Counting objects: 97173, done.
remote: Compressing objects: 100% (24020/2

sudo yum install gcc
sudo yum install gcc-c++
sudo yum install gcc-gfortran

sudo yum groupinstall 'Development tools'

[malcolm@localhost] cd ~
[malcolm@localhost] mkdir Build
[malcolm@localhost] cd Build
 [malcolm@localhost Build] git clone git://github.com/JuliaLang/julia.git
[malcolm@localhost julia] cd julia
[malcolm@localhost julia] make

After the build:
[malcolm@localhost julia] ls -l julia
lrwxrwxrwx 1 malcolm malcolm 39 Jun 10 09:11 julia -> /home/malcolm/Build/julia/usr/bin/julia

[malcolm@localhost] cd ~/bin
[malcolm@localhost bin] ln -s /home/malcolm/Build/julia/usr/bin/julia julia

[malcolm@localhost] julia -q

julia> println("I've just installed Julia")
I've just installed Julia

julia>?print
?print
Base.print(x)
   Write (to the default output stream) a canonical (un-decorated)
   text representation of a value if there is one, otherwise call
   "show". The representation used by "print" includes minimal
   formatting and tries to avoid Julia-specific details.

julia> ;ls
;ls
asian-ascplot.jl  asian-winplot.jl  asian.jl  asian.m  asian.o  asian.r  run-asian.jl  time-asian.jl

sumsq(x,y) = x^2 + y^2;
N=1000000; x = 0;
for i = 1:N
  if sumsq(rand(), rand()) < 1.0
    x += 1;
  end
end
@printf "Estimate of PI for %d trials is %8.5f\n" N 4.0*(x / N);

Let's look at the top-level folders in the source tree that we get from GitHub:

To gain some insight into Julia coding, the best folders to look at are base, examples, and test.

If you have created Julia from source, you will have all the folders available in the Git/build folder; the build process creates a new folder tree in the folder starting with usr and the executable is in the usr/bin folder.

Installing on a Mac under OS X creates Julia in /Applications/Julia-[version].app, where version is the build number being installed. The executables required are in a subfolder of Contents/Resources/julia/bin. To find the Julia sources, look into the share folder and go down one level in to the julia subfolder.

So, the complete path will be similar to /Applications/julia-0.2.1.app/Contents/Resources/julia/share/julia. This has the Julia files but not the C/C++ and Scheme files and so on; for these, you will need to checkout a source tree from GitHub.

For Windows, the situation is similar to that of Mac OS X. The installation file creates a folder called julia-[build-number] where build-number is typically an alpha string such as e44b593905. Immediately under it are the bin and share folders (among others), and the share folder contains a subfolder named julia with the Julia scripts in it.

Juno is an IDE, which is bundled, for stable distributions on the Julia website. There are different versions for most popular operating systems.

It requires unzipping into a subfolder and putting the Juno executable on the run-search path, so it is one of the easiest ways to get started on a variety of platforms. It uses Light Table, so unlike IJulia (explained in the following section), it does not need a helper task (viz. Python) to be present.

The driver is the Jewel.jl package, which is a collection of IDE-related code and is responsible for communication with Light Table. The IDE has a built-in workspace and navigator. Opening a folder in the workspace will display all the files via the navigator.

Juno handles things such as the following:

Juno's basic job is to transform expressions into values, which it does on pressing Ctrl + Enter, (Cmd + Enter on Mac OS X) inside a block. The code block is evaluated as a whole, rather than line by line, and the final result returned.

By default, the result is "collapsed." It is necessary to click on the bold text to toggle the content of the result. Graphs, from Winston and Gadfly, say, are displayed in line within Juno, not as a separate window.

IJulia is a backend interface to the Julia language, which uses the IPython interactive environment. It is now part of Jupyter, a project to port the agnostic parts of IPython for use with other programming languages.

This combination allows you to interact with the Julia language by using IPython's powerful graphical notebook, which combines code, formatted text, math support, and multimedia in a single document.

You need version 1.0 or later of IPython. Note that IPython 1.0 was released in August 2013, so the version of Python required is 2.7 and the version pre-packaged with operating-system distribution may be too old to run it. If so, you may have to install IPython manually.

On Mac OS X and Windows systems, the easiest way is to use the Anaconda Python installer. After installing Anaconda, use the conda command to install IPython:

conda update conda conda update ipython

On Ubuntu, we use the apt-get command and it's a good idea to install matplotlib (for graphics) plus a cocktail of other useful modules.

sudo apt-get install python-matplotlib python-scipy python-pandas python-sympy python-nose

IPython is available on Fedora (v18+) but not yet on CentOS (v6.5) although this should be resolved with CentOS 7. Installation is via yum as follows:

sudo yum install python-matplotlib scipy python-pandas sympy python-nose

The IPython notebook interface runs in your web browser and provides a rich multimedia environment. Furthermore, it is possible to produce some graphic output via Python's matplotlib by using a Julia to Python interface. This requires installation of the IJulia package.

Start IJulia from the command line by typing ipython notebook --profile julia, which opens a window in your browser.

This can be used as a console interface to Julia; using the PyPlot package is also a convenient way to plot some curves.

The following screenshot displays a damped sinusoid of the form x*exp(-0.1x)*cos(2.0x):

We can create a file called asian.jl by using the following code:

function run_asian(N = 100000, PutCall = 'C';)
# European Asian option.
# Uses geometric or arithmetic average.
# Euler and Milstein discretization for Black-Scholes.
# Option features.
  println("Setting option parameters");
  S0  = 100;      # Spot price
  K   = 100;      # Strike price
  r   = 0.05;     # Risk free rate
  q   = 0.0;      # Dividend yield
  v   = 0.2;      # Volatility
  tma = 0.25;     # Time to maturity

  Averaging = 'A';  # 'A'rithmetic or 'G'eometric
  OptType = (PutCall == 'C' ? "CALL" : "PUT");
  println("Option type is $OptType");
# Simulation settings.
  println("Setting simulation parameters");
  T = 100;         # Number of time steps
  dt = tma/T;      # Time increment

# Initialize the terminal stock price matrices
# for the Euler and Milstein discretization schemes.
S = zeros(Float64,N,T);
  for n=1:N
    S[n,1] = S0;
  end

# Simulate the stock price under the Euler and Milstein schemes.
# Take average of terminal stock price.
  println("Looping $N times.");
  A = zeros(Float64,N);
  for n=1:N
    for t=2:T
      dW = (randn(1)[1])*sqrt(dt);
      z0 = (r - q - 0.5*v*v)*S[n,t-1]*dt;
      z1 = v*S[n,t-1]*dW;
      z2 = 0.5*v*v*S[n,t-1]*dW*dW;
      S[n,t] = S[n,t-1] + z0 + z1 + z2;
     end
    if cmp(Averaging,'A') == 0
      A[n] = mean(S[n,:]);
    elseif cmp(Averaging,'G') == 0
      A[n] = exp(mean(log(S[n,:])));
    end
  end

# Define the payoff
  P = zeros(Float64,N);
  if cmp(PutCall,'C') == 0
    for n = 1:N
      P[n] = max(A[n] - K, 0);
    end
  elseif cmp(PutCall,'P') == 0
    for n = 1:N
      P[n] = max(K - A[n], 0);
    end
  end
# Calculate the price of the Asian option
AsianPrice = exp(-r*tma)*mean(P);
@printf "Price: %10.4f\n" AsianPrice;
end

We have wrapped the main body of the code in a function run_asian(N, PutCall) .... end statement. The reason for this is to be able to time the execution of the task in Julia, thereby eliminating the startup times associated with the Julia runtime when using the console.

The stochastic behavior of the stock is modeled by the randn function; randn(N) provides an array of N elements, normally distributed with zero mean and unit variance.

All the work is done in the inner loop; the z-variables are just written to decompose the calculation.

To store the averages for each track, use the zeros function to allocate and initialise the array.

The option would only be exercised if the average value of the stock is above the "agreed" prices. This is called the payoff and is stored for each run in the array P.

It is possible to use arithmetic or geometric averaging. The code sets this as arithmetic, but it could be parameterized.

The final price is set by applying the mean function to the P array. This is an example of vectorized coding.

So, to run this simulation, start the Julia console and load the script as follows:

julia> include("asian.jl")
julia> run_asian()

To get an estimate of the time taken to execute this command, we can use the tic()/toc() function or the @elapsed macro:

include("asian.jl")
tic(); run_asian(1000000, 'C'); toc();
Option Price: 1.6788 elapsed: time 1.277005471 seconds

If we are not interested in the execution times, there are a couple of ways in which we can proceed.

The first is just to append to the code a single line calling the function as follows:

run_asian(1000000, 'C')

Then, we can run the Asian option from the command prompt by simply typing the following: julia asian.jl.

This is pretty inflexible since we would like to pass different values of the number of trials N and to determine the price for either a call option or a put option.

Julia provides an ARG array when a script is started from the command line to hold the passed arguments. So, we add the following code at the end of asian.jl:

nArgs = length(ARGS)
if nArgs >= 2
    run_asian(ARGS[1], ARGS[2])
elseif nArgs == 1
    run_asian(ARGS[1])
else
    run_asian()
end

Julia variables are case-sensitive, so we must use ARGS (uppercase) to pass the arguments.

Because we have specified the default values in the function definition, this will run from the command line or if loaded into the console.

Arguments to Julia are passed as strings but will be converted automatically although we are not doing any checking on what is passed for the number of trials (N) or the PutCall option.

We will discuss the package system in the next section, but to get going, let's look at a simple example that produces a plot by using the ASCIIPlots package. This is not dependent on any other package and works at the console prompt.

You can find it at http://docs.julialang.org and choose the "Available Packages" link. Then, click on ASCIIPlots, which will take you to the GitHub repository.

It is always a good idea when looking at a package to read the markdown file: README.md for examples and guidance.

Let's use the same parameters as before. We will be doing a single walk so there is no need for an outer loop or for accumulating the price estimates and averaging them to produce an option price.

By compacting the inner loop, we can write this as follows:

using ASCIIPlots;

S0  = 100;      # Spot price
K   = 102;      # Strike price
r   = 0.05;     # Risk free rate
q   = 0.0;      # Dividend yield
v   = 0.2;      # Volatility
tma = 0.25;     # Time to maturity
T = 100;        # Number of time steps
dt = tma/T;     # Time increment
S = zeros(Float64,T);
S[1] = S0;
dW = randn(T)*sqrt(dt)
[ S[t] = S[t-1] * (1 + (r - q - 0.5*v*v)*dt + v*dW[t] + 0.5*v*v*dW[t]*dW[t]) for t=2:T ]
x = linspace(1,T);
scatterplot(x,S,sym='*');

Note that when adding a package the statement using its name as an ASCIIPlots string, whereas when using the package, it does not.

using Winston;

S0  = 100;      # Spot price
K   = 102;      # Strike price
r   = 0.05;     # Risk free rate
q   = 0.0;      # Dividend yield
v   = 0.2;      # Volatility
tma = 0.25;     # Time to maturity
T = 100;        # Number of time steps
dt = tma/T;     # Time increment
S = zeros(Float64,T)
S[1] = S0;
dW = randn(T)*sqrt(dt);
[ S[t] = S[t-1] * (1 + (r - q - 0.5*v*v)*dt + v*dW[t] + 0.5*v*v*dW[t]*dW[t]) for t=2:T ]

x = linspace(1, T, length(T));
p = FramedPlot(title = "Random Walk, drift 5%, volatility 2%")
add(p, Curve(x,S,color="red"))
display(p)

We compared the Asian option code above with similar implementations in the "usual" data science languages discussed earlier.

The point of these benchmarks is to compare the performance of specific algorithms for each language implementation. The code used is available to download.

The runs were executed on a Samsung RV711 laptop with an i5 processor and 4GB RAM running CentOS 6.5 (Final).

After installing Julia, the user can create a new repository by using the Pkg.init() command. This clones the metadata from a "well-known" repository and creates a local folder called .julia:

julia> Pkg.init()
INFO: Initializing package repository C:\Users\Malcolm
INFO: Cloning METADATA from git://Github.com/JuliaLang

The latest versions of all installed packages can be updated with the Pkg.update() command.

Notice that if the repository does not exist, the first use of a package command such as Pkg.update() or Pkg.add() will call Pkg.init() to create it:

julia> Pkg.update()
Pkg.update()
INFO: Updating METADATA...
INFO: Computing changes...
INFO: No packages to install, update or remove.

We previously discussed how to install the ASCIIPlots package by using the Pkg.add("ASCIIPlots") command. The Pkg.status() command can be used to show the current packages installed and Pkg.rm() to remove them:

julia> Pkg.status()
Pkg.status()
Required packages:
- ASCIIPlots 0.0.2

julia> Pkg.rm("ASCIIPlots")
Pkg.rm("ASCIIPlots")
INFO: Removing ASCIIPlots INFO: REQUIRE updated.

After adding ASCIIPlots, we added the Winston graphics package. Most packages have a set of others on which they depend and the list can be found in the REQUIRE file.

For instance, Winston requires the Cairo, Color, IniFile, and Tk packages. Some of these packages also have dependencies, so Pkg.add() will recursively resolve, clone, and install all of these. The Cairo package is interesting since it requires Homebrew on Mac OS X and WinRPM on Windows.

WinRPM further needs URLParse, HTTPClient, LibExpat, and ZLib. So, if we use Pkg.status() again on a Windows installation, we get the following:

julia> Pkg.status()
Required packages:
 - ASCIIPlots               0.0.2
 - Winston                    0.11.0
Additional packages:
 - BinDeps                    0.2.14
 - Cairo  0.2.13
 - Color 0.2.10
 - HTTPClient              0.1.0
 - IniFile 0.2.2
 - LibCURL             0.1.3
 - LibExpat0.0.4
 - Tk                            0.2.12
 - URIParser 0.0.2
 - URLParse 0.0.0
 - WinRPM                 0.0.13
 - Zlib                          0.1.5

All the packages installed as dependencies are listed as additional packages. Removing the Winston package will also remove all the additional packages. When adding complex packages, you may wish to add some of the dependent ones first. So, with Winston, you can add both Cairo and Tk, which will then show the required rather than the additional packages.

For such a young language, Julia has a rich and rapidly developing set of packages covering all aspects of use to the data scientist and the mathematical analyst. Registered packages are available on GitHub, and the list of these packages can be referenced via http://docs.julialang.org/.

Because the core language is still under review from release to release, some features are deprecated, others changed, and the others dropped, so it is possible that specific packages may be at variance with the release of Julia you are using, even if it is designated as the current "stable" one. Furthermore, it may be that a package may not work under different operating systems. In general, use under the Linux operating system fares the best and under Windows fares the worst.

How then should we select a package? The best indicators are the version number; packages designated v0.0.0 should always be viewed with some suspicion. Furthermore, the date of the last update is useful here. The docs website also lists the individual contributors to each individual package with the principal author listed first. Ones with multiple developers are clearly of interest to a variety of contributors and tend to be better discussed and maintained. There is strength here in numbers. The winner in this respect seems to be (as of July 2014) the DataFrames package, which is up to version 0.3.15 and has attracted the attention of 33 separate authors.

Even with an old relatively untouched package, there is nothing to stop you checking out the code and modifying or building on it. Any enhancements or modifications can be applied and the code returned; that's how open source grows. Furthermore, the principal author is likely to be delighted that someone else is finding the package useful and taking an interest in the work.

It is not possible to create a specific taxonomy of Julia packages but certain groupings emerge, which build on the backs of the earlier ones. We will be meeting many of these later in this book, but before that, it may be useful to quickly list a few.

Removing Julia is very simple; there is no explicit uninstallation process. It consists of deleting the source tree, which was created by the build process or from the DMG file on Mac OS X or the EXE file on Windows. Everything runs within this tree, so there are no files installed to any system folders.

In addition, we need to attend to the package folder. Recall that under Linux and Mac OS X this is a hidden folder called .julia in the user's home folder. In Windows, it is located in the user's profile typically in C:\Users\[my-user-name]. Removing this folder will erase all the packages that were previously installed.

There is another hidden file called .julia_history that should be deleted; it keeps an historical track of the commands listed.

The official repository for the registered packages in Julia is here:

https://Github.com/JuliaLang/METADATA.jl.

Any packages here will be listed using the package manager or in Julia Studio.

However, it is possible to use an unregistered package by using Pkg.clone(url), where the url is a Git URL from which the package can be cloned. The package should have the src and test folders and may have several others. If it contains a REQUIRE file at the top of the source tree, that file can be used to determine any dependent registered packages; these packages will be automatically installed.

If you are developing a package, it is possible to place the source in the .julia folder alongside packages added with Pkg.add() or Pkg.clone(). Eventually, you will wish to use GitHub in a more formal way; we will deal with that later when considering package implementation.

As a language aimed at the scientific community, it is natural that Julia should provide facilities for executing code in parallel. In running tasks on multiple processors, Julia takes a different approach to the popular message passing interface (MPI). In Julia, communication is one-sided and appears to the programmer more as a function call than the traditional message send and receive paradigm typified by pyMPI on Python and Rmpi on R.

Julia provides two in-built primitives: remote references and remote calls. A remote reference is an object that can be used from any processor to refer to an object stored on a particular processor. A remote call is a request by one processor to call a certain function or certain arguments on another, or possibly the same, processor.

Sending messages and moving data constitute most of the overhead in a parallel program, and reducing the number of messages and the amount of data sent is critical to achieving performance and scalability. We will be investigating how Julia tackles this in a subsequent chapter.

The choice of which method to execute when a function is applied is called dispatch.

Single dispatch polymorphism is a familiar feature in object-orientated languages where a method call is dynamically executed on the basis of the actual derived type of the object on which the method has been called.

Multiple dispatch is an extension of this paradigm where dispatch occurs by using all of a function's arguments rather than just the first.

Julia, like Lisp, represents its own code in memory by using a user-accessible data structure, thereby allowing programmers to both manipulate and generate code that the system can evaluate. This makes complex code generation and transformation far simpler than in systems without this feature.

We met an example of a macro earlier in @printf, which mimics the C-like printf statements. Its definition is in given in the base/printf.jl file.

We noted that Julia is often seen as a competitor to languages such as C, Python, and R, but this is not the view of the language designers and developers.

Julia makes it simple to call C and Fortran functions, which are compiled and saved as shared libraries. This is by the use of the in-built call. This means that there is no need for the traditional "wrapper code" approach, which acts on the function inputs, transforms them into an appropriate form, loads the shared library, makes the call, and then repeats the process in reverse on the return value. Julia's JIT compilation generates the same low-level machine instructions as a native C call, so there is no additional overhead in making the function call from Julia.

Additionally, we have seen that the PyCall package makes it easy to import Python libraries, and this has been seen to be an effective method of displaying the graphics from Python's matplotlib. Further, inter-cooperation with Python is evident in the provision of the IJulia IDE and an adjunction to IPython notebooks.

There is also work on calling R libraries from Julia by using the Rif package and calling Java programs from within Julia by using the JavaCall package. These packages present the exciting prospect of opening up Julia to a wealth of existing functionalities in a straightforward and elegant fashion.