Julia 1.0 Programming Cookbook

In this recipe, we present how to install and configure Julia on Windows and Linux.

All Linux examples in this book have been tested on Linux Ubuntu 18.04.1 LTS and Windows 10. All Linux Ubuntu commands have been run as the user ubuntu. Please note that users of other Linux distributions will need to update their scripts (for example, Linux distributions from the Red Hat family use yum instead of apt).

For Windows examples in this book, we use Windows 10 Professional.

For most users, the recommended way to start with Julia is to use a binary version.

In this section, we present the following options:

$ sudo apt update
$ sudo apt -y install build-essential

$ wget https://julialang-s3.julialang.org/bin/linux/x64/1.0/julia-1.0.1-linux-x86_64.tar.gz
$ tar xvfz julia-1.0.1-linux-x86_64.tar.gz
$ sudo ln -s /home/ubuntu/julia-1.0.1/bin/julia /usr/local/bin/julia

Those users who want to try Julia without the installation process should try JuliaBox.

JuliaBox is a ready-made pre-installed Julia environment accessible from the web browser. It is available at https://juliabox.com/. You can use this version to play with a preconfigured and installed version of the Julia environment. Julia is available in a web browser via the Jupyter Notebook environment. The website is free to use, though it does require registration. JuliaBox comes with a set of pre-configured popular Julia libraries and is therefore ready for immediate use. This is an ideal solution for people wanting to try out the language or for using Julia inside a classroom.

Excellent documentation on how to install Julia on various systems can be found on the JuliaLang website: https://julialang.org/downloads/platform.html.

Before installing an IDE, you should have Julia installed (either from binaries or source), following the instructions given in previous recipes.

The three most popular Julia IDEs are Juno, Microsoft Visual Studio Code, and Sublime Text. In subsequent sections, we discuss the installation process for each particular IDE. 

{
    "cmd": ["ConEmu64", "/cmd", "julia -i", "$file"],
    "selector": "source.julia"
}

For integration with other editors and IDEs, take a look at the https://github.com/JuliaEditorSupport project.

Before installing Julia support for text editors, you should have Julia preinstalled (either from binaries or source), in accordance with the instructions given in previous recipes.

The three most popular text editors used by Julia developers include Nano, Vim, and Emacs. Here, we provide some hints on how to configure Julia with these popular text-mode editors. All the following examples have been tested with Ubuntu 18.0.4.1 LTS.

$ wget -P ~/ https://raw.githubusercontent.com/Naereen/nanorc/master/julia.nanorc

$ echo include \"~/julia.nanorc\" >> ~/.nanorc

git clone git://github.com/JuliaEditorSupport/julia-vim.git
mkdir -p ~/.vim
cp -R julia-vim/* ~/.vim

wget -P ~/julia-emacs/ https://raw.githubusercontent.com/JuliaEditorSupport/julia-emacs/master/julia-mode.el
echo "(add-to-list'load-path \"~/julia-emacs\")" >> ~/.emacs
echo "(require'julia-mode)" >> ~/.emacs

For integration with other editors and IDEs, take a look at the Julia Editor Support project, which is available at https://github.com/JuliaEditorSupport.

All the following examples have been tested on Ubuntu 18.04.1 LTS.

Here is a list of steps to be followed:

$ sudo apt update
$ sudo apt install --yes build-essential python-minimal gfortran m4 cmake pkg-config libssl-dev

$ git clone git://github.com/JuliaLang/julia.git
$ cd julia
$ git checkout v1.0.1

In this section, we describe how to build Julia in three particular variations:

The libraries from Intel (MKL and LIBM) provide an implementation for a number of mathematical operations optimized for Intel's processor architecture. In particular, the Intel MKL library contains optimized routines for BLAS, LAPACK, ScaLAPACK, sparse solvers, fast Fourier transforms, and vector math (for more details see https://software.intel.com/en-us/mkl). On the other hand, the Intel LIBM library provides highly optimized scalar math functions that serve as direct replacements for the standard C calls—this includes optimized versions of standard math library functions, such as exp, log, sin, and cos). More information on Intel LIBM can be found at https://software.intel.com/en-us/articles/implement-the-libm-math-library.

Before running each of the recipes, please make sure that you are inside the folder where you ran the checkout command for Julia (see the Getting ready section).

$ make-j$((`nproc`-1))1>build_log.txt 2>build_error.txt

julia> versioninfo()
Julia Version 1.0.1
Commit 0d713926f8* (2018-09-29 19:05 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.0 (ORCJIT, skylake)

$ cd ~
# Get link from MKL website
$ wget http://registrationcenter-download.intel.com/[go to Intel MKL web site to get link]/l_mkl_2019.0.117.tgz
$ tar zxvf l_mkl_2019.0.117.tgz
$ cdl_mkl_2019.0.117
$ sudobash install.sh

cd ~/julia
echo"USEICC = 0" >> Make.user
echo"USEIFC = 0" >> Make.user
echo"USE_INTEL_MKL = 1" >> Make.user
echo"USE_INTEL_LIBM = 0" >> Make.user
​
source /opt/intel/bin/compilervars.sh intel64
​
make-j$((`nproc`-1))1>build_log.txt 2>build_error.txt

julia> versioninfo()
Julia Version 1.0.1
Commit 0d713926f8* (2018-09-29 19:05 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.0 (ORCJIT, skylake)

julia> ENV["MKL_INTERFACE_LAYER"]
"ILP64"

$ cd ~
# Get the link from Intel C++ compilers website
$ wget http://[go to Intel to get link]/parallel_studio_xe_2018_update3_professional_edition.tgz
$ tar zxvf parallel_studio_xe_2018_update3_professional_edition.tgz
$ cd parallel_studio_xe_2018_update3_professional_edition
$ sudobash install.sh

cd ~/julia
echo"USEICC = 0" >> Make.user
echo"USEIFC = 0" >> Make.user
echo"USE_INTEL_MKL = 1" >> Make.user
echo"USE_INTEL_LIBM = 1" >> Make.user
​
source /opt/intel/bin/compilervars.sh intel64
​
make-j$((`nproc`-1))1>build_log.txt 2>build_error.txt

For the highest performance, it is recommended to compile Julia with the Linux Intel MKL (Intel MKL is available at https://software.intel.com/en-us/mkl) drivers, which are available for free from Intel. The numerical performance can also be enhanced by using the Intel Math Library (Intel LIBM is available at https://software.intel.com/en-us/node/522653). However, LIBM can only be obtained with the Intel C++ compilers (see https://software.intel.com/en-us/c-compilers), which are free for academic and open source use but paid otherwise. Therefore, some users might be interested in building Julia with MKL, but without LIBM.

Please note that in all the scenarios outlined, we use GNU compilers rather than Intel compilers—even when using Intel's MKL and LIBM libraries. If you want to use Intel's compilers, you need to set the appropriate options in the Make.user file (that is, change USEICC = 0 and USEIFC = 0 to USEICC = 1 and USEIFC = 1). However, Intel compilers are currently not supported by Julia compiler scripts (see https://github.com/JuliaLang/julia/issues/23407).

Once Julia is installed on your system, it can be run by giving the full path to the Julia executable (for example,~/julia/julia). This is not always convenient—many users simply want to typejuliato get Julia running:

$ sudoln-s /home/ubuntu/julia/usr/bin/julia /usr/local/bin/julia

The preceding path assumes that you installed Julia as theubuntuuser in your home folder. If you have installed Julia to a different location, please update the path accordingly.

If you want to build Julia in a supercomputing environment (for example, Cray), please follow the online tutorial written by one of the authors of this book, available at https://github.com/pszufe/Building_Julia_On_Cray_and_Clusters.

In order to use Cloud9, you must have an active Amazon Web Services (AWS) account and log in to the AWS management console.

Cloud9 can create a new EC2 instance running Amazon Linux or can connect to any existing Linux server that allows SSH connections and has Node.js installed.In order to start working with Julia on Cloud9, complete the following steps:

Once you have prepared a server with Julia and Node.js, you can take the following steps to use Cloud9:

 {
"cmd" : ["julia", "$file", "$args"],
"info" : "Started $project_path$file_name",
"selector" : "source.jl"
 }

The Cloud9 environment runs in your web browser. The browser opens a REST connection back to Cloud9's server, which in turn opens an SSH connection to your Linux instance (see the following diagram). This functionality will, in fact, run with any Linux server that accepts incoming connections from Cloud9 (more details on configuring other Linux servers with Cloud9 can be found at https://docs.aws.amazon.com/cloud9/latest/user-guide/ssh-settings.html):

Please note that this means that the EC2 instance supporting Cloud9 should allow incoming connections from the AWS Cloud9 infrastructure. In production environments, we recommend limiting traffic to the EC2 instance (viaSecurityGroup) to the IP ranges defined for Cloud9. Detailed instructions can be found in Cloud9's documentation: https://docs.aws.amazon.com/cloud9/latest/user-guide/ip-ranges.html.

The Cloud9 environment is being continuously updated by AWS; new features are being added frequently. For the latest documentation, we recommend looking at AWS Cloud9's user guide, available athttps://docs.aws.amazon.com/cloud9/latest/user-guide/. In particular, it is worth looking athttps://docs.aws.amazon.com/cloud9/latest/user-guide/get-started.html.

Before we begin, make sure that you have the Julia executable in your search path, as explained in the Installing Julia from binaries recipe. Also, create a hello.jlfile in your working directory, containing the following line:

println("Hello " * join(ARGS, ", "))

We will later run this file on Julia startup.

Now, open your favorite terminal to execute commands.

In this recipe, we will show various options for controlling how the julia process is started: running scripts on startup, running a single command, and configuring a startup script for Julia installation. They are discussed in the consecutive subsections.

$ julia -i hello.jl Al Bo Cyd
Hello Al, Bo, Cyd
   _       _ _(_)_    | Documentation: https://docs.julialang.org
  (_)     | (_) (_)   |
   _ _   _| |_  __ _  |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` | |
  | | |_| | | | (_| | |  Version 1.0.2 (2018-11-08)
 _/ |\__'_|_|_|\__'_| |  Official https://julialang.org/ release
|__/                  |
julia>

$ julia -e "println(factorial(10))"
3628800
$

using Random
ENV["JULIA_EDITOR"] = "vim"
println("Setup successful")

$ julia --banner=no
Setup successful
julia> RandomDevice()
RandomDevice(UInt128[0x000000000000000000000000153e4040])

julia>

There are four methods you can use to parameterize the booting of Julia:

The general structure of running Julia from the command line is this:

$ julia [switches] -- [programfile] [args...]

The simplest way to control Julia startup is to pass switches to it. You can get the full list of switches by running the following in the console:

$ julia --help

In the previous examples, we have used three switches: -i, -e, and--banner=no.

The first of these switches (-i) keeps Julia in interactive mode after executing the commands. If we did not use the -i switch then Julia would terminate immediately after finishing the passed commands.

You can pass Julia a file to execute directly or a command to run following the-eswitch. The latter option is useful if you want to execute a short expression. However, if we want to start some commands repeatedly every time Julia starts, we can save them in the ~/.julia/config/startup.jl file. It will be run every time Julia starts unless you set the --startup-file=no command line switch.

In the preceding path, ~ is your home directory. Under Linux, this should simply work. Under Windows, you can check its location by running homedir() in the Julia command line (typically, it will be theC:\Users\[username]directory).

What are some useful things to put into ~/.julia/config/startup.jl?

In the recipe, we put the using Random statement in ~/.julia/config/startup.jl to load the Random package by default, since we routinely need it in our daily work. The second command, ENV["JULIA_EDITOR"]="vim", specifies a default editor used by Julia. Use of the editor in Julia is explained in the Useful options for interaction with Julia recipe.

The full list of environment variables that are scanned by Julia is described at https://docs.julialang.org/en/v1.0/manual/environment-variables/. They can all be set in the shell and then are read by Julia. In Julia, environment variables can be accessed and changed via the ENV dictionary.

In order to test how multiprocessing works, prepare two simple files that display a text message in the console. When running parallelization tests, we will see messages generated by those scripts appear asynchronously.

Create a hello.jl file in your working directory, containing the following code:

println("Hello " * join(ARGS, ", "))

And create hello2.jl with the following code:

println("Hello " * join(ARGS, ", "))
sleep(1)

Now, open your favorite terminal to execute the commands.

We will first explain how to start Julia using multiple processes. In the second part of the recipe, we will set up Julia to use multiple threads.

$ julia --banner=no -p 2

julia> using Distributed

julia> nworkers()
2

julia> exit()

$

$ julia --banner=no -p auto -L hello.jl
Hello !
 From worker 4: Hello !
 From worker 5: Hello !
julia> From worker 2: Hello !
 From worker 3: Hello !
julia> exit()

$ julia --banner=no -p auto -L hello2.jl
Hello !
 From worker 4: Hello !
 From worker 5: Hello !
 From worker 2: Hello !
 From worker 3: Hello !
julia> exit()

$

$ export JULIA_NUM_THREADS=`nproc`
$ julia -e "println(Threads.nthreads())"
4
$

C:\> set JULIA_NUM_THREADS=%NUMBER_OF_PROCESSORS%
C:\> julia -e "println(Threads.nthreads())"
 4
C:\>

A switch, -p {N|auto}, tells Julia to spin up N additional worker processes on startup. The auto option in the -p switch starts as many workers as you have cores on your machine, so julia -p auto is equivalent to:

It is important to understand that when you start N workers, where N is greater than 1, then Julia will spin up N+1 processes. You can check it using the nprocs() function—one master process and N worker processes. If N is equal to 1, then only one process is started.

We can see here that hello.jl was executed on the master process and on all of the worker processes. Additionally, observe that the execution was asynchronous. In this case, workers 4 and 5 printed their message before the Julia prompt was printed by the master process, but workers 2 and 3 executed their print method after it. By adding a sleep(1) statement in hello2.jl, we make the master process wait for one second, which is sufficient time for all workers to run their println command.

As you have seen, in order to start Julia with multiple threads, you have to set the environment variable JULIA_NUM_THREADS. It is used by Julia to determine how many threads it should use. This value—in order to have any effect—must be set before Julia is started. This means that you can access it via the ENV["JULIA_NUM_THREADS"] option but changing it when Julia is running will not add or remove threads. Therefore, before running Julia you have to type the following in a terminal session:

You can also add processes after Julia has started using the addprocs function. We are running the following code on Windows with two drives,C: and D:, present. Julia is started in the D:\directory:

D:\> julia --banner=no -p 2 -L hello2.jl
Hello
 From worker 3: Hello
 From worker 2: Hello
julia> pwd()
"D:\\"

julia> using Distributed

julia> pmap(i -> (i, myid(), pwd()), 1:nworkers())
2-element Array{Tuple{Int64,Int64,String},1}:
 (1, 2, "D:\\")
 (2, 3, "D:\\")

julia> cd("C:\\")

julia> pwd()
"C:\\"

julia> addprocs(2)
2-element Array{Int64,1}:
 4
 5

julia> pmap(i -> (i,myid(),pwd()), 1:nworkers())
4-element Array{Tuple{Int64,Int64,String},1}:
 (1, 3, "D:\\")
 (2, 2, "D:\\")
 (3, 5, "C:\\")
 (4, 4, "C:\\")

In particular, we see that each worker has its own working directory, which is initially set to the working directory of the master Julia process when it is started. Also, addprocs does not execute the script that was specified by the -L switch on Julia startup.

Additionally, we can see the simple use of the pmap and myid functions. The first one is a parallelized version of the map function. The second returns the identification number of a process that it is run on.

As we explained earlier, it is not possible to add threads to a running Julia process. The number of threads has to be specified before Julia is started.

Deciding between using multiple processes and multiple threads is not a simple decision. A rule of thumb is to use threads if there is a need for data sharing and frequent communication between tasks running in parallel.

More details about how to work with multiple processes and multiple threads are explained in the Multithreading in Julia and Distributed computing with Julia recipes in Chapter 10, Distributed Computing.

Create anexample.jlfile containing this:

println("An example was run!")

We will run this script in this recipe.

Now open your favorite terminal to execute the commands.

We will learn how to work interactively with Julia by going through the following steps:

julia> x = 10 # just a test command
10

julia> @edit sin(1.0)

After running these commands, an editor with the location of a code section containing the sin function opens. We explained earlier how to choose the editor that Julia uses in the How to customize Julia on startup recipe.

Now, let us check if example.jl is in our current working directory.

shell>

julia> inc

julia> include

julia> include("exa

julia> include("example.jl"

help?>

help?> in

help?> in
in include_string indexin indmax init_worker interrupt inv invoke
include ind2chr indexpids indmin insert! intersect invdigamma invperm
include_dependency ind2sub indices info instances intersect! invmod
help?> in

This time we see that there are multiple commands matching the in pattern and Julia lists them all (this is the reason that the Tab key had to be pressed twice).

help?> include
search: include include_string include_dependency

  include(path::AbstractString)

  Evaluate the contents of the input source file in the global scope of the
  containing module. Every module (except those defined with baremodule) has its
  own 1-argument definition of include, which evaluates the file in that module.
  Returns the result of the last evaluated expression of the input file. During
  including, a task-local include path is set to the directory containing the file.
  Nested calls to include will search relative to that path. This function is
  typically used to load source interactively, or to combine files in packages that
  are broken into multiple source files.

  Use Base.include to evaluate a file into another module.

And we understand exactly what include does. Now, what if we wanted to run the x = 10 command again (this is mostly useful for longer and complex commands in practice)?

(reverse-i-search)`x =': x = 10

julia> x = 10

Julia REPL offers you several modes, of which the most commonly used are these:

You can find more details about options for interacting with Julia in all those modes at https://docs.julialang.org/en/v1.0/stdlib/REPL/.

As you can observe, Julia is smart enough to perform tab completion in a context-sensitive manner—it understands if you are entering a command or a filename. Command history search is also very useful in interactive work.

In the How to customize Julia on startup recipe, we explained how to set up the editor. In this recipe, we saw how you can use the @edit macro to open the location of the definition of the sin function in your chosen editor. Julia recognizes the following editors: Vim, Emacs, gedit, textmate, mate, kate, Sublime Text, atom, Notepad++, and Visual Studio Code. Importantly, @edit recognizes the types of arguments you pass to a function and will show an appropriate method if your chosen editor supports line search on startup (otherwise, an appropriate file will be opened and the line number of the function at hand will be printed in the Julia command line).

Apart from the @edit macro, you can use the @less macro or the edit and less functions to see the source code of the function you wish to use (please consult the Julia help guide to understand the detailed differences between them).

If we only want to know the location of a method definition without displaying it, we can use the @which macro:

julia> @which sin(1.0)
sin(x::T) where T<:Union{Float32, Float64} in Base.Math at special/trig.jl:30

The How to customize Julia on startup recipe explains how to use the startup.jl file and how to choose the default Julia editor.

Make sure you have the PyPlot package installed by entering using PyPlot in the Julia command line. If it is not installed (an error message will be printed), then run the following command to add it:

julia> using Pkg; Pkg.add("PyPlot")

More details about package management are described in the Managing packages recipe in this chapter.

Then, create the display.jl file in your working directory:

using PyPlot, Random

function f()
    Random.seed!(1)
    r = rand(50)
    @show sum(r)
    display(transpose(r))
    print(transpose(r))
    plot(r)
end

f()

Now open your favorite terminal to execute the commands.

In the following steps, we will explain how the output of Julia depends on the context in which a command is invoked:

julia> include("display.jl")
sum(r) = 23.134209483707394
 1×50 LinearAlgebra.Transpose{Float64,Array{Float64,1}}:
 0.236033 0.346517 0.312707 0.00790928 0.488613 0.210968 0.951916 … 0.417039 0.144566 0.622403 0.872334 0.524975 0.241591 0.884837
 [0.236033 0.346517 0.312707 0.00790928 0.488613 0.210968 0.951916 0.999905 0.251662 0.986666 0.555751 0.437108 0.424718 0.773223 0.28119 0.209472 0.251379 0.0203749 0.287702 0.859512 0.0769509 0.640396 0.873544 0.278582 0.751313 0.644883 0.0778264 0.848185 0.0856352 0.553206 0.46335 0.185821 0.111981 0.976312 0.0516146 0.53803 0.455692 0.279395 0.178246 0.548983 0.370971 0.894166 0.648054 0.417039 0.144566 0.622403 0.872334 0.524975 0.241591 0.884837]
 1-element Array{PyCall.PyObject,1}:
 PyObject <matplotlib.lines.Line2D object at 0x0000000026314198>

Apart from the printed value presented earlier, a plot window will open as shown in the following figure:

$ julia display.jl
 sum(r) = 23.134209483707394
 1×50 LinearAlgebra.Transpose{Float64,Array{Float64,1}}:
 0.236033 0.346517 0.312707 0.00790928 0.488613 0.210968 0.951916 0.999905 0.251662 0.986666 0.555751 0.437108 0.424718 0.773223 0.28119 0.209472 0.251379 0.0203749 0.287702 0.859512 0.0769509 0.640396 0.873544 0.278582 0.751313 0.644883 0.0778264 0.848185 0.0856352
 0.553206 0.46335 0.185821 0.111981 0.976312 0.0516146 0.53803 0.455692 0.279395 0.178246 0.548983 0.370971 0.894166 0.648054 0.417039 0.144566 0.622403 0.872334 0.524975 0.241591 0.884837[0.236033 0.346517 0.312707 0.00790928 0.488613 0.210968 0.951916 0.999905 0.251662 0.986666 0.555751 0.437108 0.424718 0.773223 0.28119 0.209472 0.251379 0.0203749 0.287702 0.859512 0.0769509 0.640396 0.873544 0.278582 0.751313 0.644883 0.0778264 0.848185 0.0856352 0.553206 0.46335 0.185821 0.111981 0.976312 0.0516146 0.53803 0.455692 0.279395 0.178246 0.548983 0.370971 0.894166 0.648054 0.417039 0.144566 0.622403 0.872334 0.524975 0.241591 0.884837]

In this case, nothing is plotted. Additionally, observe that the display function, in this case, produced a different output than in interactive mode and no newline was inserted after the result (so the output of the print function is visually merged).

using PyPlot, Random

function f()
    Random.seed!(1)
    r = rand(50)
    @show sum(r)
    display(transpose(r))
    print(transpose(r))
    plot(r)
    show()
end

f()

$ julia display2.jl

This time the plot is shown and Julia is suspended until the plot dialog is closed.

In Julia, the display function is aware of the context in which it is called. This means that depending on what context is passed to this function, you may obtain a different result. For instance, an image passed to the display in the Julia console usually opens a new window with a plot, but in Jupyter Notebook it might embed it in a notebook. Similarly, many objects in the Julia console are displayed as plain text but in Jupyter Notebook they are converted to a nicely formatted HTML. In short, display(x) typically uses the richest supported multimedia output for x in a given context, with plain text stdout output as a fallback.

In particular, the display function is used by default in the Julia console when a command is executed, for example:

julia> transpose(1:100)
1×100 LinearAlgebra.Transpose{Int64,UnitRange{Int64}}:
 1 2 3 4 5 6 7 8 9 10 11 … 93 94 95 96 97 98 99 100

In this case, the output is adjusted to the size of the Terminal. However, as we saw in the previous example if display is run in a script in non-interactive mode, no adjustment of the output to the size of the device is performed.

An additional issue that is commonly required in practice is suppressing the display of an expression value in the Julia console. This is easily achieved by adding ; at the end of the command, for example:

julia> rand(100, 100);

julia>

And nothing is printed (otherwise, the screen would be flooded by a large matrix).

Finally, in the script in this recipe, we were able to examine the use of the @show macro, which is useful for debugging, as it prints the expression along with its value.

At the end of this recipe, we observed that the PyPlot package can alter its behavior based on whether it is run in interactive mode or not. Each custom package might have similar specific conditions for handling output to a variety of devices. Being aware of this, it is best to consult the documentation relating to a given package to understand the defaults. If you are developing your own code that needs to be sensitive to the Julia mode (REPL or script) in which it is invoked, there is a handy isinteractive function. This function allows you to dynamically check the Julia interpreter mode at runtime.

An in-depth explanation of how display management in Julia works is given at https://docs.julialang.org/en/v1.0/manual/types/#man-custom-pretty-printing-1.

In the Defining your own types - linked list recipe in Chapter 6, Metaprogramming and Advanced Typing, we explain how to specify custom printing for user-defined types.

In this recipe, we will use the Julia command line. Make sure that your computer is connected to the internet.

Now open your favorite terminal to execute the commands.

Here is a list of steps to be followed:

julia>

(v1.0) pkg>

(v1.0) pkg> status
    Status `~/.julia/environments/v1.0/Project.toml`

We can see that we currently have a clean environment with no additional packages installed.

(v1.0) pkg> add BenchmarkTools
   Cloning default registries into /home/ubuntu/.julia/registries
   Cloning registry General from "https://github.com/JuliaRegistries/General.git"
  Updating registry at `~/.julia/registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
 Resolving package versions...
 Installed BenchmarkTools ─ v0.4.1
 Installed JSON ─────────── v0.19.0
  Updating `~/.julia/environments/v1.0/Project.toml`
  [6e4b80f9] + BenchmarkTools v0.4.1
  Updating `~/.julia/environments/v1.0/Manifest.toml`
  [6e4b80f9] + BenchmarkTools v0.4.1
  [682c06a0] + JSON v0.19.0
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [8ba89e20] + Distributed
  [b77e0a4c] + InteractiveUtils
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [37e2e46d] + LinearAlgebra
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [a63ad114] + Mmap
  [44cfe95a] + Pkg
  [de0858da] + Printf
  [3fa0cd96] + REPL
  [9a3f8284] + Random
  [ea8e919c] + SHA
  [9e88b42a] + Serialization
  [6462fe0b] + Sockets
  [2f01184e] + SparseArrays
  [10745b16] + Statistics
  [8dfed614] + Test
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode

(v1.0) pkg> status
    Status `~/.julia/environments/v1.0/Project.toml`
  [6e4b80f9] BenchmarkTools v0.4.1

Notice that only the BenchmarkTools package is visible, although more packages have been installed by Julia. They reside in the package repository but are not visible to the user unless explicitly installed. Those packages are dependencies of the BenchmarkTools package (directly or via recursive dependency).

(v1.0) pkg> precompile
Precompiling project...
Precompiling BenchmarkTools
[ Info: Precompiling BenchmarkTools [6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf]

(v1.0) pkg>

julia>

julia> using BenchmarkTools

julia> @btime rand()
  4.487 ns (0 allocations: 0 bytes)
0.07253910317708079

(v1.0) pkg> add BSON@v0.2.0
[output is omitted]

Now you have version 0.2.0 of the BSON package installed

(v1.0) pkg> pin BSON
[output is omitted]

(v1.0) pkg> free BSON
[output is omitted]

The process of installing a specified version of some package (step 9 of the recipe) and pinning it (step 10) might be useful for you when you will need to install the exact versions of the packages that we use in this book, as in the future new releases of the packages might introduce breaking changes. The full list of packages used in this book along with their required versions is given in the To get the most out of this book section in the Preface of this book.  

For each environment, Julia keeps information about the required packages and their versions in the Project.tomlandManifest.tomlfiles. For the global default environment, they are placed in the ~/.julia/environments/v1.0/folder. The first file contains a list of installed packages along with their UUIDs (see https://en.wikipedia.org/wiki/Universally_unique_identifier or https://www.itu.int/ITU-T/studygroups/com17/oid.html). The second file describes detailed dependencies of the project with exact versions of the packages used.

In this recipe, we have only made use of the basic commands of the package manager. In most cases, this is all a regular user needs to know. There are many more commands available, which you can find using the help command in package manager mode. Some of the potentially useful commands are as follows:

All the commands we have described are also available programmatically using functions from the Pkg.jl package. For instance, running Pkg.add("BenchmarkTools") would install the package the same way as writing add BenchmarkTools in package manager mode in the Julia console. In order to use those functions, you have to load the Pkg package using using Pkg first.

In addition, it is important to know that many packages are preinstalled with Julia and thus do not require installation and can be directly loaded with the using command. Here is a selection of some of the more important ones, along with a brief description of their functionality:

Extensive coverage of these packages is given in the standard library section of the Julia manual, which can be accessed at https://docs.julialang.org/en/v1.0/.

The true power of the Julia package manager is realized when you need to have different versions of packages for your different projects on the same machine. In the Managing project dependencies recipe in Chapter 8, Julia Workflow, we discuss how you can achieve this.

Before installing Jupyter Notebook, perform the following steps:

(v1.0) pkg> add IJulia

Once IJulia is installed, there are two options for running Jupyter Notebook:

julia> using IJulia

julia> notebook()

$ ~/.julia/packages/Conda/hsaaN/deps/usr/bin/jupyter notebook

C:\> %userprofile%\.julia\packages\Conda\hsaaN\deps\usr\Scripts\jupyter-notebook

Copy/paste this URL into your browser when you connect for the first time,
to login with a token:
    http://localhost:8888/?token=b86b66b81a62d4be1ae34e7d6bd006a8ba5cb937e74b99cf

Jupyter Notebook runs a local web server on port 8888. Once it is started, you can simply connect to it via a web browser. It is also possible to run such environments on a different machine than that used to access the notebook—please check the Jupyter in the cloud recipe for more details.

Please note that for some packages, Julia installation might have conflicts with Windows security settings and thus prevent installation. In particular, IE Enhanced Security Configuration should be turned off (the default setting on Windows Server environments is on). In order to turn it off, open Server Manager and click Local Server, which is located on the left. In the right column, you will see the option IE Enhanced Security Configuration. Click on it to turn it off.

Another possible problem can be with the installation of IJulia, since it sometimes conflicts with an existing Python installation due to IJulia attempting to fetch and install a minimal Python environment itself. In such cases, if you get an error, run the following:

ENV["JUPYTER"] ="[path to your jupyter program]"
using Pkg
Pkg.build("IJulia")

You will have to manually find the[path to your jupyter program]. For example, on my Windows system with Anaconda installed, the path will be"C:\\Program Files\\Anaconda\\Scripts\\jupyter-notebook.exe".Note that we need to use two backslashes of the Julia string to represent a single backslash in a path. If you have provided a proper path, the build should finish successfully.

The most recent documentation can be found on IJulia's website(https://github.com/JuliaLang/IJulia.jl). It is worth checking, as some details of the process described earlier may change.

The configuration of JupyterLab requires having IJulia configured on your system:

By default, Julia does not include JupyterLab. However, it is possible to add JupyterLab by using the Conda.jl package. We will show, step-by-step, how to add JupyterLab to a Julia installation and run it from bash:

(v1.0) pkg> add Conda

julia> using Conda

julia> Conda.add("jupyterlab")

$ ~/.julia/packages/Conda/hsaaN/deps/usr/bin/jupyter lab

C:\>%userprofile%\.julia\packages\Conda\hsaaN\deps\usr
\Scripts\jupyter-lab

Copy/paste this URL into your browser when you connect for the first time,
to login with a token:
    http://localhost:8888/?token=b86b66b81a62d4be1ae34e7d6bd006a8ba5cb937e74b99cf

Once you have followed the preceding steps, you should see in the browser a screen similar to the one shown here (please note that since Python is installed with IJulia.jl, it is also available alongside Julia):

JupyterLab is an extension of Jupyter Notebook and hence it works in a very similar fashion. JupyterLab runs a local web server on port8888 (the same port that would have been used by Jupyter Notebook). Once it is started, you can simply connect to it via a web browser. It is also possible to run such environments on a separate machine. Please check the Configuring Julia with Jupyter Notebook headless cloud environments recipe for more details (that recipe is valid for both Jupyter Notebook and JupyterLab).

Yet another option is to use JupyterLab from a completely external Anaconda installation. Since the installation process differs for Linux and Windows, we present both versions.

$ wget https://repo.anaconda.com/archive/Anaconda3-5.3.0-Linux-x86_64.sh

$ sudo bash Anaconda3-5.3.0-Linux-x86_64.sh

$ /home/ubuntu/anaconda3/bin/jupyter lab

[I 11:07:55.625 LabApp] The Jupyter Notebook is running at:
[I 11:07:55.625 LabApp] http://localhost:8888/?token=c3756ac2013d780af16b0401d67ab017c5e4a17cc9bc7924
[I 11:07:55.625 LabApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[W 11:07:55.625 LabApp] No web browser found: could not locate runnable browser.

C:\> C:\ProgramData\Anaconda3\Scripts\jupyter-lab.exe

Copy/paste this URL into your browser when you connect for the first time,  to login with a token:
http://localhost:8888/?token=7f211507e188ecfc22e2858b195c8de915c7c221f012ee86

The JupyterLab project is developing rapidly, so it is worth checking the latest news on the project's website(https://github.com/jupyterlab/jupyterlab).

Before you start, you need to have a Linux machine with IJulia and either Jupyter Notebook or JupyterLab installed:

A general scenario for working with Jupyter Notebook/JupyterLab on a remote machine consists of the following three steps:

We discuss each step in detail and consider alternative configurations for Jupyter Notebook and JupyterLab:

$ ssh -i path/to/keyfile.pem -L8888:127.0.0.1:8888 ubuntu@[enter_hostname_here]

$ ~/.julia/packages/Conda/hsaaN/deps/usr/bin/jupyter lab

Please note that the Windows version of the preceding command is the following:

C:\> %userprofile%\.julia\packages\Conda\hsaaN\deps\usr\Scripts\jupyter-lab

You can also use either Jupyter Lab or Jupyter Notebook command versions (see the instructions from the previous recipe).

Copy/paste this URL into your browser when you connect for the first time,to login with a token:
    http://localhost:8888/?token=b86b66b81a62d4be1ae34e7d6bd006a8ba5cb937e74b99cf

If you have properly configured the SSH tunnel, you should see Jupyter Notebook/JupyterLab running on the remote server in the browser on the local machine. Please note that the environment will be available for only as long as the SSH connection is open. If you close the SSH connection, you will lose access to your Jupyter Notebook or JupyterLab.

Using Jupyter Notebook via the internet requires setting up an SSH tunnel to the server. This is the option we present in this tutorial. Please note that on a local network, you can connect directly to the server without SSH tunneling.

In the preceding scenario, the command-line argument-L 8888:127.0.0.1:8888 tells ssh to open a secure tunnel over SSH from your machine to port 8888 on the target machine in the cloud (as shown in the following diagram):

Yet another option for running IJulia on a text-only Terminal remote machine is to use a detached mode for the Jupyter environment. However, in order to obtain access to Jupyter Notebook, a token key is required. Please see the following example: 

julia> usingIJulia

julia> notebook(detached=true)

julia> run(`$(IJulia.notebook_cmd[1]) notebook list`)
Currently running servers:
http://localhost:8888/?token=2bb8421e3bba78c8c551a8af1f22460bb4bb3fdc5a0986bb

Now, you can simply copy and paste the preceding link into your web browser.

More information on SSH tunneling can be found at https://www.ssh.com/ssh/tunneling/example.