Julia Programming Projects

The creators of Julia have carefully picked the most successful elements of syntax from other languages, with the goal of producing expressive, concise, and readable code. Julia provides powerful and expressive language constructs for high-level numerical computing, in the same way as languages such as R, MATLAB, and Python do. It builds upon the experience brought by existing mathematical programming languages but also borrows much from popular dynamic ones, such as Lisp, Perl, Python, Lua, and Ruby.

To give you a quick taste of idiomatic Julia, here's how to open a file, read it, output it, and then have the file automatically closed by Julia:

open(".viminfo") do io
    read(io, String) |> println
end  

In the preceding snippet, we open the .viminfo file for reading passing io, an IOStream instance, into the underlying code block. The stream is then read into a String that is finally displayed onto the console by piping it into the println function. The code is very readable and easy to understand if you have some coding experience, even if this is your first time looking at Julia code.

This so-called do syntax (named after the do part following the open function) is inspired by Ruby's blocks—and it is, in fact, syntactic sugar for passing anonymous functions as method arguments. It is efficiently used in the preceding example to succinctly express a powerful design pattern for safely handling files, guaranteeing that the resources are not accidentally left open.

This goes to show the amount of attention that was put by the designers of the language to make Julia safe, beginner-friendly, expressive, concise, readable, and intuitive.

Julia's type system is a key feature of the language and one that has a major impact on both its performance and productivity. The type system is dynamic and optional, meaning that the developer can, but is not required to, provide type information to the compiler. If not provided, Julia will perform type inference, which is the process of deducing the types of later values from the types of input values. This is a very powerful technique, as it frees the programmers from having to worry about types, allowing them to focus on the application logic and making for a gentler learning curve. This is especially useful for prototyping and exploratory programming, when the complete set of constraints and requirements is not known beforehand.

However, understanding and correctly using the type system offers important performance benefits. Julia allows optionally adding type information, making it possible to indicate that a certain value must be of a specific kind. This is one of the cornerstones of the language, allowing performant method dispatching and facilitating the automatic generation of efficient, specialized code for different argument types. The type system allows the definition of rich type hierarchies, with user-defined types as fast and compact as the built-in ones.

If the languages of the 70s and 80s were designed under the strict requirements imposed by the limited CPU and RAM resources, the ones in the 90s and the 2000s had the optimistic outlook that these resources are forever expanding. However, the last decade had seen something of a stagnation in this regard, with a shift toward multi-CPU, multi-core, and distributed computing. In this regard, Julia's inception only 6 years ago gave it an edge compared to older languages, putting parallel and distributed computing at its center as one of its most important features.

One of the most serious barriers in the adoption of a new language is that it takes time for the ecosystem to catch up—and in the beginning, it cannot offer libraries of the quality and richness of the already established languages. This is less of an issue now, when Julia benefits from a large, enthusiastic and continuously growing developer community. But being able to seamlessly communicate with other languages is a very efficient way to enrich existing functionality and to effortlessly supplement any missing features.

Julia has the ability to directly call C and Fortran functions (that is, without glue code)—especially important for scientific computing, where these languages have a strong presence and a long history.

Optional packages extend this capability by adding support for calling functions written in other languages, most notably Python, via PyCall. And there are others, supporting interaction with Java, C++, MATLAB, Rust, and more.

The REPL represents a language shell, an interactive computer programming environment at the command line. Julia has an excellent REPL, supporting sophisticated code inputting and evaluation. It includes powerful editing features such as searchable history, tab-completion, and syntax highlighting, to name just a few.

It also comes with three special modes—shell, which allows executing commands as if at the OS Terminal; help, which provides access to documentation without leaving the REPL; and pkg, used for installing and managing application dependencies.

Julia comes with its own powerful package manager that resolves dependencies and handles the adding and removal of extra packages. Like most modern languages, Julia has full support for Unicode. And finally, it is distributed under the liberal MIT license—it is free and open source.

The simplest option is to download the Windows installer corresponding to your platform (32 or 64-bit), from https://julialang.org/downloads/. Get the .exe and run it. Follow the standard installation procedure and in the end, you will have Julia installed as a program. Double-clicking julia.exe opens a command prompt with the Julia REPL, just like the one illustrated here:

Chocolatey is a package manager for Windows, similar to apt or yum on Linux, or brew on Mac. If you don't have it, get it by following the instructions at https://chocolatey.org.

Chocolatey has the latest version of Julia, as can be confirmed with a search, as follows:

$ choco search julia 
Chocolatey v0.10.11 
Julia 1.0.0 [Approved] 
1 packages found. 

Installing is as simple as this:

$ choco install julia 
Chocolatey v0.10.11 
Installing the following packages: 
julia 
By installing you accept licenses for the packages. 
Progress: Downloading Julia 1.0.0... 100% 
Julia v1.0.0 [Approved] 
Chocolatey installed 1/1 packages.

One of the recent additions to Windows 10 is the Subsystem for Linux. This allows setting up a Linux development environment, including most command-line tools, utilities, and applications—directly on Windows, unmodified, and without the overhead of running a VM.

In order to be able to use the Linux Subsystem your PC must be running the 64-bit version of Windows 10 Anniversary Update or later (build 1607+). It also needs to be enabled first—so open a PowerShell as an administrator and run the following:

$ Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux 

Once the subsystem is enabled (computer restart might be required) you can choose one of the Linux versions available, directly from the Windows Store. At the time of writing, five versions were available—Ubuntu, openSUSE , SLES, Debian, and Kali.

Ubuntu is the default option for Windows 10 and has the best user ratings in the Windows Store, so let's go with that. It can be installed from https://www.microsoft.com/en-us/store/p/ubuntu/9nblggh4msv6. Alternatively, you can just open a command prompt and type $ bash. This will trigger the installation of the Ubuntu Linux Subsystem.

Once you find yourself at the shell prompt of your Linux subsystem, you can proceed and issue the commands for installing Julia. For Ubuntu you need to run the following:

$ sudo apt-get install julia

Make sure to confirm the required selections—then after a couple of minutes you should have Julia up and running.

Visit https://julialang.org/downloads/ and look for the macOS package (.dmg). Once it's downloaded, double-click the .dmg file and drag and drop the Julia app into the /Applications folder. Now you can simply open the Julia app—which in turn will launch a new Terminal session, loading the Julia environment, as follows:

Homebrew is a well-known package manager for macOS in the line of apt and yum on Linux. It's not really necessary for installing Julia, but it's worth setting it up as it can be very useful during development since it can seamlessly install database servers, libraries, and other components for your projects.

As per the instructions at https://brew.sh, it can be installed by running the following command in a Terminal window:

$ /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

It might take a while but once Homebrew is installed, a new command-line utility, brew, will become available.

Finally, $ brew cask install julia will download and install the latest version of Julia. In the process, it will also link the julia binary to /usr/local/bin/julia so you can interact with the language from the command line by simply typing $ julia.

As soon as you get the confirmation that the installation was successful you can run $ julia to start a new REPL session:

Depending on your OS and your preferences, the REPL can be started by simply invoking $ julia with no arguments, or by double-clicking the julia executable.

You will be greeted with a screen like this one (the Julia version might be different than mine):

Now Julia is waiting for us to input our code, evaluating it line by line. You can confirm that by checking the Terminal prompt, which says julia>. This is called the julian mode. Let's take it for a spin.

Input the following lines, pressing Enter after each one:

julia> 2+2 
julia> 2^3

So we can use Julia like a simple calculator. Not very useful, but this is only the beginning and illustrates how powerful this rapid input and feedback cycle can be when we deal with complex computations.

println is a very useful function that prints whatever value it receives, appending a new line afterward. Type the following code:

julia> println("Welcome to Julia")

Under each line, you should see the output generated by each expression. Your window should now look like this.

julia> 2+2 
4 
julia> 2^3 
8 
julia> println("Welcome to Julia") 
Welcome to Julia

Let's try some more. The REPL interprets one line at a time, but everything is evaluated in a common scope. This means that we can define variables and refer to them later on, as follows:

julia> greeting = "Hello" 
"Hello"

This looks great! Let's use the greeting variable with println:

julia> println(greting) 
ERROR: UndefVarError: greting not defined

Oops! A little typo there, and the REPL promptly returned an error. It's not greting, it's greeting. This also tells us that Julia does not allow using variables without properly initializing them. It just looked for the greting variable, unsuccessfully—and it threw an undefined variable error. Let's try that again, this time more carefully:

julia> println(greeting) 
Hello

OK, that's much better! We can see the output: the Hello value we stored in the greeting variable.

julia> using Dates 
 
julia> Dates.now() 
2018-09-02T21:13:03.122 
julia> ans 
2018-09-02T21:13:03.122

julia> pri[TAB] 
primitive type   print             print_shortest    print_with_color  println           printstyled 

julia> \pi[TAB] 
julia> π 
π = 3.1415926535897... 

julia> \om[TAB] \omega \ominus 
julia> \ome[TAB] 
julia> \omega[TAB] 
julia> ω

The help mode provides access to documentation without having to get out of the REPL. To access it, simply type ? at the beginning of the line. You should see the  help?> prompt. Now you can input text, and Julia will search the documentation for matching entries, as follows:

julia> ?  
help?> println 
search: println printstyled print_with_color print print_shortest sprint isprint 
 
  println([io::IO], xs...) 
 
  Print (using print) xs followed by a newline. If io is not supplied, prints to stdout. 
 
  Examples 
  ≡≡≡≡≡≡≡≡≡≡ 
 
  julia> println("Hello, world") 
  Hello, world 
 
  julia> io = IOBuffer(); 
 
  julia> println(io, "Hello, world") 
 
  julia> String(take!(io)) 
  "Hello, world\n" 

The output supports rich formatting, via Markdown:

julia> using Profile 
help?> Profile.print

Resulting a rich output as in the following screenshot:

More complex expressions can be queried, including macros, types, and variables.

For example, help?> @time:

Or help?> IO:

The shell mode is used to switch to a command-line interface similar to the system shell, for directly executing OS commands. To enter it, input a semicolon ; at the very beginning of the julian prompt:

julia> ;

Upon typing ; the prompt changes (in place) to shell>:

Now we can execute system-wide commands directly, without the need to wrap them in Julia code. This will list the last ten lines of your repl_history.jl file. This file is used by Julia to keep a history of the commands executed in the REPL, so your output will be different from mine:

julia> using REPL
shell> tail -n 10 ~/.julia/logs/repl_history.jl
IO
# time: 2018-09-02 21:56:47 CEST
# mode: julia
REPL.find_hist_file()
# time: 2018-09-02 21:58:47 CEST
# mode: shell
tail -n 10 ~/.julia/logs/repl_history.jl

While in REPL mode we can access Julia's API, making this a very powerful combo. For example, in order to programmatically get the path to the REPL history file, we can use the REPL.find_hist_file() function, as follows:

julia> REPL.find_hist_file() 
"/Users/adrian/.julia/logs/repl_history.jl"

The path to the file will be different for you.

We can use this in the shell mode by wrapping the command in $():

shell> tail -n 10 $(REPL.find_hist_file()) 
    REPL.find_hist_file() 
# time: 2018-09-02 21:58:47 CEST 
# mode: shell 
    tail -n 10 ~/.julia/logs/repl_history.jl 
# time: 2018-09-02 22:00:03 CEST 
# mode: shell 
    tail -n 10 $(REPL.find_hist_file())

Similarly to the help mode, the shell mode can be exited without executing any command by pressing backspace at the beginning of the line or typing Ctrl + C.

In IJulia, the command can be executed by prefixing the input with ;, like this:

;tail -n 10 ~/.julia/logs/repl_history.jl

Besides the help and the shell modes, there are two search modes. These are not necessarily Julia specific, being common to many *nix style editing apps.

Press the Ctrl key and the R key at the same time in order to initiate a reverse incremental search. The prompt will change to (reverse-i-search). Start typing your query and the most recent result will show. To find older results, type Ctrl + R again.

The counterpart of Ctrl + R is Ctrl + S, initiating an incremental search. The two may be used in conjunction to move through the previous or next matching results, respectively.

One of my favourite packages is called OhMyREPL. It implements a few super productive features for the Julia REPL, most notably syntax highlighting and brackets pairing. It's a great addition that makes the interactive coding experience even more pleasant and efficient.

Julia's Pkg is centered around GitHub. The creators distribute the packages as git repos, hosted on GitHub—and even the General registry is a GitHub repository itself. OhMyREPL is no exception. If you want to learn more about it before installing it—always a good idea when using code from third parties — you can check it out at https://github.com/KristofferC/OhMyREPL.jl 

Keep in mind that even if it's part of the General registry, the packages come with no guarantees and they're not necessarily checked, validated or endorsed by the Julia community. However, there are a few common sense indicators which provide insight into the quality of the package, most notably the number of stars, the status of the tests as well as the support for the most recent Julia versions.

The first thing we need to do in order to add a package is to enter the Pkg REPL-mode. We do this by typing ] at the beginning of the line:

julia>]

The cursor will change to reflect that we're now ready to manage packages:

(v1.0) pkg>

Pkg uses the concept of environments, allowing us to define distinct and independent sets of packages on a per-project basis. This is a very powerful and useful feature, as it eliminates dependency conflicts caused by projects that rely on different versions of the same package (the so-called dependency hell).

Given that we haven't created any project yet, Pkg will just use the default project, v1.0, indicated by the value between the parenthesis. This represents the Julia version that you're running on—and it's possible that you'll get a different default project depending on your very own version of Julia.

Now we can just go ahead and add OhMyREPL:

(v1.0) pkg> add OhMyREPL 
  Updating registry at `~/.julia/registries/General` 
  Updating git-repo `https://github.com/JuliaRegistries/General.git` 
 Resolving package versions... 
  Updating `~/.julia/environments/v1.0/Project.toml` 
  [5fb14364] + OhMyREPL v0.3.0 
  Updating `~/.julia/environments/v1.0/Manifest.toml` 
  [a8cc5b0e] + Crayons v1.0.0 
  [5fb14364] + OhMyREPL v0.3.0 
  [0796e94c] + Tokenize v0.5.2 
  [2a0f44e3] + Base64 
  [ade2ca70] + Dates 
  [8ba89e20] + Distributed 
  [b77e0a4c] + InteractiveUtils 
  [76f85450] + LibGit2 
  [8f399da3] + Libdl 
  [37e2e46d] + LinearAlgebra 
  [56ddb016] + Logging 
  [d6f4376e] + Markdown 
  [44cfe95a] + Pkg 
  [de0858da] + Printf 
  [3fa0cd96] + REPL 
  [9a3f8284] + Random 
  [ea8e919c] + SHA 
  [9e88b42a] + Serialization 
  [6462fe0b] + Sockets 
  [8dfed614] + Test 
  [cf7118a7] + UUIDs 
  [4ec0a83e] + Unicode 

When running pkg> add on a fresh Julia installation, Pkg will clone Julia's General registry and use it to look up the names of the package we requested. Although we only explicitly asked for OhMyREPL, most Julia packages have external dependencies that also need to be installed. As we can see, our package has quite a few—but they were promptly installed by Pkg.

Sometimes we might want to use packages that are not added to the general registry. This is usually the case with packages that are under (early) development—or private packages. For such situations, we can pass pkg> add the URL of the repository, instead of the package's name:

(v1.0) pkg> add https://github.com/JuliaLang/Example.jl.git 
   Cloning git-repo `https://github.com/JuliaLang/Example.jl.git` 
  Updating git-repo `https://github.com/JuliaLang/Example.jl.git` 
 Resolving package versions... 
  Updating `~/.julia/environments/v1.0/Project.toml` 
  [7876af07] + Example v0.5.1+ #master (https://github.com/JuliaLang/Example.jl.git) 
  Updating `~/.julia/environments/v1.0/Manifest.toml` 
  [7876af07] + Example v0.5.1+ #master (https://github.com/JuliaLang/Example.jl.git)

Another common scenario is when we want to install a certain branch of a package's repository. This can be easily achieved by appending #name_of_the_branch at the end of the package's name or URL:

(v1.0) pkg> add OhMyREPL#master 
   Cloning git-repo `https://github.com/KristofferC/OhMyREPL.jl.git` 
  Updating git-repo `https://github.com/KristofferC/OhMyREPL.jl.git` 
 Resolving package versions... 
 Installed Crayons ─ v0.5.1 
  Updating `~/.julia/environments/v1.0/Project.toml` 
  [5fb14364] ~ OhMyREPL v0.3.0 ⇒ v0.3.0 #master (https://github.com/KristofferC/OhMyREPL.jl.git) 
  Updating `~/.julia/environments/v1.0/Manifest.toml`

  [a8cc5b0e] ↓ Crayons v1.0.0 ⇒ v0.5.1 
  [5fb14364] ~ OhMyREPL v0.3.0 ⇒ v0.3.0 #master (https://github.com/KristofferC/OhMyREPL.jl.git)

Or, for unregistered packages, use the following:

(v1.0) pkg> add https://github.com/JuliaLang/Example.jl.git#master

If we want to get back to using the published branch, we need to free the package:

(v1.0) pkg> free OhMyREPL 
 Resolving package versions... 
  Updating `~/.julia/environments/v1.0/Project.toml` 
  [5fb14364] ~ OhMyREPL v0.3.0 #master (https://github.com/KristofferC/OhMyREPL.jl.git) ⇒ v0.3.0 
  Updating `~/.julia/environments/v1.0/Manifest.toml` 
  [a8cc5b0e] ↑ Crayons v0.5.1 ⇒ v1.0.0 
  [5fb14364] ~ OhMyREPL v0.3.0 #master (https://github.com/KristofferC/OhMyREPL.jl.git) ⇒ v0.3.0

(v1.0) pkg> add Revise 
Resolving package versions... 
 Installed Revise ─ v0.7.5 
  Updating `~/.julia/environments/v1.0/Project.toml` 
  [295af30f] + Revise v0.7.5 
  Updating `~/.julia/environments/v1.0/Manifest.toml` 
  [bac558e1] + OrderedCollections v0.1.0

  [295af30f] + Revise v0.7.5 
  [7b1f6079] + FileWatching 

OhMyREPL and Revise are excellent development tools and it's very useful to have them loaded automatically in all the Julia sessions. This is exactly why the startup.jl file exists—and now we have the opportunity to put it to good use (not that our heartfelt and welcoming greetings were not impressive enough!).

Here's a neat trick, to get us started—Julia provides an edit function that will open a file in the configured editor. Let's use it to open the startup.jl file:

julia> edit("~/.julia/config/startup.jl")

This will open the file in the default editor. If you haven't yet deleted our previously added welcome messages, feel free to do it now (unless you really like them and in that case, by all means, you can keep them). Now, Revise needs to be used before any other module that we want to track—so we'll want to have it at the top of the file. As for OhMyREPL, it can go next. Your startup.jl file should look like this:

using Revise 
using OhMyREPL 

Save it and close the editor. Next time you start Julia, both Revise and OhMyREPL will be already loaded.

Sometimes though we might want to ensure that certain packages will not be updated. That's when we pin them:

(v1.0) pkg> pin OhMyREPL 
 Resolving package versions... 
  Updating `~/.julia/environments/v1.0/Project.toml` 
  [5fb14364] ~ OhMyREPL v0.3.0 ⇒ v0.3.0
  Updating `~/.julia/environments/v1.0/Manifest.toml` 
  [5fb14364] ~ OhMyREPL v0.3.0 ⇒ v0.3.0

Pinned packages are marked with the ⚲ symbol—also present now when checking the status:

(v1.0) pkg> st 
    Status `~/.julia/environments/v1.0/Project.toml` 
  [5fb14364] OhMyREPL v0.3.0
  [295af30f] Revise v0.7.5

If we want to unpin a package, we can use pkg> free:

(v1.0) pkg> free OhMyREPL 
  Updating `~/.julia/environments/v1.0/Project.toml` 
  [5fb14364] ~ OhMyREPL v0.3.0 ⇒ v0.3.0 
  Updating `~/.julia/environments/v1.0/Manifest.toml` 
  [5fb14364] ~ OhMyREPL v0.3.0 ⇒ v0.3.0 
 
(v1.0) pkg> st 
    Status `~/.julia/environments/v1.0/Project.toml` 
  [5fb14364] OhMyREPL v0.3.0 
  [295af30f] Revise v0.7.5