How-To Tutorials

The frontend space is indeed crowded, but none of the more popular solutions are really convincing to me. I feel Angular is bloated and the double binding is not for me. I also do not like React and its syntax. Riot is, as stated by their creators, "A React-like user interface micro-library" with simpler syntax that is five times smaller than React. What we are going to learn We are going to build a simple Riot application backed by Express, using Jade as our template language. The backend will expose a simple REST API, which we will consume from the UI. We are not going to use any other dependency like JQuery, so this is also a good chance to try XMLHttpRequest2. I deliberately ommited the inclusion of a client package manager like webpack or jspm because I want to focus on the Expressjs + Riotjs. For the same reason, the application data is persisted in memory. Requirements You just need to have any recent version of node.js(+4), text editor of your choice, and some JS, Express and website development knowledge. Project layout Under my project directory we are going to have 3 directories: * public For assets like the riot.js library itself. * views Common in most Express setup, this is where we put the markup. * client This directory will host the Riot tags (we will see more of that later) We will also have the package.json, our project manifesto, and an app.js file, containing the Express application. Our Express server exposes a REST API; its code can be found in api.js. Here is how the layout of the final project looks: ├── api.js ├── app.js ├── client │ ├── tag-todo.jade │ └── tag-todo.js ├── package.json ├── node_modules ├── public │ └── js │ ├── client.js │ └── riot.js └── views └── index.jade Project setup Create your project directory and from there run the following to install the node.js dependencies: $ npm init -y $ npm install --save body-parser express jade And the application directories: $ mkdir -p views public/js client Start! Lets start by creating the Express application file, app.js: 'use strict'; const express = require('express'), app = express(), bodyParser = require('body-parser'); // Set the views directory and template engine app.set('views', __dirname + '/views'); app.set('view engine', 'jade'); // Set our static directory for public assets like client scripts app.use(express.static('public')); // Parses the body on incoming requests app.use(bodyParser.json()); // Pretty prints HTML output app.locals.pretty = true; // Define our main route, HTTP "GET /" which will print "hello" app.get('/', function (req, res) { res.send('hello'); }); // Start listening for connections app.listen(3000, function (err) { if (err) { console.error('Cannot listen at port 3000', err); } console.log('Todo app listening at port 3000'); }); The #app object we just created, is just a plain Express application. After setting up the application we call the listen function, which will create an HTTP server listening at port 3000. To test our application setup, we open another terminal, cd to our project directory and run $ node app.js. Open a web browser and load http://localhost:3000; can you read "hello"? Node.js will not reload the site if you change the files, so I recommend you to install nodemon. Nodemon monitors your code and reloads the site on every change you perform on the JS source code. The command,$ npm install -g nodemon, installs the program on your computer globally, so you can run it from any directory. Okay, kill our previously created server and start a new one with $ nodemon app.js. Our first Riot tag Riot allows you to encapsulate your UI logic in "custom tags". Tag syntax is pretty straightforward. Judge for yourself. <employee> <span>{ name }</span> </employee> Custom tags can contain code and can be nested as showed in the next code snippet: <employeeList> <employee each="{ items }" onclick={ gotoEmployee } /> <script> gotoEmployee (e) { var item = e.item; // do something } </script> </employeeList> This mechanism enables you to build complex functionality from simple units. Of course you can find more information at their documentation. On the next steps we will create our first tag: ./client/tag-todo.jade. Oh, we have not yet downloaded Riot! Here is the non minified Riot + compiler download. Download it to ./public/js/riot.js. Next step is to create our index view and tell our app to serve it. Locate / router handler, remove the res.send('hello) ', and update to: // Define our main route, HTTP "GET /" which will print "hello" app.get('/', function (req, res) { res.render('index'); }); Now, create the ./views/index.jade file: doctype html html head script(src="/js/riot.js") body h1 ToDo App todo Go to your browser and reload the page. You can read the big "ToDo App" but nothing else. There is a <todo></todo> tag there, but since the browser does not understand, this tag is not rendered. Let's tell Riot to mount the tag. Mount means Riot will use <todo></todo> as a placeholder for our —not yet there— todo tag. doctype html html head script(src="/js/riot.js") body h1 ToDo App script(type="riot/tag" src="/tags/todo.tag") todo script. riot.mount('todo'); Open your browser's dev console and reload the page. riot.mount failed because there was no todo.tag. Tags can be served in many ways, but I choose to serve them as regular Express templates. Of course, you can serve it as static assets or bundled. Just below the / route handler, add the /tags/:name.tag handler. // "/" route handler app.get('/', function (req, res) { res.render('index'); }); // tag route handler app.get('/tags/:name.tag', function (req, res) { var name = 'tag-' + req.params.name; res.render('../client/' + name); }); Now create the tag in ./client/tag-todo.jade: todo form(onsubmit="{ add }") input(type="text", placeholder="Needs to be done", name="todo") And reload the browser again. Errors gone and a new form in your browser. onsubmit="{ add }" is part of Riot's syntax and means on submit call the add function. You can add mix implementation with the markup, but I rather prefer to split markup from code. In Jade (and any other template language),it is trivial to include other files, which is exactly what we are going to do. Update the file as: todo form(onsubmit="{ add }") input(type="text", placeholder="Needs to be done", name="todo") script include tag-todo.js And create ./client/tag-todo.js with this snippet: 'use strict'; var self = this; var api = self.opts; When the tag gets mounted by Riot, it gets a context. That is the reason for var self = this;. That context can include the opts object. opts object can be anything of your choice, defined at the time you mount the tag. Let’s say we have an API object and we pass it to riot.mount as the second option at the time we mount the tag, that isriot.mount('todo', api). Then, at the time the tag is rendered this.opts will point to the api object. This is the mechanism we are going to use to expose our client api with the todo tag. Our form is still waiting for the add function, so edit the tag-todo.js again and append the following: self.add = function (e) { var title = self.todo.value; console.log('New ToDo', title); }; Reload the page, type something at the text field, and hit enter. The expected message should appear in your browser's dev console. Implementing our REST API We are ready to implement our REST API on the Express side. Create ./api.js file and add: 'use strict'; const express = require('express'); var app = module.exports = express(); // simple in memory DB var db = []; // handle ToDo creation app.post('/', function (req, res) { db.push({ title: req.body.title, done: false }); let todoID = db.length - 1; // mountpath = /api/todos/ res.location(app.mountpath + todoID); res.status(201).end(); }); // handle ToDo updates app.put('/', function (req, res) { db[req.body.id] = req.body; res.location('/' + req.body.id); res.status(204).end(); }); Our API supports ToDo creation/update, and it is architected as an Express sub application. To mount it, we just need to update app.js for the last time. Update the require block at app.js to: const express = require('express'), api = require('./api'), app = express(), bodyParser = require('body-parser'); ... And mount the api sub application just before the app.listen... // Mount the api sub application app.use('/api/todos/', api); We said we will implement a client for our API. It should expose two functions –create and update –located at ./public/client.js. Here is its source: 'use strict'; (function (api) { var url = '/api/todos/'; function extractIDFromResponse(xhr) { var location = xhr.getResponseHeader('location'); var result = +location.slice(url.length); return result; } api.create = function createToDo(title, callback) { var xhr = new XMLHttpRequest(); var todo = { title: title, done: false }; xhr.open('POST', url); xhr.setRequestHeader('Content-Type', 'application/json'); xhr.onload = function () { if (xhr.status === 201) { todo.id = extractIDFromResponse(xhr); } return callback(null, xhr, todo); }; xhr.send(JSON.stringify(todo)); }; api.update = function createToDo(todo, callback) { var xhr = new XMLHttpRequest(); xhr.open('PUT', url); xhr.setRequestHeader('Content-Type', 'application/json'); xhr.onload = function () { if (xhr.status === 200) { console.log('200'); } return callback(null, xhr, todo); }; xhr.send(JSON.stringify(todo)); }; })(this.todoAPI = {}); Okay, time to load the API client into the UI and share it with our tag. Modify the index view including it as a dependency: doctype html html head script(src="/js/riot.js") body h1 ToDo App script(type="riot/tag" src="/tags/todo.tag") script(src="/js/client.js") todo script. riot.mount('todo', todoAPI); We are now loading the API client and passing it as a reference to the todo tag. Our last change today is to update the add function to consume the API. Reload the browser again, type something into the textbox, and hit enter. Nothing new happens because our add function is not yet using the API. We need to update ./client/tag-todo.js as: 'use strict'; var self = this; var api = self.opts; self.items = []; self.add = function (e) { var title = self.todo.value; api.create(title, function (err, xhr, todo) { if (xhr.status === 201) { self.todo.value = ''; self.items.push(todo); self.update(); } }); }; We have augmented self with an array of items. Everytime we create a new ToDo task (after we get the 201 code from the server) we push that new ToDo object into the array because we are going to print that list of items. In Riot, we can loop the items adding each attribute to any tag. Last, update to ./client/tag-todo.jade todo form(onsubmit="{ add }") input(type="text", placeholder="Needs to be done", name="todo") ul li(each="{items}") span {title} script include tag-todo.js Finally! Reload the page and create a ToDo! Next steps You can find the complete source code for this article here. The final version of the code also implements a done/undone button, which you can try to implement by yourself. About the author Pedro NarcisoGarcíaRevington is a Senior Full Stack Developer with 10+ years experience in high scalability and availability, microservices, automated deployments, data processing, CI, (T,B,D)DD and polyglot persistence.

In this article by Jaroslaw Krochmalski, author of Developing with Docker, at the beginning, Docker was created as an internal tool by a Platform as a Service company, called dotCloud. Later on, in March 2013, it was released as open source. Apart from the Docker Inc. team, which is the main sponsor, there are some other big names contributing to the tool –Red Hat, IBM, Microsoft, Google, and Cisco Systems, just to name a few. Software development today needs to be agile and react quickly to the changes. We use methodologies such as Scrum, estimate our work in story points and attend the daily stand-ups. But what about preparing our software for shipment and the deployment? Let's see how Docker fits into that scenario and can help us being agile. (For more resources related to this topic, see here.) In this article, we will cover the following topics: The basic idea behind Docker A difference between virtualization and containerization Benefits of using Docker Components available to install We will begin with a basic idea behind this wonderful tool. The basic idea The basic idea behind Docker is to pack an application with all of its dependencies (let it be binaries, libraries, configuration files, scripts, jars, and so on) into a single, standardized unit for software development and deployment. Docker containers wrap up a piece of software in a complete filesystem that contains everything it needs to run: code, runtime, system tools, and system libraries—anything you can install on a server. This guarantees that it will always run in the same way, no matter what environment it will be deployed in. With Docker, you can build some Node.js or Java project (but you are of course not limited to those two) without having to install Node.js or Java on your host machine. Once you're done with it, you can just destroy the Docker image and it's as though nothing ever happened. It's not a programming language or a framework, rather think of it as about a tool that helps solving the common problems such as installing, distributing, and managing the software. It allows programmers and DevOps to build, ship, and run their code anywhere. You can think that maybe Docker is a virtualization engine, but it's far from it as we will explain in a while. Containerization versus virtualization To fully understand what Docker really is, first we need to understand the difference between traditional virtualization and containerization. Let's compare those two technologies now. Traditional virtualization A traditional virtual machine, which represents the hardware-level virtualization, is basically a complete operating system running on top of the host operating system. There are two types of virtualization hypervisors: Type 1 and Type 2. Type 1 hypervisors provide server virtualization on bare metal hardware—there is no traditional, end user's operating system. Type 2 hypervisors, on the other hand, are commonly used as a desktop virtualization—you run the virtualization engine on top of your own operating system. There can be a lot of use cases that would make an advantage from using virtualization—the biggest asset is that you can run many virtual machines with totally different operating systems on a single host. Virtual machines are fully isolated, hence very secure. But nothing comes without a price. There are many drawbacks—they contain all the features that an operating system needs to have: device drivers, core system libraries, and so on. They are heavyweight, usually resource-hungry and not so easy to set up—virtual machines require full installation. They require more computing resources to execute. To successfully run an application on a virtual machine, the hypervisor needs to first import the virtual machine and then power it up, and this takes time. Furthermore, their performance gets substantially degraded. As a result, only a few virtual machines can be provisioned and made available to work on a single machine. Containerization The Docker software runs in an isolated environment called a Docker container. A Docker container is not a virtual machine in the popular sense. It represents operating system virtualization. While each virtual machine image runs on an independent guest OS, the Docker images run within the same operating system kernel. A container has its own filesystem and environment variables. It's self-sufficient. Because of the containers run within the same kernel, they utilize fewer system resources. The base container can be, and usually is, very lightweight. Worth knowing is that Docker containers are isolated not only from the underlying OS, but from each other as well. There is no overhead related to a classic virtualization hypervisor and a guest operating system. This allows achieving almost bare metal, near native performance. The boot time of a dockerized application is usually very fast due to the low overhead of containers. It is also possible to speed up the roll-out of hundreds of application containers in seconds and to reduce the time taken for provisioning your software. As you can see, Docker is quite different from the traditional virtualization engines. Be aware that containers cannot substitute virtual machines for all use cases. A thoughtful evaluation is still required to determine what is best for your application. Both solutions have their advantages. On one hand we have the fully isolated, secure virtual machine with average performance and on the other hand, we have the containers that are missing some of the key features (such as total isolation), but are equipped with high performance that can be provisioned swiftly. Let's see what other benefits you will get when using Docker containerization. Benefits of using Docker When comparing the Docker containers with traditional virtual machines, we have mentioned some of its advantages. Let's summarize them now in more detail and add some more. Speed and size As we have said before, the first visible benefit of using Docker will be very satisfactory performance and short provisioning time. You can create or destroy containers quickly and easily. Docker shares only the Kernel, nothing less, nothing more. However, it reuses the image layers on which the specific image is built upon. Because of that, multiple versions of an application running in containers will be very lightweight. The result is faster deployment, easier migration, and nimble boot times. Reproducible and portable builds Using Docker enables you to deploy ready-to-run software, which is portable and extremely easy to distribute. Your containerized application simply runs within its container, there's no need for traditional installation. The key advantage of a Docker image is, it is bundled with all the dependency the containerized application needs to run. The lack of installation of dependencies process has a huge advantage. This eliminates problems such as software and library conflicts or even driver compatibility issues. Because of Docker's reproducible build environment, it's particularly well suited for testing, especially in your continuous integration flow. You can quickly boot up identical environments to run the tests. And because the container images are all identical each time, you can distribute the workload and run tests in parallel without a problem. Developers can run the same image on their machine that will be run in production later, which again has a huge advantage in testing. The use of Docker containers speeds up continuous integration. There are no more endless build-test-deploy cycles. Docker containers ensure that applications run identically in development, test, and production environments. One of Docker's greatest features is the portability. Docker containers are portable – they can be run from anywhere: your local machine, a nearby or distant server, and private or public cloud. When speaking about the cloud, all major cloud computing providers, such as Amazon Web Services, Google's Compute Platform have perceived Docker's availability and now support it. Docker containers can be run inside an Amazon EC2 instance, Google Compute Engine instance provided that the host operating system supports Docker. A container running on an Amazon EC2 instance can easily be transferred to some other environment, achieving the same consistency and functionality. Docker works very well with various other IaaS (Infrastructure-as-a-Service) providers such as Microsoft's Azure, IBM SoftLayer, or OpenStack. This additional level of abstraction from your infrastructure layer is an indispensable feature. You can just develop your software without worrying about the platform it will be run later on. It's truly a write once run everywhere solution. Immutable and agile infrastructure Maintaining a truly idempotent configuration management code base can be tricky and a time-consuming process. The code grows over time and becomes more and more troublesome. That's why the idea of an immutable infrastructure becomes more and more popular nowadays. Containerization comes to the rescue. By using containers during the process of development and deployment of your applications, you can simplify the process. Having a lightweight Docker server that needs almost no configuration management, you manage your applications simply by deploying and redeploying containers to the server. And again, because the containers are very lightweight, it takes only seconds of your time. As a starting point, you can download a prebuilt Docker image from the Docker Hub, which is like a repository of ready-to-use images. There are many choices of web servers, runtime platforms, databases, messaging servers and so on. It's like a real gold mine of software you can use for free to get a base foundation for your own project. The effect of the immutability of Docker's images is the result of a way they are being created. Docker makes use of a special file called a Dockerfile. This file contains all the setup instructions on how to create an image, like must-have components, libraries, exposed shared directories, network configuration, and so on. An image can be very basic, containing nothing but the operating system foundations, or—something that is more common—containing a whole prebuilt technology stack which is ready to launch. You can create images by hand, but it can be an automated process also. Docker creates images in a layered fashion: every feature you include will be added as another layer in the base image. This is another serious speed boost compared to the traditional virtualization techniques. Tools and APIs Of course, Docker is not just a Dockerfile processor and the runtime engine. It's a complete package with wide selection of tools and APIs that are helpful during the developer's and DevOp's daily work. First of all, there's The Docker Toolbox, which is an installer to quickly and easily install and setup a Docker environment on your own machine. The Kinematic is desktop developer environment for using Docker on Windows and Mac OS X. Docker distribution also contains a whole bunch of command-line tools. Let's look at them now. Tools overview On Windows, depending on the Windows version you use, there are two choices. It can be either Docker for Windows if you are on Windows 10 or later, or Docker Toolbox for all earlier versions of Windows. The same applies to MacOS. The newest offering is Docker for Mac, which runs as a native Mac application and uses xhyve to virtualize the Docker Engine environment and Linux kernel. For earlier version of Mac that doesn't meet the Docker for Mac requirements you should pick the Docker Toolbox for Mac. The idea behind Docker Toolbox and Docker native applications are the same—to virtualize Linux kernel and Docker engine on top of your operating system, we will be using Docker Toolbox, as it is more universal; it will run in all Windows and MacOS versions. The installation package for Windows and Mac OS is wrapped in an executable called the Docker Toolbox. The package contains all the tools you need to begin working with Docker. Of course there are tons of additional third-party utilities compatible with Docker, some of them very useful. But for now, let's focus on the default toolset. Before we start the installation, let's look at the tools that the installer package contains to better understand what changes will be made to your system. Docker Engine and Docker Engine client Docker is a client-server application. It consists of the daemon that does the important job: builds and downloads images, starts and stops containers and so on. It exposes a REST API that specifies interfaces for interacting with the daemon and is being used for remote management. Docker Engine accepts Docker commands from the command line, such as docker to run the image, docker ps to list running containers, docker images to list images, and so on. The Docker client is a command-line program that is being used to manage Docker hosts running Linux containers. It communicates with the Docker server using the REST API wrapper. You will interact with Docker by using the client to send commands to the server. Docker Engine works only on Linux. If you want run Docker on Windows or Mac OS, or want to provision multiple Docker hosts on a network or in the Cloud, you will then need the Docker Machine. Docker Machine Docker Machine is a fairly new command-line tool created by Docker team to manage Docker servers you can deploy containers to. It deprecated the old way of installing Docker with the Boot2Docker utility. The Docker Machine eliminates the need to create virtual machines manually and install Docker before starting Docker containers on them. It handles the provisioning and installation process for you behind the scenes. In other words, it's a quick way to get a new virtual machine provisioned and ready to run Docker containers. This is an indispensable tool when developing PaaS (Platform as a Service) architecture. Docker Machine not only creates a new VM with the Docker Engine installed in it, but sets up the certificate files for authentication and then configures the Docker client to talk to it. For flexibility purposes, the Docker Machine introduces the concept of drivers. Using drivers, Docker is able to communicate with various virtualization software and cloud providers. In fact, when you install Docker for Windows or Mac OS, the default VirtualBox driver will be used. The following command will be executed behind the scenes: docker-machine create --driver=virtualbox default Another available driver is amazonec2 for Amazon Web Services. It can be used to install Docker on the Amazon's cloud—we will do it later in this article. There are a lot of drivers ready to be used, and more are coming all the time. The list of existing official drivers with their documentation is always available at the Docker Drivers website: https://docs.docker.com/machine/drivers. The list contains the following drivers at the moment: Amazon Web Services Microsoft Azure Digital Ocean Exoscale Google Compute Engine Generic Microsoft Hyper-V OpenStack Rackspace IBM Softlayer Oracle VirtualBox VMware vCloud Air VMware Fusion VMware vSphere Apart from these, there is also a lot of third-party driver plugins available freely on the Internet sites such as GitHub. You can find additional drivers for different cloud providers and virtualization platforms, such as OVH Cloud or Parallels for Mac OS, for example, you are not limited to Amazon's AWS or Oracle's VirtualBox. As you can see, the choice is very broad. If you cannot find a specific driver for your Cloud provider, try looking for it on the GitHub. When installing the Docker Toolbox on Windows or Mac OS, Docker Machine will be selected by default. It's mandatory and currently the only way to run Docker on these operating systems. Installing the Docker Machine is not obligatory for Linux—there is no need to virtualize the Linux kernel there. However, if you want to deal with the Cloud providers or just want to have common runtime environment portable between Mac OS, Windows, and Linux, you can install Docker Machine for Linux as well. We will describe the process later in this article. Machine will be also used behind the scenes when using the graphical tool Kitematic, which we will present in a while. After the installation process, Docker Machine will be available as a command-line tool: docker-machine. Kitematic Kitematic is the software tool you can use to run containers through a plain, yet robust graphical user interface. In 2015, Docker acquired the Kitematic team, expecting to attract many more developers and hoping to open up the containerization solution to more developers and a wider, more general public. Kitematic is now included by default when installing Docker Toolbox on Mac OS and MS Windows. You can use it to comfortably search and fetch images you need from the Docker Hub. The tool can also be used to run your own app containers. Using the GUI, you can edit environment variables, map ports, configure volumes, study logs, and have command-line access to the containers. It is worth mentioning that you can seamlessly switch between Kitematic GUI and command-line interface to run and manage application containers. Kitematic is very convenient, however, if you want to have more control when dealing with the containers or just want to use scripting - the command line will be a better solution. In fact, Kitematic allows you to switch back and forth between the Docker CLI and the graphical interface. Any changes you make on the command-line interface will be directly reflected in Kitematic. The tool is simple to use, as you will see at the end of this article, when we are going to test our setup on Mac or Windows PC, we will be using the command-line interface for working with Docker. Docker compose Compose is a tool, executed from the command line as docker-compose. It replaces the old fig utility. It's used to define and run multicontainer Docker applications. Although it's very easy to imagine a multi-container application (like a web server in one container and a database in the other), it's not mandatory. So if you decide that your application will fit in a single Docker container, there will be no use for docker-compose. In real life, it's very likely that your application will span into multiple containers. With docker-compose, you use a compose file to configure your application's services, so they can be run together in an isolated environment. Then, using a single command, you create and start all the services from your configuration. When it comes to multicontainer applications, docker-compose is great for development and testing, as well as continuous integration workflows. Oracle VirtualBox Oracle VM VirtualBox is a free and open source hypervisor for x86 computers from Oracle. It will be installed by default when installing the Docker Toolbox. It supports the creation and management of virtual machines running Windows, Linux, BSD, OS/2, Solaris, and so on. In our case, the Docker Machine using VirtualBox driver, will use VirtualBox to create and boot a bitsy Linux distribution capable of the running docker-engine. It's worth mentioning that you can also run the teensy–weensy virtualized Linux straight from the VirtualBox itself. Every Docker Machine you have created using the docker-machine or Kitematic, will be visible and available to boot in the VirtualBox, when you run it directly, as shown in the following screenshot: You can start, stop, reset, change settings, and read logs in the same way as for other virtualized operating systems. You can use VirtualBox in Windows or Mac for other purposes than Docker. Git Git is a distributed version control system that is widely used for software development and other version control tasks. It has emphasis on speed, data integrity, and support for distributed, non-linear workflows. Docker Machine and Docker client follows the pull/push model of Git for fetching the needed dependencies from the network. For example, if you decide to run the Docker image which is not present on your local machine, Docker will fetch this image from the Docker Hub. Docker doesn't internally use Git for any kind of resource versioning. It does, however, rely on hashing to uniquely identify the filesystem layers which is very similar to what Git does. Docker also takes initial inspiration in the notion of commits, pushes, and pulls. Git is also included in the Docker Toolbox installation package. From a developer's perspective, there are tools especially useful in a programmer's daily job, be it IntelliJ IDEA Docker Integration Plugin for Java fans or Visual Studio 2015 Tools for Docker for those who prefer C#. They let you download and build Docker images, create and start containers, and carry out other related tasks straight from your favorite IDE. Apart from the tools included in the Docker's distribution package (it will be Docker Toolbox for older versions of Windows or Docker for Windows and Docker for Mac), there are hundreds of third-party tools, such as Kubernetes and Helios (for Docker orchestration), Prometheus (for monitoring of statistics) or Swarm and Shipyard for managing clusters. As Docker captures higher attention, more and more Docker-related tools pop-up almost every week. But these are not the only tools available for you. Additionally, Docker provides a set of APIs that can be very handy. One of them is the Remote API for the management of the images and containers. Using this API, you will be able to distribute your images to the runtime Docker engine. The container can be shifted to a different machine that runs Docker, and executed there without compatibility concerns. This may be especially useful when creating PaaS (Platform-as-a-Service) architectures. There's also the Stats API that will expose live resource usage information (such as CPU, memory, network I/O and block I/O) for your containers. This API endpoint can be used to create tools that show how your containers behave, for example, on a production system. Summary By now we understand the difference between the virtualization and containerization and also, I hope, we can see the advantages of using the latter. We also know what components are available for us to install and use. Let's begin our journey to the world of containers and go straight to the action by installing the software. Resources for Article: Further resources on this subject: Let's start with Extending Docker [article] Docker Hosts [article] Understanding Docker [article]

This article is by Pieter van der Westhuizen, author of the book Bootstrap for ASP.NET MVC - Second edition. As developers, we can find it difficult to create great-looking user interfaces from scratch when using HTML and CSS. This is especially hard when developers have extensive experience developing Windows Forms applications. Microsoft introduced Web Forms to remove the complexities of building websites and to ease the switch from Windows Forms to the Web. This in turn makes it very hard for Web Forms developers, and even harder for Windows Forms developers to switch to ASP.NET MVC. Bootstrap is a set of stylized components, plugins, and a layout grid that takes care of the heavy lifting. Microsoft included Bootstrap in all ASP.NET MVC project templates since 2013. In this article, we will cover the following topics: Files included in the Bootstrap distribution How to create an empty ASP.NET site and enable MVC and static files Adding the Bootstrap files using Bower Automatically compile the Bootstrap Sass files using Gulp Installing additional icon sets How to create a layout file that references the Bootstrap files (For more resources related to this topic, see here.) Files included in the Bootstrap distribution In order to get acquainted with the files inside the Bootstrap distribution, you need to download its source files. At the time of writing, Bootstrap 4 was still in Alpha, and its source files can be downloaded from http://v4-alpha.getbootstrap.com. Bootstrap style sheets Do not be alarmed by the amount of files inside the css folder. This folder contains four .css files and two .map files. We only need to include the bootstrap.css file in our project for the Bootstrap styles to be applied to our pages. The bootstrap.min.css file is simply a minified version of the aforementioned file. The .map files can be ignored for the project we'll be creating. These files are used as a type of debug symbol (similar to the .pdb files in Visual Studio), which allows developers to live edit their preprocessor source files. Bootstrap JavaScript files The js folder contains two files. All the Bootstrap plugins are contained in the bootstrap.js file. The bootstrap.min.js file is simply a minified version of the aforementioned file. Before including the file in your project, make sure that you have a reference to the jQuery library because all Bootstrap plugins require jQuery. Bootstrap fonts/icons Bootstrap 3 uses Glyphicons to display various icons and glyphs in Bootstrap sites. Bootstrap 4 will no longer ship with glyphicons included, but you still have the option to include it manually or to include your own icons. The following two icon sets are good alternatives to Glyphicons: Font Awesome, available from http://fontawesome.io/ GitHub's Octicons, available from https://octicons.github.com/ Bootstrap source files Before you can get started with Bootstrap, you first need to download the Bootstrap source files. At the time of writing, Bootstrap 4 was at version 4 Alpha 3. You have a few choices when adding Bootstrap to you project. You can download the compiled CSS and JavaScript files or you can use a number of package managers to install the Bootstrap Sass source to your project. In this article, you'll be using Bower to add the Bootstrap 4 source files to your project. For a complete list of Bootstrap 4 Alpha installation sources, visit http://v4-alpha.getbootstrap.com/getting-started/download/ CSS pre-processors CSS pre-processors process code written in a pre-processed language, such as LESS or Sass, and convert it into standard CSS, which in turn can be interpreted by any standard web browser. CSS pre-processors extend CSS by adding features that allow variables, mixins, and functions. The benefits of using CSS pre-processors are that they are not bound by any limitations of CSS. CSS pre-processors can give you more functionality and control over your style sheets and allows you to write more maintainable, flexible, and extendable CSS. CSS pre-processors can also help to reduce the amount of CSS and assist with the management of large and complex style sheets that can become harder to maintain as the size and complexity increases. In essence, CSS pre-processors such as Less and Sass enables programmatic control over your style sheets. Bootstrap moved their source files from Less to Sass with version 4.  Less and Sass are very alike in that they share a similar syntax as well as features such as variables, mixins, partials, and nesting, to name but a few. Less was influenced by Sass, and later on, Sass was influenced by Less when it adopted CSS-like block formatting, which worked very well for Less. Creating an empty ASP.NET MVC site and adding Bootstrap manually The default ASP.NET 5 project template in Visual Studio 2015 Update 3 currently adds Bootstrap 3 to the project. In order to use Bootstrap 4 in your ASP.NET project, you'll need to create an empty ASP.NET project and add the Bootstrap 4 files manually. To create a project that uses Bootstrap 4, complete the following process: In Visual Studio 2015, select New | Project from the File menu or use the keyboard shortcut Ctrl + Shift + N. From the New Project dialog window, select ASP.NET Core Web Application (.NET Core), which you'll find under Templates | Visual C# | Web. Select the Empty project template from the New ASP.NET Core Web Application (.NET Core) Project dialog window and click on OK. Enabling MVC and static files The previous steps will create a blank ASP.NET Core project. Running the project as-is will only show a simple Hello World output in your browser. In order for it to serve static files and enable MVC, we'll need to complete the following steps: Double-click on the project.json file inside the Solution Explorer in Visual Studio. Add the following two lines to the dependencies section, and save the project.json file: "Microsoft.AspNetCore.Mvc": "1.0.0", "Microsoft.AspNetCore.StaticFiles": "1.0.0" You should see a yellow colored notification inside the Visual Studio Solution Explorer with a message stating that it is busy restoring packages. Open the Startup.cs file. To enable MVC for the project, change the ConfigureServices method to the following: public void ConfigureServices(IServiceCollection services) {     services.AddMvc(); } Finally, update the Configure method to the following code: public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory) {     loggerFactory.AddConsole();       if (env.IsDevelopment())     {         app.UseDeveloperExceptionPage();     }       app.UseStaticFiles();       app.UseMvc(routes =>     {         routes.MapRoute(             name: "default",             template: "{controller=Home}/{action=Index}/{id?}");     }); } The preceding code will enable logging and the serving of static files such as images, style sheets, and JavaScript files. It will also set the default MVC route. Creating the default route controller and view When creating an empty ASP.NET Core project, no default controller or views will be created by default. In the previous steps, we've created a default route to the Index action of the Home controller. In order for this to work, we first need to complete the following steps: In the Visual Studio Solution Explorer, right-click on the project name and select Add | New Folder from the context menu. Name the new folder Controllers. Add another folder called Views. Right-click on the Controllers folder and select Add | New Item… from the context menu. Select MVC Controller Class from the Add New Item dialog, located under .NET Core | ASP.NET, and click on Add. The default name when adding a new controller will be HomeController.cs: Next, we'll need to add a subfolder for the HomeController in the Views folder. Right-click on the Views folder and select Add | New Folder from the context menu. Name the new folder Home. Right-click on the newly created Home folder and select Add | New Item… from the context menu. Select the MVC View Page item, located under .NET Core | ASP.NET; from the list, make sure the filename is Index.cshtml and click on the Add button: Your project layout should resemble the following in the Visual Studio Solution Explorer: Adding the Bootstrap 4 files using Bower With ASP.NET 5 and Visual Studio 2015, Microsoft provided the ability to use Bower as a client-side package manager. Bower is a package manager for web frameworks and libraries that is already very popular in the web development community. You can read more about Bower and search the packages it provides by visiting http://bower.io/ Microsoft's decision to allow the use of Bower and package managers other than NuGet for client-side dependencies is because it already has such a rich ecosystem. Do not fear! NuGet is not going away. You can still use NuGet to install libraries and components, including Bootstrap 4! To add the Bootstrap 4 source files to your project, you need to follow these steps: Right-click on the project name inside Visual Studio's Solution Explorer and select Add | New Item…. Under .NET Core | Client-side, select the Bower Configuration File item, make sure the filename is bower.json and click on Add, as shown here: If not already open, double-click on the bower.json file to open it and add Bootstrap 4 to the dependencies array. The code for the file should look similar to the following: {    "name": "asp.net",    "private": true,   "dependencies": {     "bootstrap": "v4.0.0-alpha.3"   } } Save the bower.json file. Once you've saved the bower.json file, Visual Studio will automatically download the dependencies into the wwwroot/lib folder of your project. In the case of Bootstrap 4 it also depends on jQuery and Tether, you'll notice that jQuery and Tether has also been downloaded as part of the Bootstrap dependency. After you've added Bootstrap to your project, your project layout should look similar to the following screenshot: Compiling the Bootstrap Sass files using Gulp When adding Bootstrap 4, you'll notice that the bootstrap folder contains a subfolder called dist. Inside the dist folder, there are ready-to-use Bootstrap CSS and JavaScript files that you can use as-is if you do not want to change any of the default Bootstrap colours or properties. However, because the source Sass files were also added, this gives you extra flexibility in customizing the look and feel of your web application. For instance, the default colour of the base Bootstrap distribution is gray; if you want to change all the default colours to shades of blue, it would be tedious work to find and replace all references to the colours in the CSS file. For example, if you open the _variables.scss file, located in wwwroot/lib/bootstrap/scss, you'll notice the following code: $gray-dark:                 #373a3c !default; $gray:                      #55595c !default; $gray-light:                #818a91 !default; $gray-lighter:              #eceeef !default; $gray-lightest:             #f7f7f9 !default; We're not going to go into too much detail regarding Sass in this article, but the $ in front of the names in the code above indicates that these are variables used to compile the final CSS file. In essence, changing the values of these variables will change the colors to the new values we've specified, when the Sass file is compiled. To learn more about Sass, head over to http://sass-lang.com/ Adding Gulp npm packages We'll need to add the gulp and gulp-sass Node packages to our solution in order to be able to perform actions using Gulp. To accomplish this, you will need to use npm. npm is the default package manager for the Node.js runtime environment. You can read more about it at https://www.npmjs.com/ To add the gulp and gulp-sass npm packages to your ASP.NET project, complete the following steps: Right-click on your project name inside the Visual Studio Solution Explorer and select Add | New Item… from the project context menu. Find the npm Configuration File item, located under .NET Core | Client-side. Keep its name as package.json and click on Add. If not already open, double-click on the newly added package.json file and add the following two dependencies to the devDependencies array inside the file: "devDependencies": {   "gulp": "3.9.1",   "gulp-sass": "2.3.2" } This will add version 3.9.1 of the gulp package and version 2.3.2 of the gulp-sass package to your project. At the time of writing, these were the latest versions. Your version numbers might differ. Enabling Gulp-Sass compilation Visual Studio does not compile Sass files to CSS by default without installing extensions, but we can enable it using Gulp. Gulp is a JavaScript toolkit used to stream client-side code through a series of processes when an event is triggered during build. Gulp can be used to automate and simplify development and repetitive tasks, such as the following: Minify CSS, JavaScript and image files Rename files Combine CSS files Learn more about Gulp at http://gulpjs.com/ Before you can use Gulp to compile your Sass files to CSS, you need to complete the following tasks: Add a new Gulp Configuration File to your project by right-cing Add | New Item… from the context menu. The location of the item is .NET Core | Client-side. Keep the filename as gulpfile.js and click on the Add button. Change the code inside the gulpfile.js file to the following: var gulp = require('gulp'); var gulpSass = require('gulp-sass'); gulp.task('compile-sass', function () {     gulp.src('./wwwroot/lib/bootstrap/scss/bootstrap.scss')         .pipe(gulpSass())         .pipe(gulp.dest('./wwwroot/css')); }); The code in the preceding step first declares that we require the gulp and gulp-sass packages, and then creates a new task called compile-sass that will compile the Sass source file located at /wwwroot/lib/bootstrap/scss/bootstrap.scss and output the result to the /wwwroot/css folder. Running Gulp tasks With the gulpfile.js properly configured, you are now ready to run your first Gulp task to compile the Bootstrap Sass to CSS. Accomplish this by completing the following steps: Right-click on gulpfile.js in the Visual Studio Solution Explorer and choose Task Runner Explorer from the context menu. You should see all tasks declared in the gulpfile.js listed underneath the Tasks node. If you do not see tasks listed, click on the Refresh button, located on the left-hand side of the Task Runner Explorer window. To run the compile-sass task, right-click on it and select Run from the context menu. Gulp will compile the Bootstrap 4 Sass files and output the CSS to the specified folder. Binding Gulp tasks to Visual Studio events Right-clicking on every task in the Task Runner Explorer in order to execute each, could involve a lot of manual steps. Luckily, Visual Studio allows us to bind tasks to the following events inside Visual Studio: Before Build After Build Clean Project Open If, for example, we would like to compile the Bootstrap 4 Sass files before building our project, simply select Before Build from the Bindings context menu of the Visual Studio Task Runner Explorer: Visual Studio will add the following line of code to the top of gulpfile.js to tell the compiler to run the task before building the project: /// <binding BeforeBuild='compile-sass' /> Installing Font Awesome Bootstrap 4 no longer comes bundled with the Glyphicons icon set. However, there are a number of free alternatives available for use with your Bootstrap and other projects. Font Awesome is a very good alternative to Glyphicons that provides you with 650 icons to use and is free for commercial use. Learn more about Font Awesome by visiting https://fortawesome.github.io/Font-Awesome/ You can add a reference to Font Awesome manually, but since we already have everything set up in our project, the quickest option is to simply install Font Awesome using Bower and compile it to the Bootstrap style sheet using Gulp. To accomplish this, follow these steps: Open the bower.json file, which is located in your project route. If you do not see the file inside the Visual Studio Solution Explorer, click on the Show All Files button on the Solution Explorer toolbar. Add font-awesome as a dependency to the file. The complete listing of the bower.json file is as follows: {   "name": "asp.net",   "private": true,   "dependencies": {     "bootstrap": "v4.0.0-alpha.3",     "font-awesome": "4.6.3"   } } Visual Studio will download the Font Awesome source files and add a font-awesome subfolder to the wwwroot/lib/ folder inside your project. Copy the fonts folder located under wwwroot/font-awesome to the wwwroot folder. Next, open the bootstrap.scss file located in the wwwroot/lib/bootstrap/scss folder and add the following line at the end of the file: $fa-font-path: "/fonts"; @import "../../font-awesome/scss/font-awesome.scss"; Run the compile-sass task via the Task Runner Explorer to recompile the Bootstrap Sass. The preceding steps will include Font Awesome in your Bootstrap CSS file, which in turn will enable you to use it inside your project by including the mark-up demonstrated here: <i class="fa fa-pied-piper-alt"></i> Creating a MVC Layout page The final step for using Bootstrap 4 in your ASP.NET MVC project is to create a Layout page that will contain all the necessary CSS and JavaScript files in order to include Bootstrap components in your pages. To create a Layout page, follow these steps: Add a new sub folder called Shared to the Views folder. Add a new MVC View Layout Page to the Shared folder. The item can be found in the .NET Core | Server-side category of the Add New Item dialog. Name the file _Layout.cshtml and click on the Add button: With the current project layout, add the following HTML to the _Layout.cshtml file: <!DOCTYPE html> <html lang="en"> <head>     <meta charset="utf-8">     <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">     <meta http-equiv="x-ua-compatible" content="ie=edge">     <title>@ViewBag.Title</title>     <link rel="stylesheet" href="~/css/bootstrap.css" /> </head> <body>     @RenderBody()       <script src="~/lib/jquery/dist/jquery.js"></script>     <script src="~/lib/bootstrap/dist/js/bootstrap.js"></script> </body> </html> Finally, add a new MVC View Start Page to the Views folder called _ViewStart.cshtml. The _ViewStart.cshtml file is used to specify common code shared by all views. Add the following Razor mark-up to the _ViewStart.cshtml file: @{     Layout = "_Layout"; } In the preceding mark-up, a reference to the Bootstrap CSS file that was generated using the Sass source files and Gulp is added to the <head> element of the file. In the <body> tag, the @RenderBody method is invoked using Razor syntax. Finally, at the bottom of the file, just before the closing </body> tag, a reference to the jQuery library and the Bootstrap JavaScript file is added. Note that jQuery must always be referenced before the Bootstrap JavaScript file. Content Delivery Networks You could also reference the jQuery and Bootstrap library from a Content Delivery Network (CDN). This is a good approach to use when adding references to the most widely used JavaScript libraries. This should allow your site to load faster if the user has already visited a site that uses the same library from the same CDN, because the library will be cached in their browser. In order to reference the Bootstrap and jQuery libraries from a CDN, change the <script> tags to the following: <script src="https://code.jquery.com/jquery-3.1.0.js"></script> <script src="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0-alpha.2/js/bootstrap.min.js"></script> There are a number of CDNs available on the Internet; listed here are some of the more popular options: MaxCDN: https://www.maxcdn.com/ Google Hosted Libraries: https://developers.google.com/speed/libraries/ CloudFlare: https://www.cloudflare.com/ Amazon CloudFront: https://aws.amazon.com/cloudfront/ Learn more about Bootstrap Frontend development with Bootstrap 4

In this article by Stephen Haney, the author of the book Swift 3 Game Development - Second Edition, we will see that apple's newest version of its flagship programming language, Swift 3, is the perfect choice for game developers. As it matures, Swift is realizing its opportunity to be something special, a revolutionary tool for app creators. Swift is the gateway for developers to create the next big game in the Apple ecosystem. We have only started to explore the wonderful potential of mobile gaming, and Swift is the modernization we need for our toolset. Swift is fast, safe, current, and attractive to developers coming from other languages. Whether you are new to the Apple world, or a seasoned veteran of Objective-C, I think you will enjoy making games with Swift. (For more resources related to this topic, see here.) Apple's website states the following: "Swift is a successor to the C and Objective-C languages." My goal is to guide you step-by-step through the creation of a 2D game for iPhones and iPads. We will start with installing the necessary software, work through each layer of game development, and ultimately publish our new game to the App Store. We will also have some fun along the way! We aim to create an endless flyer game featuring a magnificent flying penguin named Pierre. What is an endless flyer? Picture hit games like iCopter, Flappy Bird, Whale Trail, Jetpack Joyride, and many more—the list is quite long. Endless flyer games are popular on the App Store, and the genre necessitates that we cover many reusable components of 2D game design. I will show you how to modify our mechanics to create many different game styles. My hope is that our demo project will serve as a template for your own creative works. Before you know it, you will be publishing your own game ideas using the techniques we explore together. In this article, we will learn the following topics: Why you will love Swift What you will learn in this article New in Swift 3 Setting up your development environment Creating your first Swift game Why you will love Swift Swift, as a modern programming language, benefits from the collective experience of the programming community; it combines the best parts of other languages and avoids poor design decisions. Here are a few of my favorite Swift features: Beautiful syntax: Swift's syntax is modern and approachable, regardless of your existing programming experience. Apple balanced syntax with structure to make Swift concise and readable. Interoperability: Swift can plug directly into your existing projects and run side-by-side with your Objective-C code. Strong typing: Swift is a strongly typed language. This means the compiler will catch more bugs at compile time, instead of when your users are playing your game! The compiler will expect your variables to be of a certain type (int, string, and so on) and will throw a compile-time error if you try to assign a value of a different type. While this may seem rigid if you are coming from a weakly typed language, the added structure results in safer, more reliable code. Smart type inference: To make things easier, type inference will automatically detect the types of your variables and constants based upon their initial value. You do not need to explicitly declare a type for your variables. Swift is smart enough to infer variable types in most expressions. Automatic memory management: As the Apple Swift developer guide states, "memory management just works in Swift". Swift uses a method called Automatic Reference Counting (ARC) to manage your game's memory usage. Besides a few edge cases, you can rely on Swift to safely clean up and turn off the lights. An even playing field: One of my favorite things about Swift is how quickly the language is gaining mainstream adoption. We are all learning and growing together, and there is a tremendous opportunity to break new ground. Open source: From version 2.2 onwards, Apple made Swift open source, curetting it through the website www.swift.org, and launched a package manager with Swift 3. This is a welcome change as it fosters greater community involvement and a larger ecosystem of third party tools and add-ons. Eventually, we should see Swift migrate to new platforms. Prerequisites I will try to make this text easy to understand for all skill levels: I will assume you are brand new to Swift as a language Requires no prior game development experience, though it will help I will assume you have a fundamental understanding of common programming concepts What you will learn in this article You will be capable of creating and publishing your own iOS games. You will know how to combine the techniques we learned to create your own style of game, and you will be well prepared to dive into more advanced topics with a solid foundation in 2D game design. Embracing SpriteKit SpriteKit is Apple's 2D game development framework and your main tool for iOS game design. SpriteKit will handle the mechanics of our graphics rendering, physics, and sound playback. As far as game development frameworks go, SpriteKit is a terrific choice. It is built and supported by Apple and thus integrates perfectly with Xcode and iOS. You will learn to be highly proficient with SpriteKit as we will be using it exclusively in our demo game. We will learn to use SpriteKit to power the mechanics of our game in the following ways: Animate our player, enemies, and power-ups Paint and move side-scrolling environments Play sounds and music Apply physics-like gravity and impulses for movement Handle collisions between game objects Reacting to player input The control schemes in mobile games must be inventive. Mobile hardware forces us to simulate traditional controller inputs, such as directional pads and multiple buttons, on the screen. This takes up valuable visible area, and provides less precision and feedback than with physical devices. Many games operate with only a single input method: A single tap anywhere on the screen. We will learn how to make the best of mobile input, and explore new forms of control by sensing device motion and tilt. Structuring your game code It is important to write well-structured code that is easy to re-use and modify as your game design inevitably changes. You will often find mechanical improvements as you develop and test your games, and you will thank yourself for a clean working environment. Though there are many ways to approach this topic, we will explore some best practices to build an organized system with classes, protocols, inheritance, and composition. Building UI/menus/levels We will learn to switch between scenes in our game with a menu screen. We will cover the basics of user experience design and menu layout as we build our demo game. Integrating with Game Center Game Center is Apple's built-in social gaming network. Your game can tie into Game Center to store and share high scores and achievements. We will learn how to register for Game Center, tie it into our code, and create a fun achievement system. Maximizing fun If you are like me, you will have dozens of ideas for games floating around your head. Ideas come easily, but designing fun game play is difficult! It is common to find that your ideas need game play enhancements once you see your design in action. We will look at how to avoid dead-ends and see your project through to the finish line. Plus, I will share my tips and tricks to ensure your game will bring joy to your players. Crossing the finish line Creating a game is an experience you will treasure. Sharing your hard work will only sweeten the satisfaction. Once our game is polished and ready for public consumption, we will navigate the App Store submission process together. You will finish feeling confident in your ability to create games with Swift and bring them to market in the App Store. Monetizing your work Game development is a fun and rewarding process, even without compensation, but the potential exists to start a career, or side-job, selling games on the App Store. Successfully promoting and marketing your game is an important task. I will outline your options and start you down the path to monetization. New in Swift 3 The largest feature in Swift 3 is syntax compatibility and stability. Apple is trying to refine its young, shifting language into its final foundational shape. Each successive update of Swift has introduced breaking syntax changes that made older code incompatible with the newest version of Swift; this is very inconvenient for developers. Going forward, Swift 3 aims to reach maturity and maintain source compatibility with future releases of the language. Swift 3 also features the following:  A package manager that will help grow the ecosystem A more consistent, readable API that often results in less code for the same result Improved tooling and bug fixes in the IDE, Xcode Many small syntax improvements in consistency and clarity Swift has already made tremendous steps forward as a powerful, young language. Now Apple is working on polishing Swift into a mature, production-ready tool. The overall developer experience improves with Swift 3. Setting up your development environment Learning a new development environment can be a roadblock. Luckily, Apple provides some excellent tools for iOS developers. We will start our journey by installing Xcode. Introducing and installing Xcode Xcode is Apple's Integrated Development Environment (IDE). You will need Xcode to create your game projects, write and debug your code, and build your project for the App Store. Xcode also comes bundled with an iOS simulator to test your game on virtualized iPhones and iPads on your computer. Apple praises Xcode as "an incredibly productive environment for building amazing apps for Mac, iPhone, and iPad".   To install Xcode, search for Xcode in the AppStore, or visit http://developer.apple.com and select Developer and then Xcode. Swift is continually evolving, and each new Xcode release brings changes to Swift. If you run into errors because Swift has changed, you can always use Xcode's built-in syntax update tool. Simply use Xcode's Edit | Convert to Latest Syntax option to update your code. Xcode performs common IDE features to help you write better, faster code. If you have used IDEs in the past, then you are probably familiar with auto completion, live error highlighting, running and debugging a project, and using a project manager pane to create and organize your files. However, any new program can seem overwhelming at first. We will walk through some common interface functions over the next few pages. I have also found tutorial videos on YouTube to be particularly helpful if you are stuck. Most common search queries result in helpful videos. Creating our first Swift game Do you have Xcode installed? Let us see some game code in action in the simulator! We will start by creating a new project in Xcode. For our demo game, we will create a side-scrolling endless flyer featuring an astonishing flying penguin named Pierre. I am going to name this project Pierre Penguin Escapes the Antarctic, but feel free to name your project whatever you like. Follow these steps to create a new project in Xcode: Launch Xcode and navigate to File | New | Project. You will see a screen asking you to select a template for your new project. Select iOS | Application in the left pane, and Game in the right pane. It should look like this: Once you select Game, click Next. The following screen asks us to enter some basic information about our project. Don’t worry; we are almost at the fun bit. Fill in the Product Name field with the name of your game. Let us fill in the Team field. Do you have an active Apple developer account? If not, you can skip over the Team field for now. If you do, your Team is your developer account. Click Add Team and Xcode will open the accounts screen where you can log in. Enter your developer credentials as shown in the following screenshot: Once you're authenticated, you can close the accounts screen. Your developer account should appear in the Team dropdown. You will want to pick a meaningful Organization Name and Organization Identifier when you create your own games for publication. Your Organization Name is the name of your game development studio. For me, that's Joyful Games. By convention, your Organization Identifier should follow a reverse domain name style. I will use io.JoyfulGames since my website is JoyfulGames.io. After you fill out the name fields, be sure to select Swift for the Language, SpriteKit for Game Technology, and Universal for Devices. For now, uncheck Integrate GameplayKit, uncheck Include Unit Tests, uncheck Include UI Tests. We will not use these features in our demo game. Here are my final project settings: Click Next and you will see the final dialog box. Save your new project. Pick a location on your computer and click Next. And we are in! Xcode has pre-populated our project with a basic SpriteKit template. Navigating our project Now that we have created our project, you will see the project navigator on the left-hand side of Xcode. You will use the project navigator to add, remove, and rename files and generally organize your project. You might notice that Xcode has created quite a few files in our new project. We will take it slow; don’t feel that you have to know what each file does yet, but feel free to explore them if you are curious: Exploring the SpriteKit Demo Use the project navigator to open up the file named GameScene.swift. Xcode created GameScene.swift to store the default scene of our new game. What is a scene? SpriteKit uses the concept of scenes to encapsulate each unique area of a game. Think of the scenes in a movie; we will create a scene for the main menu, a scene for the Game Over screen, a scene for each level in our game, and so on. If you are on the main menu of a game and you tap Play, you move from the menu scene to the Level 1 scene. SpriteKit prepends its class names with the letters "SK"; consequently, the scene class is SKScene. You will see there is already some code in this scene. The SpriteKit project template comes with a very small demo. Let's take a quick look at this demo code and use it to test the iOS simulator. Please do not be concerned with understanding the demo code at this point. Your focus should be on learning the development environment. Look for the run toolbar at the top of the Xcode window. It should look something like the following: Select the iOS device of your choice to simulate using the dropdown on the far right. Which iOS device should you simulate? You are free to use the device of your choice. I will be using an iPhone 6 for the screenshots, so choose iPhone 6 if you want your results to match my images perfectly. Unfortunately, expect your game to play poorly in the simulator. SpriteKit suffers from poor FPS in the iOS simulator. Once our game becomes relatively complex, we will see our FPS drop, even on high-end computers. The simulator will get you through, but it is best if you can plug in a physical device to test. It is time for our first glimpse of SpriteKit in action! Press the gray play arrow in the toolbar (handy keyboard shortcut: command + r). Xcode will build the project and launch the simulator. The simulator starts in a new window, so make sure you bring it to the front. You should see a gray background with white text: Hello, World. Click around on the gray background. You will see colorful, spinning boxes spawning wherever you click: If you have made it this far, congratulations! You have successfully installed and configured everything you need to make your first Swift game. Once you have finished playing with the spinning squares, you can close the simulator down and return to Xcode. Note: You can use the keyboard command command + q to exit the simulator or press the stop button inside Xcode. If you use the stop button, the simulator will remain open and launch your next build faster. Examining the demo code Let's quickly explore the demo code. Do not worry about understanding everything just yet; we will cover each element in depth later. At this point, I am hoping you will acclimatize to the development environment and pick up a few things along the way. If you are stuck, keep going! Make sure you have GameScene.swift open in Xcode. The demo GameScene class implements some functions you will use in your games. Let’s examine these functions. Feel free to read the code inside each function, but I do not expect you to understand the specific code just yet. The game invokes the didMove function whenever it switches to the GameScene. You can think of it a bit like an initialize, or main, function for the scene. The SpriteKit demo uses it to draw the Hello, World text to the screen and set up the spinning square shape that shows up when we tap. There are seven functions involving touch which handle the user's touch input to the iOS device screen. The SpriteKit demo uses these functions to spawn the spinning square wherever we touch the screen. Do not worry about understanding these functions at this time. The update function runs once for every frame drawn to the screen. The SpriteKit demo does not use this function, but we may have reason to implement it later. Cleaning up I hope that you have absorbed some Swift syntax and gained an overview of Swift and SpriteKit. It is time to make room for our own game; let us clear all of that demo code out! We want to keep a little bit of the boilerplate, but we can delete most of what is inside the functions. To be clear, I do not expect you to understand this code yet. This is simply a necessary step towards the start of our journey. Please remove lines from your GameScene.swift file until it looks like the following code: import SpriteKit class GameScene: SKScene { override funcdidMove(to view: SKView) { } } Summary You have already accomplished a lot. You have had your first experience with Swift, installed and configured your development environment, launched code successfully into the iOS simulator. Great work! Resources for Article: Further resources on this subject: Swift for Open Source Developers [Article] Swift Power and Performance [Article] Introducing the Swift Programming Language [Article]

In this article by Mugilan TS Raghupathi author of the book Learning ASP.NET Core MVC Programming explains the setup for getting started with programming in ASP.NET MVC 6. In any development project, it is vital to set up the right kind of development environment so that you can concentrate on the developing the solution rather than solving the environment issues or configuration problems. With respect to .NET, Visual Studio is the de-facto standard IDE (Integrated Development Environment) for building web applications in .NET. In this article, you'll be learning about the following topics: Purpose of IDE Different offerings of Visual Studio Installation of Visual Studio Community 2015 Creating your first ASP.NET MVC 5 project and project structure (For more resources related to this topic, see here.) Purpose of IDE First of all, let us see why we need an IDE, when you can type the code in Notepad, compile, and execute it. When you develop a web application, you might need the following things for you to be productive: Code editor: This is the text editor where you type your code. Your code-editor should be able to recognize different constructs such as the if condition, for loop of your programming language. In Visual Studio, all of your keywords would be highlighted in blue color. Intellisense: Intellisense is a context aware code-completion feature available in most of the modern IDEs including Visual Studio. One such example is, when you type a dot after an object, this Intellisense feature lists out all the methods available on the object. This helps the developers to write code faster and easier. Build/Publish: It would be helpful if you could build or publish the application using a single click or single command. Visual Studio provides several options out of the box to build a separate project or to build the complete solution at a single click. This makes the build and deployment of your application easier. Templates: Depending on the type of the application, you might have to create different folders and files along with the boilerplate code. So, it'll be very helpful if your IDE supports the creation of different kinds of templates. Visual Studio generates different kinds of templates with the code for ASP.Net Web Forms, MVC, and Web API to get you up and running. Ease of addition of items: Your IDE should allow you to add different kinds of items with ease. For example, you should be able to add an XML file without any issues. And if there is any problem with the structure of your XML file, it should be able to highlight the issue along with the information and help you to fix the issues. Visual Studio offerings There are different versions of Visual Studio 2015 available to satisfy the various needs of the developers/organizations. Primarily, there are four versions of Visual Studio 2015: Visual Studio Community Visual Studio Professional Visual Studio Enterprise Visual Studio Test Professional System requirements Visual Studio can be installed on computers installed with Operation System Windows 7 Service Pack1 and above. You can get to know the complete list of requirements from the following URL: https://www.visualstudio.com/en-us/downloads/visual-studio-2015-system-requirements-vs.aspx Visual Studio Community 2015 This is a fully featured IDE available for building desktops, web applications, and cloud services. It is available free of cost for individual users. You can download Visual Studio Community from the following URL: https://www.visualstudio.com/en-us/products/visual-studio-community-vs.aspx Throughout this book, we will be using the Visual Studio Community version for development as it is available free of cost to individual developers. Visual Studio Professional As the name implies, Visual Studio Professional is targeted at professional developers which contains features such as Code Lens for improving your team's productivity. It also has features for greater collaboration within the team. Visual Studio Enterprise Visual Studio Enterprise is a full blown version of Visual Studio with a complete set of features for collaboration, including a team foundation server, modeling, and testing. Visual Studio Test Professional Visual Studio Test Professional is primarily aimed for the testing team or the people who are involved in the testing which might include developers. In any software development methodology—either the waterfall model or agile—developers need to execute the development suite test cases for the code they are developing. Installation of Visual Studio Community Follow the given steps to install Visual Studio Community 2015: Visit the following link to download Visual Studio Community 2015: https://www.visualstudio.com/en-us/products/visual-studio-community-vs.aspx Click on the Download Community 2015 button. Save the file in a folder where you can retrieve it easily later: Run the downloaded executable file: Click on Run and the following screen will appear: There are two types of installation—default and custom installation. Default installation installs the most commonly used features and this will cover most of the use cases of the developer. Custom installation helps you to customize the components that you want to get installed, such as the following: Click on the Install button after selecting the installation type. Depending on your memory and processor speed, it will take 1 to 2 hours to install. Once all the components are installed, you will see the following Setup completed screen: Installation of ASP.NET 5 When we install the Visual Studio Community 2015 edition, ASP.NET 5 will not have been installed by default. As the ASP.NET MVC 6 application runs on top of ASP.NET 5, we need to install ASP.NET 5. There are couple of ways to install ASP.NET 5: Get ASP.NET 5 from https://get.asp.net/ Another option is to install from the New Project template in Visual Studio This option is bit easier as you don't need to search and install. The following are the detailed steps: Create a new project by selecting File | New Project or using the shortcut Ctrl + Shift + N: Select ASP.NET Web Application and enter the project name and click on OK: The following window will appear to select the template. Select the Get ASP.NET 5 RC option as shown in the following screenshot: When you click on OK in the preceding screen, the following window will appear: When you click on the Run or Save button in the preceding dialog, you will get the following screen asking for ASP.NET 5 Setup. Select the checkbox, I agree to the license terms and conditions and click on the Install button: Installation of ASP.NET 5 might take couple of hours and once it is completed you'll get the following screen: During the process of installation of ASP.NET 5 RC1 Update 1, it might ask you to close the Visual Studio. If asked, please do so. Project structure in ASP.Net 5 application Once the ASP.NET 5 RC1 is successfully installed, open the Visual Studio and create a new project and select the ASP.NET 5 Web Application as shown in the following screenshot: A new project will be created and the structure will be like the following: File-based project Whenever you add a file or folder in your file system (inside of our ASP.NET 5 project folder), the changes will be automatically reflected in your project structure. Support for full .NET and .NET core You could see a couple of references in the preceding project: DNX 4.5.1 and DNX Core 5.0. DNX 4.5.1 provides functionalities of full-blown .NET whereas DNX Core 5.0 supports only the core functionalities—which would be used if you are deploying the application across cross-platforms such as Apple OS X, Linux. The development and deployment of an ASP.NET MVC 6 application on a Linux machine will be explained in the book. The Project.json package Usually in an ASP.NET web application, we would be having the assemblies as references and the list of references in a C# project file. But in an ASP.NET 5 application, we have a JSON file by the name of Project.json, which will contain all the necessary configuration with all its .NET dependencies in the form of NuGet packages. This makes dependency management easier. NuGet is a package manager provided by Microsoft, which makes the package installation and uninstallation easier. Prior to NuGet, all the dependencies had to be installed manually. The dependencies section identifies the list of dependent packages available for the application. The frameworks section informs about the frameworks being supported for the application. The scripts section identifies the script to be executed during the build process of the application. Include and exclude properties can be used in any section to include or exclude any item. Controllers This folder contains all of your controller files. Controllers are responsible for handling the requests and communicating the models and generating the views for the same. Models All of your classes representing the domain data will be present in this folder. Views Views are files which contain your frontend components and are presented to the end users of the application. This folder contains all of your Razor View files. Migrations Any database-related migrations will be available in this folder. Database migrations are the C# files which contain the history of any database changes done through an Entity Framework (an ORM framework). This will be explained in detail in the book. The wwwroot folder This folder acts as a root folder and it is the ideal container to place all of your static files such as CSS and JavaScript files. All the files which are placed in wwwroot folder can be directly accessed from the path without going through the controller. Other files The appsettings.json file is the config file where you can configure application level settings. Bower, npm (Node Package Manager), and gulpfile.js are client-side technologies which are supported by ASP.NET 5 applications. Summary In this article, you have learnt about the offerings in Visual Studio. Step-by-step instructions are provided for the installation of the Visual Studio Community version—which is freely available for individual developers. We have also discussed the new project structure of the ASP.Net 5 application and the changes when compared to the previous versions. In this book, we are going to discuss the controllers and their roles and functionalities. We'll also build a controller and associated action methods and see how it works. Resources for Article: Further resources on this subject: Designing your very own ASP.NET MVC Application [article] Debugging Your .NET Application [article] Using ASP.NET Controls in SharePoint [article]

"Invention is not enough. Tesla invented the electric power we use, but he struggled to get it out to people. You have to combine both things: invention and innovation focus, plus the company that can commercialize things and get them to people" – Larry Page In this article written by Fernando J Miguel, author of the book Magento 2 Theme Design Second Edition, you will learn the process of sharing, code hosting, validating, and publishing your subject as well as future components (extensions/modules) that you develop for Magento 2. (For more resources related to this topic, see here.) The following topics will be covered in this article: The packaging process Packaging your theme Hosting your theme The Magento marketplace The packaging process For every theme you develop for distribution in marketplaces and repositories through the sale and delivery of projects to clients and contractors of the service, you must follow some mandatory requirements for the theme to be packaged properly and consequently distributed to different Magento instances. Magento uses the composer.json file to define dependencies and information relevant to the developed component. Remember how the composer.json file is declared in the Bookstore theme: { "name": "packt/bookstore", "description": "BookStore theme", "require": { "php": "~5.5.0|~5.6.0|~7.0.0", "magento/theme-frontend-luma": "~100.0", "magento/framework": "~100.0" }, "type": "magento2-theme", "version": "1.0.0", "license": [ "OSL-3.0", "AFL-3.0" ], "autoload": { "files": [ "registration.php" ], "psr-4": { "Packt\BookStore\": "" } } } The main fields of the declaration components in the composer.json file are as follows: Name: A fully qualified component name Type: This declares the component type Autoload: This specifies the information necessary to be loaded in the component The three main types of Magento 2 component declarations can be described as follows: Module: Use the magento2-module type to declare modules that add to and/or modify functionalities in the Magento 2 system Theme: Use the magento2-theme type to declare themes in Magento 2 storefronts Language package: Use the magento2-language type to declare translations in the Magento 2 system Besides the composer.json file that must be declared in the root directory of your theme, you should follow these steps to meet the minimum requirements for packaging your new theme: Register the theme by declaring the registration.php file. Package the theme, following the standards set by Magento. Validate the theme before distribution. Publish the theme. From the minimum requirements mentioned, you already are familiar with the composer.json and registration.php files. Now we will look at the packaging process, validation, and publication in sequence. Packaging your theme By default, all themes should be compressed in ZIP format and contain only the root directory of the component developed, excluding any file and directory that is not part of the standard structure. The following command shows the compression standard used in Magento 2 components: zip -r vendor-name_package-name-1.0.0.zip package-path/* -x 'package-path/.git/*' Here, the name of the ZIP file has the following components: vendor: This symbolizes the vendor by which the theme was developed name_package: This is the package name name: This is the component name 1.0.0: This is the component version After formatting the component name, it defines which directory will be compressed, followed by the -x parameter, which excludes the git directory from the theme compression. How about applying ZIP compression on the Bookstore theme? To do this, follow these steps: Using a terminal or Command Prompt, access the theme's root directory: <magento_root>/app/design/frontend/Packt/bookstore. Run the zip packt-bookstore-bookstore.1.0.0.zip*-x'.git/*' command. Upon successfully executing this command, you will have packed your theme, and your directory will be as follows: After this, you will validate your new Magento theme using a verification tool. Magento component validation The Magento developer community created the validate_m2_package script to perform validation of components developed for Magento 2. This script is available on the GitHub repository of the Magento 2 development community in the marketplace-tools directory: According to the description, the idea behind Marketplace Tools is to house standalone tools that developers can use to validate and verify their extensions before submitting them to the Marketplace. Here's how to use the validation tool: Download the validate_m2_package.php script, available at https://github.com/magento/marketplace-tools. Move the script to the root directory of the Bookstore theme <magento_root>/app/design/frontend/Packt/bookstore. Open a terminal or Command Prompt. Run the validate_m2_package.php packt-bookstore-bookstore.1.0.0.zip PHP command. This command will validate the package you previously created with the ZIP command. If all goes well, you will not have any response from the command line, which will mean that your package is in line with the minimum requirements for publication. If you wish, you can use the -d parameter that enables you to debug your component by printing messages during verification. To use this option, run the following command: php validate_m2_package.php -d packt-bookstore-bookstore.1.0.0.zip If everything goes as expected, the response will be as follows: Hosting your theme You can share your Magento theme and host your code on different services to achieve greater interaction with your team or even with the Magento development community. Remembering that the standard control system software version used by the Magento development community is Git. There are some options well used in the market, so you can distribute your code and share your work. Let's look at some of these options. Hosting your project on GitHub and Packagist The most common method of hosting your code/theme is to use GitHub. Once you have created a repository, you can get help from the Magento developer community if you are working on an open source project or even one for learning purposes. The major point of using GitHub is the question of your portfolio and the publication of your Magento 2 projects developed, which certainly will make a difference when you are looking for employment opportunities and trying to get selected for new projects. GitHub has a specific help area for users that provides a collection of documentation that developers may find useful. GitHub Help can be accessed directly at https://help.github.com/: To create a GitHub repository, you can consult the official documentation, available at https://help.github.com/articles/create-a-repo/. Once you have your project published on GitHub, you can use the Packagist (https://packagist.org/) service by creating a new account and entering the link of your GitHub package on Packagist: Packagist collects information automatically from the available composer.json file in the GitHub repository, creating your reference to use in other projects. Hosting your project in a private repository In some cases, you will be developing your project for private clients and companies. In case you want to keep your version control in private mode, you can use the following procedure: Create your own package composer repository using the Toran service (https://toranproxy.com/). Create your package as previously described. Send your package to your private repository. Add the following to your composer.json file: { "repositories": [ { "type": "composer", "url": [repository url here] } ] } Magento Marketplace According to Magento, Marketplace (https://marketplace.magento.com/) is the largest global e-commerce resource for applications and services that extend Magento solutions with powerful new features and functionality. Once you have completed developing the first version of your theme, you can upload your project to be a part of the official marketplace of Magento. In addition to allowing theme uploads, Magento Marketplace also allows you to upload shared packages and extensions (modules). To learn more about shared packages, visit http://docs.magento.com/marketplace/user_guide/extensions/shared-package-submit.html. Submitting your theme After the compression and validation processes, you can send your project to be distributed to Magento Marketplace. For this, you should confirm an account on the developer portal (https://developer.magento.com/customer/account/) with a valid e-mail and personal information about the scope of your activities. After this confirmation, you will have access to the extensions area at https://developer.magento.com/extension/extension/list/, where you will find options to submit themes and extensions: After clicking on the Add Theme button, you will need to answer a questionnaire: Which Magento platform your theme will work on The name of your theme Whether your theme will have additional services Additional functionalities your theme has What makes your theme unique After the questionnaire, you will need to fill in the details of your extension, as follows: Extension title Public version Package file (upload) The submitted theme will be evaluated by a technical review, and you will be able to see the evaluation progress through your e-mail and the control panel of the Magento developer area. You can find more information about Magento Marketplace at the following link: http://docs.magento.com/marketplace/user_guide/getting-started.html Summary In this article, you learned about the theme-packaging process besides validation according to the minimum requirements for its publication on Magento Marketplace. You are now ready to develop your solutions! There is still a lot of work left, but I encourage you to seek your way as a Magento theme developer by putting a lot of study, research, and application into the area. Participate in events, be collaborative, and count on the community's support. Good luck and success in your career path! Resources for Article: Further resources on this subject: Installing Magento [article] Social Media and Magento [article] Magento 2 – the New E-commerce Era [article]

In this article by Luca Massaron and Alberto Boschetti the authors of the book Python Data Science Essentials - Second Edition we will cover steps on installing Python, the different installation packages and have a glance at the essential packages will constitute a complete Data Science Toolbox. (For more resources related to this topic, see here.) Whether you are an eager learner of data science or a well-grounded data science practitioner, you can take advantage of this essential introduction to Python for data science. You can use it to the fullest if you already have at least some previous experience in basic coding, in writing general-purpose computer programs in Python, or in some other data-analysis-specific language such as MATLAB or R. Introducing data science and Python Data science is a relatively new knowledge domain, though its core components have been studied and researched for many years by the computer science community. Its components include linear algebra, statistical modelling, visualization, computational linguistics, graph analysis, machine learning, business intelligence, and data storage and retrieval. Data science is a new domain and you have to take into consideration that currently its frontiers are still somewhat blurred and dynamic. Since data science is made of various constituent sets of disciplines, please also keep in mind that there are different profiles of data scientists depending on their competencies and areas of expertise. In such a situation, what can be the best tool of the trade that you can learn and effectively use in your career as a data scientist? We believe that the best tool is Python, and we intend to provide you with all the essential information that you will need for a quick start. In addition, other tools such as R and MATLAB provide data scientists with specialized tools to solve specific problems in statistical analysis and matrix manipulation in data science. However, only Python really completes your data scientist skill set. This multipurpose language is suitable for both development and production alike; it can handle small- to large-scale data problems and it is easy to learn and grasp no matter what your background or experience is. Created in 1991 as a general-purpose, interpreted, and object-oriented language, Python has slowly and steadily conquered the scientific community and grown into a mature ecosystem of specialized packages for data processing and analysis. It allows you to have uncountable and fast experimentations, easy theory development, and prompt deployment of scientific applications. At present, the core Python characteristics that render it an indispensable data science tool are as follows: It offers a large, mature system of packages for data analysis and machine learning. It guarantees that you will get all that you may need in the course of a data analysis, and sometimes even more. Python can easily integrate different tools and offers a truly unifying ground for different languages, data strategies, and learning algorithms that can be fitted together easily and which can concretely help data scientists forge powerful solutions. There are packages that allow you to call code in other languages (in Java, C, FORTRAN, R, or Julia), outsourcing some of the computations to them and improving your script performance. It is very versatile. No matter what your programming background or style is (object-oriented, procedural, or even functional), you will enjoy programming with Python. It is cross-platform; your solutions will work perfectly and smoothly on Windows, Linux, and Mac OS systems. You won't have to worry all that much about portability. Although interpreted, it is undoubtedly fast compared to other mainstream data analysis languages such as R and MATLAB (though it is not comparable to C, Java, and the newly emerged Julia language). Moreover, there are also static compilers such as Cython or just-in-time compilers such as PyPy that can transform Python code into C for higher performance. It can work with large in-memory data because of its minimal memory footprint and excellent memory management. The memory garbage collector will often save the day when you load, transform, dice, slice, save, or discard data using various iterations and reiterations of data wrangling. It is very simple to learn and use. After you grasp the basics, there's no better way to learn more than by immediately starting with the coding. Moreover, the number of data scientists using Python is continuously growing: new packages and improvements have been released by the community every day, making the Python ecosystem an increasingly prolific and rich language for data science. Installing Python First, let's proceed to introduce all the settings you need in order to create a fully working data science environment to test the examples and experiment with the code that we are going to provide you with. Python is an open source, object-oriented, and cross-platform programming language. Compared to some of its direct competitors (for instance, C++ or Java), Python is very concise.  It allows you to build a working software prototype in a very short time. Yet it has become the most used language in the data scientist's toolbox not just because of that. It is also a general-purpose language, and it is very flexible due to a variety of available packages that solve a wide spectrum of problems and necessities. Python 2 or Python 3? There are two main branches of Python: 2.7.x and 3.x. At the time of writing this article, the Python foundation (www.python.org) is offering downloads for Python version 2.7.11 and 3.5.1. Although the third version is the newest, the older one is still the most used version in the scientific area, since a few packages (check on the website py3readiness.org for a compatibility overview) won't run otherwise yet. In addition, there is no immediate backward compatibility between Python 3 and 2. In fact, if you try to run some code developed for Python 2 with a Python 3 interpreter, it may not work. Major changes have been made to the newest version, and that has affected past compatibility. Some data scientists, having built most of their work on Python 2 and its packages, are reluctant to switch to the new version. We intend to address a larger audience of data scientists, data analysts and developers, who may not have such a strong legacy with Python 2. Thus, we agreed that it would be better to work with Python 3 rather than the older version. We suggest using a version such as Python 3.4 or above. After all, Python 3 is the present and the future of Python. It is the only version that will be further developed and improved by the Python foundation and it will be the default version of the future on many operating systems. Anyway, if you are currently working with version 2 and you prefer to keep on working with it, you can still the examples. In fact, for the most part, our code will simply work on Python 2 after having the code itself preceded by these imports: from __future__ import (absolute_import, division, print_function, unicode_literals) from builtins import * from future import standard_library standard_library.install_aliases() The from __future__ import commands should always occur at the beginning of your scripts or else you may experience Python reporting an error. As described in the Python-future website (python-future.org), these imports will help convert several Python 3-only constructs to a form compatible with both Python 3 and Python 2 (and in any case, most Python 3 code should just simply work on Python 2 even without the aforementioned imports). In order to run the upward commands successfully, if the future package is not already available on your system, you should install it (version >= 0.15.2) using the following command to be executed from a shell: $> pip install –U future If you're interested in understanding the differences between Python 2 and Python 3 further, we recommend reading the wiki page offered by the Python foundation itself: wiki.python.org/moin/Python2orPython3. Step-by-step installation Novice data scientists who have never used Python (who likely don't have the language readily installed on their machines) need to first download the installer from the main website of the project, www.python.org/downloads/, and then install it on their local machine. We will now coversteps which will provide you with full control over what can be installed on your machine. This is very useful when you have to set up single machines to deal with different tasks in data science. Anyway, please be warned that a step-by-step installation really takes time and effort. Instead, installing a ready-made scientific distribution will lessen the burden of installation procedures and it may be well suited for first starting and learning because it saves you time and sometimes even trouble, though it will put a large number of packages (and we won't use most of them) on your computer all at once. This being a multiplatform programming language, you'll find installers for machines that either run on Windows or Unix-like operating systems. Please remember that some of the latest versions of most Linux distributions (such as CentOS, Fedora, Red Hat Enterprise, and Ubuntu) have Python 2 packaged in the repository. In such a case and in the case that you already have a Python version on your computer (since our examples run on Python 3), you first have to check what version you are exactly running. To do such a check, just follow these instructions: Open a python shell, type python in the terminal, or click on any Python icon you find on your system. Then, after having Python started, to test the installation, run the following code in the Python interactive shell or REPL: >>> import sys >>> print (sys.version_info) If you can read that your Python version has the major=2 attribute, it means that you are running a Python 2 instance. Otherwise, if the attribute is valued 3, or if the print statements reports back to you something like v3.x.x (for instance v3.5.1), you are running the right version of Python and you are ready to move forward. To clarify the operations we have just mentioned, when a command is given in the terminal command line, we prefix the command with $>. Otherwise, if it's for the Python REPL, it's preceded by >>>. The installation of packages Python won't come bundled with all you need, unless you take a specific premade distribution. Therefore, to install the packages you need, you can use either pip or easy_install. Both these two tools run in the command line and make the process of installation, upgrade, and removal of Python packages a breeze. To check which tools have been installed on your local machine, run the following command: $> pip To install pip, follow the instructions given at pip.pypa.io/en/latest/installing.html. Alternatively, you can also run this command: $> easy_install If both of these commands end up with an error, you need to install any one of them. We recommend that you use pip because it is thought of as an improvement over easy_install. Moreover, easy_install is going to be dropped in future and pip has important advantages over it. It is preferable to install everything using pip because: It is the preferred package manager for Python 3. Starting with Python 2.7.9 and Python 3.4, it is included by default with the Python binary installers. It provides an uninstall functionality. It rolls back and leaves your system clear if, for whatever reason, the package installation fails. Using easy_install in spite of pip's advantages makes sense if you are working on Windows because pip won't always install pre-compiled binary packages.Sometimes it will try to build the package's extensions directly from C source, thus requiring a properly configured compiler (and that's not an easy task on Windows). This depends on whether the package is running on eggs (and pip cannot directly use their binaries, but it needs to build from their source code) or wheels (in this case, pip can install binaries if available, as explained here: pythonwheels.com/). Instead, easy_install will always install available binaries from eggs and wheels. Therefore, if you are experiencing unexpected difficulties installing a package, easy_install can save your day (at some price anyway, as we just mentioned in the list). The most recent versions of Python should already have pip installed by default. Therefore, you may have it already installed on your system. If not, the safest way is to download the get-pi.py script from bootstrap.pypa.io/get-pip.py and then run it using the following: $> python get-pip.py The script will also install the setup tool from pypi.python.org/pypi/setuptools, which also contains easy_install. You're now ready to install the packages you need in order to run the examples provided in this article. To install the < package-name > generic package, you just need to run this command: $> pip install < package-name > Alternatively, you can run the following command: $> easy_install < package-name > Note that in some systems, pip might be named as pip3 and easy_install as easy_install-3 to stress the fact that both operate on packages for Python 3. If you're unsure, check the version of Python pip is operating on with: $> pip –V For easy_install, the command is slightly different: $> easy_install --version After this, the <pk> package and all its dependencies will be downloaded and installed. If you're not certain whether a library has been installed or not, just try to import a module inside it. If the Python interpreter raises an ImportError error, it can be concluded that the package has not been installed. This is what happens when the NumPy library has been installed: >>> import numpy This is what happens if it's not installed: >>> import numpy Traceback (most recent call last): File "<stdin>", line 1, in <module> ImportError: No module named numpy In the latter case, you'll need to first install it through pip or easy_install. Take care that you don't confuse packages with modules. With pip, you install a package; in Python, you import a module. Sometimes, the package and the module have the same name, but in many cases, they don't match. For example, the sklearn module is included in the package named Scikit-learn. Finally, to search and browse the Python packages available for Python, look at pypi.python.org. Package upgrades More often than not, you will find yourself in a situation where you have to upgrade a package because either the new version is required by a dependency or it has additional features that you would like to use. First, check the version of the library you have installed by glancing at the __version__ attribute, as shown in the following example, numpy: >>> import numpy >>> numpy.__version__ # 2 underscores before and after '1.9.2' Now, if you want to update it to a newer release, say the 1.11.0 version, you can run the following command from the command line: $> pip install -U numpy==1.11.0 Alternatively, you can use the following command: $> easy_install --upgrade numpy==1.11.0 Finally, if you're interested in upgrading it to the latest available version, simply run this command: $> pip install -U numpy You can alternatively run the following command: $> easy_install --upgrade numpy Scientific distributions As you've read so far, creating a working environment is a time-consuming operation for a data scientist. You first need to install Python and then, one by one, you can install all the libraries that you will need (sometimes, the installation procedures may not go as smoothly as you'd hoped for earlier). If you want to save time and effort and want to ensure that you have a fully working Python environment that is ready to use, you can just download, install, and use the scientific Python distribution. Apart from Python, they also include a variety of preinstalled packages, and sometimes, they even have additional tools and an IDE. A few of them are very well known among data scientists, and in the following content, you will find some of the key features of each of these packages. We suggest that you promptly download and install a scientific distribution, such as Anaconda (which is the most complete one). Anaconda (continuum.io/downloads) is a Python distribution offered by Continuum Analytics that includes nearly 200 packages, which comprises NumPy, SciPy, pandas, Jupyter, Matplotlib, Scikit-learn, and NLTK. It's a cross-platform distribution (Windows, Linux, and Mac OS X) that can be installed on machines with other existing Python distributions and versions. Its base version is free; instead, add-ons that contain advanced features are charged separately. Anaconda introduces conda, a binary package manager, as a command-line tool to manage your package installations. As stated on the website, Anaconda's goal is to provide enterprise-ready Python distribution for large-scale processing, predictive analytics, and scientific computing. Leveraging conda to install packages If you've decided to install an Anaconda distribution, you can take advantage of the conda binary installer we mentioned previously. Anyway, conda is an open source package management system, and consequently it can be installed separately from an Anaconda distribution. You can test immediately whether conda is available on your system. Open a shell and digit: $> conda -V If conda is available, there will appear the version of your conda; otherwise an error will be reported. If conda is not available, you can quickly install it on your system by going to conda.pydata.org/miniconda.html and installing the Miniconda software suitable for your computer. Miniconda is a minimal installation that only includes conda and its dependencies. conda can help you manage two tasks: installing packages and creating virtual environments. In this paragraph, we will explore how conda can help you easily install most of the packages you may need in your data science projects. Before starting, please check to have the latest version of conda at hand: $> conda update conda Now you can install any package you need. To install the <package-name> generic package, you just need to run the following command: $> conda install <package-name> You can also install a particular version of the package just by pointing it out: $> conda install <package-name>=1.11.0 Similarly you can install multiple packages at once by listing all their names: $> conda install <package-name-1> <package-name-2> If you just need to update a package that you previously installed, you can keep on using conda: $> conda update <package-name> You can update all the available packages simply by using the --all argument: $> conda update --all Finally, conda can also uninstall packages for you: $> conda remove <package-name> If you would like to know more about conda, you can read its documentation at conda.pydata.org/docs/index.html. In summary, as a main advantage, it handles binaries even better than easy_install (by always providing a successful installation on Windows without any need to compile the packages from source) but without its problems and limitations. With the use of conda, packages are easy to install (and installation is always successful), update, and even uninstall. On the other hand, conda cannot install directly from a git server (so it cannot access the latest version of many packages under development) and it doesn't cover all the packages available on PyPI as pip itself. Enthought Canopy Enthought Canopy (enthought.com/products/canopy) is a Python distribution by Enthought Inc. It includes more than 200 preinstalled packages, such as NumPy, SciPy, Matplotlib, Jupyter, and pandas. This distribution is targeted at engineers, data scientists, quantitative and data analysts, and enterprises. Its base version is free (which is named Canopy Express), but if you need advanced features, you have to buy a front version. It's a multiplatform distribution and its command-line install tool is canopy_cli. PythonXY PythonXY (python-xy.github.io) is a free, open source Python distribution maintained by the community. It includes a number of packages, which include NumPy, SciPy, NetworkX, Jupyter, and Scikit-learn. It also includes Spyder, an interactive development environment inspired by the MATLAB IDE. The distribution is free. It works only on Microsoft Windows, and its command-line installation tool is pip. WinPython WinPython (winpython.sourceforge.net) is also a free, open-source Python distribution maintained by the community. It is designed for scientists, and includes many packages such as NumPy, SciPy, Matplotlib, and Jupyter. It also includes Spyder as an IDE. It is free and portable. You can put WinPython into any directory, or even into a USB flash drive, and at the same time maintain multiple copies and versions of it on your system. It works only on Microsoft Windows, and its command-line tool is the WinPython Package Manager (WPPM). Explaining virtual environments No matter you have chosen installing a stand-alone Python or instead you used a scientific distribution, you may have noticed that you are actually bound on your system to the Python's version you have installed. The only exception, for Windows users, is to use a WinPython distribution, since it is a portable installation and you can have as many different installations as you need. A simple solution to break free of such a limitation is to use virtualenv that is a tool to create isolated Python environments. That means, by using different Python environments, you can easily achieve these things: Testing any new package installation or doing experimentation on your Python environment without any fear of breaking anything in an irreparable way. In this case, you need a version of Python that acts as a sandbox. Having at hand multiple Python versions (both Python 2 and Python 3), geared with different versions of installed packages. This can help you in dealing with different versions of Python for different purposes (for instance, some of the packages we are going to present on Windows OS only work using Python 3.4, which is not the latest release). Taking a replicable snapshot of your Python environment easily and having your data science prototypes work smoothly on any other computer or in production. In this case, your main concern is the immutability and replicability of your working environment. You can find documentation about virtualenv at virtualenv.readthedocs.io/en/stable, though we are going to provide you with all the directions you need to start using it immediately. In order to take advantage of virtualenv, you have first to install it on your system: $> pip install virtualenv After the installation completes, you can start building your virtual environments. Before proceeding, you have to take a few decisions: If you have more versions of Python installed on your system, you have to decide which version to pick up. Otherwise, virtualenv will take the Python version virtualenv was installed by on your system. In order to set a different Python version you have to digit the argument –p followed by the version of Python you want or inserting the path of the Python executable to be used (for instance, –p python2.7 or just pointing to a Python executable such as -p c:Anaconda2python.exe). With virtualenv, when required to install a certain package, it will install it from scratch, even if it is already available at a system level (on the python directory you created the virtual environment from). This default behavior makes sense because it allows you to create a completely separated empty environment. In order to save disk space and limit the time of installation of all the packages, you may instead decide to take advantage of already available packages on your system by using the argument --system-site-packages. You may want to be able to later move around your virtual environment across Python installations, even among different machines. Therefore you may want to make the functioning of all of the environment's scripts relative to the path it is placed in by using the argument --relocatable. After deciding on the Python version, the linking to existing global packages, and the relocability of the virtual environment, in order to start, you just launch the command from a shell. Declare the name you would like to assign to your new environment: $> virtualenv clone virtualenv will just create a new directory using the name you provided, in the path from which you actually launched the command. To start using it, you just enter the directory and digit activate: $> cd clone $> activate At this point, you can start working on your separated Python environment, installing packages and working with code. If you need to install multiple packages at once, you may need some special function from pip—pip freeze—which will enlist all the packages (and their version) you have installed on your system. You can record the entire list in a text file by this command: $> pip freeze > requirements.txt After saving the list in a text file, just take it into your virtual environment and install all the packages in a breeze with a single command: $> pip install -r requirements.txt Each package will be installed according to the order in the list (packages are listed in a case-insensitive sorted order). If a package requires other packages that are later in the list, that's not a big deal because pip automatically manages such situations. So if your package requires Numpy and Numpy is not yet installed, pip will install it first. When you're finished installing packages and using your environment for scripting and experimenting, in order to return to your system defaults, just issue this command: $> deactivate If you want to remove the virtual environment completely, after deactivating and getting out of the environment's directory, you just have to get rid of the environment's directory itself by a recursive deletion. For instance, on Windows you just do this: $> rd /s /q clone On Linux and Mac, the command will be: $> rm –r –f clone If you are working extensively with virtual environments, you should consider using virtualenvwrapper, which is a set of wrappers for virtualenv in order to help you manage multiple virtual environments easily. It can be found at bitbucket.org/dhellmann/virtualenvwrapper. If you are operating on a Unix system (Linux or OS X), another solution we have to quote is pyenv (which can be found at https://github.com/yyuu/pyenv). It lets you set your main Python version, allow installation of multiple versions, and create virtual environments. Its peculiarity is that it does not depend on Python to be installed and works perfectly at the user level (no need for sudo commands). conda for managing environments If you have installed the Anaconda distribution, or you have tried conda using a Miniconda installation, you can also take advantage of the conda command to run virtual environments as an alternative to virtualenv. Let's see in practice how to use conda for that. We can check what environments we have available like this: >$ conda info -e This command will report to you what environments you can use on your system based on conda. Most likely, your only environment will be just "root", pointing to your Anaconda distribution's folder. As an example, we can create an environment based on Python version 3.4, having all the necessary Anaconda-packaged libraries installed. That makes sense, for instance, for using the package Theano together with Python 3 on Windows (because of an issue we will explain in a few paragraphs). In order to create such an environment, just do: $> conda create -n python34 python=3.4 anaconda The command asks for a particular python version (3.4) and requires the installation of all packages available on the anaconda distribution (the argument anaconda). It names the environment as python34 using the argument –n. The complete installation should take a while, given the large number of packages in the Anaconda installation. After having completed all of the installation, you can activate the environment: $> activate python34 If you need to install additional packages to your environment, when activated, you just do: $> conda install -n python34 <package-name1> <package-name2> That is, you make the list of the required packages follow the name of your environment. Naturally, you can also use pip install, as you would do in a virtualenv environment. You can also use a file instead of listing all the packages by name yourself. You can create a list in an environment using the list argument and piping the output to a file: $> conda list -e > requirements.txt Then, in your target environment, you can install the entire list using: $> conda install --file requirements.txt You can even create an environment, based on a requirements' list: $> conda create -n python34 python=3.4 --file requirements.txt Finally, after having used the environment, to close the session, you simply do this: $> deactivate Contrary to virtualenv, there is a specialized argument in order to completely remove an environment from your system: $> conda remove -n python34 --all A glance at the essential packages We mentioned that the two most relevant characteristics of Python are its ability to integrate with other languages and its mature package system, which is well embodied by PyPI (the Python Package Index: pypi.python.org/pypi), a common repository for the majority of Python open source packages that is constantly maintained and updated. The packages that we are now going to introduce are strongly analytical and they will constitute a complete Data Science Toolbox. All the packages are made up of extensively tested and highly optimized functions for both memory usage and performance, ready to achieve any scripting operation with successful execution. A walkthrough on how to install them is provided next. Partially inspired by similar tools present in R and MATLAB environments, we will together explore how a few selected Python commands can allow you to efficiently handle data and then explore, transform, experiment, and learn from the same without having to write too much code or reinvent the wheel. NumPy NumPy, which is Travis Oliphant's creation, is the true analytical workhorse of the Python language. It provides the user with multidimensional arrays, along with a large set of functions to operate a multiplicity of mathematical operations on these arrays. Arrays are blocks of data arranged along multiple dimensions, which implement mathematical vectors and matrices. Characterized by optimal memory allocation, arrays are useful not just for storing data, but also for fast matrix operations (vectorization), which are indispensable when you wish to solve ad hoc data science problems: Website: www.numpy.org Version at the time of print: 1.11.0 Suggested install command: pip install numpy As a convention largely adopted by the Python community, when importing NumPy, it is suggested that you alias it as np: import numpy as np SciPy An original project by Travis Oliphant, Pearu Peterson, and Eric Jones, SciPy completes NumPy's functionalities, offering a larger variety of scientific algorithms for linear algebra, sparse matrices, signal and image processing, optimization, fast Fourier transformation, and much more: Website: www.scipy.org Version at time of print: 0.17.1 Suggested install command: pip install scipy pandas The pandas package deals with everything that NumPy and SciPy cannot do. Thanks to its specific data structures, namely DataFrames and Series, pandas allows you to handle complex tables of data of different types (which is something that NumPy's arrays cannot do) and time series. Thanks to Wes McKinney's creation, you will be able easily and smoothly to load data from a variety of sources. You can then slice, dice, handle missing elements, add, rename, aggregate, reshape, and finally visualize your data at will: Website: pandas.pydata.org Version at the time of print: 0.18.1 Suggested install command: pip install pandas Conventionally, pandas is imported as pd: import pandas as pd Scikit-learn Started as part of the SciKits (SciPy Toolkits), Scikit-learn is the core of data science operations on Python. It offers all that you may need in terms of data preprocessing, supervised and unsupervised learning, model selection, validation, and error metrics. Scikit-learn started in 2007 as a Google Summer of Code project by David Cournapeau. Since 2013, it has been taken over by the researchers at INRA (French Institute for Research in Computer Science and Automation): Website: scikit-learn.org/stable Version at the time of print: 0.17.1 Suggested install command: pip install scikit-learn Note that the imported module is named sklearn. Jupyter A scientific approach requires the fast experimentation of different hypotheses in a reproducible fashion. Initially named IPython and limited to working only with the Python language, Jupyter was created by Fernando Perez in order to address the need for an interactive Python command shell (which is based on shell, web browser, and the application interface), with graphical integration, customizable commands, rich history (in the JSON format), and computational parallelism for an enhanced performance. Jupyter is our favoured choice; it is used to clearly and effectively illustrate operations with scripts and data, and the consequent results: Website: jupyter.org Version at the time of print: 1.0.0 (ipykernel = 4.3.1) Suggested install command: pip install jupyter Matplotlib Originally developed by John Hunter, matplotlib is a library that contains all the building blocks that are required to create quality plots from arrays and to visualize them interactively. You can find all the MATLAB-like plotting frameworks inside the pylab module: Website: matplotlib.org Version at the time of print: 1.5.1 Suggested install command: pip install matplotlib You can simply import what you need for your visualization purposes with the following command: import matplotlib.pyplot as plt Statsmodels Previously part of SciKits, statsmodels was thought to be a complement to SciPy's statistical functions. It features generalized linear models, discrete choice models, time series analysis, and a series of descriptive statistics as well as parametric and nonparametric tests: Website: statsmodels.sourceforge.net Version at the time of print: 0.6.1 Suggested install command: pip install statsmodels Beautiful Soup Beautiful Soup, a creation of Leonard Richardson, is a great tool to scrap out data from HTML and XML files retrieved from the Internet. It works incredibly well, even in the case of tag soups (hence the name), which are collections of malformed, contradictory, and incorrect tags. After choosing your parser (the HTML parser included in Python's standard library works fine), thanks to Beautiful Soup, you can navigate through the objects in the page and extract text, tables, and any other information that you may find useful: Website: www.crummy.com/software/BeautifulSoup Version at the time of print: 4.4.1 Suggested install command: pip install beautifulsoup4 Note that the imported module is named bs4. NetworkX Developed by the Los Alamos National Laboratory, NetworkX is a package specialized in the creation, manipulation, analysis, and graphical representation of real-life network data (it can easily operate with graphs made up of a million nodes and edges). Besides specialized data structures for graphs and fine visualization methods (2D and 3D), it provides the user with many standard graph measures and algorithms, such as the shortest path, centrality, components, communities, clustering, and PageRank. Website: networkx.github.io Version at the time of print: 1.11 Suggested install command: pip install networkx Conventionally, NetworkX is imported as nx: import networkx as nx NLTK The Natural Language Toolkit (NLTK) provides access to corpora and lexical resources and to a complete suite of functions for statistical Natural Language Processing (NLP), ranging from tokenizers to part-of-speech taggers and from tree models to named-entity recognition. Initially, Steven Bird and Edward Loper created the package as an NLP teaching infrastructure for their course at the University of Pennsylvania. Now, it is a fantastic tool that you can use to prototype and build NLP systems: Website: www.nltk.org Version at the time of print: 3.2.1 Suggested install command: pip install nltk Gensim Gensim, programmed by Radim Rehurek, is an open source package that is suitable for the analysis of large textual collections with the help of parallel distributable online algorithms. Among advanced functionalities, it implements Latent Semantic Analysis (LSA), topic modelling by Latent Dirichlet Allocation (LDA), and Google's word2vec, a powerful algorithm that transforms text into vector features that can be used in supervised and unsupervised machine learning. Website: radimrehurek.com/gensim Version at the time of print: 0.12.4 Suggested install command: pip install gensim PyPy PyPy is not a package; it is an alternative implementation of Python 2.7.8 that supports most of the commonly used Python standard packages (unfortunately, NumPy is currently not fully supported). As an advantage, it offers enhanced speed and memory handling. Thus, it is very useful for heavy duty operations on large chunks of data and it should be part of your big data handling strategies: Website: pypy.org/ Version at time of print: 5.1 Download page: pypy.org/download.html XGBoost XGBoost is a scalable, portable, and distributed gradient boosting library (a tree ensemble machine learning algorithm). Initially created by Tianqi Chen from Washington University, it has been enriched by a Python wrapper by Bing Xu and an R interface by Tong He (you can read the story behind XGBoost directly from its principal creator at homes.cs.washington.edu/~tqchen/2016/03/10/story-and-lessons-behind-the-evolution-of-xgboost.html). XGBoost is available for Python, R, Java, Scala, Julia, and C++, and it can work on a single machine (leveraging multithreading) in both Hadoop and Spark clusters: Website: xgboost.readthedocs.io/en/latest Version at the time of print: 0.4 Download page: github.com/dmlc/xgboost Detailed instructions for installing XGBoost on your system can be found at this page: github.com/dmlc/xgboost/blob/master/doc/build.md The installation of XGBoost on both Linux and MacOS is quite straightforward, whereas it is a little bit trickier for Windows users. On a Posix system you just have For this reason, we provide specific installation steps to get XGBoost working on Windows: First download and install Git for Windows (git-for-windows.github.io). Then you need a MINGW compiler present on your system. You can download it from www.mingw.org accordingly to the characteristics of your system. From the command line, execute: $> git clone --recursive https://github.com/dmlc/xgboost $> cd xgboost $> git submodule init $> git submodule update Then, always from command line, copy the configuration for 64-byte systems to be the default one: $> copy makemingw64.mk config.mk Alternatively, you just copy the plain 32-byte version: $> copy makemingw.mk config.mk After copying the configuration file, you can run the compiler, setting it to use four threads in order to speed up the compiling procedure: $> mingw32-make -j4 In MinGW, the make command comes with the name mingw32-make. If you are using a different compiler, the previous command may not work; then you can simply try: $> make -j4 Finally, if the compiler completes its work without errors, you can install the package in your Python by this: $> cd python-package $> python setup.py install After following all the preceding instructions, if you try to import XGBoost in Python and yet it doesn't load and results in an error, it may well be that Python cannot find the MinGW's g++ runtime libraries. You just need to find the location on your computer of MinGW's binaries (in our case, it was in C:mingw-w64mingw64bin; just modify the next code to put yours) and place the following code snippet before importing XGBoost: import os mingw_path = 'C:\mingw-w64\mingw64\bin' os.environ['PATH']=mingw_path + ';' + os.environ['PATH'] import xgboost as xgb Depending on the state of the XGBoost project, similarly to many other projects under continuous development, the preceding installation commands may or may not temporarily work at the time you will try them. Usually waiting for an update of the project or opening an issue with the authors of the package may solve the problem. Theano Theano is a Python library that allows you to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays efficiently. Basically, it provides you with all the building blocks you need to create deep neural networks. Created by academics (an entire development team; you can read their names on their most recent paper at arxiv.org/pdf/1605.02688.pdf), Theano has been used for large scale and intensive computations since 2007: Website: deeplearning.net/software/theano Release at the time of print: 0.8.2 In spite of many installation problems experienced by users in the past (expecially Windows users), the installation of Theano should be straightforward, the package being now available on PyPI: $> pip install Theano If you want the most updated version of the package, you can get it by Github cloning: $> git clone git://github.com/Theano/Theano.git Then you can proceed with direct Python installation: $> cd Theano $> python setup.py install To test your installation, you can run from shell/CMD and verify the reports: $> pip install nose $> pip install nose-parameterized $> nosetests theano If you are working on a Windows OS and the previous instructions don't work, you can try these steps using the conda command provided by the Anaconda distribution: Install TDM GCC x64 (this can be found at tdm-gcc.tdragon.net) Open an Anaconda prompt interface and execute: $> conda update conda $> conda update --all $> conda install mingw libpython $> pip install git+git://github.com/Theano/Theano.git Theano needs libpython, which isn't compatible yet with the version 3.5. So if your Windows installation is not working, this could be the likely cause. Anyway, Theano installs perfectly on Python version 3.4. Our suggestion in this case is to create a virtual Python environment based on version 3.4, install, and use Theano only on that specific version. Directions on how to create virtual environments are provided in the paragraph about virtualenv and conda create. In addition, Theano's website provides some information to Windows users; it could support you when everything else fails: deeplearning.net/software/theano/install_windows.html An important requirement for Theano to scale out on GPUs is to install Nvidia CUDA drivers and SDK for code generation and execution on GPU. If you do not know too much about the CUDA Toolkit, you can actually start from this web page in order to understand more about the technology being used: developer.nvidia.com/cuda-toolkit Therefore, if your computer has an NVidia GPU, you can find all the necessary instructions in order to install CUDA using this tutorial page from NVidia itself: docs.nvidia.com/cuda/cuda-quick-start-guide/index.html Keras Keras is a minimalist and highly modular neural networks library, written in Python and capable of running on top of either Theano or TensorFlow (the source software library for numerical computation released by Google). Keras was created by François Chollet, a machine learning researcher working at Google: Website: keras.io Version at the time of print: 1.0.3 Suggested installation from PyPI: $> pip install keras As an alternative, you can install the latest available version (which is advisable since the package is in continuous development) using the command: $> pip install git+git://github.com/fchollet/keras.git Summary In this article, we performed a lot of installations, from Python packages to examples.They were installed either directly or by using a scientific distribution. We also introduced Jupyter notebooks and demonstrated how you can have access to the data run in the tutorials. Resources for Article: Further resources on this subject: Python for Driving Hardware [Article] Mining Twitter with Python – Influence and Engagement [Article] Python Data Structures [Article]

In this article by Piyush Mishra, author of the book Akka Cookbook, we will learn about supervision and monitoring of Akka actors. (For more resources related to this topic, see here.) Using supervision and monitoring, we can write fault-tolerant systems, which can run continuously for days, months, and years without stopping. Fault tolerance is a property of the systems which are intended to be always responsive rather than failing completely in case of a failure. Such systems are known as fault tolerance systems or resilient systems. In simple words, a fault-tolerant system is one which is destined to continue as more or less fully operational, with perhaps a reduction in throughput or an increase in response time because of partial failure of its components. Even if a components fails, the whole system never gets shut down, instead, it remains operational and responsive with just a decreased throughput. Similarly, while designing a distributed system, we need to care about what would happen if one or more it's components go down. So, the system design should itself be such that the system is able to take appropriate action to resolve the issue. In this article, we will cover the following recipe: Creating child actors of a parent actor Overriding the life cycle hooks of an actor Sending messages to actors and collecting responses Creating child actors of a parent actor In this recipe, we will learn how to create child actors of an actor. Akka follows a tree-like structure to create actors, and it is also the recommended practice. By following such practices, we can handle failures in actors as the parent can take care of it. Lets see how to do it. Getting ready We need to import the Hello-Akka project in the IDE of our choice. The Akka actor dependency that we added in build.sbt is sufficient for most of the recipes in this article, so we will skip the Getting ready section in our further recipes. How to do it… Create a file named ParentChild.scala in package com.packt.chapter2. Add the following imports to the top of the file: import akka.actor.{ActorSystem, Props, Actor} Create messages for sending to actors. case object CreateChild case class Greet(msg: String) Define a child actor as follows: class ChildActor extends Actor { def receive = { case Greet(msg) => println(s"My parent[${self.path.parent}] greeted to me [${self.path}] $msg") } } Define a parent actor as follows, and create a child actor in its context: class ParentActor extends Actor { def receive = { case CreateChild => val child = context.actorOf(Props[ChildActor], "child") child ! Greet("Hello Child") } } Create an application object as shown next: object ParentChild extends App { val actorSystem = ActorSystem("Supervision") val parent = actorSystem.actorOf(Props[ParentActor], "parent") parent ! CreateChild } Run the preceding application, and you will get the following output: My parent[akka://Supervision/user/parent] greeted to me [akka://Supervision/user/parent/child] Hello Child     How it works… In this recipe, we created a child actor, which receives a message, Greet, from the parent actor. We see the parent actor create a child actor using context.actorOf. This method creates a child actor under the parent actor. We can see the path of the actor in the output clearly. Overriding life cycle hooks of an actor Since we are talking about supervision and monitoring of actors, you should understand the life cycle hooks of an actor. In this recipe, you will learn how to override the life cycle hooks of an actor when it starts, stops, prestarts, and postrestarts. How to do it… Create a file called ActorLifeCycle.scala in package com.packt.chapter2. Add the following imports to the top of the file: import akka.actor import akka.actor.SupervisorStrategy import akka.util.Timeout. import scala.concurrent.Await import scala.concurrent.duration import akka.pattern.ask Create the following messages to be sent to the actors: case object Error case class StopActor(actorRef: ActorRef) Create an actor as follows, and override the life cycle methods: class LifeCycleActor extends Actor { var sum = 1 override def preRestart(reason: Throwable, message: Option[Any]): Unit = { println(s"sum in preRestart is $sum") } override def preStart(): Unit = println(s"sum in preStart is $sum") def receive = { case Error => throw new ArithmeticException() case _ => println("default msg") } override def postStop(): Unit = { println(s"sum in postStop is ${sum * 3}") } override def postRestart(reason: Throwable): Unit = { sum = sum * 2 println(s"sum in postRestart is $sum") } } Create a supervisor actor as follows: class Supervisor extends Actor { override val supervisorStrategy = OneForOneStrategy(maxNrOfRetries = 10, withinTimeRange = 1 minute) { case _: ArithmeticException => Restart case t => super.supervisorStrategy.decider.applyOrElse(t, (_: Any) => Escalate) } def receive = { case (props: Props, name: String) => sender ! context.actorOf(props, name) case StopActor(actorRef) => context.stop(actorRef) } } Create a test application as shown next, and run the application. object ActorLifeCycle extends App { implicit val timeout = Timeout(2 seconds) val actorSystem = ActorSystem("Supervision") val supervisor = actorSystem.actorOf(Props[Supervisor], "supervisor") val childFuture = supervisor ? (Props(new LifeCycleActor), "LifeCycleActor") val child = Await.result(childFuture.mapTo[ActorRef], 2 seconds) child ! Error Thread.sleep(1000) supervisor ! StopActor(child) } Create another test application as follows, and run it. object ActorLifeCycle extends App { implicit val timeout = Timeout(2 seconds) val actorSystem = ActorSystem("Supervision") val supervisor = actorSystem.actorOf(Props[Supervisor], "supervisor") val childFuture = supervisor ? (Props(new LifeCycleActor), "LifeCycleActor") val child = Await.result(childFuture.mapTo[ActorRef], 2 seconds) child ! Error Thread.sleep(1000) supervisor ! StopActor(child) } On running the preceding test application, you will get the following output: sum in preStart is 1 sum in preRestart is 1 sum in postRestart is 2 [ERROR] [07/01/2016 00:49:57.568] [Supervision-akka.actor.default-dispatcher-5] [akka://Supervision/user/supervisor/LifeCycleActor] null java.lang.ArithmeticException at com.packt.chapter2.LifeCycleActor$ $anonfun$receive$2.applyOrElse(ActorLifeCycle.scala:51) sum in postStop is 6 How it works… In this preceding recipe, we create an actor, which maintains sum as a state, and we modify its life cycle hooks. We create this actor under the parent supervisor, which handles the ArthimaticException in the child actor. Let's see what happens in life cycle hooks. When an actor starts, it calls the preStart method, so we see the following output: "sum in preStart is 1". When an actor throws an exception, it sends a message to the supervisor, and the supervisor handles the failure by restarting that actor. It clears out the accumulated state of the actor, creates a fresh new actor means, and then restores the last value assigned to the state of old actor to the preRestart value. After that postRestart method is called, and whenever the actor stops, the supervisor calls the postStop. Sending messages to actors and collecting responses In this recipe, you will learn how a parent sends messages to its child, and collects responses from them. To step through this recipe, we need to import the Hello-Akka project in the IDE. How to do it… Create a file, SendMesagesToChilds.scala, in package com.packt.chapter2. Add the following imports to the top of the file: import akka.actor.{ Props, ActorSystem, Actor, ActorRef } Create messages to be sent to the actors as follows: case class DoubleValue(x: Int) case object CreateChild case object Send case class Response(x: Int) Define a child actor. It doubles the value sent to it. class DoubleActor extends Actor { def receive = { case DoubleValue(number) => println(s"${self.path.name} Got the number $number") sender ! Response(number * 2) } } Define a parent actor. It creates child actors in its context, sends messages to them, and collects responses from them. class ParentActor extends Actor { val random = new scala.util.Random var childs = scala.collection.mutable.ListBuffer[ActorRef]() def receive = { case CreateChild => childs ++= List(context.actorOf(Props[DoubleActor])) case Send => println(s"Sending messages to child") childs.zipWithIndex map { case (child, value) => child ! DoubleValue(random.nextInt(10)) } case Response(x) => println(s"Parent: Response from child $ {sender.path.name} is $x") } } Create a test application as follows, and run it: object SendMessagesToChild extends App { val actorSystem = ActorSystem("Hello-Akka") val parent = actorSystem.actorOf(Props[ParentActor], "parent") parent ! CreateChild parent ! CreateChild parent ! CreateChild parent ! Send } On running the preceding test application, you will get the following output: $b Got the number 6 $a Got the number 5 $c Got the number 8 Parent: Response from child $a is 10 Parent: Response from child $b is 12 Parent: Response from child $c is 16 How it works… In this last recipe, we create a child actor called DoubleActor, which doubles the value it gets. We also create a parent actor, which creates a child actor when it receives a CreateChild message, and maintains it in the list. When the parent actor receives the message Send, it sends a random number to the child, and the child, in turn, sends a response to the parent. Summary In this article, you learned how to supervise and monitor Akka actors as well as create child actors of an actor. We also discussed how to override the life cycle hooks of an actor. Lastly, you learned how a parent sends messages to its child and collects responses from them. Resources for Article: Further resources on this subject: Introduction to Akka [article] Creating First Akka Application [article] Making History with Event Sourcing [article]

In this article by Eriawan Kusumawardhono, author of the book, F# High Performance explains why F# has been a first class citizen, a built in part of programming languages support in Visual Studio, starting from Visual Studio 2010. Though F# is a programming language that has its own unique trait: it is a functional programming language but at the same time it has OOP support. F# from the start has run on .NET, although we can also run F# on cross-platform, such as Android (using Mono). (For more resources related to this topic, see here.) Although F# mostly runs faster than C# or VB when doing computations, its own performance characteristics and some not so obvious bad practices and subtleties may have led to performance bottlenecks. The bottlenecks may or may not be faster than C#/VB counterparts, although some of the bottlenecks may share the same performance characteristics, such as the use of .NET APIs. The main goal of this book is to identify performance problems in F#, measuring and also optimizing F# code to run more efficiently while also maintaining the functional programming style as appropriately as possible. A basic knowledge of F# (including the functional programming concept and basic OOP) is required as a prerequisite to start understanding the performance problems and the optimization of F#. There are many ways and definitions to define F# performance characteristics and at the same time measure them, but understanding the mechanics of running F# code, especially on top of .NET, is crucial and it's also a part of the performance characteristic itself. This includes other aspects of approaches to identify concurrency problems and language constructs. Understanding the nature of F# code Understanding the nature of F# code is very crucial and it is a definitive prerequisite before we begin to measure how long it runs and its effectiveness. We can measure a running F# code by running time, but to fully understand why it may run slow or fast, there are some basic concepts we have to consider first. Before we dive more into this, we must meet the basic requirements and setup. After the requirements have been set, we need to put in place the environment setting of Visual Studio 2015. We have to set this, because we need to maintain the consistency of the default setting of Visual Studio. The setting should be set to General. These are the steps: Select the Tools menu from Visual Studio's main menu. Select Import and Export Settings... and the Import and Export Settings Wizard screen is displayed. Select Reset all Settings and then Next to proceed. Select No, just reset my settings overwriting my current setting and then Next to proceed. Select General and then Next to proceed After setting it up, we will have a consistent layout to be used throughout this book, including the menu locations and the look and feel of Visual Studio. Now we are going to scratch the surface of F# runtime with an introductory overview of common F# runtime, which will give us some insights into F# performance. F# runtime characteristics The release of Visual Studio 2015 occurred at the same time as the release of .NET 4.6 and the rest of the tools, including the F# compiler. The compiler version of F# in Visual Studio 2015 is F# 4.0. F# 4.0 has no large differences or notable new features compared to the previous version, F# 3.0 in Visual Studio 2013. Its runtime characteristic is essentially the same as F# 4.0, although there are some subtle performance improvements and bug fixes. For more information on what's new in F# 4.0 (described as release notes) visit: https://github.com/Microsoft/visualfsharp/blob/fsharp4/CHANGELOG.md. At the time of writing this book, the online and offline MSDN Library of F# in Visual Studio does not have F# 4.0 release notes documentation, but can always go to the GitHub repository of F# to check the latest update. These are the common characteristics of F# as part of managed programming language: F# must conform to .NET CLR. This includes the compatibilities, the IL emitted after compile, and support for .NET BCL (the basic class library). Therefore, F# functions and libraries can be used by other CLR compliant languages such as C#, VB, and managed C++. The debug symbols (PDB) have the same format and semantic as other CLR compliant languages. This is important, because F# code must be able to be debugged from other CLR compliant languages as well. From the managed languages perspective, measuring performance of F# is similar when measured by tools such as the CLR profiler. But from a F# unique perspective, these are F#-only unique characteristics: By default, all types in F# are immutable. Therefore, it's safe to assume it is intrinsically thread safe. F# has a distinctive collection library, and it is immutable by default. It is also safe to assume it is intrinsically thread safe. F# has a strong type inference model, and when a generic type is inferred without any concrete type, it automatically performs generalizations. Default functions in F# are implemented internally by creating an internal class derived from F#’s FastFunc. This FastFunc is essentially a delegate that is used by F# to apply functional language constructs such as currying and partial application. With tail call recursive optimization in the IL, the F# compiler may emit .tail IL, and then the CLR will recognize this and perform optimization at runtime. F# has inline functions as option F# has a computation workflow that is used to compose functions F# async computation doesn't need Task<T> to implement it. Although F# async doesn't need the Task<T> object, it can operate well with the async-await model in C# and VB. The async-await model in C# and VB is inspired by F# async, but behaves semantically differently based on more things than just the usage of Task<T>. All of those characteristics are not only unique, but they can also have performance implications when used to interoperate with C# and VB. Summary This article explained the basic introduction to F# IDE, along with runtime characteristics of F#. Resources for Article: Further resources on this subject: Creating an F# Project [article] Unit Testing [article] Working with Windows Phone Controls [article]

In this article by Behzad Ehsani, author of the book Data Acquisition using LabVIEW, after a brief introduction and a short note on installation, we will go over the most widely used pallets and objects Icon tool bar from a standard installation of LabVIEW and a brief explanation of what each object does. (For more resources related to this topic, see here.) Introduction to LabVIEW LabVIEW is a graphical developing and testing environment unlike any other test and development tool available in the industry. LabVIEW sets itself apart from traditional programming environment by its complete graphical approach to programming. As an example, while representation of a while loop in a text based language such as the C language consists of several predefined, extremely compact and sometimes extremely cryptic lines of text, a while a loop in LabVIEW, is actually a graphical loop. The environment is extremely intuitive and powerful, which makes for a short learning cure for the beginner. LabVIEW is based on what is called G language, but there are still other languages, especially C under the hood. However, the ease of use and power of LabVIEW is somewhat deceiving to a novice user. Many people have attempt to start projects in LabVIEW only because at the first glace, the graphical nature of interface and the concept of drag an drop used in LabVIEW, appears to do away with required basics of programming concepts and classical education in programming science and engineering. This is far from the reality of using LabVIEW as the predominant development environment. While it is true that in many higher level development and testing environment, specially when using complicated test equipment and complex mathematical calculations or even creating embedded software LabVIEW's approach will be much more time efficient and bug free environment which otherwise would require several lines of code in traditional text based programming environment, one must be aware of LabVIEW's strengths and possible weaknesses. LabVIEW does not completely replace the need for traditional text based languages and depending on the entire nature of a project, LabVIEW or another traditional text based language such as C may be the most suitable programming or test environment. Installing LabVIEW Installation of LabVIEW is very simple and it is just as routine as any modern day program installation; that is, Insert the DVD 1 and follow on-screen guided installation steps. LabVIEW comes in one DVD for Mac and Linux version but in four or more DVDs for the Windows edition (depending on additional software, different licensing and additional libraries and packages purchased.) In this article we will use LabVIEW 2013 Professional Development version for Windows. Given the target audience of this article, we assume the user is well capable of installation of the program. Installation is also well documented by National Instruments and the mandatory one year support purchase with each copy of LabVIEW is a valuable source of live and email help. Also, NI web site (www.ni.com) has many user support groups that are also a great source of support, example codes, discussion groups and local group events and meeting of fellow LabVIEW developers, etc. One worthy note for those who are new to installation of LabVIEW is that the installation DVDs include much more than what an average user would need and pay for. We do strongly suggest that you install additional software (beyond what has been purchased and licensed or immediate need!) These additional software are fully functional (in demo mode for 7 days) which may be extended for about a month with online registration. This is a very good opportunity to have hands on experience with even more of power and functionality that LabVIEW is capable to offer. The additional information gained by installing other available software on the DVDs may help in further development of a given project. Just imagine if the current development of a robot only encompasses mechanical movements and sensors today, optical recognition probably is going to follow sooner than one may think. If data acquisition using expensive hardware and software may be possible in one location, the need to web sharing and remote control of the setup is just around the corner. It is very helpful to at least be aware of what packages are currently available and be able to install and test them prior to a full purchase and implementation. The following screenshot shows what may be installed if almost all software on all DVDs are selected: When installing a fresh version of LabVIEW, if you do decide to observe the advice above, make sure to click on the + sign next to each package you decide to install and prevent any installation of LabWindows/CVI.... and Measurement Studio... for Visual Studio LabWindows according to National Instruments .., is an ANSI C integrated development environment. Also note that by default NI device drivers are not selected to be installed. Device drivers are an essential part of any data acquisition and appropriate drivers for communications and instrument(s) control must be installed before LabVIEW can interact with external equipments. Also, note that device drivers (on Windows installations) come on a separate DVD which means that one does not have to install device drivers at the same time that the main application and other modules are installed; they can be installed at any time later on. Almost all well established vendors are packaging their product with LabVIEW drivers and example codes. If a driver is not readily available, National Instruments has programmers that would do just that. But this would come at a cost to the user. VI Package manager, now installed as a part of standard installation is also a must these days. National Instruments distributes third party software and drivers and public domain packages via VI Package manager. Appropriate software and drivers for these microcontrollers are installed via VI Package manager. You can install many public domain packages that further installs many useful LabVIEW toolkits to a LabVIEW installation and can be used just as those that are delivered professionally by National Instruments. Finally, note that the more modules, packages and software are selected to be installed the longer it will take to complete the installation. This may sound like making an obvious point but surprisingly enough installation of all software on the three DVDs (for Windows) take up over five hours! On a standard laptop or pc we used. Obviously a more powerful PC (such as one with solid sate hard drive) my not take such log time: LabVIEW Basics Once the LabVIEW applications is launched, by default two blank windows open simultaneously; a Front Panel and a Block Diagram window and a VI is created: VIs or Virtual Instruments are heart and soul of LabVIEW. They are what separate LabVIEW from all other text based development environments. In LabVIEW everything is an object which is represented graphically. A VI may only consist of a few objects or hundreds of objects embedded in many subVIs These graphical representation of a thing, be it a simple while loop, a complex mathematical concept such as polynomial interpolation or simply a Boolean constant are all graphically represented. To use an object right-click inside the block diagram or front panel window, a pallet list appears. Follow the arrow and pick an object from the list of object from subsequent pallet an place it on the appropriate window. The selected object now can be dragged and place on different location on the appropriate window and is ready to be wired. Depending on what kind of object is selected, a graphical representation of the object appears on both windows. Of cores there are many exceptions to this rule. For example a while loop can only be selected in Block Diagram and by itself, a while loop does not have a graphical representation on the front panel window. Needless to say, LabVIEW also has keyboard combination that expedite selecting and placing any given toolkit objects onto the appropriate window. Each object has one (or several) wire connections going into as input(s) and coming out as its output(s). A VI becomes functional when a minimum number of wires are appropriately connected to input and output of one or more object. Later, we will use an example to illustrate how a basic LabVIEW VI is created and executed. Highlights LabVIEW is a complete object-oriented development and test environment based on G language. As such it is a very powerful and complex environment. In article one we went through introduction to LabVIEW and its main functionality of each of its icon by way of an actual user interactive example. Accompanied by appropriate hardware (both NI as well as many industry standard test, measurement and development hardware products) LabVIEW is capable to cover from developing embedded systems to fuzzy logic and almost everything in between! Summary In this article we cover the basics of LabVIEW, from installation to in depth explanation of each and every element in the toolbar. Resources for Article: Further resources on this subject: Python Data Analysis Utilities [article] Data mining [article] PostgreSQL in Action [article]

In this article by Jordan Krause, the author of Mastering Windows Server 2016, we will explore the Remote Access Management Console of DirectAccess and we will also look at the differences between DirectAccess and VPN. (For more resources related to this topic, see here.) You are well on your way to giving users remote access capabilities on this new server. As with many networking devices, once you have established all of your configurations on a remote access server, it is pretty common for admins to walk away and let it run. There is no need for a lot of ongoing maintenance or changes to that configuration once you have it running well. However, Remote Access Management Console in Windows Server 2016 is useful not only for configuration of the remote access parts and pieces, but for monitoring and reporting as well. Let’s take a look inside this console so that you are familiar with the different screens you will be interacting with: Configuration The configuration screen is pretty self-explanatory, this is where you would visit in order to create your initial remote access configuration, and where you go to update any settings in the future. As you can see in the screenshot, you are able to configure DirectAccess, VPN, and the Web Application Proxy right from this Remote Access Management Console. There is not a lot to configure as far as the VPN goes, you really only have one screen of options where you define what kind of IP addresses are handed down to the VPN clients connecting in, and how to handle VPN authentication. It is not immediately obvious where this screen is, so I wanted to point it out. Inside the DirectAccess and VPN configuration section, if you click on the Edit… button listed under Step 2, this will launch the Step 2 mini-wizard. The last screen of this mini-wizard is called VPN Configuration. This is the screen where you can configure these IP address and authentication settings for your VPN connections: Dashboard The Remote Access Dashboard gives you a 30,000 foot view of the Remote Access server status. You are able to view a quick status of the components running on the server, whether or not the latest configuration changes have been rolled around, and some summary numbers near the bottom about how many DirectAccess and VPN connections are happening. Operations Status If you want to drill down further into what is happening on the server side of the connections, that is what the Operations Status page is all about. Here you can see a little more detail on each of the components that are running under the hood to make your DA and VPN connections happen. If any of them have an issue, you can click on the specific component to get a little more information. For example, as a test, I have turned off the NLS web server in my lab network, and I can now see in the Operations Status page that NLS is flagged with an error. Remote Client Status Next up is the Remote Client Status screen. As indicated, this is the screen where we can monitor the client computers who are connected. It will show us both DirectAccess and VPN connections here. We will be able to see computer names, usernames, and even the resources that they are utilizing during their connections. The information on this screen is able to be filtered by simply putting any criteria into the Search bar on the top of the window. It is important to note that the Remote Client Status screen only shows live, active connections. There is no historical information stored here. Reporting You guessed it, this is the window you need to visit if you want to see historical remote access information. This screen is almost exactly the same as the Remote Client Status screen, except that you have the ability to generate reports for historical data pulled from date ranges of your choosing. Once the data is displayed, you have the same search and filtering capabilities that you had on the Remote Client Status screen. Reporting is disabled by default, but you simply need to navigate to the Reporting page and click on Configure Accounting. Once that is enabled, you will be presented with options about storing the historical information. You can choose to store the data in the local WID, or on a remote RADIUS server. You also have options here for how long to store logging data, and a mechanism that can be used to clear out old data. Tasks The last window pane of Remote Access Management Console that I want to point out is the Tasks bar on the right side of your screen. The actions and options that are displayed in this taskbar change depending on what part of the console you are navigating through. Make sure to keep an eye on this side of your screen for setting up some of the more advanced functions. Some examples of available tasks are creating usage reports, refreshing the screen, and configuring network load balancing or Multi-Site configurations if you are running multiple remote access servers. DirectAccess versus VPN VPN has been around for a very long time, making it a pretty familiar idea to anyone working in IT, and we have discussed quite a bit about DirectAccess today in order to bring you up to speed on this evolution, so to speak, of corporate remote access. Now that you know there are two great solutions built into Windows Server 2016 for enabling your mobile workforce, which one is better? While DirectAccess is certainly the newer of the technologies, we cannot say that it is better in all circumstances. Each has its pros and cons, and the ways that you use each, or both, will depend upon many variables. Your users, your client computers, and your organization’s individual needs will need to factor into your decision-making process. Let’s discuss some of the differences between DirectAccess and VPN so that you can better determine which is right for you. Domain-joined versus non-domain-joined One of the biggest requirements for a DirectAccess client computer is that it must be domain joined. While this requirement by itself doesn’t seem so major, what it implies can be pretty vast. Trusting a computer enough to be joined to your domain more than likely means that the laptop is owned by the company. It also probably means that this laptop was first in IT’s hands in order to build and prep it. Companies that are in the habit of allowing employees to purchase their own computers to be used for work purposes may not find DirectAccess to fit well with that model. DA is also not ideal for situations where employees use their existing home computers to connect into work remotely. In these kinds of situations, such as home and personally-owned computers, VPN may be better suited to the task. You can connect to a VPN from a non-domain-joined machine, and you can even establish VPN connections from many non-Microsoft devices. IOS, Android, Windows Phone—these are all platforms that have a VPN client built into them that can be used to tap into a VPN listener on a Windows Server 2016 remote access server. If your only remote access solution was DirectAccess, you would not be able to provide non-domain-joined devices with a connectivity platform. Auto versus manual launch Here, DirectAccess takes the cake. It is completely seamless. DirectAccess components are baked right into the Windows operating system, no software VPN is going to be able to touch that level of integration. With VPN, users have to log in to their computers to unlock them, then launch their VPN, then log in again to that VPN software, all before they can start working on anything. With DirectAccess, all they need to do is log in to the computer to unlock the screen. DirectAccess activates itself in the background so that as soon as the desktop loads for the user, they simply open the applications that they need to access, just like when they are inside the office. Software versus built-in I’m a fan of Ikea furniture. They do a great job of supplying quality products at a low cost, all while packaging it up in incredibly small boxes. After you pay for the product, unbox the product, put the product together, and then test the product to make sure it works—it’s great. If you can’t see where this is going, I’ll give you a hint. It’s an analogy for VPN. As in, you typically pay a vendor for their VPN product, unbox the product, implement the product at more expense, then test the product. That VPN software then has the potential to break and need reinstallation or reconfiguration, and will certainly come with software updates that need to be accomplished down the road. Maintenance, maintenance, maintenance. Maybe I have been watching too many home improvement shows lately, but I am a fan of houses with built-ins. Built-ins are essentially furniture that is permanent to the house, built right into the walls, corners, or wherever it happens to be. It adds value, and it integrates into the overall house much better than furniture that was pieced together separately and then stuck against the wall in the corner. DirectAccess is like a built-in. It is inside the operating system. There is no software to install, no software to update, no software to reinstall when it breaks. Everything that DA needs is already in Windows today, you just aren’t using it. Oh, and it’s free, well, built into the cost of your Windows license anyway. There are no user CALs, no ongoing licensing costs related to implementing Microsoft DirectAccess. Password and login issues with VPN If you have ever worked on a helpdesk for a company that uses VPN, you know what I’m talking about. There are a series of common troubleshooting calls that happen in the VPN world related to passwords. Sometimes the user forgets their password. Perhaps their password has expired and needs to be changed—ugh, VPN doesn’t handle this scenario very well either. Or maybe the employee changed their expired password on their desktop before they left work for the day, but are now trying to log in remotely from their laptop and it isn’t working. What is the solution to password problems with VPN? Reset the user’s password and then make the user come into the office in order to make it work on their laptop. Yup, these kinds of phone calls still happen every day. This is unfortunate, but a real potential problem with VPN. What’s the good news? DirectAccess doesn’t have these kinds of problems! Since DA is part of the operating system, it has the capability to be connected anytime that Windows is online. This includes the login screen! Even if I am sitting on the login or lock screen, and the system is waiting for me to input my username and password, as long as I have Internet access I also have a DirectAccess tunnel. This means that I can actively do password management tasks. If my password expires and I need to update it, it works. If I forgot my password and I can’t get into my laptop, I can call the helpdesk and simply ask them to reset my password. I can then immediately log in to my DirectAccess laptop with the new password, right from my house. Another cool function that this seamlessness enables is the ability to login with new user accounts. Have you ever logged into your laptop as a different user account in order to test something? Yup, that works over DirectAccess as well. For example, I am sitting at home and I need to help one of the sales guys troubleshoot some sort of file permission problem. I suspect it’s got something to do with his user account, so I want to log in to my laptop as him in order to test it. The problem is that his user account has never logged into my laptop before. With VPN, not a chance. This would never work. With DirectAccess, piece of cake! I simply log off, type in his username and password, and bingo. I’m logged in, while still sitting at home in my pajamas. It is important to note that you can run both DirectAccess and VPN on the same Windows Server 2016 remote access server. If both technologies have capabilities that you could benefit from, use them both! Summary The technology of today demands for most companies to enable their employees to work from wherever they are. More and more organizations are hiring a work from home workforce, and need a secure, stable, and efficient way to provide access of corporate data and applications to these mobile workers. The Remote Access role in Windows Server 2016 is designed to do exactly that. With three different ways of providing remote access to corporate resources, IT departments have never had so much remote access technology available at their fingertips, built right into the Windows operating system that they already own. If you are still supporting a third-party or legacy VPN system, you should definitely explore the new capabilities provided here and discover how much they could save for your business. DirectAccess is particularly impressive and compelling; it’s a brand new way of looking at remote access. Automatic connectivity includes always-on machines that are constantly being patched and updated because they are always connected to your management servers. You can improve user productivity and network security at the same time. These two things are usually oxymorons in the IT world, but with DirectAccess they hold hands and sing songs together. Resources for Article: Further resources on this subject: Remote Authentication [article] Configuring a MySQL linked server on SQL Server 2008 [article] Configuring sipXecs Server Features [article]

In this article by Diego Pacheco, the author of the book, Building applications with Scala, we will see the following topics: Writing a program for Scala Hello World using the REPL Scala language – the basics Scala variables – var and val Creating immutable variables (For more resources related to this topic, see here.) Scala Hello World using the REPL Let's get started. Go ahead, open your terminal, and type $ scala in order to open the Scala REPL. Once the REPL is open, you can just type "Hello World". By doing this, you are performing two operations – eval and print. The Scala REPL will create a variable called res0 and store your string there, and then it will print the content of the res0 variable. Scala REPL Hello World program $ scala Welcome to Scala 2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_77). Type in expressions for evaluation. Or try :help. scala> "Hello World" res0: String = Hello World scala> Scala is a hybrid language, which means it is both object-oriented (OO) and functional. You can create classes and objects in Scala. Next, we will create a complete Hello World application using classes. Scala OO Hello World program $ scala Welcome to Scala 2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_77). Type in expressions for evaluation. Or try :help. scala> object HelloWorld { | def main(args:Array[String]) = println("Hello World") | } defined object HelloWorld scala> HelloWorld.main(null) Hello World scala> First things first, you need to realize that we use the word object instead of class. The Scala language has different constructs, compared with Java. Object is a Singleton in Scala. It's the same as you code the Singleton pattern in Java. Next, we see the word def that is used in Scala to create functions. In this program, we create the main function just as we do in Java, and we call the built-in function, println, in order to print the String Hello World. Scala imports some java objects and packages by default. Coding in Scala does not require you to type, for instance, System.out.println("Hello World"), but you can if you want to, as shown in the following:. $ scala Welcome to Scala 2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_77). Type in expressions for evaluation. Or try :help. scala> System.out.println("Hello World") Hello World scala> We can and we will do better. Scala has some abstractions for a console application. We can write this code with less lines of code. To accomplish this goal, we need to extend the Scala class App. When we extend from App, we are performing inheritance, and we don't need to define the main function. We can just put all the code on the body of the class, which is very convenient, and which makes the code clean and simple to read. Scala HelloWorld App in the Scala REPL $ scala Welcome to Scala 2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_77). Type in expressions for evaluation. Or try :help. scala> object HelloWorld extends App { | println("Hello World") | } defined object HelloWorld scala> HelloWorld object HelloWorld scala> HelloWorld.main(null) Hello World scala> After coding the HelloWorld object in the Scala REPL, we can ask the REPL what HelloWorld is and, as you might realize, the REPL answers that HelloWorld is an object. This is a very convenient Scala way to code console applications because we can have a Hello World application with just three lines of code. Sadly, the same program in Java requires way more code, as you will see in the next section. Java is a great language for performance, but it is a verbose language compared with Scala. Java Hello World application package scalabook.javacode.chap1; public class HelloWorld { public static void main(String args[]){ System.out.println("Hello World"); } } The Java application required six lines of code, while in Scala, we were able to do the same with 50% less code(three lines of code). This is a very simple application; when we are coding complex applications, the difference gets bigger as a Scala application ends up with way lesser code than that of Java. Remember that we use an object in Scala in order to have a Singleton(Design Pattern that makes sure you have just one instance of a class), and if we want to do the same in Java, the code would be something like this: package scalabook.javacode.chap1; public class HelloWorldSingleton { private HelloWorldSingleton(){} private static class SingletonHelper{ private static final HelloWorldSingleton INSTANCE = new HelloWorldSingleton(); } public static HelloWorldSingleton getInstance(){ return SingletonHelper.INSTANCE; } public void sayHello(){ System.out.println("Hello World"); } public static void main(String[] args) { getInstance().sayHello(); } } It's not just about the size of the code, but it is all about consistency and the language providing more abstractions for you. If you write less code, you will have less bugs in your software at the end of the day. Scala language – the basics Scala is a statically typed language with a very expressive type system, which enforces abstractions in a safe yet coherent manner. All values in Scala are Java objects (but primitives that are unboxed at runtime) because at the end of the day, Scala runs on the Java JVM. Scala enforces immutability as a core functional programing principle. This enforcement happens in multiple aspects of the Scala language, for instance, when you create a variable, you do it in an immutable way, and when you use a collection, you use an immutable collection. Scala also lets you use mutable variables and mutable structures, but it favors immutable ones by design. Scala variables – var and val When you are coding in Scala, you create variables using either the var operator or the val operator. The var operator allows you to create mutable states, which is fine as long as you make it local, stick to the core functional programing principles, and avoid mutable shared state. Using var in the Scala REPL $ scala Welcome to Scala 2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_77). Type in expressions for evaluation. Or try :help. scala> var x = 10 x: Int = 10 scala> x res0: Int = 10 scala> x = 11 x: Int = 11 scala> x res1: Int = 11 scala> However, Scala has a more interesting construct called val. Using the val operator makes your variables immutable, which means that you can't change their values after you set them. If you try to change the value of a val variable in Scala, the compiler will give you an error. As a Scala developer, you should use val as much as possible because that's a good functional programing mindset, and it will make your programs better and more correct. In Scala, everything is an object; there are no primitives – the var and val rules apply for everything, be it Int, String, or even a class. Using val in the Scala REPL $ scala Welcome to Scala 2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_77). Type in expressions for evaluation. Or try :help. scala> val x = 10 x: Int = 10 scala> x res0: Int = 10 scala> x = 11 <console>:12: error: reassignment to val x = 11 ^ scala> x res1: Int = 10 scala> Creating immutable variables Right. Now let's see how we can define the most common types in Scala, such as Int, Double, Boolean, and String. Remember that you can create these variables using val or var, depending on your requirement. Scala variable types at the Scala REPL $ scala Welcome to Scala 2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_77). Type in expressions for evaluation. Or try :help. scala> val x = 10 x: Int = 10 scala> val y = 11.1 y: Double = 11.1 scala> val b = true b: Boolean = true scala> val f = false f: Boolean = false scala> val s = "A Simple String" s: String = A Simple String scala> For these variables, we did not define the type. The Scala language figures it out for us. However, it is possible to specify the type if you want. In Scala, the type comes after the name of the variable, as shown in the following section. Scala variables with explicit typing at the Scala REPL $ scala Welcome to Scala 2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_77). Type in expressions for evaluation. Or try :help. scala> val x:Int = 10 x: Int = 10 scala> val y:Double = 11.1 y: Double = 11.1 scala> val s:String = "My String " s: String = "My String " scala> val b:Boolean = true b: Boolean = true scala> Summary In this article, we learned about some basic constructs and concepts of the Scala language, with functions, collections, and OO in Scala. Resources for Article: Further resources on this subject: Making History with Event Sourcing [article] Creating Your First Plug-in [article] Content-based recommendation [article]

In this article by Nicolai Henriksen, the author of the book Microsoft System Center 1511 Endpoint Protection Cookbook, we will cover how you need to configure Endpoint Protection in Configuration Manager. (For more resources related to this topic, see here.) This is the part where you need to think through every setting you make so that it does the impact and good you want in your organization. How to configure Endpoint Protection in Configuration Manager In order to manage security and malware on your client computers with Endpoint Protection. There are a few steps you must setup and configure in order to get it working in System Center Configuration Manager (SCCM). Getting ready In this article we assume that you have SCCM in-place and working. And have setup and installed the Software Update Point Role with its prerequisites like Windows Server Update Services (WSUS). Also you have planned and thought through what impact this has in your environment, a good understanding how this should and would work in your Configuration Manager hierarchy. How to do it… First we start with installing the Endpoint Protection Role from within the SCCM console. This role must be installed before you can use and configure Endpoint Protection. It must only be installed on one site system server, and it must be installed on top of your hierarchy, meaning if you have a Central Administration Site (CAS) you install in there, or if you have a stand-alone primary site you install it there. Be aware that when you install the Endpoint Protection Role on the site server it will also install the Endpoint Protection client on that same server. This is by default and cannot be changed. However services and scans are disabled so that you can still run any other existing anti-malware solution that you may already have in place. No real-time scanning or any other form of scanning will be performed by Endpoint Protection before you enable it with a policy. So be aware of this so that you don't accidentally enable it while having another anti-malware solution installed. Installing the Endpoint Protection Role is pretty easy and straight forward; these are the steps to manage that. To install and configure Endpoint Protection Role you open the Configuration Manager console, Click Administration. And in the Administration workspace you expand Site Configuration and click on Servers and Site System Roles. You click on Add Site System Roles in the picture shown below: On the next screen I choose to use as default settings that will use the server's computer Account to install the Role on the chosen server. In my case I have a single primary site server where all the Roles reside and this will require no other preparation. However, keep in mind that if you are adding Roles to other site system servers it will require that you add the primary site server's computer Account to the local Administrators group, or you could use an installation account as shown in the following figure: Let's click Next >. This is the page where we choose the Endpoint Protection Role that we want to install. It will only list up the Roles that you have not already added to the chosen server. Pay attention that it also warns you to have software updates and anti-malware definitions already in-place and deployed. The warring will show regardless weather you have this already in-place or not as shown in the next screenshot. The next page on the wizard it about the Microsoft Active Protection Service membership. I like to think of this as the cloud feature, and I encourage you to consider setting this to Advanced membership as that will give you and Microsoft a greater chance of dealing will the unknown type of malware. This will send more information from the infected client about the surroundings of the malware. And Microsoft can investigate the bits and pieces more thoroughly in their environment in the cloud service. If it turns out that this is infectious malware like a Trojan downloader for example, it will get further removal instructions directly and try its best to remove it automatically. Now this feature will work either way on what you choose, but it will work even better if you choose to share some more information. Most other anti-virus and anti-malware products don't ask about this, they just enable it. But Microsoft has chosen to let you decide. Because there could be situations that you might not want to share this at all. You can always choose Do not join MAPS in this page and decide individually in each Endpoint Protection Policy how you want it. Setting it here simply makes this the default setting for every policy made afterwards. Clicking Next > and Finish will start the installation of the Endpoint Protection Role and finish in a few minutes. In the Monitoring | Components status shown below you can see two components starting with SMS_ENDPOINT_PROTECTION that will have a green icon on the left and will tell you that the Role is installed. How it works… So we the Endpoint Protection Role is installed in our SCCM hierarchy as simple as that. But there are more configurations to do that will be the next topic. If you remember the Endpoint Protection client will always be installed on the site server that has the Endpoint Protection Role installed. But by default, it is with no scanning or real-time protection enabled, looks like this red icon on the task-bar on the right side as shown in the figure below. Summary In this article we learned that security is key aspect for any organization. Misconfiguration may have a very bad outcome as this has to do with security. Resources for Article: Further resources on this subject: Managing Application Configuration [article] CoreOS Networking and Flannel Internals [article] Zabbix Configuration [article]

In this article by Kevin Ashton, the author of the book F# 4.0 Programming Cookbook, you will learn the following recipes: Working with optional values Working with tuples and pattern matching (For more resources related to this topic, see here.) Working with optional values In 2009, Tony Hoare, the creator of the ALGOL W programming language, called the concept of null reference his "billion-dollar mistake". Thousands of man-hours are lost each year due to bugs caused by null references. Many of these errors are avoidable, although not all developers are aware of it. The Option type, and working with it, is what this recipe is all about. This recipe covers a few different ways of using the option type, although all of the different methods shown can exist within the same script file. Getting ready Make sure that you have a text editor or IDE, and that the F# compiler and tools are installed on your system. In Visual Studio versions 2012 and 2013, F# is installed by default. If you are running Visual Studio 2015, you will need to make sure that you select Visual F# tools during the installation phase. The code in this example expects that you have a file named Alphabet.txt located in the same directory as your source code. This file consists of the letters a to z, each on a single line (with no lines before or after). How to do it… Open a new F# Script file. In Visual Studio, select File | New | File Select F# Script File Enter the following code: open System open System.IO let filePath = Path.Combine(__SOURCE_DIRECTORY__,"Alphabet.txt") let tryGet a = let lines = File.ReadAllLines(filePath) if a < 1 || a > lines.Length then None else Some(lines.[a-1]) let printResult res = let resultText = match res with | None -> "No valid letter found." | Some str -> sprintf "The letter found was %s" str printfn "%s" resultText Select the code that you have entered and press ALT + ENTER (in Visual Studio, this will send the highlighted lines of code to the F# Interactive (FSI) evaluation session). The FSI session should display: val filePath : string = "C:DevelopmentPacktBookChapter1Alphabet.txt" val tryGet : a:int -> string option To test this code, enter the following and send the result to FSI session: let badResult1, goodResult, badResult2 = tryGet 0, tryGet 5, tryGet 27 The FSI session should display: val goodResult : string option = Some "e" val badResult2 : string option = None val badResult1 : string option = None Now, enter the following code and send it to the FSI session: printResult badResult1 printResult goodResult The FSI session should display: No valid letter found The letter found was e Here, we show a different way of using the option type. In this case, it is used where there might not be a valid result from an operation (in the case of dividing by zero for instance). Enter the following code and send the result to the FSI session: let inline tryDivide numerator denominator = if denominator = LanguagePrimitives.GenericZero then None else Some(numerator / denominator) This method is defined as an inline method because that allows it take advantage of F#'s automatic generalization. If the inline keyword were to be omitted, the compiler would assume that this method only works on integer types. Notice also the use of LanguagePrimitives.GenericZero; this allows for this function to work on any type that defines a static get_Zero method. The FSI session should display: val inline tryDivide : numerator: ^a -> denominator: ^b -> ^c option when ( ^a or ^b) : (static member ( / ) : ^a * ^b -> ^c) and ^b : (static member get_Zero : -> ^b) and ^b : equality To test this function, enter the following code and send it to the FSI session: let goodDivideResult, badDivideResult = tryDivide 10 5, tryDivide 1 0 The FSI session should display: val goodDivideResult : int option = Some 2 val badDivideResult : int option = None Here is another way of using the Option type – to handle the results of parse expression where the parse may or may not succeed. In this case, the function we are writing takes advantage of an F# feature where output parameters (parameters that are marked with the out keyword in C#) can be returned as part of a tuple that includes the function result. Enter the following code and send it to the interactive session: let tryParse a f = let res,parsed = f a if res then Some parsed else None The FSI session should display: val tryParse : a:'a -> f:('a -> bool * 'b) -> 'b option To test this function, enter the following code and send it to the FSI session: let goodParseResult = tryParse "5" (Int32.TryParse) let badParseResult = tryParse "A" (Int32.TryParse) The FSI session should display: val goodParseResult : int option = Some 5 val badParseResult : int option = None How it works… This recipe shows three potential uses of the Option type. First, we show how the Option type can be used to return a safe result, where the expected value might not exist (this is particularly useful when working with databases or other IO operations, where the expected record might not be found). Then, we show a way of using the Option type to deal with operations that could potentially throw an exception (in this case, avoiding division by zero). Finally, we show a way of using the Option type to handle situations where it is valid for there to be no result, and to provide a strongly typed and explicit way of showing this. Working with tuples and pattern matching In F#, and in functional programming in general, a function is defined as taking one parameter and returning one result. F# makes it possible for developers to write functions that take more than a single parameter as input, and return more than a single value as the results. The way F# does this is by providing tuples. Tuples are heterogeneous collections of values. This recipe will show you how to work with tuples. As with the previous recipe, this recipe shows multiple ways of working with tuples, as well as ways of pattern matching over them. How to do it… Open a new F# script file. Enter the following code and send it to the FSI session: let divideWithRemainder numerator denominator = let divisionResult = numerator / denominator let remainder = numerator % denominator let inputs = (numerator, denominator) let outputs = (divisionResult, remainder) (inputs,outputs) let printResult input = let inputs = fst input let numerator, denominator = inputs let outputs = snd input let divisionResult, remainder = outputs printfn "%d divided by %d = %d (remainder: %d)" numerator denominator divisionResult remainder The FSI session should display: val divideWithRemainder :numerator:int -> denominator:int - > (int * int) * (int * int) val printResult : (int * int) * (int * int) -> unit To test the previous code, enter the following and send to the FSI session: divideWithRemainder 10 4 |> printResult The FSI session should display: 10 divided by 4 = 2 (remainder: 2) The printResult function could also be written in other ways with the same result. These ways are shown next. Send these to the interactive session to confirm that they display the same result. let printResultPartialPattern (inputs, outputs) = let numerator, denominator = inputs let divisionResult, remainder = outputs printfn "%d divided by %d = %d (remainder: %d)" numerator denominator divisionResult remainder let printResultFullPattern ((numerator, denominator), (divisionResult, remainder)) = printfn "%d divided by %d = %d (remainder: %d)" numerator denominator divisionResult remainder The FSI session should display: val printResultPartialPattern : inputs:(int * int) * outputs:(int * int) -> unit val printResultFullPattern : (int * int) * (int * int) -> unit Notice how for all of the various printResult functions, the signature displayed in the F# Interactive Session window are of the same type, (int * int) * (int * int). Whenever you see the asterisk (*) in a function signature, it is there to show that the type is a tuple, with a possible range consisting of all the possible values of the type on the left multiplied by all the possible values of the type on the right of the asterisk. This is a good way of displaying how tuples form part of the theory of algebraic data types. Now, we will show another way of using tuples. Tuples can be given type aliases so that their structure and intent is clear. This also allows for the defining of functions that have a clear purpose with regards to either their inputs or their outputs. Enter the following code and send it to the FSI session: type Gender = | Female | Male type Cat = (string * int * Gender) let printCat (cat:Cat) = let name,age,gender = cat let genderPronoun = match gender with | Female -> "She" | Male -> "He" printfn "%s is a cat. %s is %d years old." name genderPronoun age let cats: Cat list = ["Alice", 6, Female "Loki", 4, Male "Perun", 4, Male] The FSI session should display: type Gender =| Female | Male type Cat = string * int * Gender val printCat : string * int * Gender -> unit val cats : Cat list = [("Alice", 6, Female); ("Loki", 4, Male); ("Perun", 4, Male)] To test the preceding code, enter the following and send it to the FSI session: cats |> List.iter printCat The FSI session should display: How it works… This recipe shows two potential uses for tuples. Tuples can be used to allow multiple inputs to a function, or to allow a function to return multiple outputs. The values contained within a tuple do not have to be of the same type. First, we show a case for both returning multiple values from a function and providing multiple input values. The divideWithRemainder function accepts two inputsin curried form and returns a tupled output (which itself consists of two tuples). The tupled output from the divideWithRemainder function is then piped into the printResult function. Notice that the first printResult function accepts only a single input. The other ways of writing the printResult function are two different ways of taking advantage of F#'s pattern matching capabilities for simplifying the process. Then, we show you how to work with tuples with more than two elements (the functions fst and snd do not work for tuples that consist of more than two elements). As with the first section, pattern matching is able to decompose the tuple into its constituent parts. We also show how you can provide a type alias for a tuple and get a strongly typed feedback of whether the tuple matches the expected type. If you were to leave out an element from the cats list (age or gender for example), the IDE and compilers would give you an error. Summary This article intends to introduce you to many of the features of F# that will be used throughout the book. We learned more about using the Option type, Tuples, and Active Patterns, for moving from different programming styles towards a more idiomatic F# functional style. Resources for Article: Further resources on this subject: Creating an F# Project [article] Go Programming Control Flow [article] Working with Windows Phone Controls [article]

In this article by Jon Langemak, the author of the book Docker Networking Cookbook, has covered the following recipes: Verifying host based DNS configuration inside a container Overriding the default name resolution settings Configuring links for name and service resolution Leveraging Docker DNS (For more resources related to this topic, see here.) Verifying host based DNS configuration inside a container While you might not realize it but Docker, by default, is providing your containers a means to do basic name resolution. Docker passes name resolution from the Docker host, directly into the container. The result is that a spawned container can natively resolve anything that the Docker host itself can. The mechanics used by Docker to achieve name resolution in a container are elegantly simple. In this recipe, we'll walk through how this is done and how you can verify that it's working as expected. Getting Ready In this recipe we'll be demonstrating the configuration on a single Docker host. It is assumed that this host has Docker installed and that Docker is in its default configuration. We'll be altering name resolution settings on the host so you'll need root level access. How to do it… To start with, let's start a new container on our host docker1 and examine how the container handles name resolution: user@docker1:~$ docker run -d -P --name=web8 jonlangemak/web_server_8_dns d65baf205669c871d1216dc091edd1452a318b6522388e045c211344815c280a user@docker1:~$ user@docker1:~$ docker exec web8 host www.google.com www.google.com has address 216.58.216.196 www.google.com has IPv6 address 2607:f8b0:4009:80e::2004 user@docker1:~ $ It would appear that the container has the ability to resolve DNS names. If we look at our local Docker host and run the same test, we should get similar results: user@docker1:~$ host www.google.com www.google.com has address 216.58.216.196 www.google.com has IPv6 address 2607:f8b0:4009:80e::2004 user@docker1:~$ In addition, just like our Docker host, the container can also resolve local DNS records associated with the local domain lab.lab: user@docker1:~$ docker exec web8 host docker4 docker4.lab.lab has address 192.168.50.102 user@docker1:~$ You'll notice that we didn't need to specify a fully qualified domain name in order to resolve the host name docker4 in the domain lab.lab. At this point it's safe to assume that the container is receiving some sort of intelligent update from the Docker host which provides it relevant information about the local DNS configuration. In case you don't know, the resolv.conf file is generally where you define a Linux system's name resolution parameters. In many cases it is altered automatically by configuration information in other places. However – regardless of how it's altered, it should always be the source of truth for how the system handles name resolution. To see what the container is receiving, let's examine the containers resolv.conf file: user@docker1:~$ docker exec -t web8 more /etc/resolv.conf :::::::::::::: /etc/resolv.conf :::::::::::::: # Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8) # DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN nameserver 10.20.30.13 search lab.lab user@docker1:~$ As you can see, the container has learned that the local DNS server is 10.20.30.13 and that the local DNS search domain is lab.lab. Where did it get this information? The solution is rather simple. When a container starts, Docker generates instances of the following three files for each container spawned and saves it with the container configuration: /etc/hostname /etc/hosts /etc/resolv.conf These files are stored as part of the container configuration and then mounted into the container. We can use findmnt tool from within the container to examine the source of the mounts: root@docker1:~# docker exec web8 findmnt -o SOURCE …<Additional output removed for brevity>… /dev/mapper/docker1--vg-root[/var/lib/docker/containers/c803f130b7a2450609672c23762bce3499dec9abcfdc540a43a7eb560adaf62a/resolv.conf /dev/mapper/docker1--vg-root[/var/lib/docker/containers/c803f130b7a2450609672c23762bce3499dec9abcfdc540a43a7eb560adaf62a/hostname] /dev/mapper/docker1--vg-root[/var/lib/docker/containers/c803f130b7a2450609672c23762bce3499dec9abcfdc540a43a7eb560adaf62a/hosts] root@docker1:~# So while the container thinks it has local copies of the hostname, hosts, and resolv.conf, file in its /etc/ directory, the real files are actually located in the containers configuration directory (/var/lib/docker/containers/) on the Docker host. When you tell Docker to run a container, it does 3 things: It examines the Docker hosts /etc/resolv.conf file and places a copy of it in the containers directory. It creates a hostname file in the containers directory and assigns the container a unique hostname. It creates a hosts file in the containers directory and adds relevant records including localhost and a record referencing the host itself. Each time the container is restarted, the container's resolv.conf file is updated based on the values found in the Docker hosts resolv.conf file. This means that any changes made to the resolv.conf file are lost each time the container is restarted. The hostname and hosts configuration files are also rewritten each time the container is restarted losing any changes made during the previous run. To validate the configuration files a given container is using we can inspect the containers configuration for these variables: user@docker1:~$ docker inspect web8 | grep HostsPath "HostsPath": "/var/lib/docker/containers/c803f130b7a2450609672c23762bce3499dec9abcfdc540a43a7eb560adaf62a/hosts", user@docker1:~$ docker inspect web8 | grep HostnamePath "HostnamePath": "/var/lib/docker/containers/c803f130b7a2450609672c23762bce3499dec9abcfdc540a43a7eb560adaf62a/hostname", user@docker1:~$ docker inspect web8 | grep ResolvConfPath "ResolvConfPath": "/var/lib/docker/containers/c803f130b7a2450609672c23762bce3499dec9abcfdc540a43a7eb560adaf62a/resolv.conf", user@docker1:~$ As expected, these are the same mount paths we saw when we ran the findmnt command from within the container itself. These represent the exact mount path for each file into the containers /etc/ directory for each respective file. Overriding the default name resolution settings The method Docker uses for providing name resolution to containers works very well in most cases. However, there could be some instances where you want Docker to provide the containers with a DNS server other than the one the Docker host is configured to use. In these cases, Docker offers you a couple of options. You can tell the Docker service to provide a different DNS server for all of the containers the service spawns. You can also manually override this setting at container runtime by providing a DNS server as an option to the docker run subcommand. In this recipe, we'll show you your options for changing the default name resolution behavior as well as how to verify the settings worked. Getting Ready In this recipe we'll be demonstrating the configuration on a single Docker host. It is assumed that this host has Docker installed and that Docker is in its default configuration. We'll be altering name resolution settings on the host so you'll need root level access. How to do it… As we saw in the first recipe in this article, by default, Docker provides containers with the DNS server that the Docker host itself uses. This comes in the form of copying the host's resolv.conf file and providing it to each spawned container. Along with the name server setting, this file also includes definitions for DNS search domains. Both of these options can be configured at the service level to cover any spawned containers as well as at the individual level. For the purpose of comparison, let's start by examining the Docker hosts DNS configuration: root@docker1:~# more /etc/resolv.conf # Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8) # DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN nameserver 10.20.30.13 search lab.lab root@docker1:~# With this configuration, we would expect that any container spawned on this host would receive the same name server and DNS search domain. Let's spawn a container called web8 to verify this is working as expected: root@docker1:~# docker run -d -P --name=web8 jonlangemak/web_server_8_dns 156bc29d28a98e2fbccffc1352ec390bdc8b9b40b84e4c5f58cbebed6fb63474 root@docker1:~# root@docker1:~# docker exec -t web8 more /etc/resolv.conf :::::::::::::: /etc/resolv.conf :::::::::::::: # Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8) # DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN nameserver 10.20.30.13 search lab.lab As expected, the container receives the same configuration. Let's now inspect the container and see if we see any DNS related options defined: user@docker1:~$ docker inspect web8 | grep Dns "Dns": [], "DnsOptions": [], "DnsSearch": [], user@docker1:~$ Since we're using the default configuration, there is no reason to configure anything specific within the container in regards to DNS server or search domain. Each time the container starts, Docker will apply the settings for the hosts resolv.conf file to the containers DNS configuration files. If we'd prefer to have Docker give containers a different DNS server or DNS search domain, we can do so through Docker options. In this case, the two we're interested in are: --dns=<DNS Server> - Specify a DNS server address that Docker should provide to the containers. --dns-search=<DNS Search Domain> - Specify a DNS search domain that Docker should provide to the containers. Let's configure Docker to provide containers with a public DNS server (4.2.2.2) and a search domain of lab.external. We can do so by passing the following options to the Docker systemd drop-in file: ExecStart=/usr/bin/dockerd --dns=4.2.2.2 --dns-search=lab.external Once the options are configured, reload the systemd configuration, restart the service to load the new options, and restart our container web8: user@docker1:~$ sudo systemctl daemon-reload user@docker1:~$ sudo systemctl restart docker user@docker1:~$ docker start web8 web8 user@docker1:~$ docker exec -t web8 more /etc/resolv.conf search lab.external nameserver 4.2.2.2 user@docker1:~$ You'll note that despite this container initially having the hosts DNS server (10.20.30.13) and search domain (lab.lab) it now has the service level DNS options we just specified. If you recall earlier, we saw that when we inspected this container, it didn't define a specific DNS server or search domain. Since none was specified, Docker now uses the settings from the Docker options which take priority. While this provides some level of flexibility, it's not yet truly flexible. At this point any and all containers spawned on this server will be provided the same DNS server and search domain. To be truly flexible we should be able to have Docker alter the name resolution configuration on a per container level. As luck would have it, these options can also be provided directly at container runtime. The preceding image defines the priority Docker uses when deciding what name resolution settings to apply to a container when it's started. Settings defined at container runtime always take priority. If the settings aren't defined there, Docker then looks to see if they are configured at the service level. If the settings aren't there, it falls back to the default method of relying on the Docker hosts DNS settings. For instance, we can launch a container called web2 and provide different options: root@docker1:~# docker run -d --dns=8.8.8.8 --dns-search=lab.dmz -P --name=web8-2 jonlangemak/web_server_8_dns 1e46d66a47b89d541fa6b022a84d702974414925f5e2dd56eeb840c2aed4880f root@docker1:~# If we inspect the container, we'll see that we now have dns and dns-search fields defined as part of the container configuration: root@docker1:~# docker inspect web8-2 …<output removed for brevity>… "Dns": [ "8.8.8.8" ], "DnsOptions": [], "DnsSearch": [ "lab.dmz" ], …<output removed for brevity>… root@docker1:~# This ensures that if the container is restarted, it will still have the same DNS settings that were initially provided the first time the container was run. Let's make some slight changes to the Docker service to verify the priority is working as expected. Let's change our Docker options to look like this: ExecStart=/usr/bin/dockerd --dns-search=lab.external Now restart the service and run the following container: user@docker1:~$ sudo systemctl daemon-reload user@docker1:~$ sudo systemctl restart docker root@docker1:~# root@docker1:~# docker run -d -P --name=web8-3 jonlangemak/web_server_8_dns 5e380f8da17a410eaf41b772fde4e955d113d10e2794512cd20aa5e551d9b24c root@docker1:~# Since we didn't provide any DNS related options at container run time the next place we'd check would be the service level options. Our Docker service level options include a DNS search domain of lab.external, we'd expect the container to receive that search domain. However, since we don't have a DNS server defined, we'll need to fall back to the one configured on the Docker host itself. And now examine its resolv.conf file to make sure things worked as expected: user@docker1:~$ docker exec -t web8-3 more /etc/resolv.conf search lab.external nameserver 10.20.30.13 user@docker1:~$ Configuring Links for name and service resolution Container linking provides a means for one container to easily communicate with another container on the same host. As we've seen in previous examples, most container to container communication has occurred through IP addresses. Container linking improves on this by allowing linked containers to communicate with each other by name. In addition to providing basic name resolution, it also provides a means to see what services a linked container is providing. In this recipe we'll review how to create container links as well as discuss some of their limitations. Getting Ready In this recipe we'll be demonstrating the configuration on a single Docker host. It is assumed that this host has Docker installed and that Docker is in its default configuration. We'll be altering name resolution settings on the host so you'll need root level access. How to do it… The phrase container linking might imply to some that it involves some kind of network configuration or modification. In reality, container linking has very little to do with container networking. In the default mode, container linking provides a means for one container to resolve the name of another. For instance, let's start two containers on our lab host docker1: root@docker1:~# docker run -d -P --name=web1 jonlangemak/web_server_1 88f9c862966874247c8e2ba90c18ac673828b5faac93ff08090adc070f6d2922 root@docker1:~# docker run -d -P --name=web2 --link=web1 jonlangemak/web_server_2 00066ea46367c07fc73f73bdcdff043bd4c2ac1d898f4354020cbcfefd408449 root@docker1:~# Notice how when I started the second container I used a new flag called --link and referenced the container web1. We would now say that web2 was linked to web1. However, they're not really linked in any sort of way. A better description might be to say that web2 is now aware of web1. Let's connect to the container web2 to show you what I mean: root@docker1:~# docker exec -it web2 /bin/bash root@00066ea46367:/# ping web1 -c 2 PING web1 (172.17.0.2): 48 data bytes 56 bytes from 172.17.0.2: icmp_seq=0 ttl=64 time=0.163 ms 56 bytes from 172.17.0.2: icmp_seq=1 ttl=64 time=0.092 ms --- web1 ping statistics --- 2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max/stddev = 0.092/0.128/0.163/0.036 ms root@00066ea46367:/# It appears that the web2 container is now able to resolve the container web1 by name. This is because the linking process inserted records into the web2 containers hosts file: root@00066ea46367:/# more /etc/hosts 127.0.0.1 localhost ::1 localhost ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters 172.17.0.2 web1 88f9c8629668 172.17.0.3 00066ea46367 root@00066ea46367:/# With this configuration, the web2 container can reach the web1 container either by the name we gave the container at runtime (web1) or the unique hostname Docker generated for the container (88f9c8629668). In addition to the hosts file being updated, web2 also generates some new environmental variables: root@00066ea46367:/# printenv WEB1_ENV_APACHE_LOG_DIR=/var/log/apache2 HOSTNAME=00066ea46367 APACHE_RUN_USER=www-data WEB1_PORT_80_TCP=tcp://172.17.0.2:80 WEB1_PORT_80_TCP_PORT=80 LS_COLORS= WEB1_PORT=tcp://172.17.0.2:80 WEB1_ENV_APACHE_RUN_GROUP=www-data APACHE_LOG_DIR=/var/log/apache2 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PWD=/ WEB1_PORT_80_TCP_PROTO=tcp APACHE_RUN_GROUP=www-data SHLVL=1 HOME=/root WEB1_PORT_80_TCP_ADDR=172.17.0.2 WEB1_ENV_APACHE_RUN_USER=www-data WEB1_NAME=/web2/web1 _=/usr/bin/printenv root@00066ea46367:/# You'll notice many new environmental variables. Docker will copy any environmental variables from the linked container that were defined as part of the container. This includes: Environmental variables described in the docker image. More specifically, any ENV variables from the images Dockerfile Environmental variables passed to the container at runtime through the --env or -e flag. In this case, these three variables were defined as ENV variables in the images Dockerfile: APACHE_RUN_USER=www-data APACHE_RUN_GROUP=www-data APACHE_LOG_DIR=/var/log/apache2 Since both container images have the same ENV variables defined we'll see the local variables as well as the same environmental variables from the container web1 prefixed with WEB1_ENV_: WEB1_ENV_APACHE_RUN_USER=www-data WEB1_ENV_APACHE_RUN_GROUP=www-data WEB1_ENV_APACHE_LOG_DIR=/var/log/apache2 In addition, Docker also created 6 other environmental variables that describe the web1 container as well as any of its exposed ports: WEB1_PORT=tcp://172.17.0.2:80 WEB1_PORT_80_TCP=tcp://172.17.0.2:80 WEB1_PORT_80_TCP_ADDR=172.17.0.2 WEB1_PORT_80_TCP_PORT=80 WEB1_PORT_80_TCP_PROTO=tcp WEB1_NAME=/web2/web1 Linking also allows you to specify aliases. For instance let's stop, remove, and respawn container web2 using a slightly different syntax for linking… user@docker1:~$ docker stop web2 web2 user@docker1:~$ docker rm web2 web2 user@docker1:~$ docker run -d -P --name=web2 --link=web1:webserver jonlangemak/web_server_2 e102fe52f8a08a02b01329605dcada3005208d9d63acea257b8d99b3ef78e71b user@docker1:~$ Notice that after the link definition we inserted a :webserver. The name after the colon represents the alias for the link. In this case, I've specified an alias for the container web1 as webserver. If we examine the web2 container, we'll see that the alias is now also listed in the hosts file: root@c258c7a0884d:/# more /etc/hosts …<Additional output removed for brevity>… 172.17.0.2 webserver 88f9c8629668 web1 172.17.0.3 c258c7a0884d root@c258c7a0884d:/# Aliases also impact the environmental variables created during linking. Rather than using the container name they'll instead use the alias: user@docker1:~$ docker exec web2 printenv …<Additional output removed for brevity>… WEBSERVER_PORT_80_TCP_ADDR=172.17.0.2 WEBSERVER_PORT_80_TCP_PORT=80 WEBSERVER_PORT_80_TCP_PROTO=tcp …<Additional output removed for brevity>… user@docker1:~$ At this point you might be wondering how dynamic this is. After all, Docker is providing this functionality by updating static files in each container. What happens if a container's IP address changes? For instance, let's stop the container web1 and start a new container called web3 using the same image: user@docker1:~$ docker stop web1 web1 user@docker1:~$ docker run -d -P --name=web3 jonlangemak/web_server_1 69fa80be8b113a079e19ca05c8be9e18eec97b7bbb871b700da4482770482715 user@docker1:~$ If you'll recall from earlier, the container web1 had an IP address of 172.17.0.2 allocated to it. Since I stopped the container, Docker will release that IP address reservation making it available to be reassigned to the next container we start. Let's check the IP address assigned to the container web3: user@docker1:~$ docker exec web3 ip addr show dev eth0 79: eth0@if80: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP link/ether 02:42:ac:11:00:02 brd ff:ff:ff:ff:ff:ff inet 172.17.0.2/16 scope global eth0 valid_lft forever preferred_lft forever inet6 fe80::42:acff:fe11:2/64 scope link valid_lft forever preferred_lft forever user@docker1:~$ As expected, web3 took the now open IP address of 172.17.0.2 that previously belonged to the web1 container. We can also verify that the container web2 still believes that this IP address belongs to the web1 container: user@docker1:~$ docker exec –t web2 more /etc/hosts | grep 172.17.0.2 172.17.0.2 webserver 88f9c8629668 web1 user@docker1:~$ If we start the container web1 once again, we should see it will get a new IP address allocated to it: user@docker1:~$ docker start web1 web1 user@docker1:~$ docker exec web1 ip addr show dev eth0 81: eth0@if82: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP link/ether 02:42:ac:11:00:04 brd ff:ff:ff:ff:ff:ff inet 172.17.0.4/16 scope global eth0 valid_lft forever preferred_lft forever inet6 fe80::42:acff:fe11:4/64 scope link valid_lft forever preferred_lft forever user@docker1:~$ If we check the container web2 again, we should see that Docker has updated it to reference web1's new IP address… user@docker1:~$ docker exec web2 more /etc/hosts | grep web1 172.17.0.4 webserver 88f9c8629668 web1 user@docker1:~$ However, while Docker takes care of updating the host file with the new IP address, it will not take care of updating any of the environmental variables to reflect the new IP address: user@docker1:~$ docker exec web2 printenv …<Additional output removed for brevity>… WEBSERVER_PORT=tcp://172.17.0.2:80 WEBSERVER_PORT_80_TCP=tcp://172.17.0.2:80 WEBSERVER_PORT_80_TCP_ADDR=172.17.0.2 …<Additional output removed for brevity>… user@docker1:~$ Additionally it should be pointed out that the link is only one way. That is, this link does not cause the container web1 to become aware of the web2 container. The container web1 will not receive the host records or the environmental variables referencing the web2: user@docker1:~$ docker exec -it web1 ping web2 ping: unknown host user@docker1:~$ Another reason to provision links is when you use Docker Inter Container Connectivity (ICC) mode set to false. As we've discussed previously, ICC prevents any containers on the same bridge from talking directly to each other. This forces them to talk to each other only though published ports. Linking provides a mechanism to override the default ICC rules. To demonstrate, let's stop and remove all the containers on our host docker1 and then add the following Docker option to the systemd drop in file: ExecStart=/usr/bin/dockerd --icc=false Now reload the systemd configuration, restart the service, and start the following containers: docker run -d -P --name=web1 jonlangemak/web_server_1 docker run -d -P --name=web2 jonlangemak/web_server_2 With ICC mode on you'll notice containers can't talk directly to each other: user@docker1:~$ docker exec web1 ip addr show dev eth0 87: eth0@if88: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP link/ether 02:42:ac:11:00:02 brd ff:ff:ff:ff:ff:ff inet 172.17.0.2/16 scope global eth0 valid_lft forever preferred_lft forever inet6 fe80::42:acff:fe11:2/64 scope link valid_lft forever preferred_lft forever user@docker1:~$ docker exec -it web2 curl http://172.17.0.2 user@docker1:~$ In the preceding example, web2 is not able to access the web servers on web1. Now let's delete and recreate the web2 container this time linking it to web1: user@docker1:~$ docker stop web2 web2 user@docker1:~$ docker rm web2 web2 user@docker1:~$ docker run -d -P --name=web2 --link=web1 jonlangemak/web_server_2 4c77916bb08dfc586105cee7ae328c30828e25fcec1df55f8adba8545cbb2d30 user@docker1:~$ docker exec -it web2 curl http://172.17.0.2 <body> <html> <h1><span style="color:#FF0000;font-size:72px;">Web Server #1 - Running on port 80</span></h1> </body> </html> user@docker1:~$ We can see with the link in place the communication is allowed as expected. Once again, just like the link, this access is allowed in one direction. It should be noted that linking works differently when using user defined networks. In this recipe we covered what are now being called legacy links. Linking with user defined networks will be covered in the following recipes. Leveraging Docker DNS The introduction of user defined networks signaled a big change in Docker networking. While the ability to provision custom networks was the big news, there were also major enhancements in name resolution. User defined networks can benefit from what's being called embedded DNS. The Docker engine itself now has the ability to provide name resolution to all of the containers. This is a marked improvement from the legacy solution where the only means for name resolution was external DNS or linking which relied on the hosts file. In this recipe, we'll walk through how to use and configure embedded DNS. Getting Ready In this recipe we'll be demonstrating the configuration on a single Docker host. It is assumed that this host has Docker installed and that Docker is in its default configuration. We'll be altering name resolution settings on the host so you'll need root level access. How to do it… As mentioned, the embedded DNS system only works on user defined Docker networks. That being said, let's provision a user defined network and then start a simple container on it: user@docker1:~$ docker network create -d bridge mybridge1 0d75f46594eb2df57304cf3a2b55890fbf4b47058c8e43a0a99f64e4ede98f5f user@docker1:~$ docker run -d -P --name=web1 --net=mybridge1 jonlangemak/web_server_1 3a65d84a16331a5a84dbed4ec29d9b6042dde5649c37bc160bfe0b5662ad7d65 user@docker1:~$ As we saw in an earlier recipe, by default, Docker pulls the name resolution configuration from the Docker host and provides it to the container. This behavior can be changed by providing different DNS servers or search domains either at the service level or at container run time. In the case of containers connected to a user-defined network, the DNS settings provided to the container are slightly different. For instance, let's look at the resolv.conf file for the container we just connected to the user defined bridge mybridge1: user@docker1:~$ docker exec -t web1 more /etc/resolv.conf search lab.lab nameserver 127.0.0.11 options ndots:0 user@docker1:~$ Notice how the name server for this container is now 127.0.0.11. This IP address represents Docker's embedded DNS server and will be used for any container which is connected to a user-defined network. It is a requirement that any container connected to a user-defined use the embedded DNS server. Containers not initially started on a user defined network will get updated the moment they connect to a user defined network. For instance, let's start another container called web2 but have it use the default docker0 bridge: user@docker1:~$ docker run -dP --name=web2 jonlangemak/web_server_2 d0c414477881f03efac26392ffbdfb6f32914597a0a7ba578474606d5825df3f user@docker1:~$ docker exec -t web2 more /etc/resolv.conf :::::::::::::: /etc/resolv.conf :::::::::::::: # Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8) # DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN nameserver 10.20.30.13 search lab.lab user@docker1:~$ If we now connect the web2 container to our user-defined network, Docker will update the name server to reflect the embedded DNS server: user@docker1:~$ docker network connect mybridge1 web2 user@docker1:~$ docker exec -t web2 more /etc/resolv.conf search lab.lab nameserver 127.0.0.11 options ndots:0 user@docker1:~$ Since both our containers are now connected to the same user-defined network they can now reach each other by name: user@docker1:~$ docker exec -t web1 ping web2 -c 2 PING web2 (172.18.0.3): 48 data bytes 56 bytes from 172.18.0.3: icmp_seq=0 ttl=64 time=0.107 ms 56 bytes from 172.18.0.3: icmp_seq=1 ttl=64 time=0.087 ms --- web2 ping statistics --- 2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max/stddev = 0.087/0.097/0.107/0.000 ms user@docker1:~$ docker exec -t web2 ping web1 -c 2 PING web1 (172.18.0.2): 48 data bytes 56 bytes from 172.18.0.2: icmp_seq=0 ttl=64 time=0.060 ms 56 bytes from 172.18.0.2: icmp_seq=1 ttl=64 time=0.119 ms --- web1 ping statistics --- 2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max/stddev = 0.060/0.089/0.119/0.030 ms user@docker1:~$ You'll note that the name resolution is bidirectional and works inherently without the use of any links. That being said, with user defined networks, we can still define links for the purpose of creating local aliases. For instance, let's stop and remove both containers web1 and web2 and reprovision them as follows: user@docker1:~$ docker run -d -P --name=web1 --net=mybridge1 --link=web2:thesecondserver jonlangemak/web_server_1 fd21c53def0c2255fc20991fef25766db9e072c2bd503c7adf21a1bd9e0c8a0a user@docker1:~$ docker run -d -P --name=web2 --net=mybridge1 --link=web1:thefirstserver jonlangemak/web_server_2 6e8f6ab4dec7110774029abbd69df40c84f67bcb6a38a633e0a9faffb5bf625e user@docker1:~$ The first interesting item to point out is that Docker let us link to a container that did not yet exist. When we ran the container web1 we asked Docker to link it to the container web2. At that point, web2 didn't exist. This is a notable difference in how links work with the embedded DNS server. In legacy linking Docker needed to know the target containers information prior to making the link. This was because it had to manually update the source containers host file and environmental variables. The second interesting item is that aliases are no longer listed in the containers hosts file. If we look at the hosts file on each container we'll see that the linking no longer generates entries: user@docker1:~$ docker exec -t web1 more /etc/resolv.conf search lab.lab nameserver 127.0.0.11 options ndots:0 user@docker1:~$ docker exec -t web2 more /etc/resolv.conf search lab.lab nameserver 127.0.0.11 options ndots:0 user@docker1:~$ All of the resolution is now occurring in the embedded DNS server. This includes keeping track of defined aliases and their scope. So even without host records, each container is able to resolve the other containers alias through the embedded DNS server: user@docker1:~$ docker exec -t web1 ping thesecondserver -c2 PING thesecondserver (172.18.0.3): 48 data bytes 56 bytes from 172.18.0.3: icmp_seq=0 ttl=64 time=0.067 ms 56 bytes from 172.18.0.3: icmp_seq=1 ttl=64 time=0.067 ms --- thesecondserver ping statistics --- 2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max/stddev = 0.067/0.067/0.067/0.000 ms user@docker1:~$ docker exec -t web2 ping thefirstserver -c 2 PING thefirstserver (172.18.0.2): 48 data bytes 56 bytes from 172.18.0.2: icmp_seq=0 ttl=64 time=0.062 ms 56 bytes from 172.18.0.2: icmp_seq=1 ttl=64 time=0.042 ms --- thefirstserver ping statistics --- 2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max/stddev = 0.042/0.052/0.062/0.000 ms user@docker1:~$ The aliases created have a scope that is local to the container itself. For instance, a third container on the same user defined network is not able to resolve the aliases created as part of the links: user@docker1:~$ docker run -d -P --name=web3 --net=mybridge1 jonlangemak/web_server_1 d039722a155b5d0a702818ce4292270f30061b928e05740d80bb0c9cb50dd64f user@docker1:~$ docker exec -it web3 ping thefirstserver -c 2 ping: unknown host user@docker1:~$ docker exec -it web3 ping thesecondserver -c 2 ping: unknown host user@docker1:~$ You'll recall that legacy linking also automatically created a set of environmental variables on the source container. These environmental variables referenced the target container and any ports it might be exposing. Linking in user defined networks does not create these environmental variables: user@docker1:~$ docker exec web1 printenv PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin HOSTNAME=4eba77b66d60 APACHE_RUN_USER=www-data APACHE_RUN_GROUP=www-data APACHE_LOG_DIR=/var/log/apache2 HOME=/root user@docker1:~$ As we saw in the previous recipe, keeping these variables up to date wasn't achievable even with legacy links. That being said, it's not a total surprise the functionality doesn't exist when dealing with user defined networks. In addition to providing local container resolution, the embedded DNS server also handles any external requests. As we saw in the preceding example, the search domain from the Docker host (lab.lab in my case) was still being passed down to the containers and configured in their resolv.conf file. The name server learned from the host becomes a forwarder for the embedded DNS server. This allows the embedded DNS server to process any container name resolution requests and hand off external requests to the name server used by the Docker host. This behavior can be overridden either at the service level or by passing the --dns or --dns-search flag to a container at run time. For instance, we can start two more instances of the web1 container and specify a specific DNS server in either case: user@docker1:~$ docker run -dP --net=mybridge1 --name=web4 --dns=10.20.30.13 jonlangemak/web_server_1 19e157b46373d24ca5bbd3684107a41f22dea53c91e91e2b0d8404e4f2ccfd68 user@docker1:~$ docker run -dP --net=mybridge1 --name=web5 --dns=8.8.8.8 jonlangemak/web_server_1 700f8ac4e7a20204100c8f0f48710e0aab8ac0f05b86f057b04b1bbfe8141c26 user@docker1:~$ Note that web4 would receive 10.20.30.13 as a DNS forwarder even if we didn't specify it explicitly. This is because that's also the DNS server used by the Docker host and when not specified the container inherits from the host. It is specified here for the sake of the example. Now if we try to resolve a local DNS record on either container we can see that in the case of web1 it works since it has the local DNS server defined whereas the lookup on web2 fails because 8.8.8.8 doesn't know about the lab.lab domain: user@docker1:~$ docker exec -it web4 ping docker1.lab.lab -c 2 PING docker1.lab.lab (10.10.10.101): 48 data bytes 56 bytes from 10.10.10.101: icmp_seq=0 ttl=64 time=0.080 ms 56 bytes from 10.10.10.101: icmp_seq=1 ttl=64 time=0.078 ms --- docker1.lab.lab ping statistics --- 2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max/stddev = 0.078/0.079/0.080/0.000 ms user@docker1:~$ docker exec -it web5 ping docker1.lab.lab -c 2 ping: unknown host user@docker1:~$ Summary In this article we discussed the available options for container name resolution. This includes both the default name resolution behavior as well as the new embedded DNS server functionality that exists with user defined networks. You will get hold on the process used to determine name server assignment under each of these scenarios. Resources for Article: Further resources on this subject: Managing Application Configuration [article] Virtualizing Hosts and Applications [article] Deploying a Play application on CoreOS and Docker [article]