How-To Tutorials

In this article by Habib Ahmed Qureshi, Ganesan Senthilvel, and Ovais Mehboob Ahmed Khan, author of the book Enterprise Application Architecture with .NET Core, you will learn how to architect and design highly scalable, robust, clean, and highly performant applications in .NET Core 1.0. (For more resources related to this topic, see here.) In this article, we will cover the following topics: Why we need Enterprise Architecture? Knowing the role of an architect Why we need Enterprise Architecture? We will need to define, or at least provide, some basic fixed points to identify enterprise architecture specifically. Sketch Before playing an enterprise architect role, I used to get confused with so many architectural roles and terms, such as architect, solution architect, enterprise architect, data architect, blueprint, system diagram, and so on. In general, the industry perception is that the IT architect role is to draw few boxes with few suggestions; rest is with the development community. They feel that the architect role is quite easy just by drawing the diagram and not doing anything else. Like I said, it is completely a perception of few associates in the industry, and I used to be dragged by this category earlier: However, my enterprise architect job has cleared this perception and understands the true value of an enterprise architect. Definition of Enterprise Architecture In simple terms, enterprise is nothing but human endeavor. The objective of an enterprise is where people are collaborating for a particular purpose supported by a platform. Let me explain with an example of an online e-commerce company. Employees of that company are people who worked together to produce the profit of the firm using their various platforms, such as infrastructure, software, equipment, building, and so on. Enterprise has the structure/arrangements of all these pieces/components to build the complete organization. This is the exact place where enterprise architecture plays its key role. Every enterprise has an enterprise architect. EA is a process of architecting that applies the discipline to produce the prescribed output components. This process needs the experience, skill, discipline, and descriptions. Consider the following image where EA anticipates the system in two key states: Every enterprise needs an enterprise architect, not an optional. Let me give a simple example. When you need a car for business activities, you have two choices, either drive yourself or rent a driver. Still, you will need the driving capability to operate the car. EA is pretty similar to it. As depicted in the preceding diagram, EA anticipates the system in two key states, which are as follows: How it currently is How will it be in the future Basically, they work on options/alternatives to move from current to future state of an enterprise system. In this process, Enterprise Architecture does the following: Creates the frameworks to manage the architect Details the descriptions of the architect Roadmaps to lay the best way to change/improve the architecture Defines constraint/opportunity Anticipates the costs and benefits Evaluates the risks and values In this process of architecting, the system applies the discipline to produce the prescribed output components. Stakeholders of Enterprise Architecture Enterprise Architecture is so special because to its holistic view of management and evolution of an enterprise holistically. It has the unique combination of specialist technology, such as architecture frameworks and design pattern practices. Such a special EA has the following key stakeholders/users in its eco system: S.No. Stakeholders Organizational actions 1  Strategic planner Capability planning Set strategic direction Impact analysis 2  Decision makers Investment Divestment Approvals for the project Alignment with strategic direction 3  Analyst Quality assurance Compliance Alignment with business goals 4  Architects, project managers Solution development Investigate the opportunities Analysis of the existing options Business benefits Though many organizations intervened without EAs, every firm has the strong belief that it is better to architect before creating any system. It is integrated in coherent fashion with proactively designed system instead of random ad hoc and inconsistent mode. In terms of business benefits, cost is the key factor in the meaning of Return on Investment (RoI). That is how the industry business is driven in this highly competitive IT world. EA has the opportunity to prove its value for its own stakeholders with three major benefits, ranging from tactical to strategic positions. They are as follows: Cost reduction by technology standardization Business Process Improvement (BPI) Strategic differentiation Gartner's research paper on TCO: The First Justification for Enterprise IT Architecture by Colleen Young is one of the good references to justify the business benefits of an Enterprise Architecture. Check out https://www.gartner.com/doc/388268/enterprise-architecture-benefits-justification for more information. In the grand scheme of cost saving strategy, technology standardization adds a lot of efficiency to make the indirect benefits. Let me share my experience in this space. In one of my earlier legacy organization, it was noticed that the variety of technologies and products were built to server the business purpose due to the historical acquisitions and mergers. The best solution was platform standardization. All businesses have processes; few life examples are credit card processing, employee on-boarding, student enrollment, and so on. In this methodology, there are people involved with few steps for the particular system to get things done. In means of the business growth, the processes become chaotic, which leads to the duplicate efforts across the departments. Here, we miss the cross learning of the mistakes and corrections. BPI is an industry approach that is designed to support the enterprise for the realignment of the existing business operational process into the significant improved process. It helps the enterprise to identify and adopt in a better way using the industry tools and techniques. BPI is originally designed to induce a drastic game changing effect in the enterprise performance instead of bringing the changes in the incremental steps. In the current highly competitive market, strategic differentiation efforts make the firm create the perception in customers minds of receiving something of greater value than offered by the competition. An effective differentiation strategy is the best tool to highlight a business's unique features and make it stand out from the crowd. As the outcome of strategic differentiation, the business should realize the benefits on Enterprise Architecture investment. Also, it makes the business to institute the new ways of thinking to add the new customer segments along with new major competitive strategies. Knowing the role of an architect When I planned to switch my career to architecture track, I had too many questions in mind. People were referring to so many titles in the industry, such as architect, solution architect, enterprise architect, data architect, infra architect, and so on that I didn't know where exactly do I need to start and end. Industry had so many confusions to opt for. To understand it better, let me give my own work experiences as the best use cases. In the IT industry, two higher-level architects are named as follows: Solution architect (SA) Enterprise architect (EA) In my view, Enterprise Architecture is a much broader discipline than Solution Architecture with the sum of Business Architecture, Application Architecture, Data Architecture, and Technology Architecture. It will be covered in detail in the subsequent section: SA is focused on a specific solution and addresses the technological details that are compiled to the standards, roadmaps, and strategic objectives of the business. On comparing with SA, EA is at senior level. In general, EA takes a strategic, inclusive, and long term view at goals, opportunities, and challenges facing the company. However, SA is assigned to a particular project/program in an enterprise to ensure technical integrity and consistency of the solution at every stage of its life cycle. Role comparison between EA and SA Let me explain the working experiences of two different roles—EA and SA. When I played the SA role for Internet based telephony system, my role needs to build the tools, such as code generation, automation, and so on around the existing telephony system. It needs the skill set of the Microsoft platform technology and telephony domain to understand the existing system in a better way and then provide the better solution to improve the productivity and performance of the existing ecosystem. I was not really involved in the enterprise-level decision making process. Basically, it was pretty much like an individual contributor to build effective and efficient solutions to improvise the current system. As the second work, let me share my experience on the EA role for a leading financial company. The job was to build the enterprise data hub using the emerging big data technology. Degree of comparisons If we plot EA versus SA graphically, EA needs the higher degree of strategy focus and technology breath, as depicted in the following image: In terms of roles and responsibilities, EA and SA differ in their scope. Basically, the SA scope is limited within a project team and the expected delivery is to make the system quality of the solution to the business. In the same time, the EA scope is beyond SA by identifying or envisioning the future state of an organization. Summary In this article, you understood the fundamental concepts of enterprise architecture, and its related business need and benefits. Resources for Article: Further resources on this subject: Getting Started with ASP.NET Core and Bootstrap 4 [Article] Setting Up the Environment for ASP.NET MVC 6 [Article] How to Set Up CoreOS Environment [Article]

In this article by Edward Snajder, the author of Raspberry Pi Zero Cookbook, we pick up from having our operating system installed and our Raspberry Pi Zero on our home network. We can now dive into some basic Linux commands. You will find knowing these commands useful any time you are working on a Linux machine. In this article, we'll start prepping with some Linux recipes: (For more resources related to this topic, see here.) Navigating a filesystem and viewing and searching the contents of a directory Creating a new file, editing it in an editor, and changing ownership Renaming and copying/moving the file/folder into a new directory Installing and uninstalling a program Navigating a filesystem and viewing and searching the contents of a directory If you aren’t already a Linux or Mac user, getting around the filesystem can seem pretty alien at first. Truly, if you’ve only used Windows Explorer, this is going to seem like a strange, alien process. Once you start getting the hang of things, though, you’ll find that getting around the Linux filesystem is easy and fun. Getting ready The only thing you need to get started is a client connection to your Raspberry Pi Zero. I like to use SSH, but you can certainly connect using the serial connection or a terminal in X Windows. How to do it… If you want to find out where you are in the filesystem, use pwd: pi@rpz14101:~$ pwd /home/pi This tells me I’m in the /home/pi directory, which is the default home directory for the pi user. Generally, every user you create should get a /home/username directory to keep their own files in. This can be done automatically with user creation and the adduser command. To look at the contents of the directory you are in, use the ls command: pi@rpz14101:~$ ls Desktop Downloads Pictures python_games share Videos Documents Music Public Scratch Templates To look in another directory, simply specify the directory you want to list (you may need to use sudo depending on where you are looking): pi@rpz14101:~$ sudo ls /opt/ cookbook.share pigpio sonic-pi vc minecraft-pi share testsudo.deleteme Wolfram This is a nice quick summary of what files are in the directory, but you will usually want a little more information about the files and directories. The ls command has a ton of options, all of which can be displayed with ls –help and explained in more detail with man ls. Some of the best ones to know are as follows: -a show all files (regular and hidden) -l show long format (more file information, in columns) -h human readable (turns bytes into MB or GB as appropriate) -t or -tr order in time order, or reverse time order My typical command when I start looking in a directory is this one: ls -ltrh This produces all non-hidden files, with human-readable sizes, in column format, and the newest file at the bottom: pi@rpz14101:~$ ls -ltrh /opt/ total 513M drwxr-xr-x 7 root root 4.0K May 27 04:11 vc drwxr-xr-x 3 root root 4.0K May 27 04:32 Wolfram drwxr-xr-x 3 root root 4.0K May 27 04:34 pigpio drwxr-xr-x 4 root root 4.0K May 27 04:36 minecraft-pi drwxr-xr-x 5 root root 4.0K May 27 04:36 sonic-pi -rw-r--r-- 1 root root 0 Jul 4 13:41 testsudo.deleteme drwxr-xr-x 2 root root 4.0K Jul 9 13:05 share -rwxr-xr-x 1 root root 512M Jul 24 17:53 cookbook.share One last trick: if you need this format but there are a lot of files in the directory you are searching, you will see a ton of text scroll by. Maybe you just need the most recent or largest files? We can do this with a pipe (|) and the tail command. Let’s take a directory with a lot of files, such as /usr/lib/. To list the five most recently modified files, I can pipe ls -ltrh to the tail command: pi@rpz14101:~$ ls -ltrh /usr/lib/ | tail -5 lrwxrwxrwx 1 root root 22 May 27 04:40 libwiringPiDev.so -> libwiringPiDev.so.2.32 drwxr-xr-x 2 root root 4.0K Jun 5 10:38 samba drwxr-xr-x 3 root root 4.0K Jul 4 22:48 pppd drwxr-xr-x 65 root root 60K Jul 24 15:48 arm-linux-gnueabihf drwxr-xr-x 2 root root 4.0K Jul 24 15:48 tmpfiles.d What about the five largest files? Instead of the t in -ltrh, I can use S: pi@rpz14101:~$ ls -lSrh /usr/lib/ | tail -5 -rw-r--r-- 1 root root 2.8M Sep 17 2014 libmozjs185.so.1.0.0 -rw-r--r-- 1 root root 2.8M Sep 30 2014 libqscintilla2.so.11.3.0 -rw-r--r-- 1 root root 2.9M Jun 5 2014 libcmis-0.4.so.4.0.1 -rw-r--r-- 1 root root 3.4M Jun 12 2015 libv8.so.3.14.5 -rw-r--r-- 1 root root 5.1M Aug 18 2014 libmwaw-0.3.so.3.0.1 A little creative piping and you can find exactly the file you are looking for. If not, another great tool for exploring the filesystem is tree. This gives a pseudo-graphical tree that shows how the files are structured in the system. It produces a lot of text, especially if you have it print an entire directory tree. If just looking into directory structures, you can use tree with the -d flag for directories only. The -L flag will reduce how deep you dive into nested directories: pi@rpz14101:~$ tree -d -L 2 /opt/ /opt/ ├── minecraft-pi │ ├── api │ └── data ├── pigpio │ └── cgi ├── share ├── sonic-pi │ ├── app │ ├── bin │ └── etc ├── vc │ ├── bin │ ├── include │ ├── lib │ ├── sbin │ └── src └── Wolfram └── WolframEngine Last, we will look at a couple of searching utilities, find and grep. The find command is a powerful function that finds files in whatever directories you specify. It is great for trying to find that mystery piece of software that installed itself in an odd place or the needle-in-a-haystack file in a directory that contains hundreds of files. For example, if I were to run tree in the /opt/sonic-pi/ directory, it would run on for several seconds, and thousands of files would shoot by. I, however, am only interested in finding files with cowbell in the name. I can use the find command to look for it: pi@rpz14101:~$ find /opt/sonic-pi/ -name *cowbell* /opt/sonic-pi/etc/samples/drum_cowbell.flac When looking for anything with cowbell in the filename, the find command returns the exactly location of anything that matches. There are tons of options for using the find command; start with find –help, and then try man find when you want to get really deep. The grep command can be used in a couple different ways when searching for files, and it is one of those commands you will find yourself using constantly while both loving and hating its awesome power. Let’s say you need to find something inside of a file—grep is the tool for you. It can also find things like find can, but generally, find is more efficient at finding filename patterns than grep is. If I use grep to look for cowbells in my sonic-pi directory, I’ll get a different, and more colorful, output: We don’t see the file with cowbell in the name like we did using find, but we find every file that contains cowbell inside of it. The -r flag tells grep to delve into subdirectories, and -i tells it to ignore cases with cowbells (so Cowbell and cowbell are both found, as shown in the screenshot). As you use Linux more often, both find and grep become regularly used tools for administration and file management. This won’t be the last time you use them! Creating a new file, editing it in an editor, and changing ownership There are a lot of different text editors to use on a Linux system from the command line. The program vi is the Ubuntu default, and the program you will find installed on pretty much any Linux system. Emacs is another popular editor, and lots of Linux users get quite passionate about which one is better. My preference is vim, which is generally known as vi improved. The nano text editor is another one that is commonly installed on Linux distros, and it is one of the most lightweight editors available. Getting ready For this recipe, we will work with vi, since that’s definitely going to be installed on your system. If you want to try out vim, you can install it using this: sudo apt-get vim How to do it… First we will go to our share directory: cd /home/pi/share Then, we will create an empty file using the touch command: touch ch3_touchfile.txt If you use the ls command from the previous directory, you can see that the size of the file is 0. You can also display the contents of the file with the cat command, which will return nothing in this case. The touch command is a great way to test whether you have permissions to create files in a specific directory. You can also create a new file with the editor itself: vi ch3_vifile.txt This will open the vi editor with a blank file named ch3_vifile.txt: Using vi or vim (or Emacs) for the first time is completely different from using something like OpenOffice or Microsoft Word. Vi works in two modes: insert (or edit) and command. Once you learn how to use command mode, vi becomes a very efficient editor for working on scripts in bash or Python. Edit mode, more or less, is the mode where you can type and edit text like a regular WYSIWYG editor. There are books written on becoming a power user of vi, well beyond the scope of this book. Getting a handle on the basics is the best place to start: With the empty file, you can jump into edit mode by pressing the i or a keys. The editor will switch to insert mode, as shown by the -- INSERT -- in the bottom left of the screen. Then you can you start typing in your text: To get out of insert mode, press the Esc key. The :w command will save the file, and the :q command will quit. You can combine them, so :wq saves the file and quits. You can verify that the contents were saved with the cat command: pi@rpz14101:~$ cat ch3_vifile.txt Hello from the Raspberry Pi Zero Cookbook! Let’s take another look at the ls command and some of the information the -l format includes. We will take a look at the files we’ve created so far in this recipe: pi@rpz14101:~$ ls -ltrh *.txt -rw-r--r-- 1 pi pi 43 Jul 25 11:23 ch3_vifile.txt -rw-r--r-- 1 pi pi 0 Jul 25 11:24 ch3_touchfile.txt File Permissions Number of links Owner:Group Size Modification Date File Name -rw-r--r-- 1 pi:pi 43 Jul 25 11:23 ch3_vifile.txt We can see that since we made the files as the pi user, the owner of the file and the group owner are pi. By default, when a new user is created, a group container is created as well, so root has a root group, user rpz has an rpz group, and so on. We can change the ownership settings of a file with the chown command. Be careful, since you can take away your own access, though you can always sudo your way back. The chmod command will change who is allowed to do what with a file. Let’s look at ownership changes and what impact they will have with a few examples: pi@rpz14101:~ $ ls -ltrh *.txt -rw-r--r-- 1 pi pi 0 Jul 25 13:28 ch3_touchfile.txt -rwx------ 1 pi pi 43 Jul 25 13:28 ch3_vifile.txt pi@rpz14101:~ $ cat ch3_vifile.txt Hello from the Raspberry Pi Zero Cookbook! pi@rpz14101:~ $ sudo chown rpz:rpz ch3_vifile.txt pi@rpz14101:~ $ cat ch3_vifile.txt cat: ch3_vifile.txt: Permission denied pi@rpz14101:~ $ sudo cat ch3_vifile.txt Hello from the Raspberry Pi Zero Cookbook! pi@rpz14101:~ $ sudo chown rpz:pi ch3_vifile.txt pi@rpz14101:~ $ cat ch3_vifile.txt cat: ch3_vifile.txt: Permission denied pi@rpz14101:~ $ sudo chmod 750 ch3_vifile.txt pi@rpz14101:~ $ cat ch3_vifile.txt Hello from the Raspberry Pi Zero Cookbook! pi@rpz14101:~ $ sudo chown root:root ch3_vifile.txt pi@rpz14101:~ $ cat ch3_vifile.txt cat: ch3_vifile.txt: Permission denied pi@rpz14101:~ $ sudo chmod 755 ch3_vifile.txt pi@rpz14101:~ $ cat ch3_vifile.txt Hello from the Raspberry Pi Zero Cookbook! The chmod values are documented very well, and with a little practice, you can get your file permissions and ownership set up in a way that is both secure and easy to work with. Renaming and copying/moving the file/folder into a new directory A common activity on any filesystem is the practice of copying and moving files, and even directories, from one place to another. You might do it to make a backup copy of something, or you might decide that the contents should live in a more appropriate location. This recipe will explore how to manipulate files in the Raspbian system. Getting ready If you are still in your terminal from the last recipe, we are going to use the same files from the previous recipe. We should have the ownership back to pi:pi; if not, run the following: sudo chown pi:pi /home/pi/*.txt How to do it… First, let’s make a new directory. We’ll put it under the /home/pi/share/ folder so it is accessible to other computers on your home network. To make a directory, use the mkdir command: pi@rpz14101:~$ mkdir /home/pi/share/ch3 We can look at the new directory with the ls command: pi@rpz14101:~$ ls -ltrh /home/pi/share/ total 4.0K -rw-r--r-- 1 pi pi 0 Jul 24 15:56 helloNetwork.yes drwxr-xr-x 2 pi pi 4.0K Jul 25 13:06 ch3 A great flag to go with the mkdir command is -p. This will allow you to create directories and subdirectories in one command. Without it, if I try to create a subdirectory that doesn’t already exist, I’ll get an error: pi@rpz14101:~$ mkdir /home/pi/share/ch3/nested/folders mkdir: cannot create directory ‘/home/pi/share/ch3/nested/folders’: No such file or directory With the -p flag, it works without a problem: pi@rpz14101:~$ mkdir -p /home/pi/share/ch3/nested/folders The tree command shows the structure of our ch3 directory: pi@rpz14101:~$ tree /home/pi/share /home/pi/share ├── ch3 │ └── nested │ └── folders └── helloNetwork.yes 3 directories, 1 file Now, let’s move our files to our new ch3 directory. The copy and move commands—cp and mv, respectively—are the tools we will use. Copying a file from one place to another is as simple as indicating the file's source and destination. The following command will make a copy of vifile.txt and save it as vifile.txt.copy in the /home/pi/share/ch3/ directory: cp /home/pi/ch3_vifile.txt /home/pi/share/ch3/ch3_vifile.txt.copy We can copy files as well as directories and their contents as long as you have enough disk space. To move or rename a file, we use the mv command. This takes the file given in the source and moves it to the destination provided. As simple as the cp command, let’s move all of our files to the share directory: mv /home/pi/ch3_vifile.txt /home/pi/share/ch3/ch3_vifile.txt.moved mv /home/pi/ch3_touchfile.txt /home/pi/share/ch3/ch3_touchfile.txt If we look at the tree of our share directory, we will see everything nicely organized: pi@rpz14101:~$ tree /home/pi/share/ /home/pi/share/ ├── ch3 │ ├── ch3_touchfile.txt │ ├── ch3_vifile.txt.copy │ ├── ch3_vifile.txt.moved │ └── nested │ └── folders └── helloNetwork.yes 3 directories, 4 files Installing and uninstalling a program We’ve installed a few programs throughout the book so far, but have yet to delve into the apt-get command and the family of software-installation utilities. Now, we will learn how to install and uninstall any program available for Raspbian as well as how to search for new software and run updates. Getting ready Stay in your terminal window, and get ready to install some applications! How to do it… The apt-* commands are a suite of utilities that allow you to do various things with installed packages. To install a package, we use the apt-get tool, and the install command, like this: sudo apt-get install <packagename> Let’s install something cool—how about a Matrix screensaver? It is super easy and works great from the command line. To look for a package, we use the apt-cache search command. apt-cache is another tool in the apt-* family of utilities, and it checks the software database for matches. Running sudo apt-cache search matrix results tons of results! The word "matrix" is a little too popular for us computer and math nerds—we have matrixes everywhere! It would take forever to go through that list to find what we are looking for. Fortunately, we can take advantage of grep, which we touched on in an earlier recipe, to narrow down our results. One of the fun things about using Linux and the command line is the ways you can chain commands to do cool things: pi@rpz14101:~ $ sudo apt-cache search matrix | grep "The Matrix" cmatrix - simulates the display from "The Matrix" wmmatrix - View The Matrix in a Window Maker dock application That’s a bit more manageable! We could also have narrowed the list using this command: sudo apt-cache search “The Matrix” This returns fewer results than before, but a few more than the grep command. Whichever way you find it, we see that the cmatrix package is the one we are looking for. Installing is as simple as running this: pi@rpz14101:~ $ ` Reading package lists... Done Building dependency tree Reading state information... Done Suggested packages: cmatrix-xfont The following NEW packages will be installed: cmatrix 0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded. Need to get 16.2 kB of archives. After this operation, 27.6 kB of additional disk space will be used. Get:1 http://mirrordirector.raspbian.org/raspbian/ jessie/main cmatrix armhf 1.2a-5 [16.2 kB] Fetched 16.2 kB in 1s (15.3 kB/s) Selecting previously unselected package cmatrix. (Reading database ... 121906 files and directories currently installed.) Preparing to unpack .../cmatrix_1.2a-5_armhf.deb ... Unpacking cmatrix (1.2a-5) ... Processing triggers for man-db (2.7.0.2-5) ... Setting up cmatrix (1.2a-5) ... After that, we are ready to go! Channel your inner Neo and run this command: cmatrix -s -b You should be in the Matrix! Try it on your serial and SSH connections, and even in the terminal on VNC: you’ll notice differences in the rendering behavior. There are literally thousands of software packages available to install in the repositories of our awesome open source communities. Pretty much anything you think a computer should be able to do, someone, or a group of people, has worked on a solution and pushed it out to the repositories.  We will be using apt-get a lot throughout this cookbook; it is one of the commands you’ll find yourself using all the time as you get more interested in Raspberry Pis and the Linux operating system. Running sudo apt-get update will check all repositories to see whether there are any version updates available. Here, you can see all of the locations it checks to see whether there is anything new for Raspbian: pi@rpz14101:~ $ sudo apt-get update Get:1 http://archive.raspberrypi.org jessie InRelease [13.2 kB] Get:2 http://mirrordirector.raspbian.org jessie InRelease [14.9 kB] Get:3 http://archive.raspberrypi.org jessie/main armhf Packages [144 kB] Get:4 http://mirrordirector.raspbian.org jessie/main armhf Packages [8,981 kB] Hit http://archive.raspberrypi.org jessie/ui armhf Packages Ign http://archive.raspberrypi.org jessie/main Translation-en_GB Get:5 http://mirrordirector.raspbian.org jessie/contrib armhf Packages [37.5 kB] Ign http://archive.raspberrypi.org jessie/main Translation-en Get:6 http://mirrordirector.raspbian.org jessie/non-free armhf Packages [70.3 kB] Ign http://archive.raspberrypi.org jessie/ui Translation-en_GB Ign http://archive.raspberrypi.org jessie/ui Translation-en Get:7 http://mirrordirector.raspbian.org jessie/rpi armhf Packages [1,356 B] Ign http://mirrordirector.raspbian.org jessie/contrib Translation-en_GB Ign http://mirrordirector.raspbian.org jessie/contrib Translation-en Ign http://mirrordirector.raspbian.org jessie/main Translation-en_GB Ign http://mirrordirector.raspbian.org jessie/main Translation-en Ign http://mirrordirector.raspbian.org jessie/non-free Translation-en_GB Ign http://mirrordirector.raspbian.org jessie/non-free Translation-en Ign http://mirrordirector.raspbian.org jessie/rpi Translation-en_GB Ign http://mirrordirector.raspbian.org jessie/rpi Translation-en Fetched 9,263 kB in 34s (272 kB/s) Reading package lists... Done After updating, apt-get upgrade will look at the versions of everything you have installed and upgrade anything to the latest version if there is one available. Depending on how many updates you have, this can take quite a while: pi@rpz14101:~ $ sudo apt-get upgrade Reading package lists... Done Building dependency tree Reading state information... Done Calculating upgrade... Done The following packages will be upgraded: dpkg-dev gir1.2-gdkpixbuf-2.0 initramfs-tools libavcodec56 libavformat56 libavresample2 libavutil54 libdevmapper-event1.02.1 libdevmapper1.02.1 libdpkg-perl python-picamera python3-picamera raspberrypi-kernel raspberrypi-net-mods ssh tzdata xarchiver 40 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. Need to get 57.0 MB of archives. After this operation, 415 kB of additional disk space will be used. Do you want to continue? [Y/n] y Get:1 http://archive.raspberrypi.org/debian/ jessie/main nodered armhf 0.14.5 [5,578 kB] … Adding 'diversion of /boot/overlays/w1-gpio.dtbo to /usr/share/rpikernelhack/overlays/w1-gpio.dtbo by rpikernelhack' … run-parts: executing /etc/kernel/postrm.d/initramfs-tools 4.4.11-v7+ /boot/kernel7.img Preparing to unpack .../raspberrypi-net-mods_1.2.3_armhf.deb ... Unpacking raspberrypi-net-mods (1.2.3) over (1.2.2) ... Processing triggers for man-db (2.7.0.2-5) ... … Setting up libssl1.0.0:armhf (1.0.1t-1+deb8u2) ... Setting up libxml2:armhf (2.9.1+dfsg1-5+deb8u2) ... … Removing 'diversion of /boot/overlays/w1-gpio-pullup.dtbo to /usr/share/rpikernelhack/overlays/w1-gpio-pullup.dtbo by rpikernelhack' … Setting up raspberrypi-net-mods (1.2.3) ... Modified /etc/network/interfaces detected. Leaving unchanged and writing new file as interfaces.new. Processing triggers for libc-bin (2.19-18+deb8u4) ... Processing triggers for initramfs-tools (0.120+deb8u2) ... You don’t really have to understand the details of what’s going on during the upgrade, and it will let you know if there were any problems at the end (and often what to do to fix them). Regularly updating and upgrading will keep all of your software current with all of the latest bug fixes and security patches. There’s more… You can also add and remove software from the GUI. If you log on to your Pi, either directly to a monitor or over VNC Server (a recipe we covered earlier), you can find the Add / Remove Software option under Menu | Preferences: The PiPackages utility makes it very easy to find software when you only have a general idea of what you are looking for. While you can do the same things with the apt commands, if you are browsing, this is a little easier on the eyes. The utility provides categorizations so you don’t have to scroll through every package. Clicking on a package provides a detailed description: Simply check the box and click on Apply or OK, and the software will be installed. Now you can install software on your Raspberry Pi Zero from the command line or GUI. Summary In this article, we looked at the basic file manipulation functionalities of Linux on a Raspberry Pi. We saw how to navigate the filesystem and create, edit, rename, copy, and move files and folders. We also saw how to change ownership of a file and install and uninstall programs. Resources for Article: Further resources on this subject: Sending Notifications using Raspberry Pi Zero [Article] Raspberry Pi LED Blueprints [Article] Hacking a Raspberry Pi project? Understand electronics first! [Article]

In this article by Nathan Rozentals, author of the book Mastering TypeScript, Second Edition, you will learn how to build enterprise-ready, industrial web applications using TypeScript and leading JavaScript frameworks. In this article, we will cover the following topics: What is TypeScript? The benefits of TypeScript (For more resources related to this topic, see here.) What is TypeScript? TypeScript is both a language and a set of tools to generate JavaScript. It was designed by Anders Hejlsberg at Microsoft (the designer of C#) as an open source project to help developers write enterprise scale JavaScript. TypeScript generates JavaScript – it's as simple as that. Instead of requiring a completely new runtime environment, TypeScript generated JavaScript can reuse all of the existing JavaScript tools, frameworks, and wealth of libraries that are available for JavaScript. The TypeScript language and compiler, however, brings the development of JavaScript closer to a more traditional object-oriented experience. EcmaScript JavaScript as a language has been around for a long time, and is also governed by a language feature standard. The language defined in this standard is called ECMAScript, and each JavaScript interpreter must deliver functions and features that conform to this standard. The definition of this standard helped the growth of JavaScript and the web in general, and allowed websites to render correctly on many different browsers on many different operating systems. The ECMAScript standard was published in 1999 and is known as ECMA-262, third edition. With the popularity of the language, and the explosive growth of internet applications, the ECMAScript standard needed to be revised and updated. This process resulted in a draft specification for ECMAScript, called the fourth edition. Unfortunately, this draft suggested a complete overhaul of the language, and was not well received. Eventually, leaders from Yahoo, Google, and Microsoft tabled an alternate proposal which they called ECMAScript 3.1. This proposal was numbered 3.1, as it was a smaller feature set of the third edition, and sat between edition three and four of the standard. This proposal for language changes tabled earlier was eventually adopted as the fifth edition of the standard, and was called ECMAScript 5. The ECMAScript fourth edition was never published, but it was decided to merge the best features of both the fourth edition and the 3.1 feature set into a sixth edition named ECMAScript Harmony. The TypeScript compiler has a parameter that can switch between different versions of the ECMAScript standard. TypeScript currently supports ECMAScript 3, ECMAScript 5, and ECMAScript 6. When the compiler runs over your TypeScript, it will generate compile errors if the code you are attempting to compile is not valid for that standard. The team at Microsoft has committed to follow the ECMAScript standards in any new versions of the TypeScript compiler, so as new editions are adopted, the TypeScript language and compiler will follow suit. The benefits of TypeScript To give you a flavor of the benefits of TypeScript (and this is by no means the full list), let's have a very quick look at some of the things that TypeScript brings to the table: A compilation step Strong or static typing Type definitions for popular JavaScript libraries Encapsulation Private and public member variable decorators Compiling One of the most frustrating things about JavaScript development is the lack of a compilation step. JavaScript is an interpreted language, and therefore needs to be run in order to test that it is valid. Every JavaScript developer will tell horror stories of hours spent trying to find bugs in their code, only to find that they have missed a stray closing brace { , or a simple comma ,– or even a double quote " where there should have been a single quote ' . Even worse, the real headaches arrive when you misspell a property name, or unwittingly reassign a global variable. TypeScript will compile your code, and generate compilation errors where it finds these sorts of syntax errors. This is obviously very useful, and can help to highlight errors before the JavaScript is run. In large projects, programmers will often need to do large code merges and with today's tools doing automatic merges, it is surprising how often the compiler will pick up these types of errors. While tools to do this sort of syntax checking like JSLint have been around for years, it is obviously beneficial to have these tools integrated into your IDE. Using TypeScript in a continuous integration environment will also fail a build completely when compilation errors are found, further protecting your programmers against these types of bugs. Strong typing JavaScript is not strongly typed. It is a language that is very dynamic, as it allows objects to change their properties and behavior on the fly. As an example of this, consider the following code: var test = "this is a string"; test = 1; test = function(a, b) { return a + b; } On the first line of the preceding code, the variable test is bound to a string. It is then assigned a number, and finally is redefined to be a function that expects two parameters. Traditional object-oriented languages, however, will not allow the type of a variable to change, hence they are called strongly typed languages. While all of the preceding code is valid JavaScript and could be justified, it is quite easy to see how this could cause runtime errors during execution. Imagine that you were responsible for writing a library function to add two numbers, and then another developer inadvertently reassigned your function to subtract these numbers instead. These sorts of errors may be easy to spot in a few lines of code, but it becomes increasingly difficult to find and fix these as your code base and your development team grows. Another feature of strong typing is that the IDE you are working in understands what type of variable you are working with, and can bring better autocomplete or Intellisense options to the fore. TypeScript's syntactic sugar TypeScript introduces a very simple syntax to check the type of an object at compile time. This syntax has been referred to as syntactic sugar, or more formally, type annotations. Consider the following TypeScript code: var test: string = "this is a string"; test = 1; test = function(a, b) { return a + b; } Note on the first line of this code snippet, we have introduced a colon : and a string keyword between our variable and its assignment. This type annotation syntax means that we are setting the type of our variable to be of type string, and that any code that does not adhere to these rules will generate a compile error. Running the preceding code through the TypeScript compiler will generate two errors: hello.ts(3,1): error TS2322: Type 'number' is not assignable to type 'string'. hello.ts(4,1): error TS2322: Type '(a: any, b: any) => any' is not assignable to type 'string'. The first error is fairly obvious. We have specified that the variable test is a string, and therefore attempting to assign a number to it will generate a compile error. The second error is similar to the first, and is, in essence, saying that we cannot assign a function to a string. In this way, the TypeScript compiler introduces strong or static typing to your JavaScript code, giving you all of the benefits of a strongly typed language. TypeScript is therefore described as a superset of JavaScript. Type definitions for popular JavaScript libraries As we have seen, TypeScript has the ability to annotate JavaScript, and bring strong typing to the JavaScript development experience. But how do we strongly type existing JavaScript libraries? The answer is surprisingly simple—by creating a definition file. TypeScript uses files with a .d.ts extension as a sort of header file, similar to languages such as C++, to superimpose strongly typing on existing JavaScript libraries. These definition files hold information that describes each available function, and or variables, along with their associated type annotations. Let's have a quick look at what a definition would look like. As an example, I have lifted a function from the popular Jasmine unit testing framework, called describe: var describe = function(description, specDefinitions) { return jasmine.getEnv().describe(description, specDefinitions); }; Note that this function has two parameters—description and specDefinitions. But JavaScript does not tell us what sort of variables these are. We would need to have a look at the Jasmine documentation to figure out how to call this function. If we head over to http://jasmine.github.io/2.0/introduction.html, we will see an example of how to use this function: describe("A suite", function () { it("contains spec with an expectation", function () { expect(true).toBe(true); }); }); From the documentation, then, we can easily see that the first parameter is a string, and the second parameter is a function. But there is nothing in JavaScript that forces us to conform to this API. As mentioned before, we could easily call this function with two numbers or inadvertently switch the parameters around, sending a function first, and a string second. We will obviously start getting runtime errors if we do this, but TypeScript using a definition file can generate compile-time errors before we even attempt to run this code. Let's have a look at a piece of the jasmine.d.ts definition file: declare function describe( description: string, specDefinitions: () => void ): void; This is the TypeScript definition for the describe function. Firstly, declare function describe tells us that we can use a function called describe, but that the implementation of this function will be provided at runtime. Clearly, the description parameter is strongly typed to a string, and the specDefinitions parameter is strongly typed to be a function that returns void. TypeScript uses the double braces () syntax to declare functions, and the arrow syntax to show the return type of the function. So () => void is a function that does not return anything. Finally, the describe function itself will return void. If our code were to try and pass in a function as the first parameter, and a string as the second parameter (clearly breaking the definition of this function), as shown in the following example: describe(() => { /* function body */}, "description"); TypeScript will generate the following error: hello.ts(11,11): error TS2345: Argument of type '() => void' is not assignable to parameter of type 'string'. This error is telling us that we are attempting to call the describe function with invalid parameters,andit clearly shows that TypeScript will generate errors if we attempt to use external JavaScript libraries incorrectly. DefinitelyTyped Soon after TypeScript was released, Boris Yankov started a Git Hub repository to house definition files, called Definitely Typed (http://definitelytyped.org). This repository has now become the first port of call for integrating external libraries into TypeScript, and it currently holds definitions for over 1,600 JavaScript libraries. Encapsulation One of the fundamental principles of object-oriented programming is encapsulation—the ability to define data, as well as a set of functions that can operate on that data, into a single component. Most programming languages have the concept of a class for this purpose, providing a way to define a template for data and related functions. Let's first take a look at a simple TypeScript class definition: class MyClass { add(x, y) { return x + y; } } varclassInstance = new MyClass(); var result = classInstance.add(1,2); console.log(`add(1,2) returns ${result}`); This code is pretty simple to read and understand. We have created a class, named MyClass, with a simple add function. To use this class we simply create an instance of it, and call the add function with two arguments. JavaScript, unfortunately, does not have a class statement, but instead uses functions to reproduce the functionality of classes. Encapsulation through classes is accomplished by either using the prototype pattern, or by using the closure pattern. Understanding prototypes and the closure pattern, and using them correctly, is considered a fundamental skill when writing enterprise-scale JavaScript. A closure is essentially a function that refers to independent variables. This means that variables defined within a closure function remember the environment in which they were created. This provides JavaScript with a way to define local variables, and provide encapsulation. Writing the MyClass definition in the preceding code, using a closure in JavaScript, would look something like the following: varMyClass = (function () { // the self-invoking function is the // environment that will be remembered // by the closure function MyClass() { // MyClass is the inner function, // the closure } MyClass.prototype.add = function (x, y) { return x + y; }; return MyClass; }()); varclassInstance = new MyClass(); var result = classInstance.add(1, 2); console.log("add(1,2) returns " + result); We start with a variable called MyClass, and assign it to a function that is executed immediately note the })(); syntax near the bottom of the closure definition. This syntax is a common way to write JavaScript in order to avoid leaking variables into the global namespace. We then define a new function named MyClass, and return this new function to the outer calling function. We then use the prototype keyword to inject a new function into the MyClass definition. This function is named add and takes two parameters, returning their sum. The last few lines of the code show how to use this closure in JavaScript. Create an instance of the closure type, and then execute the add function. Running this code will log add(1,2) returns 3 to the console, as expected. Looking at the JavaScript code versus the TypeScript code, we can easily see how simple the TypeScript looks compared to the equivalent JavaScript. Remember how we mentioned that JavaScript programmers can easily misplace a brace {, or a bracket (? Have a look at the last line in the closure definition—})();. Getting one of these brackets or braces wrong can take hours of debugging to find. Public and private accessors A further object oriented principle that is used in Encapsulation is the concept of data hiding that is the ability to have public and private variables. Private variables are meant to be hidden to the user of a particular class as these variables should only be used by the class itself. Inadvertently exposing these variables can easily cause runtime errors. Unfortunately, JavaScript does not have a native way of declaring variables private. While this functionality can be emulated using closures, a lot of JavaScript programmers simply use the underscore character _ to denote a private variable. At runtime though, if you know the name of a private variable you can easily assign a value to it. Consider the following JavaScript code: varMyClass = (function() { function MyClass() { this._count = 0; } MyClass.prototype.countUp = function() { this._count ++; } MyClass.prototype.getCountUp = function() { return this._count; } return MyClass; }()); var test = new MyClass(); test._count = 17; console.log("countUp : " + test.getCountUp()); The MyClass variable is actually a closure with a constructor function, a countUp function, and a getCountUp function. The variable _count is supposed to be a private member variable that is used only within the scope of the closure. Using the underscore naming convention gives the user of this class some indication that the variable is private, but JavaScript will still allow you to manipulate the variable _count. Take a look at the second last line of the code snippet. We are explicitly setting the value of _count to 17 which is allowed by JavaScript, but not desired by the original creator of the class. The output of this code would be countUp : 17. TypeScript, however, introduces the public and private keywords that can be used on class member variables. Trying to access a class member variable that has been marked as private will generate a compile time error. As an example of this, the JavaScript code above can be written in TypeScript, as follows: class CountClass { private _count: number; constructor() { this._count = 0; } countUp() { this._count ++; } getCount() { return this._count; } } varcountInstance = new CountClass() ; countInstance._count = 17; On the second line of our code snippet, we have declared a private member variable named _count. Again, we have a constructor, a countUp, and a getCount function. If we compile this file, the compiler will generate an error: hello.ts(39,15): error TS2341: Property '_count' is private and only accessible within class 'CountClass'. This error is generated because we are trying to access the private variable _count in the last line of the code. The TypeScript compiler, therefore, is helping us to adhere to public and private accessors by generating a compile error when we inadvertently break this rule. Summary In this article, we took a quick look at what TypeScript is and what benefits it can bring to the JavaScript development experience. Resources for Article: Further resources on this subject: Introducing Object Oriented Programmng with TypeScript [article] Understanding Patterns and Architecturesin TypeScript [article] Writing SOLID JavaScript code with TypeScript [article]

In this article by, Daniel Reis, the author of the book Odoo 10 Development Essentials, we will create our first Odoo application and learn the steps needed make it available to Odoo and install it. (For more resources related to this topic, see here.) Inspired by the notable http://todomvc.com/ project, we will build a simple To-Do application. It should allow us to add new tasks, mark them as completed, and finally clear the task list from all the already completed tasks. Understanding applications and modules It's common to hear about Odoo modules and applications. But what exactly is the difference between them? Module add-ons are building blocks for Odoo applications. A module can add new features to Odoo, or modify existing ones. It is a directory containing a manifest, or descriptor file, named __manifest__.py, plus the remaining files that implement its features. Applications are the way major features are added to Odoo. They provide the core elements for a functional area, such as Accounting or HR, based on which additional add-on modules modify or extend features. Because of this, they are highlighted in the Odoo Apps menu. If your module is complex, and adds new or major functionality to Odoo, you might consider creating it as an application. If you module just makes changes to existing functionality in Odoo, it is likely not an application. Whether a module is an application or not is defined in the manifest. Technically is does not have any particular effect on how the add-on module behaves. It is only used for highlight on the Apps list. Creating the module basic skeleton We should have the Odoo server at ~/odoo-dev/odoo/. To keep things tidy, we will create a new directory alongside it to host our custom modules, at ~/odoo-dev/custom-addons. Odoo includes a scaffold command to automatically create a new module directory, with a basic structure already in place. You can learn more about it with: $ ~/odoo-dev/odoo/odoo-bin scaffold --help You might want to keep this in mind when you start working your next module, but we won't be using it right now, since we will prefer to manually create all the structure for our module. An Odoo add-on module is a directory containing a __manifest__.py descriptor file. In previous versions, this descriptor file was named __openerp__.py. This name is still supported, but is deprecated. It also needs to be Python-importable, so it must also have an __init__.py file. The module's directory name is its technical name. We will use todo_app for it. The technical name must be a valid Python identifier: it should begin with a letter and can only contain letters, numbers, and the underscore character. The following commands create the module directory and create an empty __init__.py file in it, ~/odoo-dev/custom-addons/todo_app/__init__.py. In case you would like to do that directly from the command line, this is what you would use: $ mkdir ~/odoo-dev/custom-addons/todo_app $ touch ~/odoo-dev/custom-addons/todo_app/__init__.py Next, we need to create the descriptor file. It should contain only a Python dictionary with about a dozen possible attributes; of this, only the name attribute is required. A longer description attribute and the author attribute also have some visibility and are advised. We should now add a __manifest__.py file alongside the __init__.py file with the following content: { 'name': 'To-Do Application', 'description': 'Manage your personal To-Do tasks.', 'author': 'Daniel Reis', 'depends': ['base'], 'application': True, } The depends attribute can have a list of other modules that are required. Odoo will have them automatically installed when this module is installed. It's not a mandatory attribute, but it's advised to always have it. If no particular dependencies are needed, we should depend on the core base module. You should be careful to ensure all dependencies are explicitly set here; otherwise, the module may fail to install in a clean database (due to missing dependencies) or have loading errors, if by chance the other required modules are loaded afterwards. For our application, we don't need any specific dependencies, so we depend on the base module only. To be concise, we chose to use very few descriptor keys, but in a real word scenario, we recommend that you also use the additional keys since they are relevant for the Odoo apps store: summary: This is displayed as a subtitle for the module. version: By default, is 1.0. It should follow semantic versioning rules (see http://semver.org/ for details). license: By default, is LGPL-3. website: This is a URL to find more information about the module. This can help people find more documentation or the issue tracker to file bugs and suggestions. category: This is the functional category of the module, which defaults to Uncategorized. The list of existing categories can be found in the security groups form (Settings | User | Groups), in the Application field drop-down list. These other descriptor keys are also available: installable: It is by default True but can be set to False to disable a module. auto_install: If the auto_install module is set to True, this module will be automatically installed, provided all its dependencies are already installed. It is used for the Glue modules. Since Odoo 8.0, instead of the description key, we can use a README.rst or README.md file in the module's top directory. A word about licenses Choosing a license for your work is very important, and you should consider carefully what is the best choice for you, and its implications. The most used licenses for Odoo modules are the GNU Lesser General Public License (LGLP) and the Affero General Public License (AGPL). The LGPL is more permissive and allows commercial derivate work, without the need to share the corresponding source code. The AGPL is a stronger open source license, and requires derivate work and service hosting to share their source code. Learn more about the GNU licenses at https://www.gnu.org/licenses/. Adding to the add-ons path Now that we have a minimalistic new module, we want to make it available to the Odoo instance. For that, we need to make sure the directory containing the module is in the add-ons path, and then update the Odoo module list. We will position in our work directory and start the server with the appropriate add-ons path configuration: $ cd ~/odoo-dev $ ./odoo/odoo-bin -d todo --addons-path="custom-addons,odoo/addons" --save The --save option saves the options you used in a config file. This spares us from repeating them every time we restart the server: just run ./odoo-bin and the last saved options will be used. Look closely at the server log. It should have an INFO ? odoo: addons paths:[...] line. It should include our custom-addons directory. Remember to also include any other add-ons directories you might be using. For instance, if you also have a ~/odoo-dev/extra directory containing additional modules to be used, you might want to include them also using the option: --addons-path="custom-addons,extra,odoo/addons" Now we need the Odoo instance to acknowledge the new module we just added. Installing the new module In the Apps top menu, select the Update Apps List option. This will update the module list, adding any modules that may have been added since the last update to the list. Remember that we need the developer mode enabled for this option to be visible. That is done in the Settings dashboard, in the link at the bottom right, below the Odoo version number information . Make sure your web client session is working with the right database. You can check that at the top right: the database name is shown in parenthesis, right after the user name. A way to enforce using the correct database is to start the server instance with the additional option --db-filter=^MYDB$. The Apps option shows us the list of available modules. By default it shows only application modules. Since we created an application module we don't need to remove that filter to see it. Type todo in the search and you should see our new module, ready to be installed. Now click on the module's Install button and we're ready! The Model layer Now that Odoo knows about our new module, let's start by adding a simple model to it. Models describe business objects, such as an opportunity, sales order, or partner (customer, supplier, and so on.). A model has a list of attributes and can also define its specific business. Models are implemented using a Python class derived from an Odoo template class. They translate directly to database objects, and Odoo automatically takes care of this when installing or upgrading the module. The mechanism responsible for this is Object Relational Model (ORM). Our module will be a very simple application to keep to-do tasks. These tasks will have a single text field for the description and a checkbox to mark them as complete. We should later add a button to clean the to-do list from the old completed tasks. Creating the data model The Odoo development guidelines state that the Python files for models should be placed inside a models subdirectory. For simplicity, we won't be following this here, so let's create a todo_model.py file in the main directory of the todo_app module. Add the following content to it: # -*- coding: utf-8 -*- from odoo import models, fields class TodoTask(models.Model): _name = 'todo.task' _description = 'To-do Task' name = fields.Char('Description', required=True) is_done = fields.Boolean('Done?') active = fields.Boolean('Active?', default=True) The first line is a special marker telling the Python interpreter that this file has UTF-8 so that it can expect and handle non-ASCII characters. We won't be using any, but it's a good practice to have it anyway. The second line is a Python import statement, making available the models and fields objects from the Odoo core. The third line declares our new model. It's a class derived from models.Model. The next line sets the _name attribute defining the identifier that will be used throughout Odoo to refer to this model. Note that the actual Python class name , TodoTask in this case, is meaningless to other Odoo modules. The _name value is what will be used as an identifier. Notice that this and the following lines are indented. If you're not familiar with Python, you should know that this is important: indentation defines a nested code block, so these four lines should all be equally indented. Then we have the _description model attribute. It is not mandatory, but it provides a user friendly name for the model records, that can be used for better user messages. The last three lines define the model's fields. It's worth noting that name and active are special field names. By default, Odoo will use the name field as the record's title when referencing it from other models. The active field is used to inactivate records, and by default, only active records will be shown. We will use it to clear away completed tasks without actually deleting them from the database. Right now, this file is not yet used by the module. We must tell Python to load it with the module in the __init__.py file. Let's edit it to add the following line: from . import todo_model That's it. For our Python code changes to take effect the server instance needs to be restarted (unless it was using the --dev mode). We won't see any menu option to access this new model, since we didn't add them yet. Still we can inspect the newly created model using the Technical menu. In the Settings top menu, go to Technical | Database Structure | Models, search for the todo.task model on the list and then click on it to see its definition: If everything goes right, it is confirmed that the model and fields were created. If you can't see them here, try a server restart with a module upgrade, as described before. We can also see some additional fields we didn't declare. These are reserved fields Odoo automatically adds to every new model. They are as follows: id: A unique, numeric identifier for each record in the model. create_date and create_uid: These specify when the record was created and who created it, respectively. write_date and write_uid: These confirm when the record was last modified and who modified it, respectively. __last_update: This is a helper that is not actually stored in the database. It is used for concurrency checks. The View layer The View layer describes the user interface. Views are defined using XML, which is used by the web client framework to generate data-aware HTML views. We have menu items that can activate the actions that can render views. For example, the Users menu item processes an action also called Users, that in turn renders a series of views. There are several view types available, such as the list and form views, and the filter options made available are also defined by particular type of view, the search view. The Odoo development guidelines state that the XML files defining the user interface should be placed inside a views/ subdirectory. Let's start creating the user interface for our To-Do application. Adding menu items Now that we have a model to store our data, we should make it available on the user interface. For that we should add a menu option to open the To-do Task model so that it can be used. Create the views/todo_menu.xml file to define a menu item and the action performed by it: <?xml version="1.0"?> <odoo> <!-- Action to open To-do Task list --> <act_window id="action_todo_task" name="To-do Task" res_model="todo.task" view_mode="tree,form" /> <!-- Menu item to open To-do Task list --> <menuitem id="menu_todo_task" name="Todos" action="action_todo_task" /> </odoo> The user interface, including menu options and actions, is stored in database tables. The XML file is a data file used to load those definitions into the database when the module is installed or upgraded. The preceding code is an Odoo data file, describing two records to add to Odoo: The <act_window> element defines a client-side window action that will open the todo.task model with the tree and form views enabled, in that order The <menuitem> defines a top menu item calling the action_todo_task action, which was defined before Both elements include an id attribute. This id , also called an XML ID, is very important: it is used to uniquely identify each data element inside the module, and can be used by other elements to reference it. In this case, the <menuitem> element needs to reference the action to process, and needs to make use of the <act_window> id for that. Our module does not know yet about the new XML data file. This is done by adding it to the data attribute in the __manifest__.py file. It holds the list of files to be loaded by the module. Add this attribute to the descriptor's dictionary: 'data': ['views/todo_menu.xml'], Now we need to upgrade the module again for these changes to take effect. Go to the Todos top menu and you should see our new menu option available: Even though we haven't defined our user interface view, clicking on the Todos menu will open an automatically generated form for our model, allowing us to add and edit records. Odoo is nice enough to automatically generate them so that we can start working with our model right away. Odoo supports several types of views, but the three most important ones are: tree (usually called list views), form, and search views. We'll add an example of each to our module. Creating the form view All views are stored in the database, in the ir.ui.view model. To add a view to a module, we declare a <record> element describing the view in an XML file, which is to be loaded into the database when the module is installed. Add this new views/todo_view.xml file to define our form view: <?xml version="1.0"?> <odoo> <record id="view_form_todo_task" model="ir.ui.view"> <field name="name">To-do Task Form</field> <field name="model">todo.task</field> <field name="arch" type="xml"> <form string="To-do Task"> <group> <field name="name"/> <field name="is_done"/> <field name="active" readonly="1"/> </group> </form> </field> </record> </odoo> Remember to add this new file to the data key in manifest file, otherwise our module won't know about it and it won't be loaded. This will add a record to the ir.ui.view model with the identifier view_form_todo_task. The view is for the todo.task model and is named To-do Task Form. The name is just for information; it does not have to be unique, but it should allow one to easily identify which record it refers to. In fact the name can be entirely omitted, in that case it will be automatically generated from the model name and the view type. The most important attribute is arch, and contains the view definition, highlighted in the XML code above. The <form> tag defines the view type, and in this case contains three fields. We also added an attribute to the active field to make it read-only. Adding action buttons Forms can have buttons to perform actions. These buttons are able to trigger workflow actions, run window actions—such as opening another form, or run Python functions defined in the model. They can be placed anywhere inside a form, but for document-style forms, the recommended place for them is the <header> section. For our application, we will add two buttons to run the methods of the todo.task model: <header> <button name="do_toggle_done" type="object" string="Toggle Done" class="oe_highlight" /> <button name="do_clear_done" type="object" string="Clear All Done" /> </header> The basic attributes of a button comprise the following: The string attribute that has the text to be displayed on the button The type attribute referring to the action it performs The name attribute referring to the identifier for that action The class attribute, which is an optional attribute to apply CSS styles, like in regular HTML The complete form view At this point, our todo.task form view should look like this: <form> <header> <button name="do_toggle_done" type="object" string="Toggle Done" class="oe_highlight" /> <button name="do_clear_done" type="object" string="Clear All Done" /> </header> <sheet> <group name="group_top"> <group name="group_left"> <field name="name"/> </group> <group name="group_right"> <field name="is_done"/> <field name="active" readonly="1" /> </group> </group> </sheet> </form> Remember that for the changes to be loaded to our Odoo database, a module upgrade is needed. To see the changes in the web client, the form needs to be reloaded: either click again on the menu option that opens it or reload the browser page (F5 in most browsers). The action buttons won't work yet, since we still need to add their business logic. The business logic layer Now we will add some logic to our buttons. This is done with Python code, using the methods in the model's Python class. Adding business logic We should edit the todo_model.py Python file to add to the class the methods called by the buttons. First we need to import the new API, so add it to the import statement at the top of the Python file: from odoo import models, fields, api The action of the Toggle Done button will be very simple: just toggle the Is Done? flag. For logic on records, use the @api.multi decorator. Here, self will represent a recordset, and we should then loop through each record. Inside the TodoTask class, add this: @api.multi def do_toggle_done(self): for task in self: task.is_done = not task.is_done return True The code loops through all the to-do task records, and for each one, modifies the is_done field, inverting its value. The method does not need to return anything, but we should have it to at least return a True value. The reason is that clients can use XML-RPC to call these methods, and this protocol does not support server functions returning just a None value. For the Clear All Done button, we want to go a little further. It should look for all active records that are done and make them inactive. Usually, form buttons are expected to act only on the selected record, but in this case, we will want it also act on records other than the current one: @api.model def do_clear_done(self): dones = self.search([('is_done', '=', True)]) dones.write({'active': False}) return True On methods decorated with @api.model, the self variable represents the model with no record in particular. We will build a dones recordset containing all the tasks that are marked as done. Then, we set on the active flag to False on them. The search method is an API method that returns the records that meet some conditions. These conditions are written in a domain, which is a list of triplets. The write method sets the values at once on all the elements of a recordset. The values to write are described using a dictionary. Using write here is more efficient than iterating through the recordset to assign the value to each of them one by one. Set up access security You might have noticed that upon loading, our module is getting a warning message in the server log: The model todo.task has no access rules, consider adding one. The message is pretty clear: our new model has no access rules, so it can't be used by anyone other than the admin super user. As a super user, the admin ignores data access rules, and that's why we were able to use the form without errors. But we must fix this before other users can use our model. Another issue yet to address is that we want the to-do tasks to be private to each user. Odoo supports row-level access rules, which we will use to implement that. Adding access control security To get a picture of what information is needed to add access rules to a model, use the web client and go to Settings | Technical | Security | Access Controls List: Here we can see the ACL for some models. It indicates, per security group, what actions are allowed on records. This information has to be provided by the module using a data file to load the lines into the ir.model.access model. We will add full access to the Employee group on the model. Employee is the basic access group nearly everyone belongs to. This is done using a CSV file named security/ir.model.access.csv. Let's add it with the following content: id,name,model_id:id,group_id:id,perm_read,perm_write,perm_create,perm_unlink acess_todo_task_group_user,todo.task.user,model_todo_task,base.group_user,1,1,1,1 The filename corresponds to the model to load the data into, and the first line of the file has the column names. These are the columns provided by the CSV file: id: It is the record external identifier (also known as XML ID). It should be unique in our module. name: This is a description title. It is only informative and it's best if it's kept unique. Official modules usually use a dot-separated string with the model name and the group. Following this convention, we used todo.task.user. model_id: This is the external identifier for the model we are giving access to. Models have XML IDs automatically generated by the ORM: for todo.task, the identifier is model_todo_task. group_id: This identifies the security group to give permissions to. The most important ones are provided by the base module. The Employee group is such a case and has the identifier base.group_user. The last four perm fields flag the access to grant read, write, create, or unlink (delete) access. We must not forget to add the reference to this new file in the __manifest__.py descriptor's data attribute It should look like this: 'data': [ 'security/ir.model.access.csv', 'views/todo_view.xml', 'views/todo_menu.xml', ], As before, upgrade the module for these additions to take effect. The warning message should be gone, and we can confirm that the permissions are OK by logging in with the user demo (password is also demo). If we run our tests now it they should only fail the test_record_rule test case. Summary We created a new module from the start, covering the most frequently used elements in a module: models, the three basic types of views (form, list, and search), business logic in model methods, and access security. Always remember, when adding model fields, an upgrade is needed. When changing Python code, including the manifest file, a restart is needed. When changing XML or CSV files, an upgrade is needed; also, when in doubt, do both: restart the server and upgrade the modules. Resources for Article: Further resources on this subject: Getting Started with Odoo Development [Article] Introduction to Odoo [Article] Web Server Development [Article]

In this article written by Suresh Kumar Gorakala, author of the book Building Recommendation Engines we will learn how to build a basic recommender system using R. In this article we will learn about various types of recommender systems in detail. This article explains neighborhood-similarity-based recommendations, personalized recommendation engines, model-based recommender systems and hybrid recommendation engines. Following are the different subtypes of recommender systems covered in this article: Neighborhood-based recommendation engines User-based collaborative filtering Item-based collaborative filtering Personalized recommendation engines Content-based recommendation engines Context-aware recommendation engines (For more resources related to this topic, see here.) Neighborhood-based recommendation engines As the name suggests, neighborhood-based recommender systems considers the preferences or likes of other users in the neighborhood before making suggestions or recommendations to the active user. While considering the preferences or tastes of neighbors, we first calculate how similar the other users are to the active user and then new items from more similar users are recommended to the user. Here the active user is the person to whom the system is serving recommendations. Since similarity calculations are involved these recommender systems are also called similarity-based recommender systems. Also since preferences or tastes are considered collaboratively from a pool of users these recommender systems are also called as collaborative filtering recommender systems. In this type of systems the main actors are the users, products and users preference information such as rating/ranking/liking towards the products. Preceding image is an example from Amazon showing collaborative filtering The collaborative filtering systems come in two flavors, They are: User-based collaborative filtering Item-based collaborative filtering Collaborative filtering When we have only the users interaction data of the products such as ratings, like/unlike, view/not viewed and we have to recommend new products then we have to choose Collaborative filtering approach. User-based Collaborative Filtering The basic intuition behind user-based collaborative filtering systems is, people with similar tastes in the past like similar items in future as well. For example, if user A and user B have very similar purchase history and if user A buys a new book which user B has not yet seen then we can suggest this book to User B as they have similar tastes. Item-based Collaborative filtering In this type of recommender systems unlike user-based collaborative filtering, we use similarity between items instead of similarity between users. Basic intuition for item-based recommender systems is if a User likes item A in past he might like item B which is similar to item A. In this approach instead of calculating similarity between users we calculate similarity between items or products. The most common similarity measure used for this approach is cosine similarity. Like user-based collaborative approach, we project the data into vector space and similarity between items is calculated using cosine angle between items. Similar to user-based collaborative filtering approach there are two steps for item-based collaborative approach. They are: Calculating the similarity between items. Predicting the ratings for the non rated item for a active user by making use of previous ratings given to other similar items Advantages of user-based collaborative filtering Easy to implement Neither the content information of the products nor users profile information is required for building recommendations New items are recommended to users giving Surprise factor to the users Disadvantages of user-based collaborative filtering This approach is computationally expensive as all the user information, product, rating information is loaded into the memory for similarity calculations. This approach fails for new users where we do not have any information about the users. This problem is called cold-start problem. This approach performs very poor if we have little data. Since we do not have content information about users or products. We cannot generate recommendations accurately only based on rating information only. Content-based recommender systems The recommendations are generated by considering only the rating or interaction information of the products by the users, that is suggesting new items for the active user are based on the ratings given to those new items by similar users to the active user. Assume the case of a person has given 4 star rating to a movie. In a collaborative filtering approach we only consider this rating information for generating recommendations. In real, a person rates a movie based on the features or content of the movie such as its genre, actor, director, story, and screenplay. Also the person watches a movie based on his personal choices. When we are building a recommendation engine to target users at personal level, the recommendations should not be based on the tastes of other similar people but should be based on the individual users’ tastes and the contents of the products. A recommendation which is targeted at personalized level and which considers individual preferences and contents of the products for generating recommendations are called content-based recommender systems. Another motivation for building content-based recommendation engines are they solve the cold start problem which new users face in collaborative filtering approach. When a new user comes based on the preferences of the person we can suggest new items which are similar to his tastes. Building content-based recommender systems involves three main steps as follows: Generating content information for products. Generating a user profile, preferences with respect to the features of the products. Generating recommendations predicting list of items which the user might like. Let us discuss each step in detail: Content extraction: In this step, we extract the features that represent the product. Most commonly the content of the products is represented in the vector space model with products name as rows and features as columns. User Profile generation: In this step, we build the user profile or preference matrix or vector space model matching the products content. Generating Recommendations: Now that, we have the generated the product content and user profile, the next step will be to generate the recommendations. Recommender systems using machine learning or any other mathematical, statistical models to generate recommendations are called as model-based systems Cosine similarity In this approach we first represent the user profiles and product content in vector forms and then we take cosine angle between each vector. The product which forms less angle with the user profile is considered as the most preferable item for the user. This approach is a standard approach while using neighborhood approach for Content based recommendations. Empirical studies shown that this approach gives more accurate results compared to other similarity measures. Classification-based approach Classification-based approaches fall under model-based recommender systems. In this approach, first we build a machine learning model by using the historical information, with user profile similar to the product content as input and the like/dislike of the product as output response classes. Supervised classification tasks such as logistic regression, KNN-classification methods, probabilistic methods and so on can be used. Advantages Content-based recommender systems are targeting at individual level Recommendations are generated using the user preferences alone unlike from user community as in collaborative filtering This approaches can be employed at real time as recommendation model doesn’t need to load the entire data for processing or generating recommendations Accuracy is high compared to collaborative approaches as they deal with the content of the products instead of rating information alone Cold start problem can be easily handled Disadvantages As the system is more personalized and the generated recommendations will become narrowed down to only user preferences with more and more user information comes into the system As a result, no new products that are not related to the user preferences will be shown to the user The user will not be able to look at what is happening around or what’s trending around Context-aware recommender Systems Over the years there has been evolution in recommender systems from neighborhood approaches to personalized recommender systems which are targeted to the individual users. These personalized recommender systems have become a huge success as this is useful at end user level and for organizations these systems become catalysts to increase their business. The personalized recommender systems, also called as content-based recommender systems are also getting evolved into Context aware recommender systems. Though the personalized recommender systems are targeted at individual user level and caters recommendations based on the personal preferences of the users, still there was scope to improve or refine the systems. Same person at different places might have different requirements. Likewise same person has different requirements at different times. Our intelligent recommender systems should be evolved enough to cater to the needs of the users for different places, at different times. Recommender System should be robust enough to suggest cotton shirts to a person during summer and suggesting Leather Jacket in winter. Similarly based on the time of the day suggesting Good restaurants serving a person’s personal choice breakfast and dinner would be very helpful. These kinds of recommender systems which considers location, time, mood, and so on that defines the context of user and suggests personalized recommendations are called context aware recommender systems. At broad level, context aware recommender systems are content-based recommenders with the inclusion of new dimension called context. In context aware systems, recommendations are generated in two steps: Generating list of recommendations of products for each user based on users’ preferences, that is content-based recommendations. Filtering out the recommendations that are specific to a current context. For example, based on past transaction history, interaction information, browsing patterns, ratings information on e-wallet mobile app, assume that User A is a movie lover, Sports lover, fitness freak. Using this information the content-based recommender systems generate recommendations of products such as Movie Tickets, 4G data offer for watching Football matches, Discount offers at GYM. Now based on the GPS co-ordinates of the mobile if the User A found to be at a 10K RUN marathon, then my Context aware recommendation engine will take this location information as the context and filters out the offers that are relevant to the current context and recommends Discount Offers at GYM to the user A. Most common approaches for building Context Aware Recommender systems are: Post filtering Approaches Pre-filtering approaches Pre-filtering approaches In pre-filtering approach, context information is applied to the User profile and product content. This step will filter out all the non relevant features and final personalized recommendations are generated on remaining feature set. Since filtering of features are made before generating personalized recommendations, these are called pre-filtering approaches. Post filtering approaches In post-filtering, firstly personalized recommendations are generated based on the user profile and product catalogue then the context information is applied for filtering out the relevant products to the user for the current context. Advantages Context aware systems are much advanced than the personalized content-based recommenders as these systems will be constantly in sync with user movements and generate recommendations as per current context. These systems are more real-time nature. Disadvantages Serendipity or surprise factor as in other personalized recommenders will be missing in this type of recommendations as well. Summary In this article, we have learned about popular recommendation engine techniques such as, collaborative filtering, content-based recommendations, context aware systems, hybrid recommendations, model-based recommendation systems with their advantages and disadvantages. Different similarity methods such as cosine similarity, Euclidean distance, Pearson-coefficient. Sub categories within each of the recommendations are also explained. Resources for Article: Further resources on this subject: Building a Recommendation Engine with Spark [article] Machine Learning Tasks [article] Machine Learning with R [article]

In this article by, Tejaswini Mandar Jog, author of the book, Learning Spring 5.0, we will cover the following topics: Introduction to Spring framework Problems address by Spring in enterprise application development Spring architecture What's new in Spring 5.0 Container Spring the fresh new start after the winter of traditional J2EE, is what Spring framework is in actual. A complete solution to the most of the problems occurred in handling the development of numerous complex modules collaborating with each other in a Java enterprise application. Spring is not a replacement to the traditional Java Development but it is a reliable solution to the companies to withstand in today's competitive and faster growing market without forcing the developers to be tightly coupled on Spring APIs. Problems addressed by Spring Java Platform is long term, complex, scalable, aggressive, and rapidly developing platform. The application development takes place on a particular version. The applications need to keep on upgrading to the latest version in order to maintain recent standards and cope up with them. These applications have numerous classes which interact with each other, reuse the APIs to take their fullest advantage so as to make the application is running smoothly. But this leads to some very common problems of as. Scalability The growth and development of each of the technologies in market is pretty fast both in hardware as well as software. The application developed, couple of years back may get outdated because of this growth in these areas. The market is so demanding that the developers need to keep on changing the application on frequent basis. That means whatever application we develop today should be capable of handling the upcoming demands and growth without affecting the working application. The scalability of an application is handling or supporting the handling of the increased load of the work to adapt to the growing environment instead of replacing them. The application when supports handling of increased traffic of website due to increase in numbers of users is a very simple example to call the application is scalable. As the code is tightly coupled, making it scalable becomes a problem. Plumbing code Let's take an example of configuring the DataSource in the Tomcat environment. Now the developers want to use this configured DataSource in the application. What will we do? Yes, we will do the JNDI lookup to get the DataSource. In order to handle JDBC we will acquire and then release the resources in try catch. The code like try catch as we discuss here, inter computer communication, collections too necessary but are not application specific are the plumbing codes. The plumbing code increases the length of the code and makes debugging complex. Boilerplate code How do we get the connection while doing JDBC? We need to register Driver class and invoke the getConnection() method on DriverManager to obtain the connection object. Is there any alternative to these steps? Actually NO! Whenever, wherever we have to do JDBC these same steps have to repeat every time. This kind of repetitive code, block of code which developer write at many places with little or no modification to achieve some task is called as boilerplate code. The boilerplate code makes the Java development unnecessarily lengthier and complex. Unavoidable non-functional code Whenever application development happens, the developer concentrate on the business logic, look and feel and persistency to be achieved. But along with these things the developers also give a rigorous thought on how to manage the transactions, how to handle increasing load on site, how to make the application secure and many more. If we give a close look, these things are not core concerns of the application but still these are unavoidable. Such kind of code which is not handling the business logic (functional) requirement but important for maintenance, trouble shooting, managing security of an application is called as non-functional code. In most of the Java application along with core concerns the developers have to write down non-functional code quite frequently. This leads to provide biased concentration on business logic development. Unit testing of the application Let's take an example. We want to test a code which is saving the data to the table in database. Here testing the database is not our motive, we just want to be sure whether the code which we have written is working fine or not. Enterprise Java application consists of many classes, which are interdependent. As there is dependency exists in the objects it becomes difficult to carry out the testing. POJO based development The class is a very basic structure of application development. If the class is getting extended or implementing an interface of the framework, reusing it becomes difficult as they are tightly coupled with API. The Plain Old Java Object (POJO) is very famous and regularly used terminology in Java application development. Unlike Struts and EJB Spring doesn't force developers to write the code which is importing or extending Spring APIs. The best thing about Spring is that developers can write the code which generally doesn't has any dependencies on framework and for this, POJOs are the favorite choice. POJOs support loosely coupled modules which are reusable and easy to test. The Spring framework is called to be non-invasive as it doesn't force the developer to use API classes or interfaces and allows to develop loosely coupled application. Loose coupling through DI Coupling, is the degree of knowledge in class has about the other. When a class is less dependent on the design of any other class, the class will be called as loosely coupled. Loose coupling can be best achieved by interface programming. In the Spring framework, we can keep the dependencies of the class separated from the code in a separate configuration file. Using interfaces and dependency injection techniques provided by Spring, developers can write loosely coupled code (Don't worry, very soon we will discuss about Dependency Injection and how to achieve it). With the help of loose coupling one can write a code which needs a frequent change, due to the change in the dependency it has. It makes the application more flexible and maintainable. Declarative programming In declarative programming, the code states what is it going to perform but not how it will be performed. This is totally opposite of imperative programming where we need to state stepwise what we will execute. The declarative programming can be achieved using XML and annotations. Spring framework keeps all configurations in XML from where it can be used by the framework to maintain the lifecycle of a bean. As the development happened in Spring framework, the 2.0 onward version gave an alternative to XML configuration with a wide range of annotations. Boilerplate code reduction using aspects and templates We just have discussed couple of pages back that repetitive code is boilerplate code. The boiler plate code is essential and without which providing transactions, security, logging, and so on, will become difficult. The framework gives solution of writing aspect which will deal with such cross cutting concerns and no need to write them along with business logic code. The use of aspect helps in reduction of boilerplate code but the developers still can achieve the same end effect. One more thing the framework provides, is the templates for different requirements. The JDBCTemplate and HibernateTemplate are couple of more useful concepts given by Spring which does reduction of boilerplate code. But as a matter of fact, you need to wait to understand and discover the actual potential. Layered architecture Unlike Struts and Hibernate which provides web persistency solutions respectively, Spring has a wide range of modules for numerous enterprise development problems. This layered architecture helps the developer to choose any one or more of the modules to write solution for his application in a coherent way. E.g. one can choose Web MVC module to handle web request efficiently without even knowing that there are many other modules available in the framework. Spring architecture Spring provides more than 20 different modules which can be broadly summaries under 7 main modules which are as follows: Spring modules What more Spring supports underneath? The following sections covers the additional features of Spring. Security module Now a days the applications alone with basic functionalities also need to provide sound ways to handle security at different levels. Spring5 support declarative security mechanism using Spring AOP. Batch module The Java Enterprise Applications needs to perform bulk processing, handling of large amount of data in many business solutions without user interactions. To handle such things in batches is the best solution available. Spring provides integration of batch processing to develop robust application. Spring integration In the development of enterprise application, the application may need interaction with them. Spring integration is extension of the core spring framework to provide integration of other enterprise applications with the help of declarative adapters. The messaging is one of such integration which is extensively supported by Spring. Mobile module The extensive use of mobiles opens the new doors in development. This module is an extension of Spring MVC which helps in developing mobile web applications known as Spring Android Project. It also provide detection of the type of device which is making the request and accordingly renders the views. LDAP module The basic aim of Spring was to simplify the development and to reduce the boilerplate code. The Spring LDAP module supports easy LDAP integration using template based development. .NEW module The new module has been introduced to support .NET platform. The modules like ADO.NET, NHibernate, ASP.NET has been in the .NET module includes to simplify the .NET development taking the advantages of features as DI, AOP, loose coupling. Container – the heart of Spring POJO development is the backbone of Spring framework. The POJO configured in the and whose object instantiation, object assembly, object management is done by Spring IoC container is called as bean or Spring bean. We use Spring IoC as it on the pattern of Inversion of Control. Inversion of Control (IoC) In every Java application, the first important thing which each developer does is, to get an object which he can use in the application. The state of an object can be obtained at runtime or it may be at compile time. But developers creates object where he use boiler plate code at a number of times. When the same developer uses Spring instead of creating object by himself he will be dependent on the framework to obtain object from. The term inversion of control comes as Spring container inverts the responsibility of object creation from developers. Spring IoC container is just a terminology, the Spring framework provides two containers: The BeanFactory The ApplicationContext Summary So in this article, we discussed about the general problems faced in Java enterprise application development and how they have been address by Spring framework. We have seen the overall major changes happened in each version of Spring from its first introduction in market. Enabling Spring Faces support Design with Spring AOP Getting Started with Spring Security

In this article by Mike Topalovich, author of the book Salesforce Lightning Application Development Essentials, we will discuss about Salesforce. As Salesforce developers, we know since Dreamforce '15, Salesforce has been all Lightning, all the time. The flagship CRM products – Sales Cloud and Service Cloud – have been rebranded to Sales Cloud Lightning and Service Cloud Lightning. In fact, many Salesforce products have undergone the Lightning treatment, the word Lightning being added to their product names overnight with few noticeable changes to the products themselves. This has led many in the Salesforce ecosystem to step back and ask, What is Lightning? (For more resources related to this topic, see here.) Lightning changes everything Lightning is not just the new Salesforce UI—it is a complete re-imagining of the entire Salesforce application and platform. Lightning represents a grand vision of unifying the Salesforce experience for everyone who interacts with Salesforce products or the Salesforce platform, on any device. In no uncertain terms, Lightning is the most important product update in the history of Salesforce as a company. Lightning represents a completely new vision for both the flagship CRM products and the platform. Salesforce is betting the company on it. Lightning changes not only how we interact with Salesforce, but how we design and develop solutions for the platform. Developers and ISV partners now have the ability to create rich, responsive applications using the same framework and design tools that Salesforce uses internally, ensuring that the user experience maintains a consistent look and feel across all applications. While the initial overuse of the term Lightning may be a source of confusion for many in the Salesforce ecosystem, don't let the noise drown out the vision. Lightning changes everything we know about Salesforce, but to understand how, we need to focus on the three key pillars of Lightning: Lightning Experience Salesforce Lightning Design System Lightning Component framework If we think about Lightning as the unified Salesforce experience across all our devices, it makes our mission much clearer. If we think about designing and developing responsive, reusable components for this new unified experience, Lightning makes a lot more sense. A brief history of Lightning The unified Lightning vision as we know it today has been rolled out in fits and starts. To understand how we arrived at the current vision for Lightning, we can look back on prior Dreamforce events as milestones in the history of Lightning. Dreamforce 2013 With Dreamforce '13, Salesforce recognized that the world was moving to a mobile-first mindset and rebranded their mobile application as Salesforce1. With Salesforce1, they also tried to sell the vision of a unified customer experience for the first time. According to a press release for Dreamforce '13, "Salesforce1 is a new social, mobile and cloud customer platform built to transform sales, service and marketing apps for the Internet of Customers." The vision was too ambitious at the time, the messaging was too confusing, and the platform was nowhere close to being ready to support any type of unified experience. Dreamforce 2014 Lightning emerged as a platform with Dreamforce '14. Branded as Salesforce1 Lightning, Salesforce opened up the underlying Aura JavaScript UI framework to developers for the first time, enabling the development of custom components for the Salesforce1 mobile application using the same technology that Salesforce had used to develop the Salesforce1 mobile application. The press release for Dreamforce '14 hinted at what was in store for Lightning, "Now customers, developers and Salesforce partners can take advantage of the new Lightning Framework to quickly build engaging apps across every device. The same framework Salesforce's internal development team uses to build apps can now be used to build custom Lightning components and create any user experience." At this point, Salesforce was still using the Salesforce1 branding for the unified end-to-end experience across Salesforce products and platforms, but we now officially had the Lightning Framework to work with. Dreamforce 2015 Dreamforce '15 may have been the official coming out party for Lightning, but in an unprecedented move for Salesforce, the company held a special pre-Dreamforce Meet the New Salesforce event on August 25, 2015, to announce the new Lightning Experience user interface as well as a complete rebranding of the end-to-end experience of Lightning. The Dreamforce event focused on strengthening the branding and educating developers, admins, and end users on what this unified experience meant for the future of Salesforce. Since then, Salesforce has been all Lightning, all the time. Dreamforce 2016 With Dreamforce '16 and the Winter '17 release of Salesforce, Lightning had finally arrived as a stable, optimized, enterprise-ready platform. Dreamforce '16 was less about hype and more about driving Lightning adoption. Sessions focused on design patterns and best practices rather than selling the platform to developers. New tooling was introduced to make the Lightning development experience something that Salesforce developers could get excited about. With Winter '17, Lightning Experience felt like a true unified end-to-end experience instead of a patchwork of functionality. The Winter '17 release notes were packed with enhancements that would get many organizations off the fence about Lightning and shift the adoption bell curve from toward an early majority from the early adopter state that it had been lingering in while Salesforce filled in the gaps in the platform. This is the Lightning we had been waiting for. Lightning Experience If someone were to ask you what the Lightning Experience was all about, your first reaction might be, "It's the new Salesforce UI." While that is technically correct, as Lightning Experience is the brand name given to the user interface that replaces what we now refer to as Salesforce Classic, the implementation of this user interface is part of the larger vision of a unified Salesforce experience across all devices. Lightning Experience takes the old way of doing things in Salesforce Classic – long, scrolling pages – and blows it up into tiny pieces. You need to reassemble those pieces in a way that makes the most sense for your business users. You can even create your own custom pieces, called components, and include those alongside components that Salesforce gives you out of the box, or components built by third parties that you download from the AppExchange. It is all completely seamless. Focusing on getting things done While the initial release of Lightning Experience focused on making salespeople more productive, the interface is rapidly evolving to improve on all areas of CRM. The ability to seamlessly transition work between desktop and mobile devices will enable every Salesforce user to find new ways to connect with customers and increase the effectiveness of business processes across the enterprise. While the initial release of Lightning Experience wasn't quite complete, it did include over 25 new features and a total redesign of many pages. Some of the notable improvements include: Component-based record pages that focused on getting work done in the context of the Salesforce object record, rather than having to scroll to find pertinent related information Completely redesigned reports and dashboards that enable visualizing customer data in new ways, with flexible layouts and additional filtering options An opportunity workspace designed to help salespeople get to closed won faster by focusing on actions instead of raw data Kanban boards for visualizing opportunities in their various deal stages and enabling salespeople to drag and drop multiple opportunities to new stages rather than having to edit and save individual records An enhanced Notes feature that enables users to create notes with rich text and attach them to multiple records at the same time Unified Salesforce Experience Following the mobile-first mindset adopted with the launch of the Salesforce1 platform, the same component framework that is used to power Salesforce1 is what underlies Lightning Experience. The incorporation of design principles from the Salesforce Lightning Design System ensures that users get the same responsive experience whether they access Salesforce from desktop browsers, mobile devices, tablets, wearables, or anything else that comes along in the near future. Developers and ISV partners can now build custom components and applications that plug right into Lightning Experience, rather than having to build custom pages and standalone applications. Blurring the lines between clicks and code Experienced Salesforce developers know that a key consideration when designing Salesforce solutions is to find the right balance between using declarative, out-of-the-box functionality and the programmatic capabilities of the platform. In the world of Salesforce development, we lovingly refer to this dichotomy as clicks versus code. With Lightning Experience and the introduction of Lightning App Builder, the discussion shifts from clicks or code to clicks and code, as developers can now build custom components and expose them to the Lightning App Builder interface, allowing admins to drag and drop these reusable components onto a canvas and assemble new Lightning pages. While this sentiment may strike fear, uncertainty, and doubt into the hearts of developers, any time we can move from code to clicks, or enable admins to maintain Salesforce customizations, it is a good thing. Lightning Experience enables a closer relationship between admins and developers. Developers can focus on building reusable components, admins can focus on maintaining the user experience. Salesforce Lightning Design System A design system is a collection of design principles, style guides, and elements that enable developers to focus on how components and applications work, while designers can focus on how the application looks and feels. The Salesforce Lightning Design System (SLDS) is a trusted, living, platform-agnostic design system that was built from the ground up to provide developers with everything needed to implement the look and feel of Lightning Experience and the Salesforce1 mobile application. SLDS ensures consistency across all components and applications, whether they are written by Salesforce developers, ISV partners, or even Salesforce itself when designing and implementing product features. Salesforce developed SLDS with four key design principles in mind: Clarity Efficiency Consistency Beauty These principles are applied to colors, typography, layout, icons, and more, throughout the accompanying CSS framework. Developers can implement the design system by including SLDS Cascading Style Sheets (CSS) and icon libraries in components and applications, and applying the appropriate CSS classes to component markup. SLDS also includes CSS style sheets for applying the design system to Visualforce components, Heroku, and native iOS applications. When SLDS was first introduced, adding the style sheets or icon libraries to a Salesforce org required installing an unmanaged package or uploading files as static resources and manually upgrading to new versions of the design system as they were released. As of the Winter '17 release of Salesforce, SLDS is included out of the box with all Salesforce orgs and no longer requires an explicit reference to the static resources from Lightning components or applications. You simply reference the appropriate SLDS class names in your component markup and it will be applied automatically, or you can use Lightning Base Components, which apply SLDS implicitly without additional markup. Lightning Component framework Traditionally, Salesforce UI design came down to two questions: Do I want to recreate the look and feel of Salesforce with custom Visualforce pages, or do I want to install a third-party framework to create rich application interfaces? The prevailing design pattern since the fall of Adobe Flash has been to use Visualforce to simply render the output from JavaScript frameworks such as Backbone, Angular, Ember, Sencha, and others. Developers could continue to follow MVC patterns by using Apex controllers to retrieve data from the Salesforce database. While this may have enabled a rich, responsive experience for certain applications developed for Salesforce, users still had to deal with a mixed experience across all of Salesforce, especially if multiple JavaScript frameworks were in use. Lighting Experience and the Lightning Component framework solve a problem that has long been a barrier to a truly unified experience for all Salesforce users: Providing a single, integrated framework for developers that enabled the creation of rich, responsive applications that could be seamlessly plugged in anywhere in the UI rather than having to stand alone in separate pages or standalone applications. Because the Lightning Component framework is what underlies Lightning Experience and the Salesforce1 mobile applications, we no longer have to choose a JavaScript framework developed and maintained outside of Salesforce. We can now create components and applications using a rich JavaScript framework, which is provided and maintained by Salesforce. What is the Lightning Component framework? The Lightning Component framework is a client-side UI framework that was built by Salesforce using the open source Aura UI framework. The framework uses JavaScript for client-side operations and exposes an API to access Salesforce data using Apex controllers. The Lightning Component framework was initially created to support development for the Salesforce1 mobile application, but is now the standard for responsive, client-side single-page applications for the end-to-end Salesforce user and developer experience across all browsers and devices. The framework provides a number of reusable out-of-the-box components for you to get started building your own Lightning components, and the platform is fully maintained and supported by Salesforce. The problem Salesforce solved with the Lightning Component framework was to give Salesforce developers a single standardized and supported JavaScript framework to move beyond the limitations of Visualforce and build rich applications with a common design system without having to select, install, and maintain a third-party framework. Eliminating the JavaScript framework sprawl within the Salesforce development ecosystem enables developers and admins to deliver customized business solutions with a standardized look and feel without having to learn and maintain yet another framework from another vendor that wasn't built specifically for the Salesforce platform. What is JavaScript? Along with HTML and CSS, JavaScript is one of the three core languages of web development. If you do not have a background in JavaScript, don't worry. Even though it will be the most difficult thing you will have to learn when coming up to speed on Lightning component-development, JavaScript has been around for decades and there are countless resources available for learning the language. JavaScript was introduced in the mid-1990s as a scripting language for the Netscape Navigator browser, with Microsoft subsequently releasing a version for Internet Explorer. By the late 1990s, JavaScript was standardized across browsers with the introduction of what is called ECMAScript. Today, all modern browsers include JavaScript engines. JavaScript will not be completely foreign to Apex developers, or anyone with an object-oriented programming (OOP) background, as it follows object-oriented principles. What will throw you off if you have an Apex background is the fact that JavaScript is loosely typed, whereas Apex is a strongly typed language. This will take some getting used to, conceptually. At its core, JavaScript is a language that is used to read and manipulate what we call the Document Object Model (DOM). The DOM is a programmatic interface that gets built when a browser reads an HTML document and converts each HTML element into what are called node objects. These HTML nodes form a tree-like structure in the DOM, and we can use JavaScript to find nodes, add nodes, remove nodes, and modify nodes to make our web applications dynamic. The other core function that JavaScript performs is listening for and handling events that occur in the browser. HTML itself is a static markup language and was not built to be dynamic in its rendering, which is why JavaScript was created to handle events such as clicking a mouse, changing a picklist value, or putting a form element into focus. You can write JavaScript functions to handle DOM events, and your JavaScript functions can in turn manipulate the DOM by adding, removing, or modifying DOM elements. Many of us have only had to learn JavaScript at a cursory level because JavaScript libraries such as jQuery and JavaScript frameworks such as Sencha ExtJS, Angular, Node, and Backbone take care of a lot of the heavy lifting for us when it comes to actual JavaScript programming. Lightning requires more direct JavaScript programming than many frameworks do, which gives you greater control over the functions in your Lightning components and applications, but unfortunately, you're going to have to bone up on your JavaScript knowledge before you can take advantage of that level of control. What are JavaScript frameworks? JavaScript frameworks handle much of the behind-the-scenes complexity of JavaScript coding and DOM manipulation, and give developers a simplified template-based approach to building web applications. While JavaScript itself does not follow the Model-View-Controller (MVC) design pattern, many JavaScript frameworks implement MVC to provide developers with a familiar architecture for separating the data model and view of an application with a logic, or controller, layer. Each component of the MVC architecture can be maintained separately and be brought together in a cohesive application. Some frameworks, such as Sencha ExtJS, may implement MVC but are more focused on enabling rich user interfaces by giving developers pre-built UI widgets that can be configured declaratively. Other frameworks are designed for touch-driven responsive mobile applications. There are dozens of JavaScript frameworks out there, the most common examples being Backbone.js, AngularJS, Ember.js, Knockout.js, React.js, and ExtJS, among others. What is Aura? Aura is an open-source JavaScript UI framework that is maintained by Salesforce. Aura is component-based, and uses JavaScript on the client-side frontend, with Java on the server-side backend. Aura is the framework that underpins the Lightning Component framework. While the Lightning Component framework and Aura have many similarities on the surface, do not try to use Aura components or functions that are not explicitly supported in the Lightning Component framework documentation. Many developers have already found that these undocumented features may work at first, but unless they are explicitly supported by Salesforce, they can be taken away at any time. Why should I use the Lightning Component framework? Why should Salesforce developers consider moving to Lightning Component development? For starters, Lightning is the future of Salesforce development. There is no way around it: Salesforce is betting the company on Lightning. If you ignore that and simply focus on the value that Lightning can provide, you will find that there are many compelling reasons for making the jump to Lightning. Responsive design From a single code base, you can create reusable components that will give your users a unified experience across all form factors. You no longer have to maintain separate desktop applications, tablet applications and mobile applications. Reusable components You can create components that can be consumed by other Lightning components and applications, allowing your component to be reused many times in many different places. Admins can use your components in the Lightning App Builder, allowing them to declaratively create new Lightning Pages with your components. This is where the line between declarative and programmatic development starts to blur! Better performance Because Lightning components render on the client and do not require expensive round trips to a server-side controller to update data and context, you can build components that are lightning fast. Have you ever used the AJAX components in Visualforce to do something as simple as create a type-ahead function for an input field? There was always a lag between the time an event was handled and the target component was re-rendered. With Lightning components, any change to an attribute value will automatically re-render a component, giving you the ability to create high-performing client-side Salesforce applications. Rendering on the client side will reduce the need for mobile applications to make expensive calls to a server API, significantly improving performance in low bandwidth situations. JavaScript + HTML5 + CSS If you have at least a cursory knowledge of web development, Lightning follows open standards and practices that you are already familiar with. While there may be a learning curve for Visualforce developers who do not have experience with JavaScript, HTML5, or CSS, the use of web standards rather than proprietary framework means there is a wealth of information and resources available to quickly get up to speed on the basics and transition to Lightning Component development. For new developers coming onto the Salesforce platform, Lightning provides an opportunity to quickly apply existing web-application development skills to hit the ground running, without having to first learn Apex or Visualforce. Event-driven architecture With Visualforce development, we are constrained to a fairly rigid architecture, which executes server-side controller actions when user-driven events occur, such as clicking a link or a button. Granted, you can use specialized Visualforce tags or component-specific event handlers to handle supported events, but this requires a significant amount of hard-coding to server-side controller methods. With Lightning, you can listen for and handle just about any DOM event or custom component event and determine what action you want to take when that event occurs. For example, you can handle the onchange event when a picklist value is selected and immediately call a function in your client-side controller to take action based on that value changing. You even have the ability to define and raise custom events, and determine how those events should be handled within your component hierarchy. Component encapsulation Encapsulation simply means that we can wall off the inner workings of a Lightning component and not expose the secret sauce behind it to another component or application that references it. This allows us to update the code within a Lightning Component without breaking any upstream components or applications. Encapsulation enables us to write reusable and efficient code that is easily maintainable. Summary In this article we have learned about Salesforce, the history about Lightning and the need to use the Lightning Component framework. Resources for Article: Further resources on this subject: Introducing Salesforce Chatter [article] Subscribing to a report [article] Learning How to Manage Records in Visualforce [article]

This article by Aleksandar Prokopec, author of the book Learning Concurrent Programming in Scala - Second Edition, explains the concepts of asynchronous programming in Scala. Asynchronous programming helps you eliminate blocking; instead of suspending the thread whenever a resource is not available, a separate computation is scheduled to proceed once the resource becomes available. In a way, many of the concurrency patterns seen so far support asynchronous programming; thread creation and scheduling execution context tasks can be used to start executing a computation concurrent to the main program flow. Still, it is not straightforward to use these facilities directly when avoiding blocking or composing asynchronous computations. In this article, we will focus on two abstractions in Scala that are specifically tailored for this task—futures and promises. More specifically, we will study the following topics: Starting asynchronous computations and using Future objects Using Promise objects to interface Blocking threads inside asynchronous computations Alternative future frameworks (For more resources related to this topic, see here.) Futures The parallel executions in a concurrent program proceed on entities called threads. At any point, the execution of a thread can be temporarily suspended until a specific condition is fulfilled. When this happens, we say that the thread is blocked. Why do we block threads in the first place in concurrent programming? One of the reasons is that we have a finite amount of resources; so, multiple computations that share these resources sometimes need to wait. In other situations, a computation needs specific data to proceed, and if that data is not yet available, threads responsible for producing the data could be slow or the source of the data could be external to the program. A classic example is waiting for the data to arrive over the network. Let's assume that we have a getWebpage method that returns that webpage's contents when given a url string with the location of the webpage: def getWebpage(url: String): String The return type of the getWebpage method is String; the method must return a string with the webpage's contents. Upon sending an HTTP request, though, the webpage's contents are not available immediately. It takes some time for the request to travel over the network to the server and back before the program can access the document. The only way for the method to return the contents of the webpage as a string value is to wait for the HTTP response to arrive. However, this can take a relatively long period of time from the program's point of view even with a high-speed Internet connection, the getWebpage method needs to wait. Since the thread that called the getWebpage method cannot proceed without the contents of the webpage, it needs to pause its execution; therefore, the only way to correctly implement the getWebpage method is for blocking. We already know that blocking can have negative side effects, so can we change the return value of the getWebpage method to some special value that can be returned immediately? The answer is yes. In Scala, this special value is called a future. The future is a placeholder, that is, a memory location for the value. This placeholder does not need to contain a value when the future is created; the value can be placed into the future eventually by getWebpage. We can change the signature of the getWebpage method to return a future as follows: def getWebpage(url: String): Future[String] Here, the Future[String] type means that the future object can eventually contain a String value. We can now implement getWebpage without blocking—we can start the HTTP request asynchronously and place the webpage's contents into the future when they become available. When this happens, we can say that the getWebpage method completes the future. Importantly, after the future is completed with some value, that value can no longer change. The Future[T] type encodes latency in the program—use it to encode values that will become available later during execution. This removes blocking from the getWebpage method, but it is not clear how the calling thread can extract the content of the future. Polling is one non-blocking way of extracting the content. In the polling approach, the calling thread calls a special method for blocking until the value becomes available. While this approach does not eliminate blocking, it transfers the responsibility of blocking from the getWebpage method to the caller thread. Java defines its own Future type to encode values that will become available later. However, as a Scala developer, you should use Scala's futures instead; they allow additional ways of handling future values and avoid blocking, as we will soon see. When programming with futures in Scala, we need to distinguish between future values and future computations. A future value of the Future[T] type denotes some value of the T type in the program that might not be currently available, but could become available later. Usually, when we say a future, we really mean a future value. In the scala.concurrent package, futures are represented with the Future[T] trait: trait Future[T] By contrast, a future computation is an asynchronous computation that produces a future value. A future computation can be started by calling the apply method on the Future companion object. This method has the following signature in the scala.concurrent package: def apply[T](b: =>T)(implicit e: ExecutionContext): Future[T] This method takes a by-name parameter of the T type. This is the body of the asynchronous computation that results in some value of type T. It also takes an implicit ExecutionContext parameter, which abstracts over where and when the thread gets executed. Recall that Scala's implicit parameters can either be specified when calling a method, in the same way as normal parameters, or they can be left out; in this case, the Scala compiler searches for a value of the ExecutionContext type in the surrounding scope. Most Future methods take an implicit execution context. Finally, the Future.apply method returns a future of the T type. This future is completed with the value resulting from the asynchronous computation, b. Starting future computations Let's see how to start a future computation in an example. We first import the contents of the scala.concurrent package. We then import the global execution context from the Implicits object. This makes sure that the future computations execute on the global context—the default execution context you can use in most cases: import scala.concurrent._ import ExecutionContext.Implicits.global object FuturesCreate extends App { Future { log("the future is here") } log("the future is coming") Thread.sleep(1000) } The order in which the log method calls (in the future computation and the main thread) execute is nondeterministic. The Future singleton object followed by a block is syntactic sugar for calling the Future.apply method. In the following example, we can use the scala.io.Source object to read the contents of our build.sbt file in a future computation. The main thread calls the isCompleted method on the future value, buildFile, returned from the future computation. Chances are that the build file was not read so fast, so isCompleted returns false. After 250 milliseconds, the main thread calls isCompleted again, and this time, isCompleted returns true. Finally, the main thread calls the value method, which returns the contents of the build file: import scala.io.Source object FuturesDataType extends App { val buildFile: Future[String] = Future { val f = Source.fromFile("build.sbt") try f.getLines.mkString("n") finally f.close() } log(s"started reading the build file asynchronously") log(s"status: ${buildFile.isCompleted}") Thread.sleep(250) log(s"status: ${buildFile.isCompleted}") log(s"value: ${buildFile.value}") } In this example, we used polling to obtain the value of the future. The Future singleton object's polling methods are non-blocking, but they are also nondeterministic; the isCompleted method will repeatedly return false until the future is completed. Importantly, completion of the future is in a happens-before relationship with the polling calls. If the future completes before the invocation of the polling method, then its effects are visible to the thread after the polling completes. Shown graphically, polling looks as shown in the following figure: Polling diagram Polling is like calling your potential employer every five minutes to ask if you're hired. What you really want to do is hand in a job application and then apply for other jobs instead of busy-waiting for the employer's response. Once your employer decides to hire you, he will give you a call on the phone number you left him. We want futures to do the same; when they are completed, they should call a specific function that we left for them. Promises Promises are objects that can be assigned a value or an exception only once. This is why promises are sometimes also called single-assignment variables. A promise is represented with the Promise[T] type in Scala. To create a promise instance, we use the Promise.apply method on the Promise companion object: def apply[T](): Promise[T] This method returns a new promise instance. Like the Future.apply method, the Promise.apply method returns immediately; it is non-blocking. However, the Promise.apply method does not start an asynchronous computation, it just creates a fresh Promise object. When the Promise object is created, it does not contain a value or an exception. To assign a value or an exception to a promise, we use the success or failure method, respectively. Perhaps you have noticed that promises are very similar to futures. Both futures and promises are initially empty and can be completed with either a value or an exception. This is intentional—every promise object corresponds to exactly one future object. To obtain the future associated with a promise, we can call the future method on the promise. Calling this method multiple times always returns the same future object. A promise and a future represent two aspects of a single-assignment variable. The promise allows you to assign a value to the future object, whereas the future allows you to read that value. In the following code snippet, we create two promises, p and q, that can hold string values. We then install a foreach callback on the future associated with the p promise and wait for 1 second. The callback is not invoked until the p promise is completed by calling the success method. We then fail the q promise in the same way and install a failed.foreach callback: object PromisesCreate extends App { val p = Promise[String] val q = Promise[String] p.future foreach { case x => log(s"p succeeded with '$x'") } Thread.sleep(1000) p success "assigned" q failure new Exception("not kept") q.future.failed foreach { case t => log(s"q failed with $t") } Thread.sleep(1000) } Alternatively, we can use the complete method and specify a Try[T] object to complete the promise. Depending on whether the Try[T] object is a success or a failure, the promise is successfully completed or failed. Importantly, after a promise is either successfully completed or failed, it cannot be assigned an exception or a value again in any way. Trying to do so results in an exception. Note that this is true even when there are multiple threads simultaneously calling success or complete. Only one thread completes the promise, and the rest throw an exception. Assigning a value or an exception to an already completed promise is not allowed and throws an exception. We can also use the trySuccess, tryFailure, and tryComplete methods that correspond to success, failure, and complete states, respectively, but return a Boolean value to indicate whether the assignment was successful. Recall that using the Future.apply method and callback methods with referentially transparent functions results in deterministic concurrent programs. As long as we do not use the trySuccess, tryFailure, and tryComplete methods, and none of the success, failure, and complete methods ever throws an exception, we can use promises and retain determinism in our programs. We now have everything we need to implement our custom Future.apply method. We call it the myFuture method in the following example. The myFuture method takes a b by-name parameter that is the asynchronous computation. First, it creates a p promise. Then, it starts an asynchronous computation on the global execution context. This computation tries to evaluate b and complete the promise. However, if the b body throws a nonfatal exception, the asynchronous computation fails the promise with that exception. In the meanwhile, the myFuture method returns the future immediately after starting the asynchronous computation: import scala.util.control.NonFatal object PromisesCustomAsync extends App { def myFuture[T](b: =>T): Future[T] = { val p = Promise[T] global.execute(new Runnable { def run() = try { p.success(b) } catch { case NonFatal(e) => p.failure(e) } }) p.future } val f = myFuture { "naa" + "na" * 8 + " Katamari Damacy!" } f foreach { case text => log(text) } } This is a common pattern when producing futures. We create a promise, let some other computation complete that promise, and return the corresponding future. However, promises were not invented just for our custom future's myFuture computation method. In the following sections, we will study use cases in which promises are useful. Futures and blocking Futures and asynchronous computations mainly exist to avoid blocking, but in some cases, we cannot live without it. It is, therefore, valid to ask how blocking interacts with futures. There are two ways to block with futures. The first way is to wait until a future is completed. The second way is by blocking from within an asynchronous computation. We will study both the topics in this section. Awaiting futures In rare situations, we cannot use callbacks or future combinators to avoid blocking. For example, the main thread that starts multiple asynchronous computations has to wait for these computations to finish. If an execution context uses daemon threads, as is the case with the global execution context, the main thread needs to block to prevent the JVM process from terminating. In these exceptional circumstances, we use the ready and result methods on the Await object from the scala.concurrent package. The ready method blocks the caller thread until the specified future is completed. The result method also blocks the caller thread, but returns the value of the future if it was completed successfully or throws the exception in the future if the future is failed. Both the methods require specifying a timeout parameter; the longest duration that the caller should wait for the completion of the future before a TimeoutException method is thrown. To specify a timeout, we import the scala.concurrent.duration package. This allows us to write expressions such as 10.seconds: import scala.concurrent.duration._ object BlockingAwait extends App { val urlSpecSizeFuture = Future { val specUrl = "http://www.w3.org/Addressing/URL/url-spec.txt" Source.fromURL(specUrl).size } val urlSpecSize = Await.result(urlSpecSizeFuture, 10.seconds) log(s"url spec contains $urlSpecSize characters") } In this example, the main thread starts a computation that retrieves the URL specification and then awaits. By this time, the World Wide Web Consortium (W3C) is worried that a DOS attack is under way, so this is the last time we download the URL specification. Blocking in asynchronous computations Waiting for the completion of a future is not the only way to block. Some legacy APIs do not use callbacks to asynchronously return results. Instead, such APIs expose the blocking methods. After we call a blocking method, we lose control over the thread; it is up to the blocking method to unblock the thread and return the control back. Execution contexts are often implemented using thread pools. By starting future computations that block, it is possible to reduce parallelism and even cause deadlocks. This is illustrated in the following example, in which 16 separate future computations call the sleep method, and the main thread waits until they complete for an unbounded amount of time: val startTime = System.nanoTime val futures = for (_ <- 0 until 16) yield Future { Thread.sleep(1000) } for (f <- futures) Await.ready(f, Duration.Inf) val endTime = System.nanoTime log(s"Total time = ${(endTime - startTime) / 1000000} ms") log(s"Total CPUs = ${Runtime.getRuntime.availableProcessors}") Assume that you have eight cores in your processor. This program does not end in one second. Instead, a first batch of eight futures started by Future.apply will block all the worker threads for one second, and then another batch of eight futures will block for another second. As a result, none of our eight processor cores can do any useful work for one second. Avoid blocking in asynchronous computations, as it can cause thread starvation. If you absolutely must block, then the part of the code that blocks should be enclosed within the blocking call. This signals to the execution context that the worker thread is blocked and allows it to temporarily spawn additional worker threads if necessary:   val futures = for (_ <- 0 until 16) yield Future { blocking { Thread.sleep(1000) } } With the blocking call around the sleep call, the global execution context spawns additional threads when it detects that there is more work than the worker threads. All 16 future computations can execute concurrently and the program terminates after one second. The Await.ready and Await.result statements block the caller thread until the future is completed and are in most cases used outside asynchronous computations. They are blocking operations. The blocking statement is used inside asynchronous code to designate that the enclosed block of code contains a blocking call. It is not a blocking operation by itself. Alternative future frameworks Scala futures and promises API resulted from an attempt to consolidate several different APIs for asynchronous programming, among them, legacy Scala futures, Akka futures, Scalaz futures, and Twitter's Finagle futures. Legacy Scala futures and Akka futures have already converged to the futures and promises API that you've learned about so far in this article. Finagle's com.twitter.util.Future type is planned to eventually implement the same interface as scala.concurrent.Future package, while the Scalaz scalaz.concurrent.Future type implements a slightly different interface. In this section, we give a brief of Scalaz futures. To use Scalaz, we add the following dependency to the build.sbt file: libraryDependencies += "org.scalaz" %% "scalaz-concurrent" % "7.0.6" We now encode an asynchronous tombola program using Scalaz. The Future type in Scalaz does not have the foreach method. Instead, we use its runAsync method, which asynchronously runs the future computation to obtain its value and then calls the specified callback: import scalaz.concurrent._ object Scalaz extends App { val tombola = Future { scala.util.Random.shuffle((0 until 10000).toVector) } tombola.runAsync { numbers => log(s"And the winner is: ${numbers.head}") } tombola.runAsync { numbers => log(s"... ahem, winner is: ${numbers.head}") } } Unless you are terribly lucky and draw the same permutation twice, running this program reveals that the two runAsync calls print different numbers. Each runAsync call separately computes the permutation of the random numbers. This is not surprising, as Scalaz futures have the pull semantics, in which the value is computed each time some callback requests it, in contrast to Finagle and Scala futures' push semantics, in which the callback is stored, and applied if and when the asynchronously computed value becomes available. To achieve the same semantics, as we would have with Scala futures, we need to use the start combinator that runs the asynchronous computation once, and caches its result: val tombola = Future { scala.util.Random.shuffle((0 until 10000).toVector) } start With this change, the two runAsync calls use the same permutation of random numbers in the tombola variable and print the same values. We will not dive further into the internals of alternate frameworks. The fundamentals about futures and promises that you learned about in this article should be sufficient to easily familiarize yourself with other asynchronous programming libraries, should the need arise. Summary This article presented some powerful abstractions for asynchronous programming. We saw how to encode latency with the Future type. You learned that futures and promises are closely tied together and that promises allow interfacing with legacy callback-based systems. In cases, where blocking was unavoidable, you learned how to use the Await object and the blocking statement. Finally, you learned that the Scala Async library is a powerful alternative for expressing future computations more concisely. Resources for Article: Further resources on this subject: Introduction to Scala [article] Concurrency in Practice [article] Integrating Scala, Groovy, and Flex Development with Apache Maven [article]

In this article by Kishore Gaddam, author of the book Building Bots with Microsoft Bot Framework, we introduced what is Microsoft Bot Framework and how it helps in the development of bots. (For more resources related to this topic, see here.) Since past several decades, the corporate, government, and business world has experienced several waves of IT architecture foundations, moving from mainframes, to minicomputers, to distributed PCs, to the Internet, to social/mobile and now the Cloud/Internet of Thuings (IoT) Stack. We call this the Sixth wave of Corporate IT, and like its predecessors, Cloud and IoT technologies are causing significant disruption and displacement, even while it drives new levels of productivity. Each architecture focused on key business processes and supported killer technology applications to drive new levels of value. Very soon we will be looking at an enormous networked interconnection of everyday machines to one another, as well as to humans. Machine-to-machine-to-human connectivity will have a profound impact on the consumer and corporate IT experience. As these machines become social and talkto us, we have enormous opportunity to greatly enhance their value proposition through improved product quality, customer experience, and lowered cost of operations. A heightened consumer expectation for more personal and real-time interactions is driving business to holistically embrace the next wave of technology innovation like Cloud, IoT, and Bots to boost business performance. In this age of billions of connected devices, there is a need for such a technology where our apps could talk back, like bots? Bots that have specific purposes and talk to any device or any app or to anyone, Bots that live in cloud, Bots that we can talk to you via any communication channel such as email, text, voice, chat, and others. Bots can go where no apps have gone before when it comes to machine-to-machine-to-human connectivity. And to make this happen we will need a whole new platform. A Platform for Conversations. Conservation as a Service (CaaS) Messaging apps in general are becoming a second home screen for many people, acting as their entry point into the internet. And where the youngins are, the brands will follow. Companies are coming to messaging apps as bots and apps, to offer everything from customer service to online shopping and banking. Conversations are shaping up be the next major human-computer interface. Thanks to advances in natural language processing and machine learning, the tech is finally getting fast and accurate enough to be viable. Imagine a platform where language is the new UI layer. When we talk about conversation as a platform, there are 3 parts: There are people talking to people – Skype translator as an ex where people can communicate across cross languages Then there is the presence or being able to enhance a conversation by the ability to be present and interact remotely Then there is personal assistance and the bots Think of Bots as the new mechanism that you can converse with. Instead of looking through multiple mobile apps or pages and pages of websites, you can call on any application as a bot within the conversational canvas. Bots are the new apps and digital assistants are the meta apps. This way intelligence is infused into all our interactions. This leads us to Microsoft Bot Framework, which is a comprehensive offering from Microsoft to build and deploy high quality bots for your users to interact using Conversation as a Platform (CaaP). This is a framework that lets you build and connect intelligent bots. The idea is that they interact naturally wherever your users are talking, like Skype, Slack, Facebook Messenger, Text/SMS, and others. Basically any kind of channel that you use today as a human being to talk to other people, well, you will be able to use them to talk to bots all using natural language. Microsoft Bot Framework is a Microsoft operated CaaP service and an open source SDK. Bot Framework is one of the many tools Microsoft is offering to for building a complete Bot. Other tools include Language Understanding Intelligent Service (LUIS), Speech APIs, Microsoft Azure, Cortana Intelligence Suit and many more. Your Bot The Microsoft Bot Builder SDK is one of three main components of the Microsoft Bot Framework. First you have to build your bot. Your bot lives in the cloud and you host it yourself. You write it just like a web service component using Node.js or C#, like a ASP.NET WebAPI component. Microsoft Bot builder SDK is open source and so you will have more languages and web stack get supported over time. Your bot will have its own logic, but you also need a conversation logic using dialogs to model a conversation. The Bot builder SDK gives you facilities for this and there are many types of dialogs that are included from simple Yes/No questions to full natural language understanding or LUIS, which is one of the API's provided in Microsoft Cognitive Services: Bot Connector Bot Connector is hosted and operated by Microsoft. Think of it as a central router between your bots and many channels to communicate with your bots. Apart from routing messages, it would be managing state within the conversation. The Bot Connector is an easy way to create a single back-end and then publish to a bunch of different platforms called channels. Bot Directory Bot Directory is where user will be able to find bots. It's like app store for mobile apps. The Bot Directory is a public directory of all reviewed bots registered through the developer portal. Users will be able to discover, try, and add bots to their favorite conversation experiences from the Bot Directory. Anyone can access it and anyone can submit Bots to the directory. As you begin your development with Microsoft Bot Framework, you might be wondering how to best get started. Bots can be built in C#, however, Microsoft's Bot Framework can also be used to build bots using Node.js. For developing any bots, we need to first setup the development environment and have the right tools installed for successfully developing and deploying a bot. Let's see how we can setup a development environment using Visual Studio. Setting up development environment Let's first look at the Prerequisites required to set up the development environment: Prerequisites To use the Microsoft Bot Framework Connector, you must have: A Microsoft Account (Hotmail, Live, or Outlook) to log into the Bot Framework developer portal, which you will use to register your Bot. An Azure subscription (Free trial: https://azure.microsoft.com/en-us/). This Azure subscription is essential for having an Azure-accessible REST endpoint exposing a callback for the Connector service. Developer accounts on one or more communication services (such as Skype, Slack, Facebook) where your Bot will communicate. In addition, you may wish to have an Azure App Insights account so you can capture telemetry from your Bot. There are additionally different ways to go about building a Bot; from scratch, coded directly to the Bot Connector REST API, the Bot Builder SDK's for Node.js and .NET, and the Bot Connector .NET template which is what this quick start guide demonstrates. Setting up Bot Framework Connector SDK .NET This is a step-by-step guide to setting up dev environment to develop a Bot in C# using the Bot Framework Connector SDK .NET template: Install prerequisite software Visual Studio 2015 (latest update) - you can download the community version here for free: www.visualstudio.com Important: Please update all Visual Studio extensions to their latest versions to do so navigate to Tools | Extensions and Updates | Updates Download and install the Bot Application template: Download the file from the direct download link at http://aka.ms/bf-bc-vstemplate Save the zip file to your Visual Studio 2015 templates directory which is traditionally in %USERPROFILE%DocumentsVisual Studio 2015TemplatesProjectTemplatesVisual C# Open Visual Studio. Create a new C# project using the new Bot Application template. The template is a fully functional Echo Bot that takes the user's text utterance as input and returns it as output. In order to run however: The bot has to be registered with Bot Connector The AppId and AppPassword from the Bot Framework registration page have to be recorded in the project's web.config The project needs to be published to the web Emulator Use the Bot Framework Emulator to test your Bot application. The Bot Framework provides a channel emulator that lets you test calls to your Bot as if it were being called by the Bot Framework cloud service. To install the Bot Framework Emulator, download it from https://download.botframework.com/bf-v3/tools/emulator/publish.html. One installed, you're ready to test. First, start your Bot in Visual Studio using a browser as the application host. The following screenshot uses Microsoft Edge: Summary In this article, we introduced what is Microsoft Bot Framework and how it helps in the development of bots. Also, we have seen how to setup development environment, Emulator and the tools needed for programming. This article is based on the thought that programming knowledge and experience grow best when they grow together.  Resources for Article: Further resources on this subject: Talking to Bot using Browser [article] Webhooks in Slack [article] Creating our first bot, WebBot [article]

This article by Gilles Crettenand, author of the book Functional PHP, covers some of the concepts explained in book in a concise manner. We will look at the following: Declarative programming Functions Recursion Composing functions Benefits of functional programming (For more resources related to this topic, see here.) Functional programming has gained a lot of traction in the last few years. Various big tech companies started using functional languages, for example: Twitter on Scala (http://www.artima.com/scalazine/articles/twitter_on_scala.html) WhatsApp being written in Erlang (http://www.fastcompany.com/3026758/inside-erlang-the-rare-programming-language-behind-whatsapps-success) Facebook using Haskell (https://code.facebook.com/posts/302060973291128/open-sourcing-haxl-a-library-for-haskell/1) There is some really wonderful and successful work done on functional languages that compile to JavaScript—the Elm and PureScript languages to name a few. There are efforts to create new languages that either extend or compile to some more traditional languages, such as Hy and Coconut for Python. Even Apple's new language for iOS development, Swift, has multiple concepts from functional programming integrated into its core. However, this article is not about using a new language or learning a whole new technology, it is about benefiting from functional techniques without having to change our whole stack. By just applying some principles to our everyday PHP, we can greatly improve the quality of our life and code. Declarative programming Functional programming is also sometimes called declarative programming in contrast to imperative programming. This languages are called programming paradigms. Object-oriented programming is also a paradigm, but it is the one that is strongly tied to the imperative programming. Instead of explaining the difference at length, let's demonstrate with an example. First an imperative programming using PHP: <?php function getPrices(array $products) { // let's assume the $products parameter is an array of products. $prices = []; foreach($products as $p) { if($p->stock > 0) { $prices[] = $p->price; } } return $prices; } Now let's see how you can do the same with SQL which, among other things, is a declarative language: SELECT price FROM products WHERE stock > 0; Notice the difference? In the first example, you tell the computer what to do step by step, taking care of storing intermediary results yourself. In the second example, you only describe what you want and it will then be the role of the database engine to return the results. In a way, functional programming looks a lot more like SQL than the PHP code we just saw. Functions Functional programming, as it names suggests, revolves around functions. In order to apply functional techniques effectively, a language must support functions as a first-class citizen or first functions. This means that functions are considered like any other value. They can be created and passed around as parameters to other functions and they can be used as return values. Luckily, PHP is such a language, you can create functions at will, pass them around as parameters, and even return them. Another fundamental concept is the idea of a pure function or, in other words, functions that only use their input to produce a result. This means that you cannot use any kind of external or internal state to perform your computation. Another way to look at this is from the angle of dependencies. All of the dependencies of your functions need to be clearly declared in the signature. This helps a lot when someone tries to understand how and what your function is doing. Higher-order functions PHP functions can take functions as parameters and return functions as return values. A function that does either of those is called a higher-order function. It is as simple as that. There are a few of those that are commonly used in any functional code base. Map The map, or array_map, method in PHP is a higher order function that applies a given callback to all the elements of a collection. The return value is a collection in the same order. A simple example is as follows: <?php function square(int $x): int { return $x * $x; } $squared = array_map('square', [1, 2, 3, 4]); /* $squared contains [1, 4, 9, 16] */ Filter The filter, or array_filter, method in PHP is a higher order function that keeps only certain elements of a collection based on a Boolean predicate. The return value is a collection that will only contain elements returning true for the predicate function. A simple example is as follows: <?php function odd(int $a): bool { return $a % 2 === 1; } $filtered = array_filter([1, 2, 3, 4, 5, 6], 'odd'); /* $filtered contains [1, 3, 5] */ Fold or reduce Folding refers to a process where you reduce a collection to a return value using a combining function. Depending on the language, this operation can have multiple names—fold, reduce, accumulate, aggregate, or compress. As with other functions related to arrays, the PHP version is the array_reduce function. You may be familiar with the array_sum function, which calculates the sum of all the values in an array. This is in fact a fold and can be easily written using the array_reduce function: <?php function sum(int $carry, int $i): int { return $carry + $i; } $summed = array_reduce([1, 2, 3, 4], 'sum', 0); /* $summed contains 10 */ You don't necessarily need to use the elements to produce a value. You could, for example, implement a naive replacement for the in_array method using fold: <?php function in_array2(string $needle, array $haystack): bool { $search = function(bool $contains, string $i) use ($needle) : bool { return $needle == $i ? true : $contains; }; return array_reduce($haystack, $search, false); } var_dump(in_array2('two', ['one', 'two', 'three'])); // bool(true) Recursion In the academic sense, recursion is the idea of dividing a problem into smaller instances of the same problem. For example, if you need to scan a directory recursively, you first scan the starting directory and then scan its children and grandchildren. Most programming languages support recursion by allowing a function to call itself. This idea is often what is described as being recursion. Let's see how we can scan a directory using recursion: <?php function searchDirectory($dir, $accumulator = []) { foreach (scandir($dir) as $path) { // Ignore hidden files, current directory and parent directory if(strpos($path, '.') === 0) { continue; } $fullPath = $dir.DIRECTORY_SEPARATOR.$path; if(is_dir($fullPath)) { $accumulator = searchDirectory($path, $accumulator); } else { $accumulator[] = $fullPath; } } return $accumulator; } We start by using the scandir method to obtain all files and directories. Then, if we encounter a child directory, we call the function on it again. Otherwise, we simply add the file to the accumulator. This function is recursive because it calls itself. You can write this using control structures, but as you don't know in advance what the depth of your folder hierarchy is, the code will probably be a lot messier and harder to understand. Trampolines Each time you call a function, information gets added to the memory. This can be an issue when doing recursion as you only have a limited amount of memory available. Until the last recursive call, memory usage will continue growing and a stack overflow can happen. The only way we can avoid stack growth is to return a value instead of calling a new function. This value can hold the information that is needed to perform a new function call, which will continue the computation. This also means that we need some cooperation from the caller of the function. This helpful caller is called a trampoline and here is how it works: The trampoline calls our f function Instead of making a recursive call, the f function returns the next call encapsulated inside a data structure with all the arguments The trampoline extracts the information and performs a new call to the f function Repeat the two last steps until the f function returns a real value The trampoline receives a value and returns those to the real caller If you want to use trampolines in your own project, I invite you to install the following library, which offers some helpers as compared to our crude implementation: composer require functional-php/trampoline Here is an example taken from the documentation: <?php use FunctionalPHPTrampoline as t; function factorial($n, $acc = 1) { return $n <= 1 ? $acc : tbounce('factorial', $n - 1, $n * $acc); }; Composing functions Previously, we discussed the idea of building blocks and small pure functions. But, so far, we haven't even hinted at how those can be used to build something bigger. What good is a building block if you cannot use it? The answer partly lies in function's composition. As it is often the case in functional programming, the concept is borrowed from mathematics. If you have two functions f and g, you can create a third function by composing them. The usual notation in mathematics is (f   g)(x), which is equivalent to calling them one after the other as f(g(x)). You can compose any two given functions really easily with PHP using a wrapper function. Say, you want to display a title in all caps and only safe HTML characters: <?php function safe_title2(string $s) { return strtoupper(htmlspecialchars($s)); } Functional libraries for PHP often come with a helper that can create new functions out of multiple subparts easily. For example, using Lars Strojny's Functional PHP library, you can write the following: <?php $titles4 = array_map(compose('htmlspecialchars', 'strtoupper', 'trim'), $titles); Partial application You might want to set some parameters of a function but leave some of them unassigned for later. For example, we might want to create a function that returns an excerpt of a blog post. The dedicated term for setting such a value is "to bind a parameter" or "bind an argument". The process itself is called partial application and the new function is set to be partially applied. The Functional PHP library also comes with helpers to partially apply a function: <?php use function Functionalpartial_right; use function Functionalpartial_left; use function Functionalpartial_any; use const Functional…; $excerpt = partial_right('substr', 0, 5); echo $excerpt('Lorem ipsum dolor si amet.'); // Lorem $fixed_string = partial_left('substr', 'Lorem ipsum dolor si amet.'); echo $fixed_string(6, 5); // ipsum $start_placeholder = partial_any('substr', 'Lorem ipsum dolor si amet.', …(), 5); echo $start_placeholder(12); // dolor Currying Currying is often used as a synonym to partial application. Although both concepts allows us to bind some parameters of a function, the core ideas are a bit different. The idea behind currying is to transform a function that takes multiple arguments into a sequence of functions that take one argument. As this might be a bit hard to grasp, let's try to curry the substr method. The result is called a curryied function. Again, a helper to create such functions is available in the Functional PHP library: <?php use function Functionalcurry; function add($a, $b, $c, $d) { return $a + $b + $c + $d; } $curryedAdd = curry('add'); $add10 = $curryedAdd(10); $add15 = $add10(5); $add42 = $add15(27); $add42(10); // -> 52 Benefits of functional programing As we just saw, the functional world is moving, adoption by the enterprise world is growing, and even new imperative languages are taking inspiration from functional languages. But why it is so? Reduce the cognitive burden on developers You've probably often read or heard that a programmer should not be interrupted because even a small interruption can lead to literally tens of minutes being lost. This is partly due to the cognitive burden or, in other words, the amount of information you have to keep in memory in order to understand the problem or function at hand. By forcing you to clearly state the dependencies of your functions and avoiding using any kind of external data, functional programming helps a lot in writing self-contained code that can be readily understood and thus reduces cognitive burden a lot. Software with fewer bugs We just saw that functional programming reduces the cognitive burden and makes your code easier to reason about. This is already a huge win when it comes to bugs because it will allow you to spot issues quickly as you will spend less time understanding how the code works to focus on what it should do. But all the benefits we've just seen have another advantage. They make testing a lot easier too! If you have a pure function and you test it with a given set of values, you have the absolute certitude that it will always return exactly the same thing in production. Easier refactoring Refactoring is never easy. However, since the only inputs of a pure function are its parameters and its sole output is the returned value, things are simpler. If you're refactored function continues to return the same output for a given input, you can have the guarantee that your software will continue to work. You cannot forget to set a few state somewhere in an object because your function are side-effect free. Enforcing good practices This article and the related book are the proof that functional programming is more about the way we do things instead of a particular language. You can use functional techniques in nearly any language that has functions. Your language still needs to have certain properties, but not that much. I like to talk about having a functional mindset. If it is so, why do companies move to functional languages? Because those languages enforce the best practice that we will learn in this book. In PHP, you will have to always remember to use functional techniques. In Haskell, you cannot do anything else, the language forces you to write pure functions. Summary This small article is by no mean a complete introduction to functional programming, this is what the Functional PHP book is for. I however hope I convinced you it is a set of techniques worth learning. We only brushed the surface here, all topics are covered more in depth in the various chapters. You will also learn about more advanced topics like the following: Functors, applicatives, and Monads Type systems Pattern matching Functional reactive programming Property-based testing Parallel execution of functional code There is also a whole chapter about using functional programming in conjunction with various frameworks like Symfony, Drupal, Laraval, and Wordpress. Resources for Article: Further resources on this subject: Understanding PHP basics [article] Developing Middleware [article] Continuous Integration [article]

In this article by Javier Fernández González, author of the book Java 9 Concurrency Cookbook - Second Edition we will cover how to run tasks asynchronously. When you execute ForkJoinTask in ForkJoinPool, you can do it in a synchronous or asynchronous way. When you do it in a synchronous way, the method that sends the task to the pool doesn't return until the task sent finishes its execution. When you do it in an asynchronous way, the method that sends the task to the executor returns immediately, so the task can continue with its execution. (For more resources related to this topic, see here.) You should be aware of a big difference between the two methods. When you use the synchronized methods, the task that calls one of these methods (for example, the invokeAll() method) is suspended until the tasks it sent to the pool finish their execution. This allows the ForkJoinPool class to use the work-stealing algorithm to assign a new task to the worker thread that executed the sleeping task. On the contrary, when you use the asynchronous methods (for example, the fork() method), the task continues with its execution, so the ForkJoinPool class can't use the work-stealing algorithm to increase the performance of the application. In this case, only when you call the join() or get() methods to wait for the finalization of a task, the ForkJoinPool class can use that algorithm. In addition to RecursiveAction and RecursiveTask classes, Java 8 introduced a new ForkJoinTask with the CountedCompleter class. With this kind of tasks you can include a completion action that will be executed when is launched and there is no child pending tasks. This mechanism is based in a method included in the class (the onCompletion() method) and a counter of pending tasks. This counter is initialized to zero by default and you can increment it when you need in an atomic way. Normally, you will increment this counter one by one when you launch a child task. Finally, when a task has finished is execution, you can try to complete the execution of the task and consequently, executes the onCompletion() method. If the pending count is bigger than zero, it is decremented by one. If it's zero, the onCompletion() method is executed and then the parent task is tried to complete. In this article, you will learn how to use the asynchronous methods provided by the ForkJoinPool and CountedCompleter classes for the management of tasks. You are going to implement a program that will search for files with a determined extension inside a folder and its subfolders. The CountedCompleter class you're going to implement will process the content of a folder. For each subfolder inside that folder, it will send a new task to the ForkJoinPool class in an asynchronous way. For each file inside that folder, the task will check the extension of the file and add it to the result list if it proceeds. When a task is completed, it will insert the result lists of all its child tasks in its result task. How to do it... Follow these steps to implement the example: Create a class named FolderProcessor and specify that it extends the CountedCompleter class parameterized with the List<String> type. public class FolderProcessor extends CountedCompleter<List<String>> { Declare the serial version UID of the class. This element is necessary because the parent class of the RecursiveTask class, the ForkJoinTask class, implements the Serializable interface. private static final long serialVersionUID = -1826436670135695513L; Declare a private String attribute named path. This attribute will store the full path of the folder this task is going to process. private String path; Declare a private String attribute named extension. This attribute will store the name of the extension of the files this task is going to look for. private String extension; Declare two List private attributes named tasks and resultList. We will use the first one to store all the child tasks launched from this task and the other one to store the list of results of this task. private List<FolderProcessor> tasks; private List<String> resultList; Implement one constructor for the class to initialize its attributes and its parent class. We declared this constructor as protected as it will only be used internally protected FolderProcessor (CountedCompleter<?> completer, String path, String extension) { super(completer); this.path=path; this.extension=extension; } We implement other public constructor to be used externally. As the task created by this constructor won't have parent task, we don't include this object as parameter. public FolderProcessor (String path, String extension) { this.path=path; this.extension=extension; } Implement the compute() method. As the base class of our task is the CountedCompleter class, the return type of this method is void. @Override protected void compute() { First, initialize the two list attributes. resultList=new ArrayList<>(); tasks=new ArrayList<>(); Get the content of the folder. File file=new File(path); File content[] = file.listFiles(); For each element in the folder, if there is a subfolder, create a new FolderProcessor object and execute it asynchronously using the fork() method. We use the first constructor of the class and pass the current task as the completer task of the new one. We also increment the counter of pending tasks using the addToPendingCount() method. if (content != null) { for (int i = 0; i < content.length; i++) { if (content[i].isDirectory()) { FolderProcessor task=new FolderProcessor(this, content[i].getAbsolutePath(), extension); task.fork(); addToPendingCount(1); tasks.add(task); Otherwise, compare the extension of the file with the extension you are looking for using the checkFile() method and, if they are equal, store the full path of the file in the list of strings declared earlier. } else { if (checkFile(content[i].getName())){ list.add(content[i].getAbsolutePath()); } } } If the list of the FolderProcessor subtasks has more than 50 elements, write a message to the console to indicate this circumstance. if (tasks.size()>50) { System.out.printf("%s: %d tasks ran.n",file.getAbsolutePath(),tasks.size()); } } Finally, try to complete the current task using the tryComplete() method: tryComplete(); } Implement the onCompletion() method. This method will be executed when all the child tasks (all the tasks that have been forked from the current task) have finished their execution. We add the result list of all the child tasks to the result list of the current task. @Override public void onCompletion(CountedCompleter<?> completer) { for (FolderProcessor childTask : tasks) { resultList.addAll(childTask.getResultList()); } } Implement the checkFile() method. This method compares if the name of a file passed as a parameter ends with the extension you are looking for. If so, the method returns the true value, otherwise it returns the false value. private boolean checkFile(String name) { return name.endsWith(extension); } Finally, implement the getResultList() method to return the result list of a task. The code of this method is very simple so it won't be included. Implement the main class of the example by creating a class named Main with a main() method. public class Main { public static void main(String[] args) { Create ForkJoinPool using the default constructor. ForkJoinPool pool=new ForkJoinPool(); Create three FolderProcessor tasks. Initialize each one with a different folder path. FolderProcessor system=new FolderProcessor("C:\Windows", "log"); FolderProcessor apps=new FolderProcessor("C:\Program Files","log"); FolderProcessor documents=new FolderProcessor("C:\Documents And Settings","log"); Execute the three tasks in the pool using the execute() method. pool.execute(system); pool.execute(apps); pool.execute(documents); Write to the console information about the status of the pool every second until the three tasks have finished their execution. do { System.out.printf("******************************************n"); System.out.printf("Main: Parallelism: %dn",pool.getParallelism()); System.out.printf("Main: Active Threads: %dn",pool.getActiveThreadCount()); System.out.printf("Main: Task Count: %dn",pool.getQueuedTaskCount()); System.out.printf("Main: Steal Count: %dn",pool.getStealCount()); System.out.printf("******************************************n"); try { TimeUnit.SECONDS.sleep(1); } catch (InterruptedException e) { e.printStackTrace(); } } while ((!system.isDone())||(!apps.isDone())||(!documents.isDone())); Shut down ForkJoinPool using the shutdown() method. pool.shutdown(); Write the number of results generated by each task to the console. List<String> results; results=system.join(); System.out.printf("System: %d files found.n",results.size()); results=apps.join(); System.out.printf("Apps: %d files found.n",results.size()); results=documents.join(); System.out.printf("Documents: %d files found.n",results.size()); How it works... The following screenshot shows part of an execution of this example: The key of this example is in the FolderProcessor class. Each task processes the content of a folder. As you know, this content has the following two kinds of elements: Files Other folders If the task finds a folder, it creates another FolderProcessor object to process that folder and sends it to the pool using the fork() method. This method sends the task to the pool that will execute it if it has a free worker-thread or it can create a new one. The method returns immediately, so the task can continue processing the content of the folder. For every file, a task compares its extension with the one it's looking for and, if they are equal, adds the name of the file to the list of results. Once the task has processed all the content of the assigned folder, we try to complete the current task. As we explained in the introduction of this article, when we try to complete a task, the code of the CountedCompleter looks for the value of the pending task counter. If this value is bigger than 0, it decrease of that counter. On the contrary, if the value is 0, the task executes the onCompletion() method and then try to completes its parent task. In our case, when a task is processing a folder and it finds a subfolder, it creates a new child task, launch that task using the fork() method and increment the counter of pending tasks. So, when a task has processed all its content, the counter of pending tasks of the task will be equal to the number of child tasks we have launched. When we call the tryComplete() method, if the folder of the current task has subfolders, this call will decrease the number of pending tasks. Only when all its child tasks have been completed, its onCompletion() method is executed. If the folder of the current task hasn't got any subfolders, the counter of pending tasks will be zero and the onComplete() method will be called immediately and then it will try to complete its parent task. By this way, we create a tree of tasks from top to bottom that are completed from bottom to top. In the onComplete() method, we process all the result lists of the child tasks and add their elements in the result list of the current task. The ForkJoinPool class also allows the execution of tasks in an asynchronous way. You have used the execute() method to send the three initial tasks to the pool. In the Main class, you also finished the pool using the shutdown() method and wrote information about the status and the evolution of the tasks that are running in it. The ForkJoinPool class includes more methods that can be useful for this purpose. There's more... In this example we have used the addToPendingCount() method to increment the counter of pending tasks, but we have other methods we can use to change the value of this counter. setPendingCount(): This method establish the value of the counter of pending tasks. compareAndSetPendingCount(): This method receives two parameters. The first one is the expected value and the second one is the new value. If the value of the counter of pending tasks is equal to the expected value, establish its value to the new one. decrementPendingCountUnlessZero(): This method decrements the value of the counter of pending tasks unless it's equal to zero. The CountedCompleter class also includes other methods to manage the completion of the tasks. These are the most significant ones: complete(): This method executes the onCompletion() method independently of the value of the counter of pending tasks try to complete its completer (parent) task. onExceptionalCompletion(): This method is executed when the completeExceptionally() method has been called or the compute() method has thrown an Exception. Override this method to include your code to process those exceptions. In this example, you have used the join() method to wait for the finalization of tasks and get their results. You can also use one of the two versions of the get() method with this purpose: get(long timeout, TimeUnit unit): This version of the get() method, if the result of the task isn't available, waits the specified time for it. If the specified period of time passes and the result isn't yet available, the method returns a null value. The TimeUnit class is an enumeration with the following constants: DAYS, HOURS, MICROSECONDS, MILLISECONDS, MINUTES, NANOSECONDS, and SECONDS. The join() method can't be interrupted. If you interrupt the thread that called the join() method, the method throws an InterruptedException exception. Summary In this article we learned how to use the asynchronous methods provided by the ForkJoinPool and CountedCompleter classes for the management of tasks. Resources for Article: Further resources on this subject: Why Bother? – Basic [article] Getting Started with Sorting Algorithms in Java [article] Saying Hello to Java EE [article]

In this article by Nic Jackson, author of the book Building Microservices with Go, we will examine the encoding/json package to see just how easy Go makes it for us to use JSON objects for our requests and responses. (For more resources related to this topic, see here.) Reading and writing JSON Thanks to the encoding/json package, which is built into the standard library, encoding and decoding JSON to and from Go types is both fast and easy. It implements the simplistic Marshal and Unmarshal functions; however, if we need them, the package also provides Encoder and Decoder types, which allow us greater control when reading and writing streams of JSON data. In this section, we are going to examine both of these approaches, but first let's take a look at how simple it is to convert a standard Go struct into its corresponding JSON string. Marshalling Go structs to JSON To encode JSON data, the encoding/json package provides the Marshal function, which has the following signature: func Marshal(v interface{}) ([]byte, error) This function takes one parameter, which is of the interface type, so that's pretty much any object you can think of, since interface represents any type in Go. It returns a tuple of ([]byte, error). You will see this return style quite frequently in Go. Some languages implement a try...catch approach, which encourages an error to be thrown when an operation cannot be performed. Go suggests the (return type, error) pattern, where the error is nil when an operation succeeds. In Go, unhanded errors are a bad thing, and while the language does implement the panic and recover functions, which resemble exception handling in other languages, the situations in which you should use them are quite different (The Go Programming Language, Donovan and Kernighan). In Go, panic causes normal execution to stop, and all deferred function calls in the Go routine are executed; the program will then crash with a log message. It is generally used for unexpected errors that indicate a bug in the code, and good, robust Go code will attempt to handle these runtime exceptions and return a detailed error object back to the calling function. This pattern is exactly what is implemented with the Marshal function. In case Marshal cannot create a JSON-encoded byte array from the given object, which could be due to a runtime panic, then this is captured and an error object detailing the problem is returned to the caller. Let's try this out, expanding on our existing example. Instead of simply printing a string from our handler, let's create a simple struct for the response and return that: 10 type helloWorldResponse struct { 11 Message string 12 } In our handler, we will create an instance of this object, set the message, and then use the Marshal function to encode it to a string before returning. Let's see what that will look like: 23 func helloWorldHandler(w http.ResponseWriter, r *http.Request) { 24 response := helloWorldResponse{Message: "HelloWorld"} 25 data, err := json.Marshal(response) 26 if err != nil { 27 panic("Ooops") 28 } 29 30 fmt.Fprint(w, string(data)) 31 } Now when we rerun our program and refresh our browser, we'll see the following output rendered in valid JSON: {"Message":"Hello World"} This is awesome, but the default behavior of Marshal is to take the literal name of the field and use that as the field in the JSON output. What if I prefer to use camel case and would rather see message—could we just rename the field in our struct message? Unfortunately, we can't because in Go, lowercase properties are not exported. Marshal will ignore these and will not include them in the output. All is not lost: the encoding/json package implements struct field attributes, which allow us to change the output for the property to anything we choose. The example code is as follows: 10 type helloWorldResponse struct { 11 Message string `json:"message"` 12 } Using the struct field's tags, we can have greater control over how the output will look. In the preceding example, when we marshal this struct, the output from our server would be the following: {"message":"Hello World"} This is exactly what we want, but we can use field tags to control the output even further. We can convert object types and even ignore a field altogether if we need to: struct helloWorldResponse { // change the output field to be "message" Message string `json:"message"` // do not output this field Author string `json:"-"` // do not output the field if the value is empty Date string `json:",omitempty"` // convert output to a string and rename "id" Id int `json:"id, string"` } The channel, complex types, and functions cannot be encoded in JSON. Attempting to encode these types will result in an UnsupportedTypeError being returned by the Marshal function. It also can't represent cyclic data structures, so if your stuct contains a circular reference, then Marshal will result in an infinite recursion, which is never a good thing for a web request. If we want to export our JSON pretty formatted with indentation, we can use the MarshallIndent function, which allows you to pass an additional string parameter to specify what you would like the indent to be—two spaces, not a tab, right? func MarshalIndent(v interface{}, prefix, indent string) ([]byte, error) The astute reader might have noticed that we are decoding our struct into a byte array and then writing that to the response stream. This does not seem to be particularly efficient, and in fact, it is not. Go provides encoders and decoders, which can write directly to a stream. Since we already have a stream with the ResponseWriter interface, let's do just that. Before we do so, I think we need to look at the ResponseWriter interface a little to see what is going on there. ResponseWriter is an interface that defines three methods: // Returns the map of headers which will be sent by the // WriteHeader method. Header() // Writes the data to the connection. If WriteHeader has not // already been called then Write will call // WriteHeader(http.StatusOK). Write([]byte) (int, error) // Sends an HTTP response header with the status code. WriteHeader(int)   If we have a ResponseWriter, how can we use this with fmt.Fprint(w io.Writer, a ...interface{})? This method requires a Writer interface as a parameter, and we have a ResponseWriter. If we look at the signature for Writer, we can see that it is the following: Write(p []byte) (n int, err error) Because the ResponseWriter interface implements this method, it also satisfies the Writer interface; therefore, any object that implements ResponseWriter can be passed to any function that expects Writer. Amazing! Go rocks—but we don't have an answer to our question: is there any better way to send our data to the output stream without marshalling to a temporary string before we return it? The encoding/json package has a function called NewEncoder. This returns an Encoder object, which can be used to write JSON straight to an open writer, and guess what—we have one of those: func NewEncoder(w io.Writer) *Encoder So instead of storing the output of Marshal into a byte array, we can write it straight to the HTTP response, as shown in the following code: func helloWorldHandler(w http.ResponseWriter, r *http.Request) { response := HelloWorldResponse{Message: "HelloWorld"} encoder := json.NewEncoder(w) encoder.Encode(&response) } We will look at benchmarking in a later chapter, but to see why this is important, here's a simple benchmark to check the two methods against each other; have a look at the output: go test -v -run="none" -bench=. -benchtime="5s" -benchmem testing: warning: no tests to run PASS BenchmarkHelloHandlerVariable 10000000 1211 ns/op 248 B/op 5 allocs/op BenchmarkHelloHandlerEncoder 10000000 662 ns/op 8 B/op 1 allocs/op ok github.com/nicholasjackson/building-microservices-in-go/chapter1/bench 20.650s Using the Encoder rather than marshalling to a byte array is nearly 50% faster. We are dealing with nanoseconds here, so that time may seem irrelevant, but it isn't; this was two lines of code. If you have that level of inefficiency throughout the rest of your code, your application will run slower, you will need more hardware to satisfy the load, and that will cost you money. There is nothing clever in the differences between the two methods—all we have done is understood how the standard packages work and chosen the correct option for our requirements. That is not performance tuning, that is understanding the framework. Unmarshalling JSON to Go structs Now that we have learned how we can send JSON back to the client, what if we need to read input before returning the output? We could use URL parameters, and we will see what that is all about in the next chapter, but usually, you will need more complex data structures, which include the service to accept JSON as part of an HTTP POST request. If we apply techniques similar to those we learned in the previous section (to write JSON), reading JSON is just as easy. To decode JSON into a stuct, the encoding/json package provides us with the Unmarshal function: func Unmarshal(data []byte, v interface{}) error The Unmarshal function works in the opposite way to Marshal: it allocates maps, slices, and pointers as required. Incoming object keys are matched using either the struct field name or its tag and will work with a case-insensitive match; however, an exact match is preferred. Like Marshal, Unmarshal will only set exported struct fields: those that start with an upper case letter. We start by adding a new struct to represent the request, while Unmarshal can decode the JSON into an interface{} array, which would be of one of the following types: map[string]interface{} // for JSON objects []interface{} // for JSON arrays Which type it is depends on whether our JSON is an object or an array. In my opinion, it is much clearer to the readers of our code if we explicitly state what we are expecting as a request. We can also save ourselves work by not having to manually cast the data when we come to use it. Remember two things: You do not write code for the compiler; you write code for humans to understand You will spend more time reading code than you do writing it We are going to do ourselves a favor by taking into account these two points and creating a simple struct to represent our request, which will look like this: 14 type helloWorldRequest struct { 15 Name string `json:"name"` 16 } Again, we are going to use struct field tags because while we could let Unmarshal do case-insensitive matching so that {"name": "World} would correctly unmarshal into the struct the same as {"Name": "World"}, when we specify a tag, we are being explicit about the request form, and that is a good thing. In terms of speed and performance, it is also about 10% faster, and remember: performance matters. To access the JSON sent with the request, we need to take a look at the http.Request object passed to our handler. The following listing does not show all the methods in the request, just the ones we are going to be immediately dealing with. For the full documentation, I recommend checking out the docs at https://godoc.org/net/http#Request. type Requests struct { … // Method specifies the HTTP method (GET, POST, PUT, etc.). Method string // Header contains the request header fields received by the server. The type Header is a link to map[string] []string. Header Header // Body is the request's body. Body io.ReadCloser … } The JSON that has been sent with the request is accessible in the Body field. The Body field implements the io.ReadCloser interface as a stream and does not return []byte or string data. If we need the data contained in the body, we can simply read it into a byte array, like the following example: 30 body, err := ioutil.ReadAll(r.Body) 31 if err != nil { 32 http.Error(w, "Bad request", http.StatusBadRequest) 33 return 34 } Here is something we'll need to remember: we are not calling Body.Close(); if we were making a call with a client, we would need to do this as it is not automatically closed; however, when used in a ServeHTTP handler, the server automatically closes the request stream. To see how this all works inside our handler, we can look at the following handler: 28 func helloWorldHandler(w http.ResponseWriter, r *http.Request) { 29 30 body, err := ioutil.ReadAll(r.Body) 31 if err != nil { 32 http.Error(w, "Bad request", http.StatusBadRequest) 33 return 34 } 35 36 var request HelloWorldRequest 37 err = json.Unmarshal(body, &request) 38 if err != nil { 39 http.Error(w, "Bad request", http.StatusBadRequest) 40 return 41 } 42 43 response := HelloWorldResponse{Message: "Hello " + request.Name} 44 45 encoder := json.NewEncoder(w) 46 encoder.Encode(response) 47 } Let's run this example and see how it works; to test it, we can simply use curl to send a request to the running server. If you feel more comfortable using a GUI tool, then Postman, which is available for the Google Chrome browser, will work just fine. Otherwise, feel free to use your preferred tool. $ curl localhost:8080/helloworld -d '{"name":"Nic"}' You should see the following response: {"message":"Hello Nic"} What do you think will happen if you do not include a body with your request? $ curl localhost:8080/helloworld If you guessed correctly that you would get an "HTTP status 400 Bad Request" error, then you win a prize. The following error replies to the request with the given message and status code: func Error(w ResponseWriter, error string, code int) Once we have sent this, we need to return, stopping further execution of the function as this does not close the ResponseWriter and return flow to the calling function automatically. You might think you are done, but have a go and see whether you can improve the performance of the handler. Think about the things we were talking about when marshaling JSON. Got it? Well, if not, here is the answer: again, all we are doing is using Decoder, which is the opposite of the Encoder function we used when writing JSON, as shown in the following code example. This nets an instant 33% performance increase, and with less code, too. 27 func helloWorldHandler(w http.ResponseWriter, r *http.Request) { 28 29 var request HelloWorldRequest 30 decoder := json.NewDecoder(r.Body) 31 32 err := decoder.Decode(&request) 33 if err != nil { 34 http.Error(w, "Bad request", http.StatusBadRequest) 35 return 36 } 37 38 response := HelloWorldResponse{Message: "Hello " + request.Name} 39 40 encoder := json.NewEncoder(w) 41 encoder.Encode(response) 42 } Now that you can see just how easy it is to encode and decode JSON with Go, I would recommend taking 5 minutes to spend some time digging through the documentation for the encoding/json package as there is a whole lot more than you can do with it: https://golang.org/pkg/encoding/json/ Summary In this article, we looked at encoding and decoding data using the encoding/json package. Resources for Article: Further resources on this subject: Microservices – Brave New World [article] A capability model for microservices [article] Breaking into Microservices Architecture [article]

In this article by Anil Mahtani, Luis Sánchez, Enrique Fernández, and Aaron Martinez, authors of the book Effective Robotics Programming with ROS, Third Edition, you will learn the structure of ROS and the parts it is made up of. Furthermore, you will start to create nodes and packages and use ROS with examples using Turtlesim. The ROS architecture has been designed and divided into three sections or levels of concepts: The Filesystem level The Computation Graph level The Community level (For more resources related to this topic, see here.) The first level is the Filesystem level. In this level, a group of concepts are used to explain how ROS is internally formed, the folder structure, and the minimum number of files that it needs to work. The second level is the Computation Graph level where communication between processes and systems happens. In this section, we will see all the concepts and mechanisms that ROS has to set up systems, handle all the processes, and communicate with more than a single computer, and so on. The third level is the Community level, which comprises a set of tools and concepts to share knowledge, algorithms, and code between developers. This level is of great importance; as with most open source software projects, having a strong community not only improves the ability of newcomers to understand the intricacies of the software as well as solve the most common issues, it is also the main force driving its growth. Understanding the ROS Filesystem level The ROS Filesystem is one of the strangest concepts to grasp when starting to develop projects in ROS, but with time and patience, the reader will easily become familiar with it and realize its value for managing projects and its dependencies. The main goal of the ROS Filesystem is to centralize the build process of a project while at the same time provide enough flexibility and tooling to decentralize its dependencies. Similar to an operating system, an ROS program is divided into folders, and these folders have files that describe their functionalities: Packages: Packages form the atomic level of ROS. A package has the minimum structure and content to create a program within ROS. It may have ROS runtime processes (nodes), configuration files, and so on. Package manifests: Package manifests provide information about a package, licenses, dependencies, compilation flags, and so on. A package manifest is managed with a file called package.xml. Metapackages: When you want to aggregate several packages in a group, you will use metapackages. In ROS Fuerte, this form for ordering packages was called Stacks. To maintain the simplicity of ROS, the stacks were removed, and now, metapackages make up this function. In ROS, there exists a lot of these metapackages; for example, the navigation stack. Metapackage manifests: Metapackage manifests (package.xml) are similar to a normal package, but with an export tag in XML. It also has certain restrictions in its structure. Message (msg) types: A message is the information that a process sends to other processes. ROS has a lot of standard types of messages. Message descriptions are stored in my_package/msg/MyMessageType.msg. Service (srv) types: Service descriptions, stored in my_package/srv/MyServiceType.srv, define the request and response data structures for services provided by each process in ROS. In the following screenshot, you can see the content of the turtlesim package. What you see is a series of files and folders with code, images, launch files, services, and messages. Keep in mind that the screenshot was edited to show a short list of files; the real package has more: The workspace In general terms, the workspace is a folder which contains packages, those packages contain our source files and the environment or workspace provides us with a way to compile those packages. It is useful when you want to compile various packages at the same time and it is a good way to centralize all of our developments. A typical workspace is shown in the following screenshot. Each folder is a different space with a different role: The source space: In the source space (the src folder), you put your packages, projects, clone packages, and so on. One of the most important files in this space is CMakeLists.txt. The src folder has this file because it is invoked by cmake when you configure the packages in the workspace. This file is created with the catkin_init_workspace command. The build space: In the build folder, cmake and catkin keep the cache information, configuration, and other intermediate files for our packages and projects. Development (devel) space: The devel folder is used to keep the compiled programs. This is used to test the programs without the installation step. Once the programs are tested, you can install or export the package to share with other developers. You have two options with regard to building packages with catkin. The first one is to use the standard CMake workflow. With this, you can compile one package at a time, as shown in the following commands: $ cmakepackageToBuild/ $ make If you want to compile all your packages, you can use the catkin_make command line, as shown in the following commands: $ cd workspace $ catkin_make Both commands build the executable in the build space directory configured in ROS. Another interesting feature of ROS is its overlays. When you are working with a package of ROS, for example, turtlesim, you can do it with the installed version, or you can download the source file and compile it to use your modified version. ROS permits you to use your version of this package instead of the installed version. This is very useful information if you are working on an upgrade of an installed package. Packages Usually, when we talk about packages, we refer to a typical structure of files and folders. This structure looks as follows: include/package_name/: This directory includes the headers of the libraries that you would need. msg/: If you develop nonstandard messages, put them here. scripts/: These are executable scripts that can be in Bash, Python, or any other scripting language. src/: This is where the source files of your programs are present. You can create a folder for nodes and nodelets or organize it as you want. srv/: This represents the service (srv) types. CMakeLists.txt: This is the CMake build file. package.xml: This is the package manifest. To create, modify, or work with packages, ROS gives us tools for assistance, some of which are as follows: rospack: This command is used to get information or find packages in the system. catkin_create_pkg: This command is used when you want to create a new package. catkin_make: This command is used to compile a workspace. rosdep: This command installs the system dependencies of a package. rqt_dep: This command is used to see the package dependencies as a graph. If you want to see the package dependencies as a graph, you will find a plugin called package graph in rqt. Select a package and see the dependencies. To move between packages and their folders and files, ROS gives us a very useful package called rosbash, which provides commands that are very similar to Linux commands. The following are a few examples: roscd: This command helps us change the directory. This is similar to the cd command in Linux. rosed: This command is used to edit a file. roscp: This command is used to copy a file from a package. rosd: This command lists the directories of a package. rosls: This command lists the files from a package. This is similar to the ls command in Linux. Every package must contain a package.xml file, as it is used to specify information about the package. If you find this file inside a folder, it is very likely that this folder is a package or a metapackage. If you open the package.xml file, you will see information about the name of the package, dependencies, and so on. All of this is to make the installation and the distribution of these packages easy. Two typical tags that are used in the package.xml file are <build_depend> and <run _depend>. The <build_depend> tag shows which packages must be installed before installing the current package. This is because the new package might use functionality contained in another package. The <run_depend> tag shows the packages that are necessary to run the code of the package. The following screenshot is an example of the package.xml file: Metapackages As we have shown earlier, metapackages are special packages with only one file inside; this file is package.xml. This package does not have other files, such as code, includes, and so on. Metapackages are used to refer to others packages that are normally grouped following a feature-like functionality, for example, navigation stack, ros_tutorials, and so on. You can convert your stacks and packages from ROS Fuerte to Kinetic and catkin using certain rules for migration. These rules can be found at http://wiki.ros.org/catkin/migrating_from_rosbuild. In the following screenshot, you can see the content from the package.xml file in the ros_tutorialsmetapackage. You can see the <export> tag and the <run_depend> tag. These are necessary in the package manifest, which is also shown in the following screenshot: If you want to locate the ros_tutorialsmetapackage, you can use the following command: $ rosstack find ros_tutorials The output will be a path, such as /opt/ros/kinetic/share/ros_tutorials. To see the code inside, you can use the following command line: $ vim /opt/ros/kinetic/ros_tutorials/package.xml Remember that Kinetic uses metapackages, not stacks, but the rosstack find command-line tool is also capable of finding metapackages. Messages ROS uses a simplified message description language to describe the data values that ROS nodes publish. With this description, ROS can generate the right source code for these types of messages in several programming languages. ROS has a lot of messages predefined, but if you develop a new message, it will be in the msg/ folder of your package. Inside that folder, certain files with the .msg extension define the messages. A message must have two main parts: fields and constants. Fields define the type of data to be transmitted in the message, for example, int32, float32, and string, or new types that you have created earlier, such as type1 and type2. Constants define the name of the fields. An example of an msg file is as follows: int32 id float32vel string name In ROS, you can find a lot of standard types to use in messages, as shown in the following table list: Primitive type Serialization C++ Python bool (1) unsigned 8-bit int uint8_t(2) bool int8 signed 8-bit int int8_t int uint8 unsigned 8-bit int uint8_t int(3) int16 signed 16-bit int int16_t int uint16 unsigned 16-bit int uint16_t int int32 signed 32-bit int int32_t int uint32 unsigned 32-bit int uint32_t int int64 signed 64-bit int int64_t long uint64 unsigned 64-bit int uint64_t long float32 32-bit IEEE float float float float64 64-bit IEEE float double float string ascii string (4) std::string string time secs/nsecs signed 32-bit ints ros::Time rospy.Time duration secs/nsecs signed 32-bit ints ros::Duration rospy.Duration A special type in ROS is the header type. This is used to add the time, frame, and sequence number. This permits you to have the messages numbered, to see who is sending the message, and to have more functions that are transparent for the user and that ROS is handling. The header type contains the following fields: uint32seq time stamp string frame_id You can see the structure using the following command: $ rosmsg show std_msgs/Header Thanks to the header type, it is possible to record the timestamp and frame of what is happening with the robot. ROS provides certain tools to work with messages. The rosmsg tool prints out the message definition information and can find the source files that use a message type. In upcoming sections, we will see how to create messages with the right tools. Services ROS uses a simplified service description language to describe ROS service types. This builds directly upon the ROS msg format to enable request/response communication between nodes. Service descriptions are stored in .srv files in the srv/ subdirectory of a package. To call a service, you need to use the package name, along with the service name; for example, you will refer to the sample_package1/srv/sample1.srv file as sample_package1/sample1. Several tools exist to perform operations on services. The rossrv tool prints out the service descriptions and packages that contain the .srv files, and finds source files that use a service type. If you want to create a service, ROS can help you with the service generator. These tools generate code from an initial specification of the service. You only need to add the gensrv() line to your CMakeLists.txt file. In upcoming sections, you will learn how to create your own services. Understanding the ROS Computation Graph level ROS creates a network where all the processes are connected. Any node in the system can access this network, interact with other nodes, see the information that they are sending, and transmit data to the network: The basic concepts in this level are nodes, the master, Parameter Server, messages, services, topics, and bags, all of which provide data to the graph in different ways and are explained in the following list: Nodes: Nodes are processes where computation is done. If you want to have a process that can interact with other nodes, you need to create a node with this process to connect it to the ROS network. Usually, a system will have many nodes to control different functions. You will see that it is better to have many nodes that provide only a single functionality, rather than have a large node that makes everything in the system. Nodes are written with an ROS client library, for example, roscpp or rospy. The master: The master provides the registration of names and the lookup service to the rest of the nodes. It also sets up connections between the nodes. If you don't have it in your system, you can't communicate with nodes, services, messages, and others. In a distributed system, you will have the master in one computer, and you can execute nodes in this or other computers. Parameter Server: Parameter Server gives us the possibility of using keys to store data in a central location. With this parameter, it is possible to configure nodes while it's running or to change the working parameters of a node. Messages: Nodes communicate with each other through messages. A message contains data that provides information to other nodes. ROS has many types of messages, and you can also develop your own type of message using standard message types. Topics: Each message must have a name to be routed by the ROS network. When a node is sending data, we say that the node is publishing a topic. Nodes can receive topics from other nodes by simply subscribing to the topic. A node can subscribe to a topic even if there aren't any other nodes publishing to this specific topic. This allows us to decouple the production from the consumption. It's important that topic names are unique to avoid problems and confusion between topics with the same name. Services: When you publish topics, you are sending data in a many-to-many fashion, but when you need a request or an answer from a node, you can't do it with topics. Services give us the possibility of interacting with nodes. Also, services must have a unique name. When a node has a service, all the nodes can communicate with it, thanks to ROS client libraries. Bags: Bags are a format to save and play back the ROS message data. Bags are an important mechanism to store data, such as sensor data, that can be difficult to collect but is necessary to develop and test algorithms. You will use bags a lot while working with complex robots. In the following diagram, you can see the graphic representation of this level. It represents a real robot working in real conditions. In the graph, you can see the nodes, the topics, which node is subscribed to a topic, and so on. This graph does not represent messages, bags, Parameter Server, and services. It is necessary for other tools to see a graphic representation of them. The tool used to create the graph is rqt_graph. These concepts are implemented in the ros_comm repository. Nodes and nodelets Nodes are executable that can communicate with other processes using topics, services, or the Parameter Server. Using nodes in ROS provides us with fault tolerance and separates the code and functionalities, making the system simpler. ROS has another type of node called nodelets. These special nodes are designed to run multiple nodes in a single process, with each nodelet being a thread (light process). This way, we avoid using the ROS network among them, but permit communication with other nodes. With that, nodes can communicate more efficiently, without overloading the network. Nodelets are especially useful for camera systems and 3D sensors, where the volume of data transferred is very high. A node must have a unique name in the system. This name is used to permit the node to communicate with another node using its name without ambiguity. A node can be written using different libraries, such as roscpp and rospy; roscpp is for C++ and rospy is for Python. Throughout we will use roscpp. ROS has tools to handle nodes and give us information about it, such as rosnode. The rosnode tool is a command-line tool used to display information about nodes, such as listing the currently running nodes. The supported commands are as follows: rosnodeinfo NODE: This prints information about a node rosnodekill NODE: This kills a running node or sends a given signal rosnodelist: This lists the active nodes rosnode machine hostname: This lists the nodes running on a particular machine or lists machines rosnode ping NODE: This tests the connectivity to the node rosnode cleanup: This purges the registration information from unreachable nodes A powerful feature of ROS nodes is the possibility of changing parameters while you start the node. This feature gives us the power to change the node name, topic names, and parameter names. We use this to reconfigure the node without recompiling the code so that we can use the node in different scenes. An example of changing a topic name is as follows: $ rosrun book_tutorials tutorialX topic1:=/level1/topic1 This command will change the topic name topic1 to /level1/topic1. To change parameters in the node, you can do something similar to changing the topic name. For this, you only need to add an underscore (_) to the parameter name; for example: $ rosrun book_tutorials tutorialX _param:=9.0 The preceding command will set param to the float number 9.0. Bear in mind that you cannot use names that are reserved by the system. They are as follows: __name: This is a special, reserved keyword for the name of the node __log: This is a reserved keyword that designates the location where the node's log file should be written __ip and __hostname: These are substitutes for ROS_IP and ROS_HOSTNAME __master: This is a substitute for ROS_MASTER_URI __ns: This is a substitute for ROS_NAMESPACE Topics Topics are buses used by nodes to transmit data. Topics can be transmitted without a direct connection between nodes, which means that the production and consumption of data is decoupled. A topic can have various subscribers and can also have various publishers, but you should be careful when publishing the same topic with different nodes as it can create conflicts. Each topic is strongly typed by the ROS message type used to publish it, and nodes can only receive messages from a matching type. A node can subscribe to a topic only if it has the same message type. The topics in ROS can be transmitted using TCP/IP and UDP. The TCP/IP-based transport is known as TCPROS and uses the persistent TCP/IP connection. This is the default transport used in ROS. The UDP-based transport is known as UDPROS and is a low-latency, lossy transport. So, it is best suited to tasks such as teleoperation. ROS has a tool to work with topics called rostopic. It is a command-line tool that gives us information about the topic or publishes data directly on the network. This tool has the following parameters: rostopicbw /topic: This displays the bandwidth used by the topic. rostopic echo /topic: This prints messages to the screen. rostopic find message_type: This finds topics by their type. rostopichz /topic: This displays the publishing rate of the topic. rostopic info /topic: This prints information about the topic, such as its message type, publishers, and subscribers. rostopic list: This prints information about active topics. rostopic pub /topic type args: This publishes data to the topic. It allows us to create and publish data in whatever topic we want, directly from the command line. rostopic type /topic: This prints the topic type, that is, the type of message it publishes. We will learn to use this command-line tool in upcoming sections. Services When you need to communicate with nodes and receive a reply, in an RPC fashion, you cannot do it with topics; you need to do it with services. Services are developed by the user, and standard services don't exist for nodes. The files with the source code of the services are stored in the srv folder. Similar to topics, services have an associated service type that is the package resource name of the .srv file. As with other ROS filesystem-based types, the service type is the package name and the name of the .srv file. ROS has two command-line tools to work with services: rossrv and rosservice. With rossrv, we can see information about the services' data structure, and it has exactly the same usage as rosmsg. With rosservice, we can list and query services. The supported commands are as follows: rosservice call /service args: This calls the service with the arguments provided rosservice find msg-type: This finds services by service type rosservice info /service: This prints information about the service rosservice list: This lists the active services rosservice type /service: This prints the service type rosserviceuri /service: This prints the ROSRPC URI service Messages A node publishes information using messages which are linked to topics. The message has a simple structure that uses standard types or types developed by the user. Message types use the following standard ROS naming convention; the name of the package, then /, and then the name of the .msg file. For example, std_msgs/ msg/String.msg has the std_msgs/String message type. ROS has the rosmsg command-line tool to get information about messages. The accepted parameters are as follows: rosmsg show: This displays the fields of a message rosmsg list: This lists all messages rosmsg package: This lists all of the messages in a package rosmsg packages: This lists all of the packages that have the message rosmsg users: This searches for code files that use the message type rosmsgmd5: This displays the MD5 sum of a message Bags A bag is a file created by ROS with the .bag format to save all of the information of the messages, topics, services, and others. You can use this data later to visualize what has happened; you can play, stop, rewind, and perform other operations with it. The bag file can be reproduced in ROS just as a real session can, sending the topics at the same time with the same data. Normally, we use this functionality to debug our algorithms. To use bag files, we have the following tools in ROS: rosbag: This is used to record, play, and perform other operations rqt_bag: This is used to visualize data in a graphic environment rostopic: This helps us see the topics sent to the nodes The ROS master The ROS master provides naming and registration services to the rest of the nodes in the ROS system. It tracks publishers and subscribers to topics as well as services. The role of the master is to enable individual ROS nodes to locate one another. Once these nodes have located each other, they communicate with each other in a peer-to-peer fashion. You can see in a graphic example the steps performed in ROS to advertise a topic, subscribe to a topic, and publish a message, in the following diagram: The master also provides Parameter Server. The master is most commonly run using the roscore command, which loads the ROS master, along with other essential components. Parameter Server Parameter Server is a shared, multivariable dictionary that is accessible via a network. Nodes use this server to store and retrieve parameters at runtime. Parameter Server is implemented using XMLRPC and runs inside the ROS master, which means that its API is accessible via normal XMLRPC libraries. XMLRPC is a Remote Procedure Call (RPC) protocol that uses XML to encode its calls and HTTP as a transport mechanism. Parameter Server uses XMLRPC data types for parameter values, which include the following: 32-bit integers Booleans Strings Doubles ISO8601 dates Lists Base64-encoded binary data ROS has the rosparam tool to work with Parameter Server. The supported parameters are as follows: rosparam list: This lists all the parameters in the server rosparam get parameter: This gets the value of a parameter rosparam set parameter value: This sets the value of a parameter rosparam delete parameter: This deletes a parameter rosparam dump file: This saves Parameter Server to a file rosparam load file: This loads a file (with parameters) on Parameter Server Understanding the ROS Community level The ROS Community level concepts are the ROS resources that enable separate communities to exchange software and knowledge. These resources include the following: Distributions: ROS distributions are collections of versioned metapackages that you can install. ROS distributions play a similar role to Linux distributions. They make it easier to install a collection of software, and they also maintain consistent versions across a set of software. Repositories: ROS relies on a federated network of code repositories, where different institutions can develop and release their own robot software components. The ROS Wiki: The ROS Wiki is the main forum for documenting information about ROS. Anyone can sign up for an account, contribute their own documentation, provide corrections or updates, write tutorials, and more. Bug ticket system: If you find a problem or want to propose a new feature, ROS has this resource to do it. Mailing lists: The ROS user-mailing list is the primary communication channel about new updates to ROS as well as a forum to ask questions about the ROS software. ROS Answers: Users can ask questions on forums using this resource. Blog: You can find regular updates, photos, and news at http://www.ros.org/news. Summary This article provided you with general information about the ROS architecture and how it works. You saw certain concepts and tools of how to interact with nodes, topics, and services. Remember that if you have queries about something, you can use the official resources of ROS from http://www.ros.org. Additionally, you can ask the ROS Community questions at http://answers.ros.org. Resources for Article: Further resources on this subject: The ROS Filesystem levels [article] Using ROS with UAVs [article] Face Detection and Tracking Using ROS, Open-CV and Dynamixel Servos [article]

In this article by Mark Brummel, David Studebaker, Christopher D Studebaker, authors of the book Programming Microsoft Dynamics NAV - Fifth Edition, we will discuss how Microsoft Dynamics NAV has one of the largest installed user bases of any enterprise resource planning (ERP) system serving over 100,000 companies and one million plus individual users. The community of supporting organizations, consultants, implementers, and developers continues to grow and prosper. The capabilities of the off-the-shelf product increase with every release. The selection of the add-on products and services expands both in variety and depth. (For more resources related to this topic, see here.) The release of Microsoft Dynamics NAV 2017 continues its 20+ year history of continuous product improvement. It provides more user options for access and output formatting. For new installations, NAV 2017 includes tools for rapid implementation. For all installations, it provides enhanced business functionality and more support for ERP computing in the cloud, including integration with Office 365. NAV 2017 is also the base foundation for Microsoft Dynamics 365 for Financials, which is a cloud-based subset of the product that you can customize using extensions. Our goal in this article is to gain a big picture understanding of NAV 2017. In this article, we will take a look at NAV 2017, including the following: A general overview of NAV 2017 NAV 2017 – An ERP system NAV 2017 is an integrated set of business applications designed to service a wide variety business operations. Microsoft Dynamics NAV 2017 is an ERP system. An ERP system integrates internal and external data across a variety of functional areas including manufacturing, accounting, supply chain management, customer relationships, service operations, and human resources management, as well as the management of other valued resources and activities. By having many related applications well integrated, a full featured ERP system provides an enter data once, use many ways information processing toolset. NAV 2017 ERP addresses the following functional areas (and more): Basic accounting functions (for example, general ledger, accounts payable, and accounts receivable) Order processing and inventory (for example, sales orders, purchase orders, shipping, inventory, and receiving) Relationship management (for example, vendors, customers, prospects, employees, and contractors) Planning (for example, MRP, sales forecasting, and production forecasting) Other critical business areas (for example, manufacturing, warehouse management, marketing, cash management, and fixed assets) A good ERP system such as NAV 2017 is modular in design, which simplifies implementation, upgrading, modification, integration with third-party products, and expansion for different types of clients. All the modules in the system share a common database and, where appropriate, common data. The groupings of individual NAV 2017 functions following is based on the Departments menu structure, supplemented by information from Microsoft marketing materials. The important thing is to understand the overall components that make up the NAV 2017 ERP system: NAV 2017 has two quite different styles of user interface (UI). One UI, the development environment, targets developers. The other UI style, the RoleTailored client, targets end users. In NAV 2017, there are four instances of the RoleTailored client – for Windows, for web interaction, for tablet use, and a phone client, which was introduced in NAV 2016. The example images in the following module descriptions are from the RoleTailored client's Departments menu in the Windows client. Financial management Financial management is the foundation of any ERP system. No matter what the business is, the money must be kept flowing, and the flow of money must be tracked. The tools that help to manage the capital resources of the business are part of NAV 2017's Financial Management module. These include all or part of the following application functions: General ledger—managing overall finances of the firm Cash management and banking—managing inventory of money Accounts receivable—tracking incoming revenue Accounts payable—tracking outgoing funds Analytical accounting—analyzing various flows of funds Inventory and fixed assets—managing inventories of goods and equipment Multi-currency and multi-language—supporting international business activities The Financial Management section of the Departments menu is as follows: Manufacturing NAV 2017 manufacturing is general-purpose enough to be appropriate for Make to Stock (MTS), Make to Order (MTO), and Assemble to Order (ATO), as well as various subsets and combinations of those. Although off-the-shelf NAV is not particularly suitable for most process manufacturing and some of the very high volume assembly line operations, there are third-party add-on and add-in enhancements available for these applications. As with most of the NAV application functions, manufacturing can be implemented to be used in a basic mode or as a full featured system. NAV manufacturing includes the following functions: Product design (BOMs and routings)—managing the structure of product components and the flow of manufacturing processes Capacity and supply requirements planning—tracking the intangible and tangible manufacturing resources Production scheduling (infinite and finite), execution, and tracking quantities and costs, plus tracking the planned use manufacturing resources, both on an unconstrained and constrained basis The Manufacturing section of the Departments menu is as follows: Supply chain management Obviously, some of the functions categorized as part of NAV 2017 supply chain management (SCM), for example, sales and purchasing) are actively used in almost every NAV implementation. The supply chain applications in NAV include all or parts of the following applications: Sales order processing and pricing—supporting the heart of every business Purchasing (including Requisitions)—planning, entering, pricing, and processing purchase orders Inventory management—managing inventories of goods and materials Warehouse management including receiving and shipping—managing the receipt, storage, retrieval, and shipment of material and goods in warehouses Even though we might consider Assembly part of Manufacturing, the standard NAV 2017 Departments menu includes it in the Warehouse section: As a whole, these functions constitute the base components of a system appropriate for distribution operations, including those that operate on an ATO basis. Business Intelligence and reporting Although Microsoft marketing materials identify Business Intelligence (BI) and reporting as though it were a separate module within NAV, it's difficult to physically identify it as such. Most of the components used for BI and reporting purposes are (appropriately) scattered throughout various application areas. In the words of one Microsoft document, Business Intelligence is a strategy, not a product. Functions within NAV that support a BI strategy include the following: Standard reports—distributed ready-to-use by end users Account schedules and analysis reports—a specialized report writer for General Ledger data Query, XML port and Report designers—developer tools to support the creation of a wide variety of report formats, charts, and XML and CSV files Analysis by dimensions—a capability embedded in many of the other tools Interfaces into Microsoft Office and Microsoft Office 365 including Excel—communications of data either into NAV or out of NAV RDLC report viewer—provides the ability to present NAV data in a variety of textual and graphic formats, including user interactive capabilities Interface capabilities such as DotNet interoperability and web services—technologies to support interfaces between NAV 2017 and external software products NAV 2017 has standard packages for Power BI, both integrated on the role center as well as dashboards Artificial Intelligence A new feature of NAV 2017 is integration with Cortana intelligence, which is used for forecasting. The algorithms used in Artificial Intelligence (AI) can be used to give users extra information to make business decisions. Relationship Management NAV's Relationship Management (RM) functionality is definitely the little sister (or, if you prefer, little brother) of the fully featured standalone Microsoft CRM system. The big advantage of NAV RM is its tight integration with NAV customer and sales data. With NAV 2016, Microsoft introduced a new way of integrating with CRM using OData. Also falling under the heading of the customer relationship module is the NAV Service Management (SM) functionality. While the RM component shows up in the menu as part of sales and marketing, the SM component is identified as an independent function in the menu structure: Relationship management: Marketing campaigns—plan and manage promotions Customer activity tracking—analyze customer orders To do lists—manage what is to be done and track what has been done Service management: Service contracts—support service operations Labor and part consumption tracking—track the resources consumed by the service business Planning and dispatching—managing service calls Human resource management NAV Human Resources is a very small module, but it relates to a critical component of the business, employees. Basic employee data can be stored and reported via the master table (in fact, one can use human resources (HR) to manage data about individual contractors in addition to employees). A wide variety of individual employee attributes can be tracked by the use of dimensions fields: Employee tracking—maintain basic employee description data Skills inventory—inventory of the capabilities of employees Absence tracking—maintain basic attendance information Employee statistics—tracking government required employee attribute data such as age, gender, length of service Project management The NAV project management module consists of the jobs functionality supported by the resources functionality. Projects can be short or long term. They can be external (in other words - billable) or internal. This module is often used by third parties as the base for vertical market add-ons (such as construction or job oriented manufacturing). This application area includes parts or all of the following functions: Budgeting and cost tracking—managing project finances Scheduling—planning project activities Resource requirements and usage tracking—managing people and equipment Project accounting—tracking the results Summary In this article, we covered some basic definitions of terms related to NAV. We had a quick overview highlighting NAV 2017 components and how NAV 2017 is designed to service a wide variety of business operations. Resources for Article:  Further resources on this subject: The Sales and Purchase Process [article] Manage Security in Excel [article] Code Analysis and Debugging Tools in Microsoft Dynamics NAV 2009 [article]

In this article by Steven F. Daniel, author of the book Mastering Xamarin UI Development, you will learn how to build stunning, maintainable, cross-platform mobile application user interfaces with the power of Xamarin. In this article, we will cover the following topics: Understanding the MVVM pattern architecture Implement the MVVM ViewModels within the app (For more resources related to this topic, see here.) Understanding the MVVM pattern architecture In this section we will be taking a look at the MVVM pattern architecture and the communication between the components that make up the architecture. The MVVM design pattern is designed to control the separation between the user interfaces (Views), the ViewModels that contain the actual binding to the Model, and the models that contain the actual structure of the entities representing information stored on a database or from a web service. The following screenshot shows the communication between each of the components contained within the MVVM design pattern architecture: The MVVM design pattern is divided into three main areas, as you can see from the preceding screenshot and these are explained in the following table: MVVM type Description Model The Model is basically a representation of business related entities used by an application, and is responsible for fetching data from either a database, or web service, and then de-serialized to the entities contained within the Model. View The View component of the MVVM model basically represents the actual screens that make up the application, along with any custom control components, and control elements, such as buttons, labels, and text fields. The Views contained within the MVVM pattern are platform-specific and are dependent on the platform APIs that are used to render the information that is contained within the application's user interface. ViewModel The ViewModel essentially controls, and manipulates the Views by acting as their main data context. The ViewModel contains a series of properties that are bound to the information contained within each Model, and those properties are then bound to each of the Views to represent this information within the user interface. ViewModels can also contain command objects that provide action-based events that can trigger the execution of event methods that occur within the View. For example, when the user taps on a toolbar item, or a button. ViewModels generally implement the INotifyPropertyChanged interface. Such a class fires a PropertyChanged event whenever one of their properties change. The data binding mechanism in Xamarin.Forms attaches a handler to this PropertyChanged event so it can be notified when a property changes and keep the target updated with the new value. Now that you have a good understanding of the components that are contained within MVVM design pattern architecture, we can begin to create our entity models and update our user interface files. In Xamarin.Forms, the term View is used to describe form controls, such as buttons and labels, and uses the term Page to describe the user interface or screen. Whereas, in MVVM, Views are used to describe the user interface, or screen. Implementing the MVVM ViewModels within your app In this section, we will begin by setting up the basic structure for our TrackMyWalks solution to include the folder that will be used to represent our ViewModels. Let's take a look at how we can achieve this, by following the steps: Launch the Xamarin Studio application and ensure that the TrackMyWalks solution is loaded within the Xamarin Studio IDE. Next, create a new folder within the TrackMyWalks PCL project, called ViewModels as shown in the following screenshot: Creating the WalkBaseViewModel for the TrackMyWalks app In this section, we will begin by creating a base MVVM ViewModel that will be used by each of our ViewModels when we create these, and then the Views (pages) will implement those ViewModels and use them as their BindingContext. Let's take a look at how we can achieve this, by following the steps: Create an empty class within the ViewModels folder, shown in the following screenshot: Next, choose the Empty Class option located within the General section, and enter in WalkBaseViewModel for the name of the new class file to create, as shown in the following screenshot: Next, click on the New button to allow the wizard to proceed and create the new empty class file, as shown in the preceding screenshot. Up until this point, all we have done is create our WalkBaseViewModel class file. This abstract class will act as the base ViewModel class that will contain the basic functionality that each of our ViewModels will inherit from. As we start to build the base class, you will see that it contains a couple of members and it will implement the INotifyPropertyChangedInterface,. As we progress through this article, we will build to this class, which will be used by the TrackMyWalks application. To proceed with creating the base ViewModel class, perform the following step as shown: Ensure that the WalkBaseViewModel.cs file is displayed within the code editor, and enter in the following code snippet: // // WalkBaseViewModel.cs // TrackMyWalks Base ViewModel // // Created by Steven F. Daniel on 22/08/2016. // Copyright © 2016 GENIESOFT STUDIOS. All rights reserved. // using System.ComponentModel; using System.Runtime.CompilerServices; namespace TrackMyWalks.ViewModels { public abstract class WalkBaseViewModel : INotifyPropertyChanged { protected WalkBaseViewModel() { } public event PropertyChangedEventHandler PropertyChanged; protected virtual void OnPropertyChanged([CallerMemberName] string propertyName = null) { var handler = PropertyChanged; if (handler != null) { handler(this, new PropertyChangedEventArgs(propertyName)); } } } } In the preceding code snippet, we begin by creating a new abstract class for our WalkBaseViewModel that implements from the INotifyPropertyChanged interface class, that allows the View or page to be notified whenever properties contained within the ViewModel have changed. Next, we declare a variable PropertyChanged that inherits from the PropertyChangedEventHandler that will be used to indicate whenever properties on the object have changed. Finally, within the OnPropertyChanged method, this will be called when it has determined that a change has occurred on a property within the ViewModel from a child class. The INotifyPropertyChanged interface is used to notify clients, typically binding clients, when the value of a property has changed. Implementing the WalksPageViewModel In the previous section, we built our base class ViewModel for our TrackMyWalks application, and this will act as the main class that will allow our View or pages to be notified whenever changes to properties within the ViewModel have been changed. In this section, we will need to begin building the ViewModel for our WalksPage,. This model will be used to store the WalkEntries, which will later be used and displayed within the ListView on the WalksPage content page. Let's take a look at how we can achieve this, by following the steps: First, create a new class file within the ViewModels folder called WalksPageViewModel, as you did in the previous section, entitled Creating the WalkBaseViewModel located within this article. Next, ensure that the WalksPageViewModel.cs file is displayed within the code editor, and enter in the following code snippet: // // WalksPageViewModel.cs // TrackMyWalks ViewModels // // Created by Steven F. Daniel on 22/08/2016. // Copyright © 2016 GENIESOFT STUDIOS. All rights reserved. // using System.Collections.ObjectModel; using TrackMyWalks.Models; namespace TrackMyWalks.ViewModels { public class WalksPageViewModel : WalkBaseViewModel { ObservableCollection<WalkEntries> _walkEntries; public ObservableCollection<WalkEntries> walkEntries { get { return _walkEntries; } set { _walkEntries = value; OnPropertyChanged(); } } In the above code snippet, we begin by ensuring that our ViewModel inherits from the WalkBaseViewModel class. Next, we create an ObservableCollection variable _walkEntries which is very useful when you want to know when the collection has changed, and an event is triggered that will tell the user what entries have been added or removed from the WalkEntries model. In our next step, we create the ObservableCollection constructor WalkEntries, that is defined within the System.Collections.ObjectModel class, and accepts a List parameter containing our WalkEntries model. The WalkEntries property will be used to bind to the ItemSource property of the ListView within the WalksMainPage. Finally, we define the getter (get) and setter (set) methods that will return and set the content of our _walkEntries when it has been determined when a property has been modified or not. Next, locate the WalksPageViewModel class constructor, and enter the following highlighted code sections:         public WalksPageViewModel() { walkEntries = new ObservableCollection<WalkEntries>() { new WalkEntries { Title = "10 Mile Brook Trail, Margaret River", Notes = "The 10 Mile Brook Trail starts in the Rotary Park near Old Kate, a preserved steam " + "engine at the northern edge of Margaret River. ", Latitude = -33.9727604, Longitude = 115.0861599, Kilometers = 7.5, Distance = 0, Difficulty = "Medium", ImageUrl = "http://trailswa.com.au/media/cache/media/images/trails/_mid/" + "FullSizeRender1_600_480_c1.jpg" }, new WalkEntries { Title = "Ancient Empire Walk, Valley of the Giants", Notes = "The Ancient Empire is a 450 metre walk trail that takes you around and through some of " + "the giant tingle trees including the most popular of the gnarled veterans, known as " + "Grandma Tingle.", Latitude = -34.9749188, Longitude = 117.3560796, Kilometers = 450, Distance = 0, Difficulty = "Hard", ImageUrl = "http://trailswa.com.au/media/cache/media/images/trails/_mid/" + "Ancient_Empire_534_480_c1.jpg" }, }; } } } In the preceding code snippet, we began by creating a new ObservableCollection for our walkEntries method and then added each of the walk list items that we would like to store within our model. As each item is added, the ObservableCollection, constructor is called, and the setter (set) method is invoked to add the item, and then the INotifyPropertyChanged event will be triggered to notify that a change has occurred. Summary In this article, you learned about the MVVM pattern architecture; we also implemented the MVVM ViewModels within the app. Additionally, we created and implemented the WalkBaseViewModel for the TrackMyWalks application. Resources for Article: Further resources on this subject: A cross-platform solution with Xamarin.Forms and MVVM architecture [article] Building a Gallery Application [article] Heads up to MvvmCross [article]