How-To Tutorials

In logistic regression, input features are linearly scaled just as with linear regression; however, the result is then fed as an input to the logistic function. This function provides a nonlinear transformation on its input and ensures that the range of the output, which is interpreted as the probability of the input belonging to class 1, lies in the interval [0,1]. (For more resources related to this topic, see here.) The form of the logistic function is as follows: The plot of the logistic function is as follows: When x = 0, the logistic function takes the value 0.5. As x tends to +∞, the exponential in the denominator vanishes and the function approaches the value 1. As x tends to -∞, the exponential, and hence the denominator, tends to move toward infinity and the function approaches the value 0. Thus, our output is guaranteed to be in the interval [0,1], which is necessary for it to be a probability. Generalized linear models Logistic regression belongs to a class of models known as generalized linear models (GLMs). Generalized linear models have three unifying characteristics. The first of these is that they all involve a linear combination of the input features, thus explaining part of their name. The second characteristic is that the output is considered to have an underlying probability distribution belonging to the family of exponential distributions. These include the normal distribution, the Poisson and the binomial distribution. Finally, the mean of the output distribution is related to the linear combination of input features by way of a function, known as the link function. Let's see how this all ties in with logistic regression, which is just one of many examples of a GLM. We know that we begin with a linear combination of input features, so for example, in the case of one input feature, we can build up an x term as follows: Note that in the case of logistic regression, we are modeling a probability that the output belongs to class 1, rather the output directly as we were in linear regression. As a result, we do not need to model the error term because our output, which is a probability, incorporates nondeterministic aspects of our model, such as measurement uncertainties, directly. Next, we apply the logistic function to this term in order to produce our model's output: Here, the left term tells us directly that we are computing the probability that our output belongs to class 1 based on our evidence of seeing the values of the input feature X1. For logistic regression, the underlying probability distribution of the output is the Bernoulli distribution. This is the same as the binomial distribution with a single trial and is the distribution we would obtain in an experiment with only two possible outcomes having constant probability, such as a coin flip. The mean of the Bernoulli distribution, μy, is the probability of the (arbitrarily chosen) outcome for success, in this case, class 1. Consequently, the left-hand side in the previous equation is also the mean of our underlying output distribution. For this reason, the function that transforms our linear combination of input features is sometimes known as the mean function, and we just saw that this function is the logistic function for logistic regression. Now, to determine the link function for logistic regression, we can perform some simple algebraic manipulations in order to isolate our linear combination of input features. The term on the left-hand side is known as the log-odds or logit function and is the link function for logistic regression. The denominator of the fraction inside the logarithm is the probability of the output being class 0 given the data. Consequently, this fraction represents the ratio of probability between class 1 and class 0, which is also known as the odds ratio. A good reference for logistic regression along with examples of other GLMs such as Poisson regression is Extending the Linear Model with R, Julian J. Faraway, CRC Press. Interpreting coefficients in logistic regression Looking at the right-hand side of the last equation, we can see that we have almost exactly the same form as we had for simple linear regression, barring the error term. The fact that we have the logit function on the left-hand side, however, means we cannot interpret our regression coefficients in the same way that we did with linear regression. In logistic regression, a unit increase in feature Xi results in multiplying the odds ratio by an amount, { QUOTE  }. When a coefficient βi is positive, then we multiply the odds ratio by a number greater than 1, so we know that increasing the feature Xi will effectively increase the probability of the output being labeled as class 1. Similarly, increasing a feature with a negative coefficient shifts the balance toward predicting class 0. Finally, note that when we change the value of an input feature, the effect is a multiplication on the odds ratio and not on the model output itself, which we saw is the probability of predicting class 1. In absolute terms, the change in the output of our model as a result of a change in the input is not constant throughout but depends on the current value of our input features. This is, again, different from linear regression, where no matter what the values of the input features, the regression coefficients always represent a fixed increase in the output per unit increase of an input feature. Assumptions of logistic regression Logistic regression makes fewer assumptions about the input than linear regression. In particular, the nonlinear transformation of the logistic function means that we can model more complex input-output relationships. We still have a linearity assumption, but in this case, it is between the features and the log-odds. We no longer require a normality assumption for residuals and nor do we need the homoscedastic assumption. On the other hand, our error terms still need to be independent. Strictly speaking, the features themselves no longer need to be independent but in practice, our model will still face issues if the features exhibit a high degree of multicollinearity. Finally, we'll note that just like with unregularized linear regression, feature scaling does not affect the logistic regression model. This means that centering and scaling a particular input feature will simply result in an adjusted coefficient in the output model without any repercussions on the model performance. It turns out that for logistic regression, this is the result of a property known as the invariance property of maximum likelihood, which is the method used to select the coefficients and will be the focus of the next section. It should be noted, however, that centering and scaling features might still be a good idea if they are on very different scales. This is done to assist the optimization procedure during training. In short, we should turn to feature scaling only if we run into model convergence issues. Maximum likelihood estimation When we studied linear regression, we found our coefficients by minimizing the sum of squared error terms. For logistic regression, we do this by maximizing the likelihood of the data. The likelihood of an observation is the probability of seeing that observation under a particular model. In our case, the likelihood of seeing an observation X for class 1 is simply given by the probability P(Y=1|X), the form of which was given earlier in this article. As we only have two classes, the likelihood of seeing an observation for class 0 is given by 1 - P(Y=1|X). The overall likelihood of seeing our entire data set of observations is the product of all the individual likelihoods for each data point as we consider our observations to be independently obtained. As the likelihood of each observation is parameterized by the regression coefficients βi, the likelihood function for our entire data set is also, therefore, parameterized by these coefficients. We can express our likelihood function as an equation, as shown in the following equation: Now, this equation simply computes the probability that a logistic regression model with a particular set of regression coefficients could have generated our training data. The idea is to choose our regression coefficients so that this likelihood function is maximized. We can see that the form of the likelihood function is a product of two large products from the two big π symbols. The first product contains the likelihood of all our observations for class 1, and the second product contains the likelihood of all our observations for class 0. We often refer to the log likelihood of the data, which is computed by taking the logarithm of the likelihood function and using the fact that the logarithm of a product of terms is the sum of the logarithm of each term: We can simplify this even further using a classic trick to form just a single sum: To see why this is true, note that for the observations where the actual value of the output variable y is 1, the right term inside the summation is zero, so we are effectively left with the first sum from the previous equation. Similarly, when the actual value of y is 0, then we are left with the second summation from the previous equation. Note that maximizing the likelihood is equivalent to maximizing the log likelihood. Maximum likelihood estimation is a fundamental technique of parameter fitting and we will encounter it in other models in this book. Despite its popularity, it should be noted that maximum likelihood is not a panacea. Alternative training criteria on which to build a model exist, and there are some well-known scenarios under which this approach does not lead to a good model. Finally, note that the details of the actual optimization procedure that finds the values of the regression coefficients for maximum likelihood are beyond the scope of this book and in general, we can rely on R to implement this for us. Summary In this article, we demonstrated why logistic regression is a better way to approach classification problems compared to linear regression with a threshold by showing that the least squares criterion is not the most appropriate criterion to use when trying to separate two classes. It turns out that logistic regression is not a great choice for multiclass settings in general. To learn more about Predictive Analysis, the following books published by Packt Publishing (https://www.packtpub.com/) are recommended: Learning Predictive Analytics with R (https://www.packtpub.com/big-data-and-business-intelligence/learning-predictive-analytics-r) Resources for Article: Further resources on this subject: Machine learning in practice [article] Introduction to Machine Learning with R [article] Training and Visualizing a neural network with R [article]

In this article by Satheesh PV, author of the book Unreal Engine 4 Game Development Essentials, we will learn that lighting is an important factor in your game, which can be easily overlooked, and wrong usage can severely impact on performance. But with proper settings, combined with post process, you can create very beautiful and realistic scenes. We will see how to place lights and how to adjust some important values. (For more resources related to this topic, see here.) Placing lights In Unreal Engine 4, lights can be placed in two different ways. Through the modes tab or by right-clicking in the level: Modes tab: In the Modes tab, go to place tab (Shift + 1) and go to the Lights section. From there you can drag and drop various lights. Right-clicking: Right-click in viewport and in Place Actor you can select your light. Once a light is added to the level, you can use transform tool (W to move, E to rotate) to change the position and rotation of your selected light. Since Directional Light casts light from an infinite source, updating their location has no effect. Various lights Unreal Engine 4 features four different types of light Actors. They are: Directional Light: Simulates light from a source that is infinitely far away. Since all shadows casted by this light will be parallel, this is the ideal choice for simulating sunlight. Spot Light: Emits light from a single point in a cone shape. There are two cones (inner cone and outer cone). Within inner cone, light achieves full brightness and between inner and outer cone a falloff takes place, which softens the illumination. That means after inner cone, light slowly loses its illumination as it goes to outer cone. Point Light: Emits light from a single point to all directions, much like a real-world light bulb. Sky Light: Does not really emit light, but instead captures the distant parts of your scene (for example, Actors that are placed beyond Sky Distance Threshold) and applies them as light. That means you can have lights coming from your atmosphere, distant mountains, and so on. Note that Sky Light will only update when you rebuild your lighting or by pressing Recapture Scene (in the Details panel with Sky Light selected). Common light settings Now that we know how to place lights into a scene, let's take a look at some of the common settings of a light. Select your light in a scene and in the Details panel you will see these settings: Intensity: Determines the intensity (energy) of the light. This is in lumens units so, for example, 1700 lm (lumen units) corresponds to a 100 W bulb. Light Color: Determines the color of light. Attenuation Radius: Sets the limit of light. It also calculates the falloff of light. This setting is only available in Point Lights and Spot Lights. Attenuation Radius from left to right: 100, 200, 500. Source Radius: Defines the size of specular highlights on surfaces. This effect can be subdued by adjusting the Min Roughness setting. This also affects building light using Lightmass. Larger Source Radius will cast softer shadows. Since this is processed by Lightmass, it will only work on Lights with mobility set to Static Source Radius 0. Notice the sharp edges of the shadow. Source Radius 0. Notice the sharp edges of the shadow. Source Length: Same as Source Radius. Light mobility Light mobility is an important setting to keep in mind when placing lights in your level because this changes the way light works and impacts on performance. There are three settings that you can choose. They are as follows: Static: A completely static light that has no impact on performance. This type of light will not cast shadows or specular on dynamic objects (for example, characters, movable objects, and so on). Example usage: Use this light where the player will never reach, such as distant cityscapes, ceilings, and so on. You can literally have millions of lights with static mobility. Stationary: This is a mix of static and dynamic light and can change its color and brightness while running the game, but cannot move or rotate. Stationary lights can interact with dynamic objects and is used where the player can go. Movable: This is a completely dynamic light and all properties can be changed at runtime. Movable lights are heavier on performance so use them sparingly. Only four or fewer stationary lights are allowed to overlap each other. If you have more than four stationary lights overlapping each other the light icon will change to red X, which indicates that the light is using dynamic shadows at a severe performance cost! In the following screenshot, you can easily see the overlapping light. Under View Mode, you can change to Stationary Light Overlap to see which light is causing an issue. Summary We will look into different light mobilities and learn more about Lightmass Global Illumination, which is the static Global Illumination solver created by Epic games. We will also learn how to prepare assets to be used with it. Resources for Article:   Further resources on this subject: Understanding Material Design [article] Build a First Person Shooter [article] Machine Learning With R [article]

In this article by Julien Moreau-Mathis, the author of the book, Babylon.JS Essentials, you will learn about creating and customizing the scenes using the materials and meshes provided by the designer. (For more resources related to this topic, see here.) In this article, let's play with the gameplay itself and create interactions with the objects in the scene by creating collisions and physics simulations. Collisions are important to add realism in your scenes if you want to walk without crossing the walls. Moreover, to make your scenes more alive, let's introduce the physics simulation with Babylon.js and, finally, see how it is easy to integrate these two notions in your scenes. We are going to discuss the following topics: Checking collisions in a scene Physics simulation Checking collisions in a scene Starting from the concept, configuring and checking collisions in a scene can be done without mathematical notions. We all have the notions of gravity and ellipsoid even if the ellipsoid word is not necessarily familiar. How collisions work in Babylon.js? Let's start with the following scene (a camera, light, plane, and box): The goal is to block the current camera of the scene in order not to cross the objects in the scene. In other words, we want the camera to stay above the plane and stop moving forward if it collides with the box. To perform these actions, the collisions system provided by Babylon.js uses what we call a collider. A collider is based on the bounding box of an object that looks as follows: A bounding box simply represents the minimum and maximum positions of a mesh's vertices and is computed automatically by Babylon.js. This means that you have to do nothing particularly tricky to configure collisions on objects. All the collisions are based on these bounding boxes of each mesh to determine if the camera should be blocked or not. You'll also see that the physics engines use the same kind of collider to simulate physics. If the geometry of a mesh has to be changed by modifying the vertices' positions, the bounding box information can be refreshed by calling the following code: myMesh.refreshBoundingInfo(); To draw the bounding box of an object, simply set the .showBoundingBox property of the mesh instance to true with the following code: myMesh.showBoundingBox = true; Configuring collisions in a scene Let's practice with the Babylon.js collisions engine itself. You'll see that the engine is particularly well hidden as you have to enable checks on scenes and objects only. First, configure the scene to enable collisions and then wake the engine up. If the following property is false, all the next properties will be ignored by Babylon.js and the collisions engine will be in stand-by mode. Then, it's easy to enable or disable collisions in a scene without modifying more properties: scene.collisionsEnabled = true; // Enable collisions in scene Next, configure the camera to check collisions. Then, the collisions engine will check collisions for all the rendered cameras that have collisions enabled. Here, we have only one camera to configure: camera.checkCollisions = true; // Check collisions for THIS camera To finish, just set the .checkCollisions property of each mesh to true to active collisions (the plane and box): plane.checkCollisions = true; box.checkCollisions = true; Now, the collisions engine will check the collisions in the scene for the camera on both the plane and box meshes. You guessed it; if you want to enable collisions only on the plane and you want the camera to walk across the box, you'll have to set the .checkCollisions property of the box to false: plane.checkCollisions = true; box.checkCollisions = false; Configuring gravity and ellipsoid In the previous section, the camera checks the collision on the plane and box but is not submitted to a famous force named gravity. To enrich the collisions in your scene, you can apply the gravitational force, for example, to go down the stairs. First, enable the gravity on the camera by setting the .applyGravity property to true: camera.applyGravity = true; // Enable gravity on the camera Customize the gravity direction by setting BABYLON.Vector3 to the .gravity property of your scene: scene.gravity = new BABYLON.Vector3(0.0, -9.81, 0.0); // To stay on earth Of course, the gravity in space should be as follows: scene.gravity = BABYLON.Vector3.Zero(); // No gravity in space Don't hesitate to play with the values in order to adjust the gravity to your scene referential. The last parameter to enrich the collisions in your scene is the camera's ellipsoid. The ellipsoid represents the camera's dimensions in the scene. In other words, it adjusts the collisions according to the X, Y, and Z axes of the ellipsoid (an ellipsoid is represented by BABYLON.Vector3). For example, the camera must measure 1m80 (Y axis) and the minimum distance to collide on the X (sides) and Z (forward) axes must be 1 m. Then, the ellipsoid must be (X=1, Y=1.8, Z=1). Simply set the .ellipsoid property of the camera as follows: camera.ellipsoid = new BABYLON.Vector3(1, 1.8, 1); The default value of the cameras ellipsoids is (X=0.5, Y=1.0, Z=0.5) As for the gravity, don't hesitate to adjust the X, Y, and Z values to your scene referential. Physics simulation Physics simulation is pretty different from the collisions system as it does not occur on the cameras but on the objects of the scene themselves. In other words, if physics is enabled (and configured) on a box, the box will interact with the other meshes in the scene and will try to represent the real physical movements. For example, let's take a sphere in the air. If you apply physics on the sphere, the sphere will fall until it collides with another mesh and, according to the given parameters, will tend to bounce and roll in the scene. The example files reproduce the behavior with a sphere that falls on the box in the middle of the sphere. Enabling physics in Babylon.js In Babylon.js, physics simulations can be done using plugins only. Two plugins are available using either the Cannon.js framework or the Oimo.js framework. These two frameworks are included in the Babylon.js GitHub repository in the dist folder. Each scene has its own physics simulations system and can be enabled with the following lines: var gravity = new BABYLON.Vector3(0, -9.81, 0); scene.enablePhysics(gravity, new BABYLON.OimoJSPlugin()); // or scene.enablePhysics(gravity, new BABYLON.CannonJSPlugin()); The .enablePhysics(gravity, plugin) function takes the following two arguments: The gravitational force of the scene to apply on objects The plugin to use: Oimo.js: This is the new BABYLON.OimoJSPlugin() plugin Cannon.js: This is the new BABYLON.CannonJSPlugin() plugin To disable physics in a scene, simply call the .disablePhysicsEngine() function using the following code: scene.disablePhysicsEngine(); Impostors Once the physics simulations are enabled in a scene, you can configure the physics properties (or physics states) of the scene meshes. To configure the physics properties of a mesh, the BABYLON.Mesh class provides a function named .setPhysicsState(impostor, options). The parameters are as follows: The impostor: Each kind of mesh has its own impostor according to their forms. For example, a box will tend to slip while a sphere will tend to roll. There are several types of impostors. The options: These options define the values used in physics equations. They count the mass, friction, and restitution. Let's have a box named box with a mass of 1 and set its physics properties: box.setPhysicsState(BABYLON.PhysicsEngine.BoxImpostor, { mass: 1 }); This is all; the box is now configured to interact with other configured meshes by following the physics equations. Let's imagine that the box is in the air. It will fall until it collides with another configured mesh. Now, let's have a sphere named sphere with a mass of 2 and set its physics properties: sphere.setPhysicsState(BABYLON.PhysicsEngine.SphereImpostor, { mass: 2 }); You'll notice that the sphere, which is a particular kind of mesh, has its own impostor (sphere impostor). In contrast to the box, the physics equations of the plugin will make the sphere roll while the box will slip on other meshes. According to their mass, if the box and sphere collide together, then the sphere will tend to push harder the box. The available impostors in Babylon.js are as follows: The box impostor: BABYLON.PhysicsEngine.BoxImpostor The sphere impostor: BABYLON.PhysicsEngine.SphereImpostor The plane impostor: BABYLON.PhysicsEngine.PlaneImpostor The cylinder impostor: BABYLON.PhysicsEngine.CylinderImpostor In fact, in Babylon.js, the box, plane, and cylinder impostors are the same according to the Cannon.js and Oimo.js plugins. Regardless of the impostor parameter, the options parameter is the same. In these options, there are three parameters, which are as follows: The mass: This represents the mass of the mesh in the world. The heavier the mesh is, the harder it is to stop its movement. The friction: This represents the force opposed to the meshes in contact. In other words, it represents how the mesh is slippery. To give you an order, the glass as a friction equals to 1. We can determine that the friction is in the range [0, 1]. The restitution: This represents how the mesh will bounce on others. Imagine a ping-pong ball and its table. If the table's material is a carpet, the restitution will be small, but if the table's material is glass, the restitution will be maximum. A real interval for the restitution is in [0, 1]. In the example files, these parameters are set and if you play, you'll see that these three parameters are linked together in the physics equations. Appling a force (impulse) on a mesh At any moment, you can apply a new force or impulse to a configured mesh. Let's take an explosion: a box is located at the coordinates (X=0, Y=0, Z=0) and an explosion happens above the box at the coordinates (X=0, Y=-5, Z=0). In real life, the box should be pushed up: this action is possible by calling a function named .applyImpulse(force, contactPoint) provided by the BABYLON.Mesh class. Once the mesh is configured with its options and impostor, you can, at any moment, call this function to apply a force to the object. The parameters are as follows: The force: This represents the force in the X, Y, and Z axes The contact point: This represents the origin of the force located in the X, Y and Z axes For example, the explosion generates a force only on Y that is equal to 10 (10 is an arbitrary value) and has its origin at the coordinates (X=0, Y=-5, Z=0): mesh.applyImpulse(new BABYLON.Vector3(0, 10, 0), new BABYLON.Vector3(0, -5, 0)); Once the impulse is applied to the mesh (only once), the box is pushed up and will fall according to its physics options (mass, friction, and restitution). Summary You are now ready to configure collisions in your scene and simulate physics. Whether by code or by the artists, you learned the pipeline to make your scenes more alive. Resources for Article: Further resources on this subject: Building Games with HTML5 and Dart[article] Fundamentals of XHTML MP in Mobile Web Development[article] MODx Web Development: Creating Lists[article]

Recommender systems are common these days. You may not have noticed, but you might already be a user or receiver of such a system somewhere. Most of the well-performing e-commerce platforms use recommendation systems to recommend items to their users. When you see on the Amazon website that a book is recommended to you based on your earlier preferences, purchases, and browse history, Amazon is actually using such a recommendation system. Similarly, Netflix uses its recommendation system to suggest movies for you. (For more resources related to this topic, see here.) A recommender or recommendation system is used to recommend a product or information often based on user characteristics, preferences, history, and so on. So, a recommendation is always personalized. Until recently, it was not so easy or straightforward to build a recommender, but Azure ML makes it really easy to build one as long as you have your data ready. This article introduces you to the concept of recommendation systems and also the model available in ML Studio for you to build your own recommender system. It then walks you through the process of building a recommendation system with a simple example. The Matchbox recommender Microsoft has developed a large-scale recommender system based on a probabilistic model (Bayesian) called Matchbox. This model can learn about a user's preferences through observations made on how they rate items, such as movies, content, or other products. Based on those observations, it recommends new items to the users when requested. Matchbox uses the available data for each user in the most efficient way possible. The learning algorithm it uses is designed specifically for big data. However, its main feature is that Matchbox takes advantage of metadata available for both users and items. This means that the things it learns about one user or item can be transferred across to other users or items. You can find more information about the Matchbox model at the Microsoft Research project link. Kinds of recommendations The Matchbox recommender supports the building of four kinds of recommenders, which will include most of the scenarios. Let's take a look at the following list: Rating Prediction: This predicts ratings for a given user and item, for example, if a new movie is released, the system will predict what will be your rating for that movie out of 1-5. Item Recommendation: This recommends items to a given user, for example, Amazon suggests you books or YouTube suggests you videos to watch on its home page (especially when you are logged in). Related Users: This finds users that are related to a given user, for example, LinkedIn suggests people that you can get connected to or Facebook suggests friends to you. Related Items: This finds the items related to a given item, for example, a blog site suggests you related posts when you are reading a blog post. Understanding the recommender modules The Matchbox recommender comes with three components; as you might have guessed, a module each to train, score, and evaluate the data. The modules are described as follows. The train Matchbox recommender This module contains the algorithm and generates the trained algorithm, as shown in the following screenshot: This module takes the values for the following two parameters. The number of traits This value decides how many implicit features (traits) the algorithm will learn about that are related to every user and item. The higher this value, the precise it would be as it would lead to better prediction. Typically, it takes a value in the range of 2 to 20. The number of recommendation algorithm iterations It is the number of times the algorithm iterates over the data. The higher this value, the better would the predictions be. Typically, it takes a value in the range of 1 to 10. The score matchbox recommender This module lets you specify the kind of recommendation and corresponding parameters you want: Rating Prediction Item Prediction Related Users Related Items Let's take a look at the following screenshot: The ML Studio help page for the module provides details of all the corresponding parameters. The evaluate recommender This module takes a test and a scored dataset and generates evaluation metrics, as shown in the following screenshot: It also lets you specify the kind of recommendation, such as the score module and corresponding parameters. Building a recommendation system Now, it would be worthwhile that you learn to build one by yourself. We will build a simple recommender system to recommend restaurants to a given user. ML Studio includes three sample datasets, described as follows: Restaurant customer data: This is a set of metadata about customers, including demographics and preferences, for example, latitude, longitude, interest, and personality. Restaurant feature data: This is a set of metadata about restaurants and their features, such as food type, dining style, and location, for example, placeID, latitude, longitude, price. Restaurant ratings: This contains the ratings given by users to restaurants on a scale of 0 to 2. It contains the columns: userID, placeID, and rating. Now, we will build a recommender that will recommend a given number of restaurants to a user (userID). To build a recommender perform the following steps: Create a new experiment. In the Search box in the modules palette, type Restaurant. The preceding three datasets get listed. Drag them all to the canvas one after another. Drag a Split module and connect it to the output port of the Restaurant ratings module. On the properties section to the right, choose Splitting mode as Recommender Split. Leave the other parameters at their default values. Drag a Project Columns module to the canvas and select the columns: userID, latitude, longitude, interest, and personality. Similarly, drag another Project Columns module and connect it to the Restaurant feature data module and select the columns: placeID, latitude, longitude, price, the_geom_meter, and address, zip. Drag a Train Matchbox Recommender module to the canvas and make connections to the three input ports, as shown in the following screenshot: Drag a Score Matchbox Recommender module to the canvas and make connections to the three input ports and set the property's values, as shown in the following screenshot: Run the experiment and when it gets completed, right-click on the output of the Score Matchbox Recommender module and click on Visualize to explore the scored data. You can note the different restaurants (IDs) recommended as items for a user from the test dataset. The next step is to evaluate the scored prediction. Drag the Evaluate Recommender module to the canvas and connect the second output of the Split module to its first input port and connect the output of the Score Matchbox Recommender module to its second input. Leave the module at its default properties. Run the experiment again and when finished, right-click on the output port of the Evaluate Recommender module and click on Visualize to find the evaluation metric. The evaluation metric Normalized Discounted Cumulative Gain (NDCG) is estimated from the ground truth ratings given in the test set. Its value ranges from 0.0 to 1.0, where 1.0 represents the most ideal ranking of the entities. Summary You started with gaining the basic knowledge about a recommender system. You then understood the Matchbox recommender that comes with ML Studio along with its components. You also explored different kinds of recommendations that you can make with it. Finally, you ended up building a simple recommendation system to recommend restaurants to a given user. For more information on Azure, take a look at the following books also by Packt Publishing: Learning Microsoft Azure (https://www.packtpub.com/networking-and-servers/learning-microsoft-azure) Microsoft Windows Azure Development Cookbook (https://www.packtpub.com/application-development/microsoft-windows-azure-development-cookbook) Resources for Article: Further resources on this subject: Introduction to Microsoft Azure Cloud Services[article] Microsoft Azure – Developing Web API for Mobile Apps[article] Security in Microsoft Azure[article]

In this article, we will cover: Twitter and it's importance Getting hands on with Twitter's data and using various Twitter APIs Use of data to solve business problems—comparison of various businesses based on tweets (For more resources related to this topic, see here.) Twitter and its importance Twitter can be considered as extension of the short messages service or SMS but on an Internet-based platform. In the words of Jack Dorsey, co-founder and co-creator of Twitter: "...We came across the word 'twitter', and it was just perfect. The definition was 'a short burst of inconsequential information,' and 'chirps from birds'. And that's exactly what the product was" Twitter acts as a utility where one can send their SMSs to the whole world. It enables people to instantaneously get heard and get a response. Since the audience of this SMS is so large, many a times responses are very quick. So, Twitter facilitates the basic social instincts of humans. By sharing on Twitter, a user can easily express his/her opinion for just about everything and at anytime. Friends who are connected or, in case of Twitter, followers, immediately get the information about what's going on in someone's life. This in turn severs another humanemotion—the innate need to know about what is going on in someone's life. Apart from being real time, Twitter's UI is really easy to work with. It's naturally and instinctively understood, that is, the UI is very intuitive in nature. Each tweet on Twitter is a short message with maximum of 140 characters. Twitter is an excellent example of a microblogging service. As of July 2014, the Twitter user base reached above 500 million, with more than 271 million active users. Around 23 percent are adult Internet users, which is also about 19 percent of the entire adult population. If we can properly mine what users are tweeting about, Twitter can act as a great tool for advertisement and marketing. But this not the only information Twitter provides. Because of its non-symmetric nature in terms of followers and followings, Twitter assists better in terms of understanding user interests rather than its impact on the social network. An interest graph can be thought of as a method to learn the links between individuals and their diverse interests. Computing the degree of association or correlations between individual's interests and the potential advertisements are one of the most important applications of the interest graphs. Based on these correlations, a user can be targeted so as to attain a maximum response to an advertisement campaign along with followers' recommendations. One interesting fact about Twitter (and Facebook) is that the user does not need to be a real person. A user on Twitter (or on Facebook) can be anything and anyone, for example, an organization, a campaign itself, a famous but imaginary personality (a fictional character recognizable in the media) apart from a real/actual person. If a real person follows these users on Twitter, a lot can be inferred about their personality and hence they can be recommended ads or other followers based on such information. For example, @fakingnews is an Indian blog that publishes news satires ranging from Indian politics to typical Indian mindsets. People who follow @fakingnews are the ones who, in general, like to read sarcasm news. Hence, these people can be thought of as to belonging to the same cluster or a community. If we have another sarcastic blog, we can always recommend it to this community and improve on advertisement return on investment. The chances of getting more hits via people belonging to this community will be higher than a community who don't follows @fakingnews, or any such news, in general. Once you have comprehended that Twitter allows you to create, link, and investigate a community of interest for a random topic, the influence of Twitter and the knowledge one can find from mining it becomes clearer. Understanding Twitter's API Twitter APIs provide a means to access the Twitter data, that is, tweets sent by its millions of users. Let's get to know these APIs a bit better. Twitter vocabulary As described earlier, Twitter is a microblogging service with social aspect associated. It allows its users to express their views/sentiments with the means of Internet SMS, called tweets in the context of Twitter. These tweets are entities formed of maximum of 140 characters. The content of these tweets can be anything ranging from a person's mood to person's location to a person's curiosity. The platform where these tweets are posted is called Timeline. To use Twitter's APIs, one must understand the basic terminology. Tweets are the crux of Twitter. Theoretically, a tweet is just 140 characters of text content tweeted by a user, but there is more to it than just that. There is more metadata associated with the same tweet, which are classified by Twitter as entities and places. The entities constitute of hash tags, URLs, and other media data that users have included in their tweet. The places are nothing but locations from where the tweet originated. It possible the place is a real world location from where the tweet was sent, or it is a location mentioned in the text of the tweet. Take the following tweet as an example: Learn how to consume millions of tweets with @twitterapi at #TDC2014 in São Paulo #bigdata tomorrow at 2:10pm http://t.co/pTBlWzTvVd The preceding tweet was tweeted by @TwitterDev and it's about 132 characters long. The following are the entities mentioned in this tweet: Handle: @twitterapi Hashtags: #TDC2014, #bigdata URL: http://t.co/pTBlWzTvVd São Paulo is the place mentioned in this tweet. This is a one such example of a tweet with a fairly good amount of metadata. Although the actual tweet's length is well within the 140-character limit, it contains more information than one can think of. This actually enables us to figure out that this tweet belongs to a specific community based on the cross referencing the topics presents in the hash tags, the URL to the website, the different users mentioned in it, and so on. The interface (web or mobile) on to which the tweets are displayed is called timeline. The tweets are, in general, arranged in chronological order of posting time. On a specific user's account, only certain number of tweets are displayed by Twitter. This is generally based on users the given user is following and is being followed by. This is the interface a user will see when he/she login his/her Twitter account. A Twitter stream is different from Twitter timeline in the sense that they are not for a specific user. The Tweets on a user's Twitter timeline will be displayed from only certain number of users will be displayed/updated less frequently while the Twitter stream is chronological collection of the all the tweets posted by all the users. The number of active users on Twitter is in orders of hundreds of millions. All the users tweeting during some public events of widespread interest such as presidential debates can achieve speeds of several hundreds of thousands of tweets per minute. The behavior is very similar to a stream; hence the name of such collection is Twitter stream. You can try the following by creating a Twitter account (it would be more insightful if you have less number of followers already with you). Before creating the account, it is advised that you read all the terms and conditions of the same. You can also start reading its API's documentation. Creating a Twitter API connection We need to have an app created at https://dev.twitter.com/apps before making any API requests to Twitter. It's a standard method for developers to gain API access and more important it helps Twitter to observe and restricts developer from making high load API requests. The ROAuth package is the one we are going to use in our experiments. Tokens allow users to authorize third-party apps to access the data from any user account without the need to have their passwords (or other sensitive information). ROAuth basically facilitates the same. Creating new app The first step to getting any kind of token access from twitter is to create an app on it. The user has to go to https://dev.twitter.com/ and log in with their Twitter credentials. With you logged in using your credentials, the step for creating app are as follows: Go to https://apps.twitter.com/app/new. Put the name of your application in the Name field. This name can be anything you like. Similarly, enter the description in the Description field. The Website field needs to be filled with a valid URL, but again that can be any random URL. You can leave the Callback URL field blank. After the creation of this app, we need to find the API Key and API Secret values from the Key and Access Token tab. Consider the example shown in the following figure:   Under the Key and Access Tokens tab, you will find a button to generate access tokens. Click on it and you will be provided with an Access Token and Access Token Secret value. Before using the preceding keys, we need to install twitteRto access the data in R using the app we just created, using following code: Install.packages(c("devtools", "rjson", "bit64", "httr")) library(devtools) install_github("geoffjentry/twitteR"). library(twitteR) Here's sample code that helps us access the tweets posted since any give date and which contain a specific keyword. In this example, we are searching for tweeting containing the word Earthquake in the tweets posted since September 29, 2014. In order to get this information, we provide four special types of information to get the authorization token: key secret access token access token secret We'll show you how to use the preceding information to get an app authorized by the user and access its resources on Twitter. The ROAuh function in twitteR will make our next steps very smooth and clear: api_key<- "your_api_key" api_secret<- "your_api_secret" access_token<- "your_access_token" access_token_secret<- "your_access_token_secret" setup_twitter_oauth (api_key,api_secret,access_token,access_token_secret) EarthQuakeTweets = searchTwitter("EarthQuake", since='2014-09-29') The results of this example should simply display Using direct authentication with 25 tweets loaded in the EarthQuakeTweets variable as shown here. head(EarthQuakeTweets,2) [[1]] [1] "TamamiJapan: RT @HistoricalPics: Japan. Top: One Month After Hiroshima, 1945. Bottom: One Month After The Earthquake and Tsunami, 2011. Incredible. http…" [[2]] [1] "OldhamDs: RT @HistoricalPics: Japan. Top: One Month After Hiroshima, 1945. Bottom: One Month After The Earthquake and Tsunami, 2011. Incredible. http…" We have shown in the first two of the 25 tweets containing the word Earthquake since September 29, 2014. If you closely observe the results, you'll find all the metadata using str(EarthQuakeTweets[1]). Finding trending topics Now that we understand how to create API connections to Twitter and fetch data using it, we will see how to get answer to what is trending on Twitter to list what topic (worldwide or local) is being talked about the most right now. Using the same API, we can easily access the trending information: #return data frame with name, country & woeid. Locs <- availableTrendLocations() # Where woeid is a numerical identification code describing a location ID # Filter the data frame for Delhi (India) and extract the woeid of the same LocsIndia = subset(Locs, country == "India") woeidDelhi = subset(LocsIndia, name == "Delhi")$woeid # getTrends takes a specified woeid and returns the trending topics associated with that woeid trends = getTrends(woeid=woeidDelhi) The function availableTrendLocations() returns R data frame containing the name, country, and woeid parameters. We than filter this data frame for a location of our choosing; in this example, its Delhi, India. The function getTrends() fetches the top 10 trends in the location determined by the woeid. Here are the top four trending hash tags in the region defined by woeid = 20070458, that is, Delhi, India. head(trends) name url query woeid 1 #AntiHinduNGOsExposed http://twitter.com/search?q=%23AntiHinduNGOsExposed %23AntiHinduNGOsExposed 20070458 2 #KhaasAadmi http://twitter.com/search?q=%23KhaasAadmi %23KhaasAadmi 20070458 3 #WinGOSF14 http://twitter.com/search?q=%23WinGOSF14 %23WinGOSF14 20070458 4 #ItsForRealONeBay http://twitter.com/search?q=%23ItsForRealONeBay %23ItsForRealONeBay 20070458 Searching tweets Now, similar to the trends there is one more important function that comes with the TwitteR package: searchTwitter(). This function will return tweets containing the searched string along with the other constraints. Some of the constraints that can be imposed are as follows: lang: This constraints the tweets of given language. since/until: This constraints the tweets to be since the given date or until the given date. geocode: This constraints tweets to be from only those users who are located within certain distance from the given latitude/longitude. For example, extracting tweets about the cricketer Sachin Tendulkar in the month of November 2014: head(searchTwitter('Sachin Tendulkar', since='2014-11-01', until= '2014-11-30')) [[1]] [1] "TendulkarFC: RT @Moulinparikh: Sachin Tendulkar had a long session with the Mumbai Ranji Trophy team after today's loss." [[2]] [1] "tyagi_niharika: @WahidRuba @Anuj_dvn @Neel_D_ @alishatariq3 @VWellwishers @Meenal_Rathore oh... Yaadaaya....hmaraesachuuu sirxedxa0xbdxedxb8x8d..i mean sachin Tendulkar" [[3]] [1] "Meenal_Rathore: @WahidRuba @Anuj_dvn @tyagi_niharika @Neel_D_ @alishatariq3 @AliaaFcc @VWellwishers .. Sachin Tendulkar xedxa0xbdxedxb8x8a☺️" [[4]] [1] "MishraVidyanand: Vidyanand Mishra is following the Interest "The Living Legend SachinTendu..." on http://t.co/tveHXMB4BM - http://t.co/CocNMcxFge" [[5]] [1] "CSKalwaysWin: I have never tried to compare myself to anyone else.n - Sachin Tendulkar" Twitter sentiment analysis Depending on the objective and based on the functionality to search any type of tweets from the public timeline, one can always collect the required corpus. For example, you may want to learn about customer satisfaction levels with various cab services, which are coming in Indian market. These start-ups are offering various discounts and coupons to attract customers but at the end of the day, the service quality determines the business of any organization. These startups are constantly promoting themselves on various social media websites. Customers are showing various levels of sentiments on the same platform. Let's target the following: Meru Cabs: A radio cabs service based in Mumbai, India. Launched in 2007. Ola Cabs: A taxi aggregator company based in Bangalore, India. Launched in 2011. TaxiForSure: A taxi aggregator company based in Bangalore, India. Launched in 2011. Uber India: A taxi aggregator company headquartered in San Francisco, California. Launched in India in 2014. Let's set our goal to get the general sentiments about each of the preceding services providers based on the customer sentiments present in the tweets on Twitter. Collecting tweets as a corpus We'll start with the searchTwitter()function (discussed previously) on the TwitteR package to gather the tweets for each of the preceding organizations. Now, in order to avoid writing same code again and again, we pushed the following authorization code in the file called authenticate.R. library(twitteR) api_key<- "xx" api_secret<- "xx" access_token<- "xx" access_token_secret<- "xx" setup_twitter_oauth(api_key,api_secret,access_token, access_token_secret) We run the following scripts to get the required tweets: # Load the necessary packages source('authenticate.R') Meru_tweets = searchTwitter("MeruCabs", n=2000, lang="en") Ola_tweets = searchTwitter("OlaCabs", n=2000, lang="en") TaxiForSure_tweets = searchTwitter("TaxiForSure", n=2000, lang="en") Uber_tweets = searchTwitter("Uber_Delhi", n=2000, lang="en") Now, as mentioned in Twitter's Rest API documentation, we get the message "Due to capacity constraints, the index currently only covers about a week's worth of tweets". We do not always get the desired number of tweets (for example, here it's 2000). Instead, the following are the size of each of the above Tweet lists we get the following: >length(Meru_tweets) [1] 393 >length(Ola_tweets) [1] 984 > length(TaxiForSure_tweets) [1] 720 > length(Uber_tweets) [1] 2000 As you can see from the preceding code, the length of these tweets is not equal to the number of tweets we had asked for in our query scripts. There are many takeaways from this information. Since these tweets are only from last one week's tweets on Twitter, they suggest there is more discussion about these taxi services in the following order: Uber India Ola Cabs TaxiForSure Meru Cabs A ban was imposed on Uber India after an alleged rape incident by one Uber India driver. The decision to put a ban on the entire organization because one of its drivers committed a crime became a matter of public outcry. Hence, the number of tweets about Uber increased on social media. Now, Meru Cabs have been in India for almost 7 years now. Hence, they are quite a stable organization. They amount of promotion Ola Cabs and TaxiForSure are doing is way higher than that of Meru Cabs. This can be one reason for Meru Cabs having theleast number (393) of tweets in last week. The number of tweets in last week is comparable for Ola Cabs (984) and TaxiForSure (720). There can be several numbers of reasons for the same. They were both started their business in same year and more importantly they follow the same business model. While Meru Cabs is a radio taxi service and they own and manage a fleet of cars while Ola Cabs, TaxiForSure, or Uber are a marketplace for users to compare the offerings of various operators and book easily. Let's dive deep into the data and get more insights. Cleaning the corpus Before applying any intelligent algorithms to gather more insights out of the tweets collected so far, let's first clean it. In order to clean up, we should understand how the list of tweets looks like: head(Meru_tweets) [[1]] [1] "MeruCares: @KapilTwitts 2&gt;...and other details at feedback@merucabs.com We'll check back and reach out soon." [[2]] [1] "vikasraidhan: @MeruCabs really disappointed with @GenieCabs. Cab is never assigned on time. Driver calls after 30 minutes. Why would I ride with Meru?" [[3]] [1] "shiprachowdhary: fallback of #ubershame , #MERUCABS taking customers for a ride" [[4]] [1] "shiprachowdhary: They book Genie, but JIT inform of cancellation &amp; send full fare #MERUCABS . Very disappointed.Always used these guys 4 and recommend them." [[5]] [1] "shiprachowdhary: No choice bt to take the #merucabs premium service. Driver told me that this happens a lot with #merucabs." [[6]] [1] "shiprachowdhary: booked #Merucabsyestrdy. Asked for Meru Genie. 10 mins 4 pick up time, they call to say Genie not available, so sending the full fare cab" The first tweet here is a grievance solution, while the second, fourth and fifth are actually customer sentiments about the services provided by Meru Cabs. We see: Lots of meta information such as @people, URLs and #hashtags Punctuation marks, numbers, and unnecessary spaces Some of these tweets are retweets from other users; for the given application, we would not like to consider retweets (RTs) in sentiment analysis We clean all these data using the following code block: MeruTweets <- sapply(Meru_tweets, function(x) x$getText()) OlaTweets = sapply(Ola_tweets, function(x) x$getText()) TaxiForSureTweets = sapply(TaxiForSure_tweets, function(x) x$getText()) UberTweets = sapply(Uber_tweets, function(x) x$getText()) catch.error = function(x) { # let us create a missing value for test purpose y = NA # Try to catch that error (NA) we just created catch_error = tryCatch(tolower(x), error=function(e) e) # if not an error if (!inherits(catch_error, "error")) y = tolower(x) # check result if error exists, otherwise the function works fine. return(y) } cleanTweets<- function(tweet){ # Clean the tweet for sentiment analysis # remove html links, which are not required for sentiment analysis tweet = gsub("(f|ht)(tp)(s?)(://)(.*)[.|/](.*)", " ", tweet) # First we will remove retweet entities from the stored tweets (text) tweet = gsub("(RT|via)((?:\b\W*@\w+)+)", " ", tweet) # Then remove all "#Hashtag" tweet = gsub("#\w+", " ", tweet) # Then remove all "@people" tweet = gsub("@\w+", " ", tweet) # Then remove all the punctuation tweet = gsub("[[:punct:]]", " ", tweet) # Then remove numbers, we need only text for analytics tweet = gsub("[[:digit:]]", " ", tweet) # finally, we remove unnecessary spaces (white spaces, tabs etc) tweet = gsub("[ t]{2,}", " ", tweet) tweet = gsub("^\s+|\s+$", "", tweet) # if anything else, you feel, should be removed, you can. For example "slang words" etc using the above function and methods. # Next we'll convert all the word in lower case. This makes uniform pattern. tweet = catch.error(tweet) tweet } cleanTweetsAndRemoveNAs<- function(Tweets) { TweetsCleaned = sapply(Tweets, cleanTweets) # Remove the "NA" tweets from this tweet list TweetsCleaned = TweetsCleaned[!is.na(TweetsCleaned)] names(TweetsCleaned) = NULL # Remove the repetitive tweets from this tweet list TweetsCleaned = unique(TweetsCleaned) TweetsCleaned } MeruTweetsCleaned = cleanTweetsAndRemoveNAs(MeruTweets) OlaTweetsCleaned = cleanTweetsAndRemoveNAs(OlaTweets) TaxiForSureTweetsCleaned <- cleanTweetsAndRemoveNAs(TaxiForSureTweets) UberTweetsCleaned = cleanTweetsAndRemoveNAs(UberTweets) Here's the size of each of the cleaned tweet lists: > length(MeruTweetsCleaned) [1] 309 > length(OlaTweetsCleaned) [1] 811 > length(TaxiForSureTweetsCleaned) [1] 574 > length(UberTweetsCleaned) [1] 1355 Estimating sentiment (A) There are many sophisticated resources available to estimate sentiments. Many research papers and software packages are available open source,and they implement very complex algorithms for sentiments analysis. After getting the cleaned Twitter data, we are going to use few of such R packages available to assess the sentiments in the tweets. It's worth mentioning here that not all the tweets represent a sentiment. Few tweets can be just information/facts, while others can be customer care responses. Ideally, they should not be used to assess the customer sentiment about a particular organization. As a first step, we'll use a Naïve algorithm, which gives a score based on the number of times a positive or a negative word occurred in the given sentence (and in our case, in a tweet). Please download the positive and negative opinion/sentiment (nearly 68, 000) words from English language. These opinion lexicon will be used as a first example in our sentiment analysis experiment. The good thing about this approach is that we are relying on a highly researched upon and at the same time customizable input parameters. Here are a few examples of existing positive and negative sentiments words: Positive: Love, best, cool, great, good, and amazing Negative: Hate, worst, sucks, awful, and nightmare >opinion.lexicon.pos = scan('opinion-lexicon-English/positive-words.txt', what='character', comment.char=';') >opinion.lexicon.neg = scan('opinion-lexicon-English/negative-words.txt', what='character', comment.char=';') > head(opinion.lexicon.neg) [1] "2-faced" "2-faces" "abnormal" "abolish" "abominable" "abominably" > head(opinion.lexicon.pos) [1] "a+" "abound" "abounds" "abundance" "abundant" "accessable" We'll add a few industry-specific and/or especially emphatic terms based on our requirements: pos.words = c(opinion.lexicon.pos,'upgrade') neg.words = c(opinion.lexicon.neg,'wait', 'waiting', 'wtf', 'cancellation') Now, we create a function score.sentiment(), which computes the raw sentiment based on the simple matching algorithm: getSentimentScore = function(sentences, words.positive, words.negative, .progress='none') { require(plyr) require(stringr) scores = laply(sentences, function(sentence, words.positive, words.negative) { # Let first remove the Digit, Punctuation character and Control characters: sentence = gsub('[[:cntrl:]]', '', gsub('[[:punct:]]', '', gsub('\d+', '', sentence))) # Then lets convert all to lower sentence case: sentence = tolower(sentence) # Now lets split each sentence by the space delimiter words = unlist(str_split(sentence, '\s+')) # Get the boolean match of each words with the positive & negative opinion-lexicon pos.matches = !is.na(match(words, words.positive)) neg.matches = !is.na(match(words, words.negative)) # Now get the score as total positive sentiment minus the total negatives score = sum(pos.matches) - sum(neg.matches) return(score) }, words.positive, words.negative, .progress=.progress ) # Return a data frame with respective sentence and the score return(data.frame(text=sentences, score=scores)) } Now, we apply the preceding function on the corpus of tweets collected and cleaned so far: MeruResult = getSentimentScore(MeruTweetsCleaned, words.positive , words.negative) OlaResult = getSentimentScore(OlaTweetsCleaned, words.positive , words.negative) TaxiForSureResult = getSentimentScore(TaxiForSureTweetsCleaned, words.positive , words.negative) UberResult = getSentimentScore(UberTweetsCleaned, words.positive , words.negative) Here are some sample results: Tweet for Meru Cabs Score gt and other details at feedback com we ll check back and reach out soon 0 really disappointed with cab is never assigned on time driver calls after minutes why would i ride with meru -1 so after years of bashing today i m pleasantly surprised clean car courteous driver prompt pickup mins efficient route 4 a min drive cost hrs used to cost less ur unreliable and expensive trying to lose ur customers -3 Tweet For Ola Cabs Score the service is going from bad to worse the drivers deny to come after a confirmed booking -3 love the olacabs app give it a swirl sign up with my referral code dxf n and earn rs download the app from 1 crn kept me waiting for mins amp at last moment driver refused pickup so unreliable amp irresponsible -4 this is not the first time has delighted me punctuality and free upgrade awesome that 4 Tweet For TaxiForSure Score great service now i have become a regular customer of tfs thank you for the upgrade as well happy taxi ing saving 5 really disappointed with cab is never assigned on time driver calls after minutes why would i ride with meru -1 horrible taxi service had to wait for one hour with a new born in the chilly weather of new delhi waiting for them -4 what do i get now if you resolve the issue after i lost a crucial business because of the taxi delay -3 Tweet For Uber India Score that s good uber s fares will prob be competitive til they gain local monopoly then will go sky high as in new york amp delhi saving 3 from a shabby backend app stack to daily pr fuck ups its increasingly obvious that is run by child minded blow hards -3 you say that uber is illegally running were you stupid to not ban earlier and only ban it now after the rape -3 perhaps uber biz model does need some looking into it s not just in delhi that this happens but in boston too 0 From the preceding observations, it's clear that this basic sentiment analysis method works fine in normal circumstances, but in case of Uber India the results deviated too much from a subjective score. It's safe to say that basic word matching gives a good indicator of overall customer sentiments, except in the case when the data itself is not reliable. In our case, the tweets from Uber India are not really related to the services that Uber provides, rather the one incident of crime by its driver and whole score went haywire. Let's not compute a point statistic of the scores we have computed so far. Since the numbers of tweets are not equal for each of the four organizations, we compute a mean and standard deviation for each. Organization Mean Sentiment Score Standard Deviation Meru Cabs -0.2218543 1.301846 Ola Cabs 0.197724 1.170334 TaxiForSure -0.09841828 1.154056 Uber India -0.6132666 1.071094 Estimating sentiment (B) Let's now move one step further. Now instead of using simple matching of opinion lexicon, we'll use something called Naive Bayes to decide on the emotion present in any tweet. We would require packages called Rstem and sentiment to assist in this. It's important to mention here that both these packages are no longer available in CRAN and hence we have to provide either the repository location as a parameter install.package() function. Here's the R script to install the required packages: install.packages("Rstem", repos = "http://www.omegahat.org/R", type="source") require(devtools) install_url("http://cran.r-project.org/src/contrib/Archive/sentiment/sentiment_0.2.tar.gz") require(sentiment) ls("package:sentiment") Now that we have the sentiment and Rstem packages installed in our R workspace, we can build the bayes classifier for sentiment analysis: library(sentiment) # classify_emotion function returns an object of class data frame # with seven columns (anger, disgust, fear, joy, sadness, surprise, # # best_fit) and one row for each document: MeruTweetsClassEmo = classify_emotion(MeruTweetsCleaned, algorithm="bayes", prior=1.0) OlaTweetsClassEmo = classify_emotion(OlaTweetsCleaned, algorithm="bayes", prior=1.0) TaxiForSureTweetsClassEmo = classify_emotion(TaxiForSureTweetsCleaned, algorithm="bayes", prior=1.0) UberTweetsClassEmo = classify_emotion(UberTweetsCleaned, algorithm="bayes", prior=1.0) The following figure shows few results from Bayesian analysis using thesentiment package for Meru Cabs tweets. Similarly, we generated results for other cab-services from our problem setup. The sentiment package was built to use a trained dataset of emotion words (nearly 1500 words). The function classify_emotion() generates results belonging to one of the following six emotions: anger, disgust, fear, joy, sadness, and surprise. Hence, when the system is not able to classify the overall emotion to any of the six,NA is returned: Let's substitute these NA values with the word unknown to make the further analysis easier: # we will fetch emotion category best_fit for our analysis purposes. MeruEmotion = MeruTweetsClassEmo[,7] OlaEmotion = OlaTweetsClassEmo[,7] TaxiForSureEmotion = TaxiForSureTweetsClassEmo[,7] UberEmotion = UberTweetsClassEmo[,7] MeruEmotion[is.na(MeruEmotion)] = "unknown" OlaEmotion[is.na(OlaEmotion)] = "unknown" TaxiForSureEmotion[is.na(TaxiForSureEmotion)] = "unknown" UberEmotion[is.na(UberEmotion)] = "unknown" The best-fit emotions present in these tweets are as follows: Further, we'll use another function classify_polarity() provided by the sentiment package to classify the tweets into two classes, pos (positive sentiment) or neg (negative sentiment). The idea is to compute the log likelihood of a tweet assuming it to belong to either of two classes. Once these likelihoods are calculated, a ratio of the pos-likelihood to neg-likelihood is calculated and based on this ratio the tweets are classified to belong to a particular class. It's important to note that if this ratio turns out to be 1, then the overall sentiment of the tweet is assumed to be "neutral". The code is as follows: MeruTweetsClassPol = classify_polarity(MeruTweetsCleaned, algorithm="bayes") OlaTweetsClassPol = classify_polarity(OlaTweetsCleaned, algorithm="bayes") TaxiForSureTweetsClassPol = classify_polarity(TaxiForSureTweetsCleaned, algorithm="bayes") UberTweetsClassPol = classify_polarity(UberTweetsCleaned, algorithm="bayes") We get the following output: The preceding figure shows few results from obtained using the classify_polarity() function of sentiment package for Meru Cabs tweets. We'll now generate consolidated results from the two functions in a data frame for each cab service for plotting purposes: # we will fetch polarity category best_fit for our analysis purposes, MeruPol = MeruTweetsClassPol[,4] OlaPol = OlaTweetsClassPol[,4] TaxiForSurePol = TaxiForSureTweetsClassPol[,4] UberPol = UberTweetsClassPol[,4] # Let us now create a data frame with the above results MeruSentimentDataFrame = data.frame(text=MeruTweetsCleaned, emotion=MeruEmotion, polarity=MeruPol, stringsAsFactors=FALSE) OlaSentimentDataFrame = data.frame(text=OlaTweetsCleaned, emotion=OlaEmotion, polarity=OlaPol, stringsAsFactors=FALSE) TaxiForSureSentimentDataFrame = data.frame(text=TaxiForSureTweetsCleaned, emotion=TaxiForSureEmotion, polarity=TaxiForSurePol, stringsAsFactors=FALSE) UberSentimentDataFrame = data.frame(text=UberTweetsCleaned, emotion=UberEmotion, polarity=UberPol, stringsAsFactors=FALSE) # rearrange data inside the frame by sorting it MeruSentimentDataFrame = within(MeruSentimentDataFrame, emotion <- factor(emotion, levels=names(sort(table(emotion), decreasing=TRUE)))) OlaSentimentDataFrame = within(OlaSentimentDataFrame, emotion <- factor(emotion, levels=names(sort(table(emotion), decreasing=TRUE)))) TaxiForSureSentimentDataFrame = within(TaxiForSureSentimentDataFrame, emotion <- factor(emotion, levels=names(sort(table(emotion), decreasing=TRUE)))) UberSentimentDataFrame = within(UberSentimentDataFrame, emotion <- factor(emotion, levels=names(sort(table(emotion), decreasing=TRUE)))) plotSentiments1<- function (sentiment_dataframe,title) { library(ggplot2) ggplot(sentiment_dataframe, aes(x=emotion)) + geom_bar(aes(y=..count.., fill=emotion)) + scale_fill_brewer(palette="Dark2") + ggtitle(title) + theme(legend.position='right') + ylab('Number of Tweets') + xlab('Emotion Categories') } plotSentiments1(MeruSentimentDataFrame, 'Sentiment Analysis of Tweets on Twitter about MeruCabs') plotSentiments1(OlaSentimentDataFrame, 'Sentiment Analysis of Tweets on Twitter about OlaCabs') plotSentiments1(TaxiForSureSentimentDataFrame, 'Sentiment Analysis of Tweets on Twitter about TaxiForSure') plotSentiments1(UberSentimentDataFrame, 'Sentiment Analysis of Tweets on Twitter about UberIndia') The output is as follows: In the preceding figure, we showed sample results using generated results on Meru Cabs tweets using both the functions. Let's now plot them one by one. First, let's create a single function to be used by each business's tweets. We call it plotSentiments1() and then we plot it for each business: The following dashboard shows the analysis for Ola Cabs: The following dashboard shows the analysis for TaxiForSure: The following dashboard shows the analysis for Uber India: These sentiments basically reflect the more or less the same observations as we did with the basic word-matching algorithm. The number of tweets with joy constitute the largest part of tweets for all these organizations, indicating that these organizations are trying their best to provide good business in the country. The sadness tweets are less numerous than the joy tweets. However, if compared with each other, they indicate the overall market share versus level of customer satisfaction of each service provider in question. Similarly, these graphs can be used to assess the level of dissatisfaction in terms of anger and disgust in the tweets. Let's now consider only the positive and negative sentiments present in the tweets: # Similarly we will plot distribution of polarity in the tweets plotSentiments2 <- function (sentiment_dataframe,title) { library(ggplot2) ggplot(sentiment_dataframe, aes(x=polarity)) + geom_bar(aes(y=..count.., fill=polarity)) + scale_fill_brewer(palette="RdGy") + ggtitle(title) + theme(legend.position='right') + ylab('Number of Tweets') + xlab('Polarity Categories') } plotSentiments2(MeruSentimentDataFrame, 'Polarity Analysis of Tweets on Twitter about MeruCabs') plotSentiments2(OlaSentimentDataFrame, 'Polarity Analysis of Tweets on Twitter about OlaCabs') plotSentiments2(TaxiForSureSentimentDataFrame, 'Polarity Analysis of Tweets on Twitter about TaxiForSure') plotSentiments2(UberSentimentDataFrame, 'Polarity Analysis of Tweets on Twitter about UberIndia') The output is as follows: The following dashboard shows the polarity analysis for Ola Cabs: The following dashboard shows the analysis for TaxiForSure: The following dashboard shows the analysis for Uber India: It's a basic human trait to inform about other's what's wrong rather than informing if there was something right. That is say that we tend to tweets/report if something bad had happened rather reporting/tweeting if the experience was rather good. Hence, the negative tweets are supposed to be larger than the positive tweets in general. Still over a period of time (a week in our case) the ratio of the two easily reflect the overall market share versus the level of customer satisfaction of each service provider in question. Next, we try to get the sense of the overall content of the tweets using the word clouds. removeCustomeWords <- function (TweetsCleaned) { for(i in 1:length(TweetsCleaned)){ TweetsCleaned[i] <- tryCatch({ TweetsCleaned[i] = removeWords(TweetsCleaned[i], c(stopwords("english"), "care", "guys", "can", "dis", "didn", "guy" ,"booked", "plz")) TweetsCleaned[i] }, error=function(cond) { TweetsCleaned[i] }, warning=function(cond) { TweetsCleaned[i] }) } return(TweetsCleaned) } getWordCloud <- function (sentiment_dataframe, TweetsCleaned, Emotion) { emos = levels(factor(sentiment_dataframe$emotion)) n_emos = length(emos) emo.docs = rep("", n_emos) TweetsCleaned = removeCustomeWords(TweetsCleaned) for (i in 1:n_emos){ emo.docs[i] = paste(TweetsCleaned[Emotion == emos[i]], collapse=" ") } corpus = Corpus(VectorSource(emo.docs)) tdm = TermDocumentMatrix(corpus) tdm = as.matrix(tdm) colnames(tdm) = emos require(wordcloud) suppressWarnings(comparison.cloud(tdm, colors = brewer.pal(n_emos, "Dark2"), scale = c(3,.5), random.order = FALSE, title.size = 1.5)) } getWordCloud(MeruSentimentDataFrame, MeruTweetsCleaned, MeruEmotion) getWordCloud(OlaSentimentDataFrame, OlaTweetsCleaned, OlaEmotion) getWordCloud(TaxiForSureSentimentDataFrame, TaxiForSureTweetsCleaned, TaxiForSureEmotion) getWordCloud(UberSentimentDataFrame, UberTweetsCleaned, UberEmotion) The preceding figure shows word cloud from tweets about Meru Cabs. The preceding figure shows word cloud from tweets about Ola Cabs. The preceding figure shows word cloud from tweets about TaxiForSure. The preceding figure shows word cloud from tweets about Uber India. Summary In this article, we gained knowledge of the various Twitter APIs, we discussed how to create a connection with Twitter, and we saw how to retrieve the tweets with various attributes. We saw the power of Twitter in helping us determine the customer attitude toward today's various businesses. The activity can be done on the weekly basis and one can easily get the monthly or quarterly or yearly changes in customer sentiments. This can not only help the customer decide the trending businesses, but the business itself can get a well-defined metric of its own performance. It can use such scores/graphs to improve. We also discussed various methods of sentiment analysis varying from basic word matching to the advanced Bayesian algorithms. Resources for Article: Further resources on this subject: Find Friends on Facebook [article] Supervised learning[article] Warming Up [article]

Node.js is a JavaScript-driven technology. The language has been in development for more than 15 years, and it was first used in Netscape. Over the years, they've found interesting and useful design patterns, which will be of use to us in this book. All this knowledge is now available to Node.js coders. Of course, there are some differences because we are running the code in different environments, but we are still able to apply all these good practices, techniques, and paradigms. I always say that it is important to have a good basis to your applications. No matter how big your application is, it should rely on flexible and well-tested code (For more resources related to this topic, see here.) Node.js fundamentals Node.js is a single-threaded technology. This means that every request is processed in only one thread. In other languages, for example, Java, the web server instantiates a new thread for every request. However, Node.js is meant to use asynchronous processing, and there is a theory that doing this in a single thread could bring good performance. The problem of the single-threaded applications is the blocking I/O operations; for example, when we need to read a file from the hard disk to respond to the client. Once a new request lands on our server, we open the file and start reading from it. The problem occurs when another request is generated, and the application is still processing the first one. Let's elucidate the issue with the following example: var http = require('http'); var getTime = function() { var d = new Date(); return d.getHours() + ':' + d.getMinutes() + ':' + d.getSeconds() + ':' + d.getMilliseconds(); } var respond = function(res, str) { res.writeHead(200, {'Content-Type': 'text/plain'}); res.end(str + 'n'); console.log(str + ' ' + getTime()); } var handleRequest = function (req, res) { console.log('new request: ' + req.url + ' - ' + getTime()); if(req.url == '/immediately') { respond(res, 'A'); } else { var now = new Date().getTime(); while(new Date().getTime() < now + 5000) { // synchronous reading of the file } respond(res, 'B'); } } http.createServer(handleRequest).listen(9000, '127.0.0.1'); The http module, which we initialize on the first line, is needed for running the web server. The getTime function returns the current time as a string, and the respond function sends a simple text to the browser of the client and reports that the incoming request is processed. The most interesting function is handleRequest, which is the entry point of our logic. To simulate the reading of a large file, we will create a while cycle for 5 seconds. Once we run the server, we will be able to make an HTTP request to http://localhost:9000. In order to demonstrate the single-thread behavior we will send two requests at the same time. These requests are as follows: One request will be sent to http://localhost:9000, where the server will perform a synchronous operation that takes 5 seconds The other request will be sent to http://localhost:9000/immediately, where the server should respond immediately The following screenshot is the output printed from the server, after pinging both the URLs: As we can see, the first request came at 16:58:30:434, and its response was sent at 16:58:35:440, that is, 5 seconds later. However, the problem is that the second request is registered when the first one finishes. That's because the thread belonging to Node.js was busy processing the while loop. Of course, Node.js has a solution for the blocking I/O operations. They are transformed to asynchronous functions that accept callback. Once the operation finishes, Node.js fires the callback, notifying that the job is done. A huge benefit of this approach is that while it waits to get the result of the I/O, the server can process another request. The entity that handles the external events and converts them into callback invocations is called the event loop. The event loop acts as a really good manager and delegates tasks to various workers. It never blocks and just waits for something to happen; for example, a notification that the file is written successfully. Now, instead of reading a file synchronously, we will transform our brief example to use asynchronous code. The modified example looks like the following code: var handleRequest = function (req, res) { console.log('new request: ' + req.url + ' - ' + getTime()); if(req.url == '/immediately') { respond(res, 'A'); } else { setTimeout(function() { // reading the file respond(res, 'B'); }, 5000); } } The while loop is replaced with the setTimeout invocation. The result of this change is clearly visible in the server's output, which can be seen in the following screenshot: The first request still gets its response after 5 seconds. However, the second one is processed immediately. Summary In this article, we went through the most common programming paradigms in Node.js. We learned how Node.js handles parallel requests. We understood how to write modules and make them communicative. We saw the problems of the asynchronous code and their most popular solutions. For more information on Node.js you can refer to the following URLs: https://www.packtpub.com/web-development/mastering-nodejs https://www.packtpub.com/web-development/deploying-nodejs Resources for Article: Further resources on this subject: Node.js Fundamentals [Article] AngularJS Project [Article] Working with Live Data and AngularJS [Article]

In this article we will learn how to apply Ragdoll physics to a character. (For more resources related to this topic, see here.) Applying Ragdoll physics to a character Action games often make use of Ragdoll physics to simulate the character's body reaction to being unconsciously under the effect of a hit or explosion. In this recipe, we will learn how to set up and activate Ragdoll physics to our character whenever she steps in a landmine object. We will also use the opportunity to reset the character's position and animations a number of seconds after that event has occurred. Getting ready For this recipe, we have prepared a Unity Package named Ragdoll, containing a basic scene that features an animated character and two prefabs, already placed into the scene, named Landmine and Spawnpoint. The package can be found inside the 1362_07_08 folder. How to do it... To apply ragdoll physics to your character, follow these steps: Create a new project and import the Ragdoll Unity Package. Then, from the Project view, open the mecanimPlayground level. You will see the animated book character and two discs: Landmine and Spawnpoint. First, let's set up our Ragdoll. Access the GameObject | 3D Object | Ragdoll... menu and the Ragdoll wizard will pop-up. Assign the transforms as follows:     Root: mixamorig:Hips     Left Hips: mixamorig:LeftUpLeg     Left Knee: mixamorig:LeftLeg     Left Foot: mixamorig:LeftFoot     Right Hips: mixamorig:RightUpLeg     Right Knee: mixamorig:RightLeg     Right Foot: mixamorig:RightFoot     Left Arm: mixamorig:LeftArm     Left Elbow: mixamorig:LeftForeArm     Right Arm: mixamorig:RightArm     Right Elbow: mixamorig:RightForeArm     Middle Spine: mixamorig:Spine1     Head: mixamorig:Head     Total Mass: 20     Strength: 50 Insert image 1362OT_07_45.png From the Project view, create a new C# Script named RagdollCharacter.cs. Open the script and add the following code: using UnityEngine; using System.Collections; public class RagdollCharacter : MonoBehaviour { void Start () { DeactivateRagdoll(); } public void ActivateRagdoll(){ gameObject.GetComponent<CharacterController> ().enabled = false; gameObject.GetComponent<BasicController> ().enabled = false; gameObject.GetComponent<Animator> ().enabled = false; foreach (Rigidbody bone in GetComponentsInChildren<Rigidbody>()) { bone.isKinematic = false; bone.detectCollisions = true; } foreach (Collider col in GetComponentsInChildren<Collider>()) { col.enabled = true; } StartCoroutine (Restore ()); } public void DeactivateRagdoll(){ gameObject.GetComponent<BasicController>().enabled = true; gameObject.GetComponent<Animator>().enabled = true; transform.position = GameObject.Find("Spawnpoint").transform.position; transform.rotation = GameObject.Find("Spawnpoint").transform.rotation; foreach(Rigidbody bone in GetComponentsInChildren<Rigidbody>()){ bone.isKinematic = true; bone.detectCollisions = false; } foreach (CharacterJoint joint in GetComponentsInChildren<CharacterJoint>()) { joint.enableProjection = true; } foreach(Collider col in GetComponentsInChildren<Collider>()){ col.enabled = false; } gameObject.GetComponent<CharacterController>().enabled = true; } IEnumerator Restore(){ yield return new WaitForSeconds(5); DeactivateRagdoll(); } } Save and close the script. Attach the RagdollCharacter.cs script to the book Game Object. Then, select the book character and, from the top of the Inspector view, change its tag to Player. From the Project view, create a new C# Script named Landmine.cs. Open the script and add the following code: using UnityEngine; using System.Collections; public class Landmine : MonoBehaviour { public float range = 2f; public float force = 2f; public float up = 4f; private bool active = true; void OnTriggerEnter ( Collider collision ){ if(collision.gameObject.tag == "Player" && active){ active = false; StartCoroutine(Reactivate()); collision.gameObject.GetComponent<RagdollCharacter>().ActivateRagdoll(); Vector3 explosionPos = transform.position; Collider[] colliders = Physics.OverlapSphere(explosionPos, range); foreach (Collider hit in colliders) { if (hit.GetComponent<Rigidbody>()) hit.GetComponent<Rigidbody>().AddExplosionForce(force, explosionPos, range, up); } } } IEnumerator Reactivate(){ yield return new WaitForSeconds(2); active = true; } } Save and close the script. Attach the script to the Landmine Game Object. Play the scene. Using the WASD keyboard control scheme, direct the character to the Landmine Game Object. Colliding with it will activate the character's Ragdoll physics and apply an explosion force to it. As a result, the character will be thrown away to a considerable distance and will no longer be in the control of its body movements, akin to a ragdoll. How it works... Unity's Ragdoll Wizard assigns, to selected transforms, the components Collider, Rigidbody, and Character Joint. In conjunction, those components make ragdoll physics possible. However, those components must be disabled whenever we want our character to be animated and controlled by the player. In our case, we switch those components on and off using the RagdollCharacter script and its two functions: ActivateRagdoll() and DeactivateRagdoll(), the latter including instructions to re-spawn our character in the appropriate place. For the testing purposes, we have also created the Landmine script, which calls RagdollCharacter script's function named ActivateRagdoll(). It also applies an explosion force to our ragdoll character, throwing it outside the explosion site. There's more... Instead of resetting the character's transform settings, you could have destroyed its gameObject and instantiated a new one over the respawn point using Tags. For more information on this subject, check Unity's documentation at: http://docs.unity3d.com/ScriptReference/GameObject.FindGameObjectsWithTag.html. Summary In this article we learned how to apply Ragdoll physics to a character. We also learned how to setup Ragdoll for the character of the game. To learn more please refer to the following books: Learning Unity 2D Game Development by Examplehttps://www.packtpub.com/game-development/learning-unity-2d-game-development-example. Unity Game Development Blueprintshttps://www.packtpub.com/game-development/unity-game-development-blueprints. Getting Started with Unityhttps://www.packtpub.com/game-development/getting-started-unity. Resources for Article: Further resources on this subject: Using a collider-based system [article] Looking Back, Looking Forward [article] The Vertex Functions [article]

Python is an easy to learn yet a powerful programming language. It has efficient high-level data structures and effective approach to object-oriented programming. Let's talk a little bit about how Python code is organized. In this paragraph, we'll start going down the rabbit hole a little bit more and introduce a bit more technical names and concepts. Starting with the basics, how is Python code organized? Of course, you write your code into files. When you save a file with the extension .py, that file is said to be a Python module. If you're on Windows or Mac, which typically hide file extensions to the user, please make sure you change the configuration so that you can see the complete name of the files. This is not strictly a requirement, but a hearty suggestion. It would be impractical to save all the code that it is required for software to work within one single file. That solution works for scripts, which are usually not longer than a few hundred lines (and often they are quite shorter than that). A complete Python application can be made of hundreds of thousands of lines of code, so you will have to scatter it through different modules. Better, but not nearly good enough. It turns out that even like this it would still be impractical to work with the code. So Python gives you another structure, called package, which allows you to group modules together. A package is nothing more than a folder, which must contain a special file, __init__.py that doesn't need to hold any code but whose presence is required to tell Python that the folder is not just some folder, but it's actually a package (note that as of Python 3.3 __init__.py is not strictly required any more). As always, an example will make all of this much clearer. I have created an example structure in my project, and when I type in my Linux console: $ tree -v example Here's how a structure of a real simple application could look like: example/ ├── core.py ├── run.py └── util ├── __init__.py ├── db.py ├── math.py └── network.py You can see that within the root of this example, we have two modules, core.py and run.py, and one package: util. Within core.py, there may be the core logic of our application. On the other hand, within the run.py module, we can probably find the logic to start the application. Within the util package, I expect to find various utility tools, and in fact, we can guess that the modules there are called by the type of tools they hold: db.py would hold tools to work with databases, math.py would of course hold mathematical tools (maybe our application deals with financial data), and network.py would probably hold tools to send/receive data on networks. As explained before, the __init__.py file is there just to tell Python that util is a package and not just a mere folder. Had this software been organized within modules only, it would have been much harder to infer its structure. I put a module only example under the ch1/files_only folder, see it for yourself: $ tree -v files_only This shows us a completely different picture: files_only/ ├── core.py ├── db.py ├── math.py ├── network.py └── run.py It is a little harder to guess what each module does, right? Now, consider that this is just a simple example, so you can guess how much harder it would be to understand a real application if we couldn't organize the code in packages and modules. How do we use modules and packages When a developer is writing an application, it is very likely that they will need to apply the same piece of logic in different parts of it. For example, when writing a parser for the data that comes from a form that a user can fill in a web page, the application will have to validate whether a certain field is holding a number or not. Regardless of how the logic for this kind of validation is written, it's very likely that it will be needed in more than one place. For example in a poll application, where the user is asked many question, it's likely that several of them will require a numeric answer. For example: What is your age How many pets do you own How many children do you have How many times have you been married It would be very bad practice to copy paste (or, more properly said: duplicate) the validation logic in every place where we expect a numeric answer. This would violate the DRY (Don't Repeat Yourself) principle, which states that you should never repeat the same piece of code more than once in your application. I feel the need to stress the importance of this principle: you should never repeat the same piece of code more than once in your application (got the irony?). There are several reasons why repeating the same piece of logic can be very bad, the most important ones being: There could be a bug in the logic, and therefore, you would have to correct it in every place that logic is applied. You may want to amend the way you carry out the validation, and again you would have to change it in every place it is applied. You may forget to fix/amend a piece of logic because you missed it when searching for all its occurrences. This would leave wrong/inconsistent behavior in your application. Your code would be longer than needed, for no good reason. Python is a wonderful language and provides you with all the tools you need to apply all the coding best practices. For this particular example, we need to be able to reuse a piece of code. To be able to reuse a piece of code, we need to have a construct that will hold the code for us so that we can call that construct every time we need to repeat the logic inside it. That construct exists, and it's called function. I'm not going too deep into the specifics here, so please just remember that a function is a block of organized, reusable code which is used to perform a task. Functions can assume many forms and names, according to what kind of environment they belong to, but for now this is not important. Functions are the building blocks of modularity in your application, and they are almost indispensable (unless you're writing a super simple script, you'll use functions all the time). Python comes with a very extensive library, as I already said a few pages ago. Now, maybe it's a good time to define what a library is: a library is a collection of functions and objects that provide functionalities that enrich the abilities of a language. For example, within Python's math library we can find a plethora of functions, one of which is the factorial function, which of course calculates the factorial of a number. In mathematics, the factorial of a non-negative integer number N, denoted as N!, is defined as the product of all positive integers less than or equal to N. For example, the factorial of 5 is calculated as: 5! = 5 * 4 * 3 * 2 * 1 = 120 The factorial of 0 is 0! = 1, to respect the convention for an empty product. So, if you wanted to use this function in your code, all you would have to do is to import it and call it with the right input values. Don't worry too much if input values and the concept of calling is not very clear for now, please just concentrate on the import part. We use a library by importing what we need from it, and then we use it. In Python, to calculate the factorial of number 5, we just need the following code: >>> from math import factorial >>> factorial(5) 120 Whatever we type in the shell, if it has a printable representation, will be printed on the console for us (in this case, the result of the function call: 120). So, let's go back to our example, the one with core.py, run.py, util, and so on. In our example, the package util is our utility library. Our custom utility belt that holds all those reusable tools (that is, functions), which we need in our application. Some of them will deal with databases (db.py), some with the network (network.py), and some will perform mathematical calculations (math.py) that are outside the scope of Python's standard math library and therefore, we had to code them for ourselves. Summary In this article, we started to explore the world of programming and that of Python. We saw how Python code can be organized using modules and packages. For more information on Python, refer the following books recomended by Packt Publishing: Learning Python (https://www.packtpub.com/application-development/learning-python) Python 3 Object-oriented Programming - Second Edition (https://www.packtpub.com/application-development/python-3-object-oriented-programming-second-edition) Python Essentials (https://www.packtpub.com/application-development/python-essentials) Resources for Article: Further resources on this subject: Test all the things with Python [article] Putting the Fun in Functional Python [article] Scraping the Web with Python - Quick Start [article]

In this article by Joe Larson, the author of the book 3D Printing Designs: Octopus Pencil Holder, we are going to look at some of the basics, while setting the project for 3D printing. In this article we used Blender as our 3D editing tool. (For more resources related to this topic, see here.) Editing the basic shape This project is going to take advantage of several powerful editing tools that Blender provides. The first one is going to be the Extrude operator. Extruding takes its name from the process of creating things in real life, but in 3D modeling, extruding takes a selected part of an existing model and creates new geometry on the edge of the selected parts so that the original can be moved away but remain attached to where it came from. The result is a new shape that can then be edited. Extruding is a very powerful tool that's used to alter the shape of an object and create new faces that can be extruded themselves: Enter Edit Mode (Tab) and switch to face the select mode (Ctrl + Tab). Deselect all faces (A). Then, select one of the vertical sides of the cylinder. Extrude it either by navigating to Mesh | Extrude | Region in the 3D View menu or pressing E on the keyboard. Extrude the face about 40 mm by moving the mouse or typing 40 on the keyboard. Press Enter or click on the select mouse button to complete the extrude action. Like all actions in Blender, if a mistake is made in the process of extruding, pressing Esc or click on the right mouse button to cancel the action. If a mistake is made after this, undoing the action with Ctrl + Z is always possible. Then, scale the face (S) down to about 20% (0.2) in order to create a tentacle. Repeat the extruding and scaling process with the other seven vertical faces of the cylinder to create all eight tentacles. Select the top face of the cylinder and extrude (E) it about 30 mm. Then, scale (S) it up just a little bit to make the head bulbous. Extrude (E) the top again—this time, about 20 mm and—and scale (S) it in order to give the top a more rounded shape. Now, the cylinder has been changed into something more of an octopus-like shape. And it was mostly accomplished with the Extrude command, a truly powerful tool used to modify the shape of an object. Summary In this article we learned the basics of 3D editing and setting the project for initial stage. We learned basics of 3D modeling and used Octopus Pencil holder as our project. We also come across various keyboard shortcuts. Happy designing! Resources for Article:   Further resources on this subject: Build a First Person Shooter [article] Audio and Animation: Hand in Hand [article] Metal API: Get closer to the bare metal with Metal API [article]

A design pattern is a reusable solution to a recurring problem; the term is really broad in its definition and can span multiple domains of application. However, the term is often associated with a well-known set of object-oriented patterns that were popularized in the 90's by the book, Design Patterns: Elements of Reusable Object-Oriented Software, Pearson Education by the almost legendary Gang of Four (GoF): Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides. We will often refer to these specific set of patterns as traditional design patterns, or GoF design patterns. (For more resources related to this topic, see here.) Applying this set of object-oriented design patterns in JavaScript is not as linear and formal as it would happen in a class-based object-oriented language. As we know, JavaScript is multi-paradigm, object-oriented, and prototype-based, and has dynamic typing; it treats functions as first class citizens, and allows functional programming styles. These characteristics make JavaScript a very versatile language, which gives tremendous power to the developer, but at the same time, it causes a fragmentation of programming styles, conventions, techniques, and ultimately the patterns of its ecosystem. There are so many ways to achieve the same result using JavaScript that everybody has their own opinion on what the best way is to approach a problem. A clear demonstration of this phenomenon is the abundance of frameworks and opinionated libraries in the JavaScript ecosystem; probably, no other language has ever seen so many, especially now that Node.js has given new astonishing possibilities to JavaScript and has created so many new scenarios. In this context, the traditional design patterns too are affected by the nature of JavaScript. There are so many ways in which they can be implemented so that their traditional, strongly object-oriented implementation is not a pattern anymore, and in some cases, not even possible because JavaScript, as we know, doesn't have real classes or abstract interfaces. What doesn't change though, is the original idea at the base of each pattern, the problem it solves, and the concepts at the heart of the solution. In this article, we will see how some of the most important GoF design patterns apply to Node.js and its philosophy, thus rediscovering their importance from another perspective. The design patterns explored in this article are as follows: Factory Proxy Decorator Adapter Strategy State Template Middleware Command This article assumes that the reader has some notion of how inheritance works in JavaScript. Please also be advised that throughout this article we will often use generic and more intuitive diagrams to describe a pattern in place of standard UML, since many patterns can have an implementation based not only on classes, but also on objects and even functions. Factory We begin our journey starting from what probably is the most simple and common design pattern in Node.js: Factory. A generic interface for creating objects We already stressed the fact that, in JavaScript, the functional paradigm is often preferred to a purely object-oriented design, for its simplicity, usability, and small surface area. This is especially true when creating new object instances. In fact, invoking a factory, instead of directly creating a new object from a prototype using the new operator or Object.create(), is so much more convenient and flexible under several aspects. First and foremost, a factory allows us to separate the object creation from its implementation; essentially, a factory wraps the creation of a new instance, giving us more flexibility and control in the way we do it. Inside the factory, we can create a new instance leveraging closures, using a prototype and the new operator, using Object.create(), or even returning a different instance based on a particular condition. The consumer of the factory is totally agnostic about how the creation of the instance is carried out. The truth is that, by using new, we are binding our code to one specific way of creating an object, while in JavaScript we can have much more flexibility, almost for free. As a quick example, let's consider a simple factory that creates an Image object: function createImage(name) { return new Image(name); } var image = createImage('photo.jpeg'); The createImage() factory might look totally unnecessary, why not instantiate the Image class by using the new operator directly? Something like the following line of code: var image = new Image(name); As we already mentioned, using new binds our code to one particular type of object; in the preceding case, to objects of type, Image. A factory instead, gives us much more flexibility; imagine that we want to refactor the Image class, splitting it into smaller classes, one for each image format that we support. If we exposed a factory as the only means to create new images, we can simply rewrite it as follows, without breaking any of the existing code: function createImage(name) { if(name.match(/.jpeg$/)) { return new JpegImage(name); } else if(name.match(/.gif$/)) { return new GifImage(name); } else if(name.match(/.png$/)) { return new PngImage(name); } else { throw new Exception('Unsupported format'); } } Our factory also allows us to not expose the constructors of the objects it creates, and prevents them from being extended or modified (remember the principle of small surface area?). In Node.js, this can be achieved by exporting only the factory, while keeping each constructor private. A mechanism to enforce encapsulation A factory can also be used as an encapsulation mechanism, thanks to closures. Encapsulation refers to the technique of controlling the access to some internal details of an object by preventing the external code from manipulating them directly. The interaction with the object happens only through its public interface, isolating the external code from the changes in the implementation details of the object. This practice is also referred to as information hiding. Encapsulation is also a fundamental principle of object-oriented design, together with inheritance, polymorphism, and abstraction. As we know, in JavaScript, we don't have access level modifiers (for example, we can't declare a private variable), so the only way to enforce encapsulation is through function scopes and closures. A factory makes it straightforward to enforce private variables, consider the following code for example: function createPerson(name) { var privateProperties = {}; var person = { setName: function(name) { if(!name) throw new Error('A person must have a name'); privateProperties.name = name; }, getName: function() { return privateProperties.name; } }; person.setName(name); return person; } In the preceding code, we leverage closures to create two objects: a person object which represents the public interface returned by the factory and a group of privateProperties that are inaccessible from the outside and that can be manipulated only through the interface provided by the person object. For example, in the preceding code, we make sure that a person's name is never empty, this would not be possible to enforce if name was just a property of the person object. Factories are only one of the techniques that we have for creating private members; in fact, other possible approaches are as follows: Defining private variables in a constructor (as recommended by Douglas Crockford: http://javascript.crockford.com/private.html) Using conventions, for example, prefixing the name of a property with an underscore "_" or the dollar sign "$" (this however, does not technically prevent a member from being accessed from the outside) Using ES6 WeakMaps (http://fitzgeraldnick.com/weblog/53/) Building a simple code profiler Now, let's work on a complete example using a factory. Let's build a simple code profiler, an object with the following properties: A start() method that triggers the start of a profiling session An end() method to terminate the session and log its execution time to the console Let's start by creating a file named profiler.js, which will have the following content: function Profiler(label) { this.label = label; this.lastTime = null; } Profiler.prototype.start = function() { this.lastTime = process.hrtime(); } Profiler.prototype.end = function() { var diff = process.hrtime(this.lastTime); console.log('Timer "' + this.label + '" took ' + diff[0] + ' seconds and ' + diff[1] + ' nanoseconds.'); } There is nothing fancy in the preceding class; we simply use the default high resolution timer to save the current time when start() is invoked, and then calculate the elapsed time when end() is executed, printing the result to the console. Now, if we are going to use such a profiler in a real-world application to calculate the execution time of the different routines, we can easily imagine the huge amount of logging we will generate to the standard output, especially in a production environment. What we might want to do instead is redirect the profiling information to another source, for example, a database, or alternatively, disabling the profiler altogether if the application is running in the production mode. It's clear that if we were to instantiate a Profiler object directly by using the new operator, we would need some extra logic in the client code or in the Profiler object itself in order to switch between the different logics. We can instead use a factory to abstract the creation of the Profiler object, so that, depending on whether the application runs in the production or development mode, we can return a fully working Profiler object, or alternatively, a mock object with the same interface, but with empty methods. Let's do this then, in the profiler.js module instead of exporting the Profiler constructor, we will export only a function, our factory. The following is its code: module.exports = function(label) { if(process.env.NODE_ENV === 'development') { return new Profiler(label); //[1] } else if(process.env.NODE_ENV === 'production') { return { //[2] start: function() {}, end: function() {} } } else { throw new Error('Must set NODE_ENV'); } } The factory that we created abstracts the creation of a profiler object from its implementation: If the application is running in the development mode, we return a new, fully functional Profiler object If instead the application is running in the production mode, we return a mock object where the start() and stop() methods are empty functions The nice feature to highlight is that, thanks to the JavaScript dynamic typing, we were able to return an object instantiated with the new operator in one circumstance and a simple object literal in the other (this is also known as duck typing http://en.wikipedia.org/wiki/Duck_typing). Our factory is doing its job perfectly; we can really create objects in any way that we like inside the factory function, and we can execute additional initialization steps or return a different type of object based on particular conditions, and all of this while isolating the consumer of the object from all these details. We can easily understand the power of this simple pattern. Now, we can play with our profiler; this is a possible use case for the factory that we just created earlier: var profiler = require('./profiler'); function getRandomArray(len) { var p = profiler('Generating a ' + len + ' items long array'); p.start(); var arr = []; for(var i = 0; i < len; i++) { arr.push(Math.random()); } p.end(); } getRandomArray(1e6); console.log('Done'); The p variable contains the instance of our Profiler object, but we don't know how it's created and what its implementation is at this point in the code. If we include the preceding code in a file named profilerTest.js, we can easily test these assumptions. To try the program with profiling enabled, run the following command: export NODE_ENV=development; node profilerTest The preceding command enables the real profiler and prints the profiling information to the console. If we want to try the mock profiler instead, we can run the following command: export NODE_ENV=production; node profilerTest The example that we just presented is just a simple application of the factory function pattern, but it clearly shows the advantages of separating an object's creation from its implementation. In the wild As we said, factories are very popular in Node.js. Many packages offer only a factory for creating new instances, some examples are the following: Dnode (https://npmjs.org/package/dnode): This is an RPC system for Node.js. If we look into its source code, we will see that its logic is implemented into a class named D; however, this is never exposed to the outside as the only exported interface is a factory, which allows us to create new instances of the class. You can take a look at its source code at https://github.com/substack/dnode/blob/34d1c9aa9696f13bdf8fb99d9d039367ad873f90/index.js#L7-9. Restify (https://npmjs.org/package/restify): This is a framework to build REST API that allows us to create new instances of a server using the restify.createServer()factory, which internally creates a new instance of the Server class (which is not exported). You can take a look at its source code at https://github.com/mcavage/node-restify/blob/5f31e2334b38361ac7ac1a5e5d852b7206ef7d94/lib/index.js#L91-116. Other modules expose both a class and a factory, but document the factory as the main method—or the most convenient way—to create new instances; some of the examples are as follows: http-proxy (https://npmjs.org/package/http-proxy): This is a programmable proxying library, where new instances are created with httpProxy.createProxyServer(options). The core Node.js HTTP server: This is where new instances are mostly created using http.createServer(), even though this is essentially a shortcut for new http.Server(). bunyan (https://npmjs.org/package/bunyan): This is a popular logging library; in its readme file the contributors propose a factory, bunyan.createLogger(), as the main method to create new instances, even though this would be equivalent to running new bunyan(). Some other modules provide a factory to wrap the creation of other components. Popular examples are through2 and from2 , which allow us to simplify the creation of new streams using a factory approach, freeing the developer from explicitly using inheritance and the new operator. Proxy A proxy is an object that controls the access to another object called subject. The proxy and the subject have an identical interface and this allows us to transparently swap one for the other; in fact, the alternative name for this pattern is surrogate. A proxy intercepts all or some of the operations that are meant to be executed on the subject, augmenting or complementing their behavior. The following figure shows the diagrammatic representation: The preceding figure shows us how the Proxy and the Subject have the same interface and how this is totally transparent to the client, who can use one or the other interchangeably. The Proxy forwards each operation to the subject, enhancing its behavior with additional preprocessing or post-processing. It's important to observe that we are not talking about proxying between classes; the Proxy pattern involves wrapping actual instances of the subject, thus preserving its state. A proxy is useful in several circumstances, for example, consider the following ones: Data validation: The proxy validates the input before forwarding it to the subject Security: The proxy verifies that the client is authorized to perform the operation and it passes the request to the subject only if the outcome of the check is positive Caching: The proxy keeps an internal cache so that the operations are executed on the subject only if the data is not yet present in the cache Lazy initialization: If the creation of the subject is expensive, the proxy can delay it to when it's really necessary Logging: The proxy intercepts the method invocations and the relative parameters, recoding them as they happen Remote objects: A proxy can take an object that is located remotely, and make it appear local Of course, there are many more applications for the Proxy pattern, but these should give us an idea of the extent of its purpose. Techniques for implementing proxies When proxying an object, we can decide to intercept all its methods or only part of them, while delegating the rest of them directly to the subject. There are several ways in which this can be achieved; let's analyze some of them. Object composition Composition is the technique whereby an object is combined with another object for the purpose of extending or using its functionality. In the specific case of the Proxy pattern, a new object with the same interface as the subject is created, and a reference to the subject is stored internally in the proxy in the form of an instance variable or a closure variable. The subject can be injected from the client at creation time or created by the proxy itself. The following is one example of this technique using a pseudo class and a factory: function createProxy(subject) { var proto = Object.getPrototypeOf(subject); function Proxy(subject) { this.subject = subject; } Proxy.prototype = Object.create(proto); //proxied method Proxy.prototype.hello = function() { return this.subject.hello() + ' world!'; } //delegated method Proxy.prototype.goodbye = function() { return this.subject.goodbye .apply(this.subject, arguments); } return new Proxy(subject); } To implement a proxy using composition, we have to intercept the methods that we are interested in manipulating (such as hello()), while simply delegating the rest of them to the subject (as we did with goodbye()). The preceding code also shows the particular case where the subject has a prototype and we want to maintain the correct prototype chain, so that, executing proxy instanceof Subject will return true; we used pseudo-classical inheritance to achieve this. This is just an extra step, required only if we are interested in maintaining the prototype chain, which can be useful in order to improve the compatibility of the proxy with code initially meant to work with the subject. However, as JavaScript has dynamic typing, most of the time we can avoid using inheritance and use more immediate approaches. For example, an alternative implementation of the proxy presented in the preceding code, might just use an object literal and a factory: function createProxy(subject) { return { //proxied method hello: function() { return subject.hello() + ' world!'; }, //delegated method goodbye: function() { return subject.goodbye.apply(subject, arguments); } }; } If we want to create a proxy that delegates most of its methods, it would be convenient to generate these automatically using a library, such as delegates (https://npmjs.org/package/delegates). Object augmentation Object augmentation (or monkey patching)is probably the most pragmatic way of proxying individual methods of an object and consists of modifying the subject directly by replacing a method with its proxied implementation; consider the following example: function createProxy(subject) { var helloOrig = subject.hello; subject.hello = function() { return helloOrig.call(this) + ' world!'; } return subject; } This technique is definitely the most convenient one when we need to proxy only one or a few methods, but it has the drawback of modifying the subject object directly. A comparison of the different techniques Composition can be considered the safest way of creating a proxy, because it leaves the subject untouched without mutating its original behavior. Its only drawback is that we have to manually delegate all the methods, even if we want to proxy only one of them. If needed, we might also have to delegate the access to the properties of the subject. The object properties can be delegated using Object.defineProperty(). Find out more at https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/defineProperty. Object augmentation, on the other hand, modifies the subject, which might not always be what we want, but it does not present the various inconveniences related to delegation. For this reason, object augmentation is definitely the most pragmatic way to implement proxies in JavaScript, and it's the preferred technique in all those circumstances where modifying the subject is not a big concern. However, there is at least one situation where composition is almost necessary; this is when we want to control the initialization of the subject as for example, to create it only when needed (lazy initialization). It is worth pointing out that by using a factory function (createProxy() in our examples), we can shield our code from the technique used to generate the proxy. Creating a logging Writable stream To see the proxy pattern in a real example, we will now build an object that acts as a proxy to a Writable stream, by intercepting all the calls to the write() method and logging a message every time this happens. We will use an object composition to implement our proxy; this is how the loggingWritable.js file looks: function createLoggingWritable(writableOrig) { var proto = Object.getPrototypeOf(writableOrig); function LoggingWritable(subject) { this.writableOrig = writableOrig; } LoggingWritable.prototype = Object.create(proto); LoggingWritable.prototype.write = function(chunk, encoding, callback) { if(!callback && typeof encoding === 'function') { callback = encoding; encoding = undefined; } console.log('Writing ', chunk); return this.writableOrig.write(chunk, encoding, function() { console.log('Finished writing ', chunk); callback && callback(); }); }; LoggingWritable.prototype.on = function() { return this.writableOrig.on .apply(this.writableOrig, arguments); }; LoggingWritable.prototype.end = function() { return this.writableOrig.end .apply(this.writableOrig, arguments); } return new LoggingWritable(this.writableOrig); } In the preceding code, we created a factory that returns a proxied version of the writable object passed as an argument. We provide an override for the write() method that logs a message to the standard output every time it is invoked and every time the asynchronous operation completes. This is also a good example to demonstrate the particular case of creating proxies of asynchronous functions, which makes necessary to proxy the callback as well; this is an important detail to be considered in a platform such as Node.js. The remaining methods, on() and end(), are simply delegated to the original writable stream (to keep the code leaner we are not considering the other methods of the Writable interface). We can now include a few more lines of code into the logginWritable.js module to test the proxy that we just created: var fs = require('fs'); var writable = fs.createWriteStream('test.txt'); var writableProxy = createProxy(writable); writableProxy.write('First chunk'); writableProxy.write('Second chunk'); writable.write('This is not logged'); writableProxy.end(); The proxy did not change the original interface of the stream or its external behavior, but if we run the preceding code, we will now see that every chunk that is written into the stream is transparently logged to the console. Proxy in the ecosystem – function hooks and AOP In its numerous forms, Proxy is a quite popular pattern in Node.js and in the ecosystem. In fact, we can find several libraries that allow us to simplify the creation of proxies, most of the time leveraging object augmentation as an implementation approach. In the community, this pattern can be also referred to as function hooking or sometimes also as Aspect Oriented Programming (AOP), which is actually a common area of application for proxies. As it happens in AOP, these libraries usually allow the developer to set pre or post execution hooks for a specific method (or a set of methods) that allow us to execute some custom code before and after the execution of the advised method respectively. Sometimes proxies are also called middleware, because, as it happens in the middleware pattern, they allow us to preprocess and post-process the input/output of a function. Sometimes, they also allow to register multiple hooks for the same method using a middleware-like pipeline. There are several libraries on npm that allow us to implement function hooks with little effort. Among them there are hooks (https://npmjs.org/package/hooks), hooker (https://npmjs.org/package/hooker), and meld (https://npmjs.org/package/meld). In the wild Mongoose (http://mongoosejs.com) is a popular Object-Document Mapping (ODM) library for MongoDB. Internally, it uses the hooks package (https://npmjs.org/package/hooks) to provide pre and post execution hooks for the init, validate, save, and remove methods of its Document objects. Find out more on the official documentation at http://mongoosejs.com/docs/middleware.html. Decorator Decorator is a structural pattern that consists of dynamically augmenting the behavior of an existing object. It's different from classical inheritance, because the behavior is not added to all the objects of the same class but only to the instances that are explicitly decorated. Implementation-wise, it is very similar to the Proxy pattern, but instead of enhancing or modifying the behavior of the existing interface of an object, it augments it with new functionalities, as described in the following figure: In the previous figure, the Decorator object is extending the Component object by adding the methodC() operation. The existing methods are usually delegated to the decorated object, without further processing. Of course, if necessary we can easily combine the Proxy pattern, so that also the calls to the existing methods can be intercepted and manipulated. Techniques for implementing decorators Although Proxy and Decorator are conceptually two different patterns, with different intents, they practically share the same implementation strategies. Let's revise them. Composition Using composition, the decorated component is wrapped around a new object that usually inherits from it. The Decorator in this case simply needs to define the new methods while delegating the existing ones to the original component: function decorate(component) { var proto = Object.getPrototypeOf(component); function Decorator(component) { this.component = component; } Decorator.prototype = Object.create(proto); //new method Decorator.prototype.greetings = function() { //... }; //delegated method Decorator.prototype.hello = function() { this.component.hello.apply(this.component, arguments); }; return new Decorator(component); } Object augmentation Object decoration can also be achieved by simply attaching new methods directly to the decorated object, as follows: function decorate(component) { //new method component.greetings = function() { //... }; return component; } The same caveats discussed during the analysis of the Proxy pattern are also valid for Decorator. Let's now practice the pattern with a working example! Decorating a LevelUP database Before we start coding with the next example, let's spend a few words to introduce LevelUP, the module that we are now going to work with. Introducing LevelUP and LevelDB LevelUP (https://npmjs.org/package/levelup) is a Node.js wrapper around Google's LevelDB, a key-value store originally built to implement IndexedDB in the Chrome browser, but it's much more than that. LevelDB has been defined by Dominic Tarr as the "Node.js of databases", because of its minimalism and extensibility. Like Node.js, LevelDB provides blazing fast performances and only the most basic set of features, allowing developers to build any kind of database on top of it. The Node.js community, and in this case Rod Vagg, did not miss the chance to bring the power of this database into Node.js by creating LevelUP. Born as a wrapper for LevelDB, it then evolved to support several kinds of backends, from in-memory stores, to other NoSQL databases such as Riak and Redis, to web storage engines such as IndexedDB and localStorage, allowing to use the same API on both the server and the client, opening up some really interesting scenarios. Today, there is a full-fledged ecosystem around LevelUp made of plugins and modules that extend the tiny core to implement features such as replication, secondary indexes, live updates, query engines, and more. Also, complete databases were built on top of LevelUP, including CouchDB clones such as PouchDB (https://npmjs.org/package/pouchdb) and CouchUP (https://npmjs.org/package/couchup), and even a graph database, levelgraph (https://npmjs.org/package/levelgraph) that can work both on Node.js and the browser! Find out more about the LevelUP ecosystem at https://github.com/rvagg/node-levelup/wiki/Modules. Implementing a LevelUP plugin In the next example, we are going to show how we can create a simple plugin for LevelUp using the Decorator pattern, and in particular, the object augmentation technique, which is the simplest but nonetheless the most pragmatic and effective way to decorate objects with additional capabilities. For convenience, we are going to use the level package (http://npmjs.org/package/level) that bundles both levelup and the default adapter called leveldown, which uses LevelDB as the backend. What we want to build is a plugin for LevelUP that allows to receive notifications every time an object with a certain pattern is saved into the database. For example, if we subscribe to a pattern such as {a: 1}, we want to receive a notification when objects such as {a: 1, b: 3} or {a: 1, c: 'x'} are saved into the database. Let's start to build our small plugin by creating a new module called levelSubscribe.js. We will then insert the following code: module.exports = function levelSubscribe(db) { db.subscribe = function(pattern, listener) { //[1] db.on('put', function(key, val) { //[2] var match = Object.keys(pattern).every(function(k) { //[3] return pattern[k] === val[k]; }); if(match) { listener(key, val); //[4] } }); }; return db; } That's it for our plugin, and it's extremely simple. Let's see what happens in the preceding code briefly: We decorated the db object with a new method named subscribe(). We simply attached the method directly to the provided db instance (object augmentation). We listen for any put operation performed on the database. We performed a very simple pattern-matching algorithm, which verified that all the properties in the provided pattern are also available on the data being inserted. If we have a match, we notify the listener. Let's now create some code—in a new file named levelSubscribeTest.js—to try out our new plugin: var level = require('level'); //[1] var db = level(__dirname + '/db', {valueEncoding: 'json'}); var levelSubscribe = require('./levelSubscribe'); //[2] db = levelSubscribe(db); db.subscribe({doctype: 'tweet', language: 'en'}, //[3] function(k, val){ console.log(val); }); //[4] db.put('1', {doctype: 'tweet', text: 'Hi', language: 'en'}); db.put('2', {doctype: 'company', name: 'ACME Co.'}); This is what we did in the preceding code: First, we initialize our LevelUP database, choosing the directory where the files will be stored and the default encoding for the values. Then, we attach our plugin, which decorates the original db object. At this point, we are ready to use the new feature provided by our plugin, the subscribe() method, where we specify that we are interested in all the objects with doctype: 'tweet' and language: 'en'. Finally, we save some values in the database, so that we can see our plugin in action: db.put('1', {doctype: 'tweet', text: 'Hi', language: 'en'}); db.put('2', {doctype: 'company', name: 'ACME Co.'}); This example shows a real application of the decorator pattern in its most simple implementation: object augmentation. It might look like a trivial pattern but it has undoubted power if used appropriately. For simplicity, our plugin will work only in combination with the put operations, but it can be easily expanded to work even with the batch operations (https://github.com/rvagg/node-levelup#batch). In the wild For more examples of how Decorator is used in the real world, we might want to inspect the code of some more LevelUp plugins: level-inverted-index (https://github.com/dominictarr/level-inverted-index): This is a plugin that adds inverted indexes to a LevelUP database, allowing to perform simple text searches across the values stored in the database level-plus (https://github.com/eugeneware/levelplus): This is a plugin that adds atomic updates to a LevelUP database Summary To learn more about Node.js, the following books published by Packt Publishing (https://www.packtpub.com/) are recommended: Node.js Essentials (https://www.packtpub.com/web-development/nodejs-essentials) Node.js Blueprints (https://www.packtpub.com/web-development/nodejs-blueprints) Learning Node.js for Mobile Application Development (https://www.packtpub.com/web-development/learning-nodejs-mobile-application-development) Mastering Node.js (https://www.packtpub.com/web-development/mastering-nodejs) Resources for Article: Further resources on this subject: Node.js Fundamentals [Article] Learning Node.js for Mobile Application Development [Article] Developing a Basic Site with Node.js and Express [Article]

In this article, we will learn how we can implement machine learning in practice. To apply the learning process to real-world tasks, we'll use a five-step process. Regardless of the task at hand, any machine learning algorithm can be deployed by following this plan: Data collection: The data collection step involves gathering the learning material an algorithm will use to generate actionable knowledge. In most cases, the data will need to be combined into a single source like a text file, spreadsheet, or database. Data exploration and and preparation: The quality of any machine learning project is based largely on the quality of its input data. Thus, it is important to learn more about the data and its nuances during a practice called data exploration. Additional work is required to prepare the data for the learning process. This involves fixing or cleaning so-called "messy" data, eliminating unnecessary data, and recoding the data to conform to the learner's expected inputs. Model training: By the time the data has been prepared for analysis, you are likely to have a sense of what you are capable of learning from the data. The specific machine learning task chosen will inform the selection of an appropriate algorithm and the algorithm will represent the data in the form of a model. Model evaluation: Because each machine learning model results in a biased solution to the learning problem, it is important to evaluate how well the algorithm learns from its experience. Depending on the type of model used, you might be able to evaluate the accuracy of the model using a test dataset or you may need to develop measures of performance specific to the intended application. Model improvement: If better performance is needed, it becomes necessary to utilize more advanced strategies to augment the performance of the model. Sometimes, it may be necessary to switch to a different type of model altogether. You may need to supplement your data with additional data or perform additional preparatory work as in step two of this process. (For more resources related to this topic, see here.) After these steps are completed, if the model appears to be performing well, it can be deployed for its intended task. As the case may be, you might utilize your model to provide score data for predictions (possibly in real time), for projections of financial data, to generate useful insight for marketing or research, or to automate tasks such as mail delivery or flying aircraft. The successes and failures of the deployed model might even provide additional data to train your next generation learner. Types of input data The practice of machine learning involves matching the characteristics of input data to the biases of the available approaches. Thus, before applying machine learning to real-world problems, it is important to understand the terminology that distinguishes among input datasets. The phrase unit of observation is used to describe the smallest entity with measured properties of interest for a study. Commonly, the unit of observation is in the form of persons, objects or things, transactions, time points, geographic regions, or measurements. Sometimes, units of observation are combined to form units such as person-years, which denote cases where the same person is tracked over multiple years; each person-year comprises of a person's data for one year. The unit of observation is related but not identical to the unit of analysis, which is the smallest unit from which the inference is made. Although it is often the case, the observed and analyzed units are not always the same. For example, data observed from people might be used to analyze trends across countries. Datasets that store the units of observation and their properties can be imagined as collections of data consisting of: Examples: Instances of the unit of observation for which properties have been recorded Features: Recorded properties or attributes of examples that may be useful for learning It is the easiest to understand features and examples through real-world cases. To build a learning algorithm to identify spam e-mail, the unit of observation could be e-mail messages, examples would be specific messages, and its features might consist of the words used in the messages. For a cancer detection algorithm, the unit of observation could be patients, the examples might include a random sample of cancer patients, and its features may be the genomic markers from biopsied cells as well as the characteristics of patient such as weight, height, or blood pressure. While examples and features do not have to be collected in any specific form, they are commonly gathered in the matrix format, which means that each example has exactly the same features. The following spreadsheet shows a dataset in the matrix format. In the matrix data, each row in the spreadsheet is an example and each column is a feature. Here, the rows indicate examples of automobiles, while the columns record various each automobile's feature, such as price, mileage, color, and transmission type. Matrix format data is by far the most common form used in machine learning, though other forms are used occasionally in specialized cases: Features also come in various forms. If a feature represents a characteristic measured in numbers, it is unsurprisingly called numeric. Alternatively, if a feature is an attribute that consists of a set of categories, the feature is called categorical or nominal. A special case of categorical variables is called ordinal, which designates a nominal variable to the categories falling in an ordered list. Some examples of ordinal variables include clothing sizes such as small, medium, and large, or a measurement of customer satisfaction on a scale from "not at all happy" to "very happy." It is important to consider what the features represent, as the type and number of features in your dataset will assist in determining an appropriate machine learning algorithm for your task. Types of machine learning algorithms Machine learning algorithms are divided into categories according to their purpose. Understanding the categories of learning algorithms is an essential first step towards using data to drive the desired action. A predictive model is used for tasks that involve, as the name implies, the prediction of one value using other values in the dataset. The learning algorithm attempts to discover and model the relationship between the target feature (the feature being predicted) and the other features. Despite the common use of the word "prediction" to imply forecasting, predictive models need not necessarily foresee events in the future. For instance, a predictive model could be used to predict past events such as the date of a baby's conception using the mother's present-day hormone levels. Predictive models can also be used in real time to control traffic lights during rush hours. Because predictive models are given clear instruction on what they need to learn and how they are intended to learn it, the process of training a predictive model is known as supervised learning. The supervision does not refer to human involvement, but rather to the fact that the target values provide a way for the learner to know how well it has learned the desired task. Stated more formally, given a set of data, a supervised learning algorithm attempts to optimize a function (the model) to find the combination of feature values that result in the target output. The often used supervised machine learning task of predicting which category an example belongs to is known as classification. It is easy to think of potential uses for a classifier. For instance, you could predict whether: An e-mail message is spam A person has cancer A football team will win or lose An applicant will default on a loan In classification, the target feature to be predicted is a categorical feature known as the class and is divided into categories called levels. A class can have two or more levels, and the levels may or may not be ordinal. Because classification is so widely used in machine learning, there are many types of classification algorithms, with strengths and weaknesses suited for different types of input data. Supervised learners can also be used to predict numeric data such as income, laboratory values, test scores, or counts of items. To predict such numeric values, a common form of numeric prediction fits linear regression models to the input data. Although regression models are not the only type of numeric models, they are, by far, the most widely used. Regression methods are widely used for forecasting, as they quantify in exact terms the association between inputs and the target, including both, the magnitude and uncertainty of the relationship. Since it is easy to convert numbers into categories (for example, ages 13 to 19 are teenagers) and categories into numbers (for example, assign 1 to all males, 0 to all females), the boundary between classification models and numeric prediction models is not necessarily firm. A descriptive model is used for tasks that would benefit from the insight gained from summarizing data in new and interesting ways. As opposed to predictive models that predict a target of interest, in a descriptive model, no single feature is more important than the other. In fact, because there is no target to learn, the process of training a descriptive model is called unsupervised learning. Although it can be more difficult to think of applications for descriptive models—after all, what good is a learner that isn't learning anything in particular—they are used quite regularly for data mining. For example, the descriptive modeling task called pattern discovery is used to identify useful associations within data. Pattern discovery is often used for market basket analysis on retailers' transactional purchase data. Here, the goal is to identify items that are frequently purchased together, such that the learned information can be used to refine marketing tactics. For instance, if a retailer learns that swimming trunks are commonly purchased at the same time as sunscreens, the retailer might reposition the items more closely in the store or run a promotion to "up-sell" customers on associated items. Originally used only in retail contexts, pattern discovery is now starting to be used in quite innovative ways. For instance, it can be used to detect patterns of fraudulent behavior, screen for genetic defects, or identify hot spots for criminal activity. The descriptive modeling task of dividing a dataset into homogeneous groups is called clustering. This is, sometimes, used for segmentation analysis that identifies groups of individuals with similar behavior or demographic information so that advertising campaigns could be tailored for particular audiences. Although the machine is capable of identifying the clusters, human intervention is required to interpret them. For example, given five different clusters of shoppers at a grocery store, the marketing team will need to understand the differences among the groups in order to create a promotion that best suits each group. This is almost certainly easier than trying to create a unique appeal for each customer. Lastly, a class of machine learning algorithms known as meta-learners is not tied to a specific learning task, but is rather focused on learning how to learn more effectively. A meta-learning algorithm uses the result of some learnings to inform additional learning. This can be beneficial for very challenging problems or when a predictive algorithm's performance needs to be as accurate as possible. Matching input data to algorithms The following table lists the general types of machine learning algorithms, each of which may be implemented in several ways. These are the basis on which nearly all the other more advanced methods are based. Although this covers only a fraction of the entire set of machine learning algorithms, learning these methods will provide a sufficient foundation to make sense of any other method you may encounter in the future. Model Learning task Supervised Learning Algorithms Nearest Neighbor Classification Naive Bayes Classification Decision Trees Classification Classification Rule Learners Classification Linear Regression Numeric prediction Regression Trees Numeric prediction Model Trees Numeric prediction Neural Networks Dual use Support Vector Machines Dual use Unsupervised Learning Algorithms Association Rules Pattern detection k-means clustering Clustering Meta-Learning Algorithms Bagging Dual use Boosting Dual use Random Forests Dual use  To begin applying machine learning to a real-world project, you will need to determine which of the four learning tasks your project represents: classification, numeric prediction, pattern detection, or clustering. The task will drive the choice of algorithm. For instance, if you are undertaking pattern detection, you are likely to employ association rules. Similarly, a clustering problem will likely utilize the k-means algorithm and numeric prediction will utilize regression analysis or regression trees. For classification, more thought is needed to match a learning problem to an appropriate classifier. In these cases, it is helpful to consider various distinctions among algorithms—distinctions that will only be apparent by studying each of the classifiers in depth. For instance, within classification problems, decision trees result in models that are readily understood, while the models of neural networks are notoriously difficult to interpret. If you were designing a credit-scoring model, this could be an important distinction because law often requires that the applicant must be notified about the reasons he or she was rejected for the loan. Even if the neural network is better at predicting loan defaults, if its predictions cannot be explained, then it is useless for this application. Although you will sometimes find that these characteristics exclude certain models from consideration. In many cases, the choice of algorithm is arbitrary. When this is true, feel free to use whichever algorithm you are most comfortable with. Other times, when predictive accuracy is primary, you may need to test several algorithms and choose the one that fits the best or use a meta-learning algorithm that combines several different learners to utilize the strengths of each. Summary Machine learning originated at the intersection of statistics, database science, and computer science. It is a powerful tool, capable of finding actionable insight in large quantities of data. Still, caution must be used in order to avoid common abuses of machine learning in the real world. Conceptually, learning involves the abstraction of data into a structured representation and the generalization of this structure into action that can be evaluated for utility. In practical terms, a machine learner uses data containing examples and features of the concept to be learned and summarizes this data in the form of a model, which is then used for predictive or descriptive purposes. These purposes can be grouped into tasks, including classification, numeric prediction, pattern detection, and clustering. Among the many options, machine learning algorithms are chosen on the basis of the input data and the learning task. R provides support for machine learning in the form of community-authored packages. These powerful tools are free to download, but need to be installed before they can be used. Resources for Article:   Further resources on this subject: Introduction to Machine Learning with R [article] Machine Learning with R [article] Spark – Architecture and First Program [article]

In software development, design is often considered as the step done before programming. This isn't true; in reality, analysis, programming, and design tend to overlap, combine, and interweave. (For more resources related to this topic, see here.) Introducing object-oriented Everyone knows what an object is—a tangible thing that we can sense, feel, and manipulate. The earliest objects we interact with are typically baby toys. Wooden blocks, plastic shapes, and over-sized puzzle pieces are common first objects. Babies learn quickly that certain objects do certain things: bells ring, buttons press, and levers pull. The definition of an object in software development is not terribly different. Software objects are not typically tangible things that you can pick up, sense, or feel, but they are models of something that can do certain things and have certain things done to them. Formally, an object is a collection of data and associated behaviors. So, knowing what an object is, what does it mean to be object-oriented? Oriented simply means directed toward. So object-oriented means functionally directed towards modeling objects. This is one of the many techniques used for modeling complex systems by describing a collection of interacting objects via their data and behavior. If you've read any hype, you've probably come across the terms object-oriented analysis, object-oriented design, object-oriented analysis and design, and object-oriented programming. These are all highly related concepts under the general object-oriented umbrella. In fact, analysis, design, and programming are all stages of software development. Calling them object-oriented simply specifies what style of software development is being pursued. Object-oriented analysis (OOA) is the process of looking at a problem, system, or task (that somebody wants to turn into an application) and identifying the objects and interactions between those objects. The analysis stage is all about what needs to be done. The output of the analysis stage is a set of requirements. If we were to complete the analysis stage in one step, we would have turned a task, such as, I need a website, into a set of requirements. For example: Website visitors need to be able to (italic represents actions, bold represents objects): review our history apply for jobs browse, compare, and order products In some ways, analysis is a misnomer. The baby we discussed earlier doesn't analyze the blocks and puzzle pieces. Rather, it will explore its environment, manipulate shapes, and see where they might fit. A better turn of phrase might be object-oriented exploration. In software development, the initial stages of analysis include interviewing customers, studying their processes, and eliminating possibilities. Object-oriented design (OOD) is the process of converting such requirements into an implementation specification. The designer must name the objects, define the behaviors, and formally specify which objects can activate specific behaviors on other objects. The design stage is all about how things should be done. The output of the design stage is an implementation specification. If we were to complete the design stage in a single step, we would have turned the requirements defined during object-oriented analysis into a set of classes and interfaces that could be implemented in (ideally) any object-oriented programming language. Object-oriented programming (OOP) is the process of converting this perfectly defined design into a working program that does exactly what the CEO originally requested. Yeah, right! It would be lovely if the world met this ideal and we could follow these stages one by one, in perfect order, like all the old textbooks told us to. As usual, the real world is much murkier. No matter how hard we try to separate these stages, we'll always find things that need further analysis while we're designing. When we're programming, we find features that need clarification in the design. Most twenty-first century development happens in an iterative development model. In iterative development, a small part of the task is modeled, designed, and programmed, then the program is reviewed and expanded to improve each feature and include new features in a series of short development cycles. Objects and classes So, an object is a collection of data with associated behaviors. How do we differentiate between types of objects? Apples and oranges are both objects, but it is a common adage that they cannot be compared. Apples and oranges aren't modeled very often in computer programming, but let's pretend we're doing an inventory application for a fruit farm. To facilitate the example, we can assume that apples go in barrels and oranges go in baskets. Now, we have four kinds of objects: apples, oranges, baskets, and barrels. In object-oriented modeling, the term used for kind of object is class. So, in technical terms, we now have four classes of objects. What's the difference between an object and a class? Classes describe objects. They are like blueprints for creating an object. You might have three oranges sitting on the table in front of you. Each orange is a distinct object, but all three have the attributes and behaviors associated with one class: the general class of oranges. The relationship between the four classes of objects in our inventory system can be described using a Unified Modeling Language (invariably referred to as UML, because three letter acronyms never go out of style) class diagram. Here is our first class diagram:   This diagram shows that an Orange is somehow associated with a Basket and that an Apple is also somehow associated with a Barrel. Association is the most basic way for two classes to be related. UML is very popular among managers, and occasionally disparaged by programmers. The syntax of a UML diagram is generally pretty obvious; you don't have to read a tutorial to (mostly) understand what is going on when you see one. UML is also fairly easy to draw, and quite intuitive. After all, many people, when describing classes and their relationships, will naturally draw boxes with lines between them. Having a standard based on these intuitive diagrams makes it easy for programmers to communicate with designers, managers, and each other. However, some programmers think UML is a waste of time. Citing iterative development, they will argue that formal specifications done up in fancy UML diagrams are going to be redundant before they're implemented, and that maintaining these formal diagrams will only waste time and not benefit anyone. Depending on the corporate structure involved, this may or may not be true. However, every programming team consisting of more than one person will occasionally has to sit down and hash out the details of the subsystem it is currently working on. UML is extremely useful in these brainstorming sessions for quick and easy communication. Even those organizations that scoff at formal class diagrams tend to use some informal version of UML in their design meetings or team discussions. Further, the most important person you will ever have to communicate with is yourself. We all think we can remember the design decisions we've made, but there will always be the Why did I do that? moments hiding in our future. If we keep the scraps of papers we did our initial diagramming on when we started a design, we'll eventually find them a useful reference. This article, however, is not meant to be a tutorial in UML. There are many of these available on the Internet, as well as numerous books available on the topic. UML covers far more than class and object diagrams; it also has a syntax for use cases, deployment, state changes, and activities. We'll be dealing with some common class diagram syntax in this discussion of object-oriented design. You'll find that you can pick up the structure by example, and you'll subconsciously choose the UML-inspired syntax in your own team or personal design sessions. Our initial diagram, while correct, does not remind us that apples go in barrels or how many barrels a single apple can go in. It only tells us that apples are somehow associated with barrels. The association between classes is often obvious and needs no further explanation, but we have the option to add further clarification as needed. The beauty of UML is that most things are optional. We only need to specify as much information in a diagram as makes sense for the current situation. In a quick whiteboard session, we might just quickly draw lines between boxes. In a formal document, we might go into more detail. In the case of apples and barrels, we can be fairly confident that the association is, many apples go in one barrel, but just to make sure nobody confuses it with, one apple spoils one barrel, we can enhance the diagram as shown: This diagram tells us that oranges go in baskets with a little arrow showing what goes in what. It also tells us the number of that object that can be used in the association on both sides of the relationship. One Basket can hold many (represented by a *) Orange objects. Any one Orange can go in exactly one Basket. This number is referred to as the multiplicity of the object. You may also hear it described as the cardinality. These are actually slightly distinct terms. Cardinality refers to the actual number of items in the set, whereas multiplicity specifies how small or how large this number could be. I frequently forget which side of a relationship the multiplicity goes on. The multiplicity nearest to a class is the number of objects of that class that can be associated with any one object at the other end of the association. For the apple goes in barrel association, reading from left to right, many instances of the Apple class (that is many Apple objects) can go in any one Barrel. Reading from right to left, exactly one Barrel can be associated with any one Apple. Specifying attributes and behaviors We now have a grasp of some basic object-oriented terminology. Objects are instances of classes that can be associated with each other. An object instance is a specific object with its own set of data and behaviors; a specific orange on the table in front of us is said to be an instance of the general class of oranges. That's simple enough, but what are these data and behaviors that are associated with each object? Data describes objects Let's start with data. Data typically represents the individual characteristics of a certain object. A class can define specific sets of characteristics that are shared by all objects of that class. Any specific object can have different data values for the given characteristics. For example, our three oranges on the table (if we haven't eaten any) could each weigh a different amount. The orange class could then have a weight attribute. All instances of the orange class have a weight attribute, but each orange has a different value for this attribute. Attributes don't have to be unique, though; any two oranges may weigh the same amount. As a more realistic example, two objects representing different customers might have the same value for a first name attribute. Attributes are frequently referred to as members or properties. Some authors suggest that the terms have different meanings, usually that attributes are settable, while properties are read-only. In Python, the concept of "read-only" is rather pointless, so throughout this article, we'll see the two terms used interchangeably. In our fruit inventory application, the fruit farmer may want to know what orchard the orange came from, when it was picked, and how much it weighs. They might also want to keep track of where each basket is stored. Apples might have a color attribute, and barrels might come in different sizes. Some of these properties may also belong to multiple classes (we may want to know when apples are picked, too), but for this first example, let's just add a few different attributes to our class diagram: Depending on how detailed our design needs to be, we can also specify the type for each attribute. Attribute types are often primitives that are standard to most programming languages, such as integer, floating-point number, string, byte, or Boolean. However, they can also represent data structures such as lists, trees, or graphs, or most notably, other classes. This is one area where the design stage can overlap with the programming stage. The various primitives or objects available in one programming language may be somewhat different from what is available in other languages. Usually, we don't need to be overly concerned with data types at the design stage, as implementation-specific details are chosen during the programming stage. Generic names are normally sufficient for design. If our design calls for a list container type, the Java programmers can choose to use a LinkedList or an ArrayList when implementing it, while the Python programmers (that's us!) can choose between the list built-in and a tuple. In our fruit-farming example so far, our attributes are all basic primitives. However, there are some implicit attributes that we can make explicit—the associations. For a given orange, we might have an attribute containing the basket that holds that orange. Behaviors are actions Now, we know what data is, but what are behaviors? Behaviors are actions that can occur on an object. The behaviors that can be performed on a specific class of objects are called methods. At the programming level, methods are like functions in structured programming, but they magically have access to all the data associated with this object. Like functions, methods can also accept parameters and return values. Parameters to a method are a list of objects that need to be passed into the method that is being called (the objects that are passed in from the calling object are usually referred to as arguments). These objects are used by the method to perform whatever behavior or task it is meant to do. Returned values are the results of that task. We've stretched our "comparing apples and oranges" example into a basic (if far-fetched) inventory application. Let's stretch it a little further and see if it breaks. One action that can be associated with oranges is the pick action. If you think about implementation, pick would place the orange in a basket by updating the basket attribute of the orange, and by adding the orange to the oranges list on the Basket. So, pick needs to know what basket it is dealing with. We do this by giving the pick method a basket parameter. Since our fruit farmer also sells juice, we can add a squeeze method to Orange. When squeezed, squeeze might return the amount of juice retrieved, while also removing the Orange from the basket it was in. Basket can have a sell action. When a basket is sold, our inventory system might update some data on as-yet unspecified objects for accounting and profit calculations. Alternatively, our basket of oranges might go bad before we can sell them, so we add a discard method. Let's add these methods to our diagram: Adding models and methods to individual objects allows us to create a system of interacting objects. Each object in the system is a member of a certain class. These classes specify what types of data the object can hold and what methods can be invoked on it. The data in each object can be in a different state from other objects of the same class, and each object may react to method calls differently because of the differences in state. Object-oriented analysis and design is all about figuring out what those objects are and how they should interact. The next section describes principles that can be used to make those interactions as simple and intuitive as possible. Hiding details and creating the public interface The key purpose of modeling an object in object-oriented design is to determine what the public interface of that object will be. The interface is the collection of attributes and methods that other objects can use to interact with that object. They do not need, and are often not allowed, to access the internal workings of the object. A common real-world example is the television. Our interface to the television is the remote control. Each button on the remote control represents a method that can be called on the television object. When we, as the calling object, access these methods, we do not know or care if the television is getting its signal from an antenna, a cable connection, or a satellite dish. We don't care what electronic signals are being sent to adjust the volume, or whether the sound is destined to speakers or headphones. If we open the television to access the internal workings, for example, to split the output signal to both external speakers and a set of headphones, we will void the warranty. This process of hiding the implementation, or functional details, of an object is suitably called information hiding. It is also sometimes referred to as encapsulation, but encapsulation is actually a more all-encompassing term. Encapsulated data is not necessarily hidden. Encapsulation is, literally, creating a capsule and so think of creating a time capsule. If you put a bunch of information into a time capsule, lock and bury it, it is both encapsulated and the information is hidden. On the other hand, if the time capsule has not been buried and is unlocked or made of clear plastic, the items inside it are still encapsulated, but there is no information hiding. The distinction between encapsulation and information hiding is largely irrelevant, especially at the design level. Many practical references use these terms interchangeably. As Python programmers, we don't actually have or need true information hiding, so the more encompassing definition for encapsulation is suitable. The public interface, however, is very important. It needs to be carefully designed as it is difficult to change it in the future. Changing the interface will break any client objects that are calling it. We can change the internals all we like, for example, to make it more efficient, or to access data over the network as well as locally, and the client objects will still be able to talk to it, unmodified, using the public interface. On the other hand, if we change the interface by changing attribute names that are publicly accessed, or by altering the order or types of arguments that a method can accept, all client objects will also have to be modified. While on the topic of public interfaces, keep it simple. Always design the interface of an object based on how easy it is to use, not how hard it is to code (this advice applies to user interfaces as well). Remember, program objects may represent real objects, but that does not make them real objects. They are models. One of the greatest gifts of modeling is the ability to ignore irrelevant details. The model car I built as a child may look like a real 1956 Thunderbird on the outside, but it doesn't run and the driveshaft doesn't turn. These details were overly complex and irrelevant before I started driving. The model is an abstraction of a real concept. Abstraction is another object-oriented concept related to encapsulation and information hiding. Simply put, abstraction means dealing with the level of detail that is most appropriate to a given task. It is the process of extracting a public interface from the inner details. A driver of a car needs to interact with steering, gas pedal, and brakes. The workings of the motor, drive train, and brake subsystem don't matter to the driver. A mechanic, on the other hand, works at a different level of abstraction, tuning the engine and bleeding the breaks. Here's an example of two abstraction levels for a car: Now, we have several new terms that refer to similar concepts. Condensing all this jargon into a couple of sentences: abstraction is the process of encapsulating information with separate public and private interfaces. The private interfaces can be subject to information hiding. The important lesson to take from all these definitions is to make our models understandable to other objects that have to interact with them. This means paying careful attention to small details. Ensure methods and properties have sensible names. When analyzing a system, objects typically represent nouns in the original problem, while methods are normally verbs. Attributes can often be picked up as adjectives, although if the attribute refers to another object that is part of the current object, it will still likely be a noun. Name classes, attributes, and methods accordingly. Don't try to model objects or actions that might be useful in the future. Model exactly those tasks that the system needs to perform, and the design will naturally gravitate towards the one that has an appropriate level of abstraction. This is not to say we should not think about possible future design modifications. Our designs should be open ended so that future requirements can be satisfied. However, when abstracting interfaces, try to model exactly what needs to be modeled and nothing more. When designing the interface, try placing yourself in the object's shoes and imagine that the object has a strong preference for privacy. Don't let other objects have access to data about you unless you feel it is in your best interest for them to have it. Don't give them an interface to force you to perform a specific task unless you are certain you want them to be able to do that to you. Composition So far, we learned to design systems as a group of interacting objects, where each interaction involves viewing objects at an appropriate level of abstraction. But we don't know yet how to create these levels of abstraction. But even most design patterns rely on two basic object-oriented principles known as composition and inheritance. Composition is simpler, so let's start with it. Composition is the act of collecting several objects together to create a new one. Composition is usually a good choice when one object is part of another object. We've already seen a first hint of composition in the mechanic example. A car is composed of an engine, transmission, starter, headlights, and windshield, among numerous other parts. The engine, in turn, is composed of pistons, a crank shaft, and valves. In this example, composition is a good way to provide levels of abstraction. The car object can provide the interface required by a driver, while also providing access to its component parts, which offers the deeper level of abstraction suitable for a mechanic. Those component parts can, of course, be further broken down if the mechanic needs more information to diagnose a problem or tune the engine. This is a common introductory example of composition, but it's not overly useful when it comes to designing computer systems. Physical objects are easy to break into component objects. People have been doing this at least since the ancient Greeks originally postulated that atoms were the smallest units of matter (they, of course, didn't have access to particle accelerators). Computer systems are generally less complicated than physical objects, yet identifying the component objects in such systems does not happen as naturally. The objects in an object-oriented system occasionally represent physical objects such as people, books, or telephones. More often, however, they represent abstract ideas. People have names, books have titles, and telephones are used to make calls. Calls, titles, accounts, names, appointments, and payments are not usually considered objects in the physical world, but they are all frequently-modeled components in computer systems. Let's try modeling a more computer-oriented example to see composition in action. We'll be looking at the design of a computerized chess game. This was a very popular pastime among academics in the 80s and 90s. People were predicting that computers would one day be able to defeat a human chess master. When this happened in 1997 (IBM's Deep Blue defeated world chess champion, Gary Kasparov), interest in the problem waned, although there are still contests between computer and human chess players. (The computers usually win.) As a basic, high-level analysis, a game of chess is played between two players, using a chess set featuring a board containing sixty-four positions in an 8 X 8 grid. The board can have two sets of sixteen pieces that can be moved, in alternating turns by the two players in different ways. Each piece can take other pieces. The board will be required to draw itself on the computer screen after each turn. I've identified some of the possible objects in the description using italics, and a few key methods using bold. This is a common first step in turning an object-oriented analysis into a design. At this point, to emphasize composition, we'll focus on the board, without worrying too much about the players or the different types of pieces. Let's start at the highest level of abstraction possible. We have two players interacting with a chess set by taking turns making moves: What is this? It doesn't quite look like our earlier class diagrams. That's because it isn't a class diagram! This is an object diagram, also called an instance diagram. It describes the system at a specific state in time, and is describing specific instances of objects, not the interaction between classes. Remember, both players are members of the same class, so the class diagram looks a little different: The diagram shows that exactly two players can interact with one chess set. This also indicates that any one player can be playing with only one chess set at a time. However, we're discussing composition, not UML, so let's think about what the Chess Set is composed of. We don't care what the player is composed of at this time. We can assume that the player has a heart and brain, among other organs, but these are irrelevant to our model. Indeed, there is nothing stopping said player from being Deep Blue itself, which has neither a heart nor a brain. The chess set, then, is composed of a board and 32 pieces. The board further comprises 64 positions. You could argue that pieces are not part of the chess set because you could replace the pieces in a chess set with a different set of pieces. While this is unlikely or impossible in a computerized version of chess, it introduces us to aggregation. Aggregation is almost exactly like composition. The difference is that aggregate objects can exist independently. It would be impossible for a position to be associated with a different chess board, so we say the board is composed of positions. But the pieces, which might exist independently of the chess set, are said to be in an aggregate relationship with that set. Another way to differentiate between aggregation and composition is to think about the lifespan of the object. If the composite (outside) object controls when the related (inside) objects are created and destroyed, composition is most suitable. If the related object is created independently of the composite object, or can outlast that object, an aggregate relationship makes more sense. Also, keep in mind that composition is aggregation; aggregation is simply a more general form of composition. Any composite relationship is also an aggregate relationship, but not vice versa. Let's describe our current chess set composition and add some attributes to the objects to hold the composite relationships: The composition relationship is represented in UML as a solid diamond. The hollow diamond represents the aggregate relationship. You'll notice that the board and pieces are stored as part of the chess set in exactly the same way a reference to them is stored as an attribute on the chess set. This shows that, once again, in practice, the distinction between aggregation and composition is often irrelevant once you get past the design stage. When implemented, they behave in much the same way. However, it can help to differentiate between the two when your team is discussing how the different objects interact. Often, you can treat them as the same thing, but when you need to distinguish between them, it's great to know the difference (this is abstraction at work). Inheritance We discussed three types of relationships between objects: association, composition, and aggregation. However, we have not fully specified our chess set, and these tools don't seem to give us all the power we need. We discussed the possibility that a player might be a human or it might be a piece of software featuring artificial intelligence. It doesn't seem right to say that a player is associated with a human, or that the artificial intelligence implementation is part of the player object. What we really need is the ability to say that "Deep Blue is a player" or that "Gary Kasparov is a player". The is a relationship is formed by inheritance. Inheritance is the most famous, well-known, and over-used relationship in object-oriented programming. Inheritance is sort of like a family tree. My grandfather's last name was Phillips and my father inherited that name. I inherited it from him (along with blue eyes and a penchant for writing). In object-oriented programming, instead of inheriting features and behaviors from a person, one class can inherit attributes and methods from another class. For example, there are 32 chess pieces in our chess set, but there are only six different types of pieces (pawns, rooks, bishops, knights, king, and queen), each of which behaves differently when it is moved. All of these classes of piece have properties, such as color and the chess set they are part of, but they also have unique shapes when drawn on the chess board, and make different moves. Let's see how the six types of pieces can inherit from a Piece class: The hollow arrows indicate that the individual classes of pieces inherit from the Piece class. All the subtypes automatically have a chess_set and color attribute inherited from the base class. Each piece provides a different shape property (to be drawn on the screen when rendering the board), and a different move method to move the piece to a new position on the board at each turn. We actually know that all subclasses of the Piece class need to have a move method; otherwise, when the board tries to move the piece, it will get confused. It is possible that we would want to create a new version of the game of chess that has one additional piece (the wizard). Our current design will allow us to design this piece without giving it a move method. The board would then choke when it asked the piece to move itself. We can implement this by creating a dummy move method on the Piece class. The subclasses can then override this method with a more specific implementation. The default implementation might, for example, pop up an error message that says: That piece cannot be moved. Overriding methods in subtypes allows very powerful object-oriented systems to be developed. For example, if we wanted to implement a player class with artificial intelligence, we might provide a calculate_move method that takes a Board object and decides which piece to move where. A very basic class might randomly choose a piece and direction and move it accordingly. We could then override this method in a subclass with the Deep Blue implementation. The first class would be suitable for play against a raw beginner, the latter would challenge a grand master. The important thing is that other methods in the class, such as the ones that inform the board as to which move was chose need not be changed; this implementation can be shared between the two classes. In the case of chess pieces, it doesn't really make sense to provide a default implementation of the move method. All we need to do is specify that the move method is required in any subclasses. This can be done by making Piece an abstract class with the move method declared abstract. Abstract methods basically say, "We demand this method exist in any non-abstract subclass, but we are declining to specify an implementation in this class." Indeed, it is possible to make a class that does not implement any methods at all. Such a class would simply tell us what the class should do, but provides absolutely no advice on how to do it. In object-oriented parlance, such classes are called interfaces. Inheritance provides abstraction Let's explore the longest word in object-oriented argot. Polymorphism is the ability to treat a class differently depending on which subclass is implemented. We've already seen it in action with the pieces system we've described. If we took the design a bit further, we'd probably see that the Board object can accept a move from the player and call the move function on the piece. The board need not ever know what type of piece it is dealing with. All it has to do is call the move method, and the proper subclass will take care of moving it as a Knight or a Pawn. Polymorphism is pretty cool, but it is a word that is rarely used in Python programming. Python goes an extra step past allowing a subclass of an object to be treated like a parent class. A board implemented in Python could take any object that has a move method, whether it is a bishop piece, a car, or a duck. When move is called, the Bishop will move diagonally on the board, the car will drive someplace, and the duck will swim or fly, depending on its mood. This sort of polymorphism in Python is typically referred to as duck typing: "If it walks like a duck or swims like a duck, it's a duck". We don't care if it really is a duck (inheritance), only that it swims or walks. Geese and swans might easily be able to provide the duck-like behavior we are looking for. This allows future designers to create new types of birds without actually specifying an inheritance hierarchy for aquatic birds. It also allows them to create completely different drop-in behaviors that the original designers never planned for. For example, future designers might be able to make a walking, swimming penguin that works with the same interface without ever suggesting that penguins are ducks. Multiple inheritance When we think of inheritance in our own family tree, we can see that we inherit features from more than just one parent. When strangers tell a proud mother that her son has, "his fathers eyes", she will typically respond along the lines of, "yes, but he got my nose." Object-oriented design can also feature such multiple inheritance, which allows a subclass to inherit functionality from multiple parent classes. In practice, multiple inheritance can be a tricky business, and some programming languages (most notably, Java) strictly prohibit it. However, multiple inheritance can have its uses. Most often, it can be used to create objects that have two distinct sets of behaviors. For example, an object designed to connect to a scanner and send a fax of the scanned document might be created by inheriting from two separate scanner and faxer objects. As long as two classes have distinct interfaces, it is not normally harmful for a subclass to inherit from both of them. However, it gets messy if we inherit from two classes that provide overlapping interfaces. For example, if we have a motorcycle class that has a move method, and a boat class also featuring a move method, and we want to merge them into the ultimate amphibious vehicle, how does the resulting class know what to do when we call move? At the design level, this needs to be explained, and at the implementation level, each programming language has different ways of deciding which parent class's method is called, or in what order. Often, the best way to deal with it is to avoid it. If you have a design showing up like this, you're probably doing it wrong. Take a step back, analyze the system again, and see if you can remove the multiple inheritance relationship in favor of some other association or composite design. Inheritance is a very powerful tool for extending behavior. It is also one of the most marketable advancements of object-oriented design over earlier paradigms. Therefore, it is often the first tool that object-oriented programmers reach for. However, it is important to recognize that owning a hammer does not turn screws into nails. Inheritance is the perfect solution for obvious is a relationships, but it can be abused. Programmers often use inheritance to share code between two kinds of objects that are only distantly related, with no is a relationship in sight. While this is not necessarily a bad design, it is a terrific opportunity to ask just why they decided to design it that way, and whether a different relationship or design pattern would have been more suitable. Summary To learn more about object oriented programming, the following books published by Packt Publishing (https://www.packtpub.com/) are recommended: Python 3 Object Oriented Programming (https://www.packtpub.com/application-development/python-3-object-oriented-programming) Learning Object-Oriented Programming (https://www.packtpub.com/application-development/learning-object-oriented-programming) Resources for Article: Further resources on this subject: Putting the Fun in Functional Python [article] Scraping the Web with Python - Quick Start [article] Introduction to Object-Oriented Programming using Python, JavaScript, and C# [article]

With so many different screen sizes, so little space to work with, and no real standard in the videogame industry, interface design is one of the toughest parts of creating a successful game. We will provide you with what you need to know to address the task properly and come up with optimal solutions for your mobile games.a (For more resources related to this topic, see here.) Best practices in UI design In this article we will provide a list of useful hints to approach the creation of a well-designed UI for your next game. The first golden rule is that the better the game interface is designed, the less it will be noticed by players, as it allows users to navigate through the game in a way that feels natural and easy to grasp. To achieve that, always opt for the simplest solution possible, as simplicity means that controls are clean and easy to learn and that the necessary info is displayed clearly. A little bit of psychology helps when deciding the positioning of buttons in the interface. Human beings have typical cognitive biases and they easily develop habits. Learning how to exploit such psychological aspects can really make a difference in the ergonomics of your game interface. We mentioned The Theory of Affordances by Gibson, but there are other theories of visual perception that should be taken into consideration. A good starting point is the Gestalt Principles that you will find at the following link: http://graphicdesign.spokanefalls.edu/tutorials/process/gestaltprinciples/gestaltprinc.htm Finally, it may seem trivial, but never assume anything. Design your game interface so that the most prominent options available on the main game screen lead your players to the game. Choose the target platform. When you begin designing the interface for a game, spend some time thinking about the main target platform for your game, in order to have a reference resolution to begin working with. The following table describes the screen resolutions of the most popular devices you are likely to work with: Device model Screen resolution (pixels) iPhone 3GS and equivalent 480x320 iPhone 4(S) and equivalent 960x640 at 326 ppi iPhone 5 1136x640 at 326 ppi iPad 1 and 2 1024x768 iPad 3 retina 2048x1536 at 264 ppi iPad Mini 1024x768 at 163 ppi Android devices variable Tablets variable As you may notice, when working with iOS, things are almost straightforward, as with the exception of the latest iPhone 5, there are only two main aspect ratios, and retina displays simply doubles the number of pixels. By designing your interface separately for the iPhone/iPod touch, the iPad at retina resolution, and scaling it down for older models, you basically cover almost all the Apple-equipped customers. For Android-based devices, on the other hand, things are more complicated, as there are tons of devices and they can widely differ from each other in screen size and resolution. The best thing to do in this case is to choose a reference, high-end model with HD display, such as the HTC One X+ or the Samsung Galaxy S4 (as we write), and design the interface to match their resolution. Scale it as required to adapt to others: though this way, the graphics won't be perfect for any device, 90 percent of your gamers won't notice any difference. The following is a list of sites where you can find useful information to deal with the Android screens variety dilemma: http://developer.android.com/about/dashboards/index.html http://anidea.com/technology/designer%E2%80%99s-guide-to-supporting-multiple-android-device-screens/ http://unitid.nl/2012/07/10-free-tips-to-create-perfect-visual-assets-for-ios-and-android-tablet-and-mobile/ Search for references There is no need to reinvent the wheel every time you design a new game interface. Games can be easily classified by genre and different genres tend to adopt general solutions for the interface that are shared among different titles in the same genre. Whenever you are planning the interface for your new game, look at others' work first. Play games and study their UI, especially from a functional perspective. When studying others' game interfaces, always ask yourself: What info is necessary to the player to play this title? What kind of functionality is needed to achieve the game goals? Which are the important components that need to stand out from the rest? What's the purpose and context of each window? By answering such questions, you will be able to make a deep analysis of the interface of other games, compare them, and then choose the solutions to better fit your specific needs. The screen flow The options available to the players of your game will need to be located in a game screen of some sort. So the questions you should ask yourself are: How many screens does my game need? Which options will be available to players? Where will these options be located? Once you come up with a list of the required options and game screens, create a flow chart to describe how the player navigates through the different screens and which options are available in each one. The resulting visual map will help you understand if the screen flow is clear and intuitive, if game options are located where the players expect to find them, and if there are doubles, which should be avoided. Be sure that each game screen is provided with a BACK button to go back to a previous game screen. It can be useful to add hyperlinks to your screen mockups so that you can try navigating through them early on. Functionality It is now time to define how the interface you are designing will help users to play your game. At this point, you should already have a clear idea of what the player will be doing in your game and the mechanics of your game. With that information in mind, think about what actions are required and what info must be displayed to deal with them. For every piece of information that you can deliver to the player, ask yourself if it is really necessary and where it should be displayed for optimal fruition. Try to be as conservative as you can when doing that, it is much too easy to lose the grip on the interface of your game if new options, buttons, and functions keep proliferating. The following is a list of useful hints to keep in mind when defining the functionality of your game interface: Keep the number of buttons as low as possible Stick to one primary purpose for each game screen Refer to the screen flow to check the context for each game screen Split complex info into small chunks and/or multiple screens to avoid overburdening your players Wireframes Now that the flow and basic contents of the game interface is set, it is time to start drawing with a graphic editor, such as Photoshop. Create a template for your game interface which can support the different resolutions you expect to develop your game for, and start defining the size and positioning of each button and any pieces of information that must be available on screen. Try not to use colors yet, or just use them to highlight very important buttons available in each game screen. This operation should involve at least three members of the team: the game designer, the artist, and the programmer. If you are a game designer, never plan the interface without conferring with your artist and programmer: the first is responsible for creating the right assets for the job, so it is important that he/she understands the ideas behind your design choices. The programmer is responsible for implementing the solutions you designed, so it is good practice to ask for his/her opinion too, to avoid designing solutions which in the end cannot be implemented. There are also many tools that can be used by web and app developers to quickly create wireframes and prototypes for user interfaces. A good selection of the most appreciated tools can be found at the following link: http://www.dezinerfolio.com/2011/02/21/14-prototyping-and-wireframing-tools-for-designers The button size We suggest you put an extra amount of attention to defining the proper size for your game buttons. There's no point in having buttons on the screen if the player can't press them. This is especially true with games that use virtual pads. As virtual buttons tend to shadow a remarkable portion of a mobile device, there is a tendency to make them as small as possible. If they are too small, the consequences can be catastrophic, as the players won't be able to even play your game, let alone enjoy it. Street Fighter IV for the iPhone, for example, implements the biggest virtual buttons available on the Apple Store. Check them in the following figure: When designing buttons for your game interface, take your time to make tests and find an optimal balance between the opposing necessities of displaying buttons and saving as much screen space as possible for gameplay. The main screen The main goal of the first interactive game screen of a title should be to make it easy to play. It is thus very important that the PLAY button is large and distinctive enough for players to easily find it on the main screen. The other options that should be available on the main screen may vary depending on the characteristics of each specific game, although some are expected despite the game genre, such as OPTIONS, LEADERBOARDS, ACHIEVEMENTS, BUY, and SUPPORT. The following image represents the main screen of Angry Birds, which is a perfect example of a well-designed main screen. Notice, for example, that optional buttons on the bottom part of the screen are displayed as symbols that make it clear what is the purpose of each one of them. This is a smart way to reduce issues related with translating your game text into different languages. Test and iterate Once the former steps are completed, start testing the game interface. Try every option available to check that the game interface actually provides users with everything they need to correctly play and enjoy your game. Then ask other people to try it and get feedback from them. As you collect feedback, list the most requested modifications, implement them, and repeat the cycle until you are happy with the actual configuration you came up with for your game interface. Evergreen options In the last section of this article, we will provide some considerations about game options that should always be implemented in a well-designed mobile game UI, regardless of its genre or distinctive features. Multiple save slots Though extremely fit for gaming, today's smartphones are first of all phones and multi-purpose devices in general, so it is very common to be forced to suddenly quit a match due to an incoming call or other common activities. All apps quit when there is an incoming call or when the player presses the home button and mobile games offer an auto-saving feature in case of such events. What not all games do is to keep separate save states for every mode the game offers or for multiple users. Plants vs. Zombies, for example, offers such a feature: both the adventure and the quick play modes, in all their variations, are stored in separate save slots, so that the player never loses his/her progresses, regardless of the game mode he/she last played or the game level he/she would like to challenge. The following is a screenshot taken from the main screen of the game: A multiple save option is also much appreciated because it makes it safe for your friends to try your newly downloaded game without destroying your previous progresses. Screen rotation The accelerometer included in a large number of smartphones detects the rotation of the device in the 3D space and most software running on those devices rotate their interface as well, according to the portrait or landscape mode in which the smartphone is held. With games, it is not as easy to deal with such a feature as it would be for an image viewer or a text editor. Some games are designed to exploit the vertical or horizontal dimension of the screen with a purpose, and rotating the phone is an action that might not be accommodated by the game altogether. Should the game allow rotating the device, it is then necessary to adapt the game interface to the orientation of the phone as well, and this generally means designing an alternate version of the interface altogether. It is also an interesting (and not much exploited) feature to have the action of rotating the device as part of the actual gameplay and a core mechanic for a game. Calibrations and reconfigurations It is always a good idea to let players have the opportunity to calibrate and/or reconfigure the game controls in the options screen. For example, left-handed players would appreciate the possibility of switching the game controls orientation. When the accelerometer is involved, it can make a lot of difference for a player to be able to set the sensibility of the device to rotation. Different models with different hardware and software detect the rotation in the space differently and there's no single configuration which is good for all smartphones. So let players calibrate their phones according to their personal tastes and the capabilities of the device they are handling. Several games offer this option. Challenges As games become more and more social, several options have been introduced to allow players to display their score on public leaderboards and compete against friends. One game which does that pretty well is Super 7, an iPhone title that displays, on the very top of the screen, a rainbow bar which increases with the player's score. When the bar reaches its end on the right half of the screen, it means some other player's score has been beaten. It is a nice example of a game feature which continually rewards the player and motivates him as he plays the title. Experiment The touch screen is a relatively new control scheme for games. Feel free to try out new ideas. For example, would it be possible to design a first person shooter that uses the gestures, instead of traditional virtual buttons and D-pad? The trick is to keep the playfield as open as possible since the majority of smart devices have relatively small screens. Summary To learn more about mobile game desgin, the following books published by Packt Publishing (https://www.packtpub.com/) are recommended: Mobile First Design with HTML5 and CSS3 (https://www.packtpub.com/web-development/mobile-first-design-html5-and-css3) Gideros Mobile Game Development (https://www.packtpub.com/game-development/gideros-mobile-game-development) Resources for Article: Further resources on this subject: Elucidating the Game-changing Phenomenon of the Docker-inspired Containerization Paradigm [article] Getting Started with Multiplayer Game Programming [article] Introduction to Mobile Web ArcGIS Development [article]

Node.js is a platform that supports various types of applications, but the most popular kind is the development of web applications. Node's style of coding depends on the community to extend the platform through third-party modules; these modules are then built upon to create new modules, and so on. Companies and single developers around the globe are participating in this process by creating modules that wrap the basic Node APIs and deliver a better starting point for application development. (For more resources related to this topic, see here.) There are many modules to support web application development but none as popular as the Connect module. The Connect module delivers a set of wrappers around the Node.js low-level APIs to enable the development of rich web application frameworks. To understand what Connect is all about, let's begin with a basic example of a basic Node web server. In your working folder, create a file named server.js, which contains the following code snippet: var http = require('http'); http.createServer(function(req, res) { res.writeHead(200, { 'Content-Type': 'text/plain' }); res.end('Hello World'); }).listen(3000); console.log('Server running at http://localhost:3000/'); To start your web server, use your command-line tool, and navigate to your working folder. Then, run the node CLI tool and run the server.js file as follows: $ node server Now open http://localhost:3000 in your browser, and you'll see the Hello World response. So how does this work? In this example, the http module is used to create a small web server listening to the 3000 port. You begin by requiring the http module and use the createServer() method to return a new server object. The listen() method is then used to listen to the 3000 port. Notice the callback function that is passed as an argument to the createServer() method. The callback function gets called whenever there's an HTTP request sent to the web server. The server object will then pass the req and res arguments, which contain the information and functionality needed to send back an HTTP response. The callback function will then do the following two steps: First, it will call the writeHead() method of the response object. This method is used to set the response HTTP headers. In this example, it will set the Content-Type header value to text/plain. For instance, when responding with HTML, you just need to replace text/plain with html/plain. Then, it will call the end() method of the response object. This method is used to finalize the response. The end() method takes a single string argument that it will use as the HTTP response body. Another common way of writing this is to add a write() method before the end() method and then call the end() method, as follows: res.write('Hello World'); res.end(); This simple application illustrates the Node coding style where low-level APIs are used to simply achieve certain functionality. While this is a nice example, running a full web application using the low-level APIs will require you to write a lot of supplementary code to support common requirements. Fortunately, a company called Sencha has already created this scaffolding code for you in the form of a Node module called Connect. Meet the Connect module Connect is a module built to support interception of requests in a more modular approach. In the first web server example, you learned how to build a simple web server using the http module. If you wish to extend this example, you'd have to write code that manages the different HTTP requests sent to your server, handles them properly, and responds to each request with the correct response. Connect creates an API exactly for that purpose. It uses a modular component called middleware, which allows you to simply register your application logic to predefined HTTP request scenarios. Connect middleware are basically callback functions, which get executed when an HTTP request occurs. The middleware can then perform some logic, return a response, or call the next registered middleware. While you will mostly write custom middleware to support your application needs, Connect also includes some common middleware to support logging, static file serving, and more. The way a Connect application works is by using an object called dispatcher. The dispatcher object handles each HTTP request received by the server and then decides, in a cascading way, the order of middleware execution. To understand Connect better, take a look at the following diagram: The preceding diagram illustrates two calls made to the Connect application: the first one should be handled by a custom middleware and the second is handled by the static files middleware. Connect's dispatcher initiates the process, moving on to the next handler using the next() method, until it gets to middleware responding with the res.end() method, which will end the request handling. Express is based on Connect's approach, so in order to understand how Express works, we'll begin with creating a Connect application. In your working folder, create a file named server.js that contains the following code snippet: var connect = require('connect'); var app = connect(); app.listen(3000); console.log('Server running at http://localhost:3000/'); As you can see, your application file is using the connect module to create a new web server. However, Connect isn't a core module, so you'll have to install it using NPM. As you already know, there are several ways of installing third-party modules. The easiest one is to install it directly using the npm install command. To do so, use your command-line tool, and navigate to your working folder. Then execute the following command: $ npm install connect NPM will install the connect module inside a node_modules folder, which will enable you to require it in your application file. To run your Connect web server, just use Node's CLI and execute the following command: $ node server Node will run your application, reporting the server status using the console.log() method. You can try reaching your application in the browser by visiting http://localhost:3000. However, you should get a response similar to what is shown in the following screenshot: Connect application's empty response What this response means is that there isn't any middleware registered to handle the GET HTTP request. This means two things: You've successfully managed to install and use the Connect module It's time for you to write your first Connect middleware Connect middleware Connect middleware is just JavaScript function with a unique signature. Each middleware function is defined with the following three arguments: req: This is an object that holds the HTTP request information res: This is an object that holds the HTTP response information and allows you to set the response properties next: This is the next middleware function defined in the ordered set of Connect middleware When you have a middleware defined, you'll just have to register it with the Connect application using the app.use() method. Let's revise the previous example to include your first middleware. Change your server.js file to look like the following code snippet: var connect = require('connect'); var app = connect(); var helloWorld = function(req, res, next) { res.setHeader('Content-Type', 'text/plain'); res.end('Hello World'); }; app.use(helloWorld); app.listen(3000); console.log('Server running at http://localhost:3000/'); Then, start your connect server again by issuing the following command in your command-line tool: $ node server Try visiting http://localhost:3000 again. You will now get a response similar to that in the following screenshot: Connect application's response Congratulations, you've just created your first Connect middleware! Let's recap. First, you added a middleware function named helloWorld(), which has three arguments: req, res, and next. In your middleware, you used the res.setHeader() method to set the response Content-Type header and the res.end() method to set the response text. Finally, you used the app.use() method to register your middleware with the Connect application. Understanding the order of Connect middleware One of Connect's greatest features is the ability to register as many middleware functions as you want. Using the app.use() method, you'll be able to set a series of middleware functions that will be executed in a row to achieve maximum flexibility when writing your application. Connect will then pass the next middleware function to the currently executing middleware function using the next argument. In each middleware function, you can decide whether to call the next middleware function or stop at the current one. Notice that each Connect middleware function will be executed in first-in-first-out (FIFO) order using the next arguments until there are no more middleware functions to execute or the next middleware function is not called. To understand this better, we will go back to the previous example and add a logger function that will log all the requests made to the server in the command line. To do so, go back to the server.js file and update it to look like the following code snippet: var connect = require('connect'); var app = connect(); var logger = function(req, res, next) { console.log(req.method, req.url); next(); }; var helloWorld = function(req, res, next) { res.setHeader('Content-Type', 'text/plain'); res.end('Hello World'); }; app.use(logger); app.use(helloWorld); app.listen(3000); console.log('Server running at http://localhost:3000/'); In the preceding example, you added another middleware called logger(). The logger() middleware uses the console.log() method to simply log the request information to the console. Notice how the logger() middleware is registered before the helloWorld() middleware. This is important as it determines the order in which each middleware is executed. Another thing to notice is the next() call in the logger() middleware, which is responsible for calling the helloWorld() middleware. Removing the next() call would stop the execution of middleware function at the logger() middleware, which means that the request would hang forever as the response is never ended by calling the res.end() method. To test your changes, start your connect server again by issuing the following command in your command-line tool: $ node server Then, visit http://localhost:3000 in your browser and notice the console output in your command-line tool. Mounting Connect middleware As you may have noticed, the middleware you registered responds to any request regardless of the request path. This does not comply with modern web application development because responding to different paths is an integral part of all web applications. Fortunately, Connect middleware supports a feature called mounting, which enables you to determine which request path is required for the middleware function to get executed. Mounting is done by adding the path argument to the app.use() method. To understand this better, let's revisit our previous example. Modify your server.js file to look like the following code snippet: var connect = require('connect'); var app = connect(); var logger = function(req, res, next) { console.log(req.method, req.url); next(); }; var helloWorld = function(req, res, next) { res.setHeader('Content-Type', 'text/plain'); res.end('Hello World'); }; var goodbyeWorld = function(req, res, next) { res.setHeader('Content-Type', 'text/plain'); res.end('Goodbye World'); }; app.use(logger); app.use('/hello', helloWorld); app.use('/goodbye', goodbyeWorld); app.listen(3000); console.log('Server running at http://localhost:3000/'); A few things have been changed in the previous example. First, you mounted the helloWorld() middleware to respond only to requests made to the /hello path. Then, you added another (a bit morbid) middleware called goodbyeWorld() that will respond to requests made to the /goodbye path. Notice how, as a logger should do, we left the logger() middleware to respond to all the requests made to the server. Another thing you should be aware of is that any requests made to the base path will not be responded by any middleware because we mounted the helloWorld() middleware to a specific path. Connect is a great module that supports various features of common web applications. Connect middleware is super simple as it is built with a JavaScript style in mind. It allows the endless extension of your application logic without breaking the nimble philosophy of the Node platform. While Connect is a great improvement over writing your web application infrastructure, it deliberately lacks some basic features you're used to having in other web frameworks. The reason lies in one of the basic principles of the Node community: create your modules lean and let other developers build their modules on top of the module you created. The community is supposed to extend Connect with its own modules and create its own web infrastructures. In fact, one very energetic developer named TJ Holowaychuk, did it better than most when he released a Connect-based web framework known as Express. Summary In this article, you learned about the basic principles of Node.js web applications and discovered the Connect web module. You created your first Connect application and learned how to use middleware functions. To learn more about Node.js and Web Development, the following books published by Packt Publishing (https://www.packtpub.com/) are recommended: Node.js Design Patterns (https://www.packtpub.com/web-development/nodejs-design-patterns) Web Development with MongoDB and Node.js (https://www.packtpub.com/web-development/web-development-mongodb-and-nodejs) Resources for Article: Further resources on this subject: So, what is Node.js?[article] AngularJS[article] So, what is MongoDB?[article]

Material can be thought of as something like smart paper. Like paper, it has surfaces and edges that reflect light and cast shadows, but unlike paper, material has properties that real paper does not, such as its ability to move, change its shape and size, and merge with other material. Despite this seemingly magical behavior, material should be treated like a physical object with a physicality of its own. Material can be seen as existing in a three-dimensional space, and it is this that gives its interfaces a reassuring sense of depth and structure. Hierarchies become obvious when it is instantly clear whether an object is above or below another. Based largely on age-old principles taken from color theory, animation, traditional print design, and physics, material design provides a virtual space where developers can use surface and light to create meaningful interfaces and movement to design intuitive user interactions. (For more resources related to this topic, see here.) Material properties As mentioned in the introduction, material can be thought of as being bound by physical laws. There are things it can do and things it cannot. It can split apart and heal again, and change color and shape, but it cannot occupy the same space as another sheet of material or rotate around two of its axes. We will be dealing with these properties throughout the book, but it is a good idea to begin with a quick look at the things material can and can't do. The third dimension is fundamental when it comes to material. This is what gives the user the illusion that they are interacting with something more tangible than a rectangle of light. The illusion is generated by the widening and softening of shadows beneath material that is closer to the user. Material exists in virtual space, but a space that, nevertheless, represents the real dimensions of a phone or tablet. The x axis can be thought of as existing between the top and bottom of the screen, the y axis between the right and left edges, and the z axis confined to the space between the back of the handset and the glass of the screen. It is for this reason that material should not rotate around the x or y axes, as this would break the illusion of a space inside the phone. The basic laws of the physics of material are outlined, as follows, in the form of a list: All material is 1 dp thick (along the z axis). Material is solid, only one sheet can exist in one place at a time and material cannot pass through other material. For example, if a card needs to move past another, it must move over it. Elevation, or position along the z axis, is portrayed by shadow, with higher objects having wider, softer shadows. The z axis should be used to prompt interaction. For example, an action button rising up toward the user to demonstrate that it can be used to perform some action. Material does not fold or bend. Material cannot appear to rise higher than the screen surface. Material can grow and shrink along both x and y axes. Material can move along any axis. Material can be spontaneously created and destroyed, but this must not be without movement. The arrivals and departures of material components must be animated. For example, a card growing from the point that it was summoned from or sliding off the screen when dismissed. A sheet of material can split apart anywhere along the x or y axes, and join together again with its original partner or with other material. This covers the basic rules of material behavior but we have said nothing of its content. If material can be thought of as smart paper, then its content can only be described as smart ink. The rules governing how ink behaves are a little simpler: Material content can be text, imagery, or any other form of visual digital content Content can be of any shape or color and behaves independently from its container material It cannot be displayed beyond the edges of its material container It adds nothing to the thickness (z axis) of the material it is displayed on Setting up a development environment The Android development environment consists mainly of two distinct components: the SDK, which provides the code libraries behind Android and Android Studio, and a powerful code editor that is used for constructing and testing applications for Android phones and tablets, Wear, TV, Auto, Glass, and Cardboard. Both these components can both be downloaded as a single package from http://developer.android.com/sdk/index.html. Installing Android Studio The installation is very straightforward. Run the Android Studio bundle and follow the on-screen instructions, installing HAXM hardware acceleration if prompted, and selecting all SDK components, as shown here: Android Studio is dependent on the Java JDK. If you have not previously installed it, this will be detected while you are installing Android Studio, and you will be prompted to download and install it. If for some reason it does not, it can be found at http://www.oracle.com/technetwork/java/javase/downloads/index.html, from where you should download the latest version. This is not quite the end of the installation process. There are still some SDK components that we will need to download manually before we can build our first app. As we will see next, this is done using the Android SDK Manager. Configuring the Android SDK People often refer to Android versions by name, such as Lollipop, or an identity number, such as 5.1.1. As developers, it makes more sense to use the API level, which in the case of Android 5.1.1 would be API level 22. The SDK provides a platform for every API level since API level 8 (Android 2.2). In this section, we will use the SDK Manager to take a closer look at Android platforms, along with the other tools included in the SDK. Start a new Android Studio project or open an existing one with the minimum SDK at 21 or higher. You can then open the SDK manager from the menu via Tools | Android | SDK Manager or the matching icon on the main toolbar. The Android SDK Manager can also be started as a stand alone program. It can be found in the /Android/sdk directory, as can the Android Virtual Device (AVD) manager. As can be seen in the preceding screenshot, there are really three main sections in the SDK: A Tools folder A collection of platforms An Extras folder All these require a closer look. The Tools directory contains exactly what it says, that is, tools. There are a handful of these but the ones that will concern us are the SDK manager that we are using now, and the AVD manager that we will be using shortly to create a virtual device. Open the Tools folder. You should find the latest revisions of the SDK tools and the SDK Platform-tools already installed. If not, select these items, along with the latest Build-tools, that is, if they too have not been installed. These tools are often revised, and it is well worth it to regularly check the SDK manager for updates. When it comes to the platforms themselves, it is usually enough to simply install the latest one. This does not mean that these apps will not work on or be available to devices running older versions, as we can set a minimum SDK level when setting up a project, and along with the use of support libraries, we can bring material design to almost any Android device out there. If you open up the folder for the latest platform, you will see that some items have already been installed. Strictly speaking, the only things you need to install are the SDK platform itself and at least one system image. System images are copies of the hard drives of actual Android devices and are used with the AVD to create emulators. Which images you use will depend on your system and the form factors that you are developing for. In this book, we will be building apps for phones and tablets, so make sure you use one of these at least. Although they are not required to develop apps, the documentation and samples packages can be extremely useful. At the bottom of each platform folder are the Google APIs and corresponding system images. Install these if you are going to include Google services, such as Maps and Cloud, in your apps. You will also need to install the Google support libraries from the Extras directory, and this is what we will cover next. The Extras folder contains various miscellaneous packages with a range of functions. The ones you are most likely to want to download are listed as follows: Android support libraries are invaluable extensions to the SDK that provide APIs that not only facilitate backwards compatibility, but also provide a lot of extra components and functions, and most importantly for us, the design library. As we are developing on Android Studio, we need only install the Android Support Repository, as this contains the Android Support Library and is designed for use with Android. The Google Play services and Google Repository packages are required, along with the Google APIs mentioned a moment ago, to incorporate Google Services into an application. You will most likely need the Google USB Driver if you are intending to test your apps on a real device. How to do this will be explained later in this chapter. The HAXM installer is invaluable if you have a recent Intel processor. Android emulators can be notoriously slow, and this hardware acceleration can make a noticeable difference. Once you have downloaded your selected SDK components, depending on your system and/or project plans, you should have a list of installed packages similar to the one shown next: The SDK is finally ready, and we can start developing material interfaces. All that is required now is a device to test it on. This can, of course, be done on an actual device, but generally speaking, we will need to test our apps on as many devices as possible. Being able to emulate Android devices allows us to do this. Emulating Android devices The AVD allows us to test our designs across the entire range of form factors. There are an enormous number of screen sizes, shapes, and densities around. It is vital that we get to test our apps on as many device configurations as possible. This is actually more important for design than it is for functionality. An app might operate perfectly well on an exceptionally small or narrow screen, but not look as good as we had wanted, making the AVD one of the most useful tools available to us. This section covers how to create a virtual device using the AVD Manager. The AVD Manager can be opened from within Android Studio by navigating to Tools | Android | AVD Manager from the menu or the corresponding icon on the toolbar. Here, you should click on the Create Virtual Device... button. The easiest way to create an emulator is to simply pick a device definition from the list of hardware images and keep clicking on Next until you reach Finish. However, it is much more fun and instructive to either clone and edit an existing profile, or create one from scratch. Click on the New Hardware Profile button. This takes you to the Configure Hardware Profile window where you will be able to create a virtual device from scratch, configuring everything from cameras and sensors, to storage and screen resolution. When you are done, click on Finish and you will be returned to the hardware selection screen where your new device will have been added: As you will have seen from the Import Hardware Profiles button, it is possible to download system images for many devices not included with the SDK. Check the developer sections of device vendor's web sites to see which models are available. So far, we have only configured the hardware for our virtual device. We must now select all the software it will use. To do this, select the hardware profile you just created and press Next. In the following window, select one of the system images you installed earlier and press Next again. This takes us to the Verify Configuration screen where the emulator can be fine-tuned. Most of these configurations can be safely left as they are, but you will certainly need to play with the scale when developing for high density devices. It can also be very useful to be able to use a real SD card. Once you click on Finish, the emulator will be ready to run. An emulator can be rotated through 90 degrees with left Ctrl + F12. The menu can be called with F2, and the back button with ESC. Keyboard commands to emulate most physical buttons, such as call, power, and volume, and a complete list can be found at http://developer.android.com/tools/help/emulator.html. Android emulators are notoriously slow, during both loading and operating, even on quite powerful machines. The Intel hardware accelerator we encountered earlier can make a significant difference. Between the two choices offered, the one that you use should depend on how often you need to open and close a particular emulator. More often than not, taking advantage of your GPU is the more helpful of the two. Apart from this built-in assistance, there are a few other things you can do to improve performance, such as setting lower pixel densities, increasing the device's memory, and building the website for lower API levels. If you are comfortable doing so, set up exclusions in your anti-virus software for the Android Studio and SDK directories. There are several third-party emulators, such as Genymotion, that are not only faster, but also behave more like real devices. The slowness of Android emulators is not necessarily a big problem, as most early development needs only one device, and real devices suffer none of the performance issues found on emulators. As we shall see next, real devices can be connected to our development environment with very little effort. Connecting a real device Using an actual physical device to run and test applications does not have the flexibility that emulators provide, but it does have one or two advantages of its own. Real devices are faster than any emulator, and you can test features unavailable to a virtual device, such as accessing sensors, and making and receiving calls. There are two steps involved in setting up a real phone or tablet. We need to set developer options on the handset and configure the USB connection with our development computer: To enable developer options on your handset, navigate to Settings | About phone. Tap on Build number 7 times to enable Developer options, which will now be available from the previous screen. Open this to enable USB debugging and Allow mock locations. Connect the device to your computer and check that it is connected as a Media device (MTP). Your handset can now be used as a test device. Depending on your We need only install the Google USB. Connect the device to your computer with a USB cable, start Android Studio, and open a project. Depending on your setup, it is quite possible that you are already connected. If not, you can install the Google USB driver by following these steps: From the Windows start menu, open the device manager. Your handset can be found under Other Devices or Portable Devices. Open its Properties window and select the Driver tab. Update the driver with the Google version, which can be found in the sdkextrasgoogleusb_driver directory. An application can be compiled and run from Android Studio by selecting Run 'app' from the Run menu, pressing Shift + F10, or clicking on the green play icon on the toolbar. Once the project has finished building, you will be asked to confirm your choice of device before the app loads and then opens on your handset. With a fully set up development environment and devices to test on, we can now start taking a look at material design, beginning with the material theme that is included as the default in all SDKs with APIs higher than 21. The material theme Since API level 21 (Android 5.0), the material theme has been the built-in user interface. It can be utilized and customized, simplifying the building of material interfaces. However, it is more than just a new look; the material theme also provides the automatic touch feedback and transition animations that we associate with material design. To better understand Android themes and how to apply them, we need to understand how Android styles work, and a little about how screen components, such as buttons and text boxes, are defined. Most individual screen components are referred to as widgets or views. Views that contain other views are called view groups, and they generally take the form of a layout, such as the relative layout we will use in a moment. An Android style is a set of graphical properties defining the appearance of a particular screen component. Styles allow us to define everything from font size and background color, to padding elevation, and much more. An Android theme is simply a style applied across a whole screen or application. The best way to understand how this works is to put it into action and apply a style to a working project. This will also provide a great opportunity to become more familiar with Android Studio. Applying styles Styles are defined as XML files and are stored in the resources (res) directory of Android Studio projects. So that we can apply different styles to a variety of platforms and devices, they are kept separate from the layout code. To see how this is done, start a new project, selecting a minimum SDK of 21 or higher, and using the blank activity template. To the left of the editor is the project explorer pane. This is your access point to every branch of your project. Take a look at the activity_main.xml file, which would have been opened in the editor pane when the project was created. At the bottom of the pane, you will see a Text tab and a Design tab. It should be quite clear, from examining these, how the XML code defines a text box (TextView) nested inside a window (RelativeLayout). Layouts can be created in two ways: textually and graphically. Usually, they are built using a combination of both techniques. In the design view, widgets can be dragged and dropped to form layout designs. Any changes made using the graphical interface are immediately reflected in the code, and experimenting with this is a fantastic way to learn how various widgets and layouts are put together. We will return to both these subjects in detail later on in the book, but for now, we will continue with styles and themes by defining a custom style for the text view in our Hello world app. Open the res node in the project explorer; you can then right-click on the values node and select the New | Values resource file from the menu. Call this file my_style and fill it out as follows: <?xml version="1.0" encoding="utf-8"?> <resources>     <style name="myStyle">         <item name="android:layout_width">match_parent</item>         <item name="android:layout_height">wrap_content</item>         <item name="android:elevation">4dp</item>         <item name="android:gravity">center_horizontal</item>         <item name="android:padding">8dp</item>         <item name="android:background">#e6e6e6</item>         <item name="android:textSize">32sp</item>         <item name="android:textColor">#727272</item>     </style> </resources> This style defines several graphical properties, most of which should be self-explanatory with the possible exception of gravity, which here refers to how content is justified within the view. We will cover measurements and units later in the book, but for now, it is useful to understand dp and sp: Density-independent pixel (dp): Android runs on an enormous number of devices, with screen densities ranging from 120 dpi to 480 dpi and more. To simplify the process of developing for such a wide variety, Android uses a virtual pixel unit based on a 160 dpi screen. This allows us to develop for a particular screen size without having to worry about screen density. Scale-independent pixel (sp): This unit is designed to be applied to text. The reason it is scale-independent is because the actual text size on a user's device will depend on their font size settings. To apply the style we just defined, open the activity_main.xml file (from res/layouts, if you have closed it) and edit the TextView node so that it matches this: <TextView     style="@style/myStyle"     android_text="@string/hello_world" /> The effects of applying this style can be seen immediately from the design tab or preview pane, and having seen how styles are applied, we can now go ahead and create a style to customize the material theme palette. Customizing the material theme One of the most useful features of the material theme is the way it can take a small palette made of only a handful of colors and incorporate these colors into every aspect of a UI. Text and cursor colors, the way things are highlighted, and even system features such as the status and navigation bars can be customized to give our apps brand colors and an easily recognizable look. The use of color in material design is a topic in itself, and there are strict guidelines regarding color, shade, and text, and these will be covered in detail later in the book. For now, we will just look at how we can use a style to apply our own colors to a material theme. So as to keep our resources separate, and therefore easier to manage, we will define our palette in its own XML file. As we did earlier with the my_style.xml file, create a new values resource file in the values directory and call it colors. Complete the code as shown next: <?xml version="1.0" encoding="utf-8"?> <resources>     <color name="primary">#FFC107</color>     <color name="primary_dark">#FFA000</color>     <color name="primary_light">#FFECB3</color>     <color name="accent">#03A9F4</color>     <color name="text_primary">#212121</color>     <color name="text_secondary">#727272</color>     <color name="icons">#212121</color>     <color name="divider">#B6B6B6</color> </resources> In the gutter to the left of the code, you will see small, colored squares. Clicking on these will take you to a dialog with a color wheel and other color selection tools for quick color editing. We are going to apply our style to the entire app, so rather than creating a separate file, we will include our style in the theme that was set up by the project template wizard when we started the project. This theme is called AppTheme, as can be seen by opening the res/values/styles/styles.xml (v21) file. Edit the code in this file so that it looks like the following: <?xml version="1.0" encoding="utf-8"?> <resources>     <style name="AppTheme" parent="android:Theme.Material.Light">         <item name="android:colorPrimary">@color/primary</item>         <item name="android:colorPrimaryDark">@color/primary_dark</item>         <item name="android:colorAccent">@color/accent</item>         <item name="android:textColorPrimary">@color/text_primary</item>         <item name="android:textColor">@color/text_secondary</item>     </style> </resources> Being able to set key colors, such as colorPrimary and colorAccent, allows us to incorporate our brand colors throughout the app, although the project template only shows us how we have changed the color of the status bar and app bar. Try adding radio buttons or text edit boxes to see how the accent color is applied. In the following figure, a timepicker replaces the original text view: The XML for this looks like the following lines: <TimePicker     android_layout_width="wrap_content"     android_layout_height="wrap_content"     android_layout_alignParentBottom="true"     android_layout_centerHorizontal="true" /> For now, it is not necessary to know all the color guidelines. Until we get to them, there is an online material color palette generator at http://www.materialpalette.com/ that lets you try out different palette combinations and download color XML files that can just be cut and pasted into the editor. With a complete and up-to-date development environment constructed, and a way to customize and adapt the material theme, we are now ready to look into how material specific widgets, such as card views, are implemented. Summary The Android SDK, Android Studio, and AVD comprise a sophisticated development toolkit, and even setting them up is no simple task. But, with our tools in place, we were able to take a first look at one of material design's major components: the material theme. We have seen how themes and styles relate, and how to create and edit styles in XML. Finally, we have touched on material palettes, and how to customize a theme to utilize our own brand colors across an app. With these basics covered, we can move on to explore material design further, and in the next chapter, we will look at layouts and material components in greater detail. To learn more about material design, the following books published by Packt Publishing (https://www.packtpub.com/) are recommended: Instant Responsive Web Design (https://www.packtpub.com/web-development/instant-responsive-web-design-instant) Mobile Game Design Essentials (https://www.packtpub.com/game-development/mobile-game-design-essentials) Resources for Article: Further resources on this subject: Speaking Java – Your First Game [article] Metal API: Get closer to the bare metal with Metal API [article] Looking Good – The Graphical Interface [article]