How-To Tutorials

So, this is it! Finally, we move from the 2D world to 3D. With SceneKit, we can make 3D games quite easily, especially since the syntax for SceneKit is quite similar to SpriteKit. When we say 3D games, we don't mean that you get to put on your 3D glasses to make the game. In 2D games, we mostly work in the x and y coordinates. In 3D games, we deal with all three axes x, y, and z. Additionally, in 3D games, we have different types of lights that we can use. Also, SceneKit has an inbuilt physics engine that will take care of forces such as gravity and will also aid collision detection. We can also use SpriteKit in SceneKit for GUI and buttons so that we can add scores and interactivity to the game. So, there is a lot to cover in this article. Let's get started. The topics covered in this article by Siddharth Shekar, the author of Learning iOS 8 Game Development Using Swift, are as follows: Creating a scene with SCNScene Adding objects to a scene Importing scenes from external 3D applications Adding physics to the scene Adding an enemy (For more resources related to this topic, see here.) Creating a scene with SCNScene First, we create a new SceneKit project. It is very similar to creating other projects. Only this time, make sure you select SceneKit from the Game Technology drop-down list. Don't forget to select Swift for the language field. Choose iPad as the device and click on Next to create the project in the selected directory, as shown in the following screenshot: Once the project is created, open it. Click on the GameViewController class, and delete all the contents in the viewDidLoad function, delete the handleTap function, as we will be creating a separate class, and add touch behavior. Create a new class called GameSCNScene and import the following headers. Inherit from the SCNScene class and add an init function that takes in a parameter called view of type SCNView: import Foundation import UIKit import SceneKit   class GameSCNScene: SCNScene{      let scnView: SCNView!    let _size:CGSize!    var scene: SCNScene!           required init(coder aDecoder: NSCoder) {        fatalError("init(coder:) has not been implemented")    }       init(currentview view: SCNView) {               super.init()    } } Also, create two new constants scnView and _size of type SCNView and CGSize, respectively. Also, add a variable called scene of type SCNScene. Since we are making a SceneKit game, we have to get the current view, which is the type SCNView, similar to how we got the view in SpriteKit where we typecasted the current view in SpriteKit to SKView. We create a _size constant to get the current size of the view. We then create a new variable scene of type SCNScene. SCNScene is the class used to make scenes in SceneKit, similar to how we would use SKScene to create scenes in SpriteKit. Swift would automatically ask to create the required init function, so we might as well include it in the class. Now, move to the GameViewController class and create a global variable called gameSCNScene of type GameSCNScene and assign it in the viewDidLoad function, as follows: class GameViewController: UIViewController { var gameSCNScene:GameSCNScene!    override func viewDidLoad() {      super.viewDidLoad()      let scnView = view as SCNView      gameSCNScene = GameSCNScene(currentview: scnView)    } }// UIViewController Class Great! Now we can add objects in the GameSCNScene class. It is better to move all the code to a single class so that we can keep the GameSceneController class clean. In the init function of GameSCNScene, add the following after the super.init function: scnView = view _size = scnView.bounds.size                         // retrieve the SCNView scene = SCNScene() scnView.scene = scene scnView.allowsCameraControl = true scnView.showsStatistics = true scnView.backgroundColor = UIColor.yellowColor() Here, we first assign the current view to the scnView constant. Next, we set the _size constant to the dimensions of the current view. Next we initialize the scene variable. Then, assign the scene to the scene of scnView. Next, enable allowCameraControls and showStatistics. This will enable us to control the camera and move it around to have a better look at the scene. Also, with statistics enabled, we will see the performance of the game to make sure that the FPS is maintained. The backgroundColor property of scnView enables us to set the color of the view. I have set it to yellow so that objects are easily visible in the scene, as shown in the following screenshot. With all this set we can run the scene. Well, it is not all that awesome yet. One thing to notice is that we have still not added a camera or a light, but we still see the yellow scene. This is because while we have not added anything to the scene yet, SceneKit automatically provides a default light and camera for the scene created. Adding objects to a scene Let us next add geometry to the scene. We can create some basic geometry such as spheres, boxes, cones, tori, and so on in SceneKit with ease. Let us create a sphere first and add it to the scene. Adding a sphere to the scene Create a function called addGeometryNode in the class and add the following code in it: func addGeometryNode(){      let sphereGeometry = SCNSphere(radius: 1.0)    sphereGeometry.firstMaterial?.diffuse.contents = UIColor.orangeColor()           let sphereNode = SCNNode(geometry: sphereGeometry)    sphereNode.position = SCNVector3Make(0.0, 0.0, 0.0)    scene.rootNode.addChildNode(sphereNode)       } For creating geometry, we use the SCNSphere class to create a sphere shape. We can also call SCNBox, SCNCone, SCNTorus, and so on to create box, cone, or torus shapes respectively. While creating the sphere, we have to provide the radius as a parameter, which will determine the size of the sphere. Although to place the shape, we have to attach it to a node so that we can place and add it to the scene. So, create a new constant called sphereNode of type SCNNode and pass in the sphere geometry as a parameter. For positioning the node, we have to use the SCNvector3Make property to place our object in 3D space by providing the values for x, y, and z. Finally, to add the node to the scene, we have to call scene.rootNode to add the sphereNode to scene, unlike SpriteKit where we would simply use addChild to add objects to the scene. With the sphere added, let us run the scene. Don't forget to add self.addGeometryNode() in the init function. We did add a sphere, so why are we getting a circle (shown in the following screenshot)? Well, the basic light source used by SceneKit just enables to us to see objects in the scene. If we want to see the actual sphere, we have to improve the light source of the scene. Adding light sources Let us create a new function called addLightSourceNode as follows so that we can add custom lights to our scene: func addLightSourceNode(){           let lightNode = SCNNode()    lightNode.light = SCNLight()    lightNode.light!.type = SCNLightTypeOmni    lightNode.position = SCNVector3(x: 10, y: 10, z: 10)    scene.rootNode.addChildNode(lightNode)         let ambientLightNode = SCNNode()    ambientLightNode.light = SCNLight()    ambientLightNode.light!.type = SCNLightTypeAmbient    ambientLightNode.light!.color = UIColor.darkGrayColor()    scene.rootNode.addChildNode(ambientLightNode) } We can add some light sources to see some depth in our sphere object. Here we add two types of light source. The first is an omni light. Omni lights start at a point and then the light is scattered equally in all directions. We also add an ambient light source. An ambient light is the light that is reflected by other objects, such as moonlight. There are two more types of light sources: directional and spotlight. Spotlight is easy to understand, and we usually use it if a certain object needs to be brought to attention like a singer on a stage. Directional lights are used if you want light to go in a single direction, such as sunlight. The Sun is so far from the Earth that the light rays are almost parallel to each other when we see them. For creating a light source, we create a node called lightNode of type SCNNode. We then assign SCNLight to the light property of lightNode. We assign the omni light type to be the type of the light. We assign position of the light source to be at 10 in all three x, y, and z coordinates. Then, we add it to the rootnode of the scene. Next we add an ambient light to the scene. The first two steps of the process are the same as for creating any light source: For the type of light we have to assign SCNLightTypeAmbient to assign an ambient type light source. Since we don't want the light source to be very strong, as it is reflected, we assign a darkGrayColor to the color. Finally, we add the light source to the scene. There is no need to add the ambient light source to the scene but it will make the scene have softer shadows. You can remove the ambient light source to see the difference. Call the addLightSourceNode function in the init function. Now, build and run the scene to see an actual sphere with proper lighting, as shown in the following screenshot: You can place a finger on the screen and move it to rotate the cameras as we have enabled camera control. You can use two fingers to pan the camera and you can double tap to reset the camera to its original position and direction. Adding a camera to the scene Next let us add a camera to the scene, as the default camera is very close. Create a new function called addCameraNode to the class and add the following code in it: func addCameraNode(){        let cameraNode = SCNNode()    cameraNode.camera = SCNCamera()    cameraNode.position = SCNVector3(x: 0, y: 0, z: 15)    scene.rootNode.addChildNode(cameraNode)       } Here, again we create an empty node called cameraNode. We assign SCNCamera to the camera property of cameraNode. Next we position the camera such that we keep the x and y values at zero and move the camera back in the z direction by 15 units. Then we add the camera to the rootnode of the scene. Call the addCameraNode at the bottom of the init function. In this scene, the origin is at the center of the scene, unlike SpriteKit where the origin of a scene is always at bottom right of the scene. Here the positive x and y are to the right and up from the center. The positive z direction is toward you. We didn't move the sphere back or reduce its size here. This is purely because we brought the camera backward in the scene. Let us next create a floor so that we can have a better understanding of the depth in the scene. Also, in this way, we will learn how to create floors in the scene. Adding a floor In the class, add a new function called addFloorNode and add the following code: func addFloorNode(){                   var floorNode = SCNNode()      floorNode.geometry = SCNFloor()      floorNode.position.y = -1.0      scene.rootNode.addChildNode(floorNode) } For creating a floor, we create a variable called floorNode of type SCNNode. We then assign SCNFloor to the geometry property of floorNode. For the position, we assign the y value to -1 as we want the sphere to appear above the floor. At the end, as usual, we assign the floorNode to the root node of the scene. In the following screenshot, I have rotated the camera to show the scene in full action. Here we can see the floor is gray in color and the sphere is casting its reflection on the floor, and we can also see the bright omni light at the top left of the sphere. Importing scenes from external 3D applications Although we can add objects, cameras, and lights through code, it will become very tedious and confusing when we have a lot of objects added to the scene. In SceneKit, this problem can be easily overcome by importing scenes prebuilt in other 3D applications. All 3D applications such as 3D StudioMax, Maya, Cheetah 3D, and Blender have the ability to export scenes in Collada (.dae) and Alembic (.abc) format. We can import these scenes with lighting, camera, and textured objects into SceneKit directly, without the need for setting up the scene. In this section, we will import a Collada file into the scene. Drag this file into the current project. Along with the DAE file, also add the monster.png file to the project, otherwise you will see only the untextured monster mesh in the scene. Click on the monsterScene.DAE file. If the textured monster is not automatically loaded, drag the monster.png file from the project into the monster mesh in the preview window. Release the mouse button once you see a (+) sign while over the monster mesh. Now you will be able to see the monster properly textured. The panel on the left shows the entities in the scene. Below the entities, the scene graph is shown and the view on the right is the preview pane. Entities show all the objects in the scene and the scene graph shows the relation between these entities. If you have certain objects that are children to other objects, the scene graph will show them as a tree. For example, if you open the triangle next to CATRigHub001, you will see all the child objects under it. You can use the scene graph to move and rotate objects in the scene to fine-tune your scene. You can also add nodes, which can be accessed by code. You can see that we already have a camera and a spotlight in the scene. You can select each object and move it around using the arrow at the pivot point of the object. You can also rotate the scene to get a better view by clicking and dragging the left mouse button on the preview scene. For zooming, scroll your mouse wheel up and down. To pan, hold the Alt button on the keyboard and left-click and drag on the preview pane. One thing to note is that rotating, zooming, and panning in the preview pane won't actually move your camera. The camera is still at the same position and angle. To view from the camera, again select the Camera001 option from the drop-down list in the preview pane and the view will reset to the camera view. At the bottom of the preview window, we can either choose to see the view through the camera or spotlight, or click-and-drag to rotate the free camera. If you have more than one camera in your scene, then you will have Camera002, Camera003, and so on in the drop-down list. Below the view selection dropdown in the preview panel you also have a play button. If you click on the play button, you can look at the default animation of the monster getting played in the preview window. The preview panel is just that; it is just to aid you in having a better understanding of the objects in the scene. In no way is it a replacement for a regular 3D package such as 3DSMax, Maya, or Blender. You can create cameras, lights, and empty nodes in the scene graph, but you can't add geometry such as boxes and spheres. You can add an empty node and position it in the scene graph, and then add geometry in code and attach it to the node. Now that we have an understanding of the scene graph, let us see how we can run this scene in SceneKit. In the init function, delete the line where we initialized the scene and add the following line instead. Also delete the objects, light, and camera we added earlier. init(currentview view:SCNView){    super.init()    scnView = view    _size = scnView.bounds.size       //retrieve the SCNView    //scene = SCNScene()    scene = SCNScene(named: "monsterScene.DAE")       scnView.scene = scene    scnView.allowsCameraControl = true    scnView.showsStatistics = true    scnView.backgroundColor = UIColor.yellowColor()    //   self.addGeometryNode() //   self.addLightSourceNode() //   self.addCameraNode() //   self.addFloorNode() //   } Build and run the game to see the following screenshot: You will see the monster running and the yellow background that we initially assigned to the scene. While exporting the scene, if you export the animations as well, once the scene loads in SceneKit the animation starts playing automatically. Also, you will notice that we have deleted the camera and light in the scene. So, how come the default camera and the light aren't loaded in the scene? What is happening here is that while I exported the file, I inserted a camera in the scene and also added a spotlight. So, when we imported the file into the scene, SceneKit automatically understood that there is a camera already present, so it will use the camera as its default camera. Similarly, a spotlight is already added in the scene, which is taken as the default light source, and lighting is calculated accordingly. Adding objects and physics to the scene Let us now see how we can access each of the objects in the scene graph and add gravity to the monster. Accessing the hero object and adding a physics body So, create a new function called addColladaObjects and call an addHero function in it. Create a global variable called heroNode of type SCNNode. We will use this node to access the hero object in the scene. In the addHero function, add the following code: init(currentview view:SCNView){    super.init()    scnView = view    _size = scnView.bounds.size       //retrieve the SCNView    //scene = SCNScene()    scene = SCNScene(named: "monster.scnassets/monsterScene.DAE")       scnView.scene = scene    scnView.allowsCameraControl = true    scnView.showsStatistics = true    scnView.backgroundColor = UIColor.yellowColor()       self.addColladaObjects()    //   self.addGeometryNode() //   self.addLightSourceNode() //   self.addCameraNode() //   self.addFloorNode()    }   func addHero(){      heroNode = SCNNode()        var monsterNode = scene.rootNode.childNodeWithName( "CATRigHub001", recursively: false)    heroNode.addChildNode(monsterNode!) heroNode.position = SCNVector3Make(0, 0, 0)                     let collisionBox = SCNBox(width: 10.0, height: 10.0,            length: 10.0, chamferRadius: 0)      heroNode.physicsBody?.physicsShape = SCNPhysicsShape(geometry: collisionBox, options: nil)    heroNode.physicsBody = SCNPhysicsBody.dynamicBody()      heroNode.physicsBody?.mass = 20    heroNode.physicsBody?.angularVelocityFactor = SCNVector3Zero heroNode.name = "hero"           scene.rootNode.addChildNode(heroNode) } First, we call the addColladaObjects function in the init function, as highlighted. Then we create the addHero function. In it we initiate the heroNode. Then, to actually move the monster, we need access to the CatRibHub001 node to move the monster. We gain access to it through the ChildWithName property of scene.rootNode. For each object that we wish to gain access to through code, we will have to use the ChildWithName property of the rootNode of the scene and pass in the name of the object. If recursively is set to true, to get said object, SceneKit will go through all the child nodes to get access to the specific node. Since the node that we are looking for is right on top, we said false to save processing time. We create a temporary variable called monsterNode. In the next step, we add the monsterNode variable to heroNode. We then set the position of the hero node to the origin. For heroNode to interact with other physics bodies in the scene, we have to assign a shape to the physics body of heroNode. We could use the mesh of the monster, but the shape might not be calculated properly and a box is a much simpler shape than the mesh of the monster. For creating a box collider, we create a new box geometry roughly the width, height, and depth of the monster. Then, using the physicsBody.physicsShape property of the heroNode, we assign the shape of the collisionBox we created for it. Since we want the body to be affected by gravity, we assign the physics body type to be dynamic. Later we will see other body types. Since we want the body to be highly responsive to gravity, we assign a value of 20 to the mass of the body. In the next step, we set the angularVelocityFactor to 0 in all three directions, as we want the body to move straight up and down when a vertical force is applied. If we don't do this, the body will flip-flop around. We also assign the name hero to the monster to check if the collided object is the hero or not. This will come in handy when we check for collision with other objects. Finally, we add heroNode to the scene. Add the addColladaObjects to the init function and comment or delete the self.addGeometryNode, self.addLightSourceNode, self.addCameraNode, and self.addFloorNode functions if you haven't already. Then, run the game to see the monster slowly falling through. We will create a small patch of ground right underneath the monster so that it doesn't fall down. Adding ground Create a new function called addGround and add the following: func addGround(){           let groundBox = SCNBox(width: 10, height: 2,                            length: 10, chamferRadius: 0)      let groundNode = SCNNode(geometry: groundBox)           groundNode.position = SCNVector3Make(0, -1.01, 0)    groundNode.physicsBody = SCNPhysicsBody.staticBody()    groundNode.physicsBody?.restitution = 0.0      scene.rootNode.addChildNode(groundNode) } We create a new constant called groundBox of type SCNBox, with a width and length of 10, and height of 2. Chamfer is the rounding of the edges of the box. Since we didn't want any rounding of the corners, it is set to 0. Next we create a SCNNode called groundNode and assign groundBox to it. We place it slightly below the origin. Since the height of the box is 2, we place it at –1.01 so that heroNode will be (0, 0, 0) when the monster rests on the ground. Next we assign the physics body of type static body. Also, since we don't want the hero to bounce off the ground when he falls on it, we set the restitution to 0. Finally, we then add the ground to the scene's rootnode. The reason we made this body static instead of dynamic is because a dynamic body gets affected by gravity and other forces but a static one doesn't. So, in this scene, even though gravity is acting downward, the hero will fall but groundBox won't as it is a static body. You will see that the physics syntax is very similar to SpriteKit with static bodies and dynamic bodies, gravity, and so on. And once again, similar to SpriteKit, the physics simulation is automatically turned on when we run the scene. Add the addGround function in the addColladaObjects functions and run the game to see the monster getting affected by gravity and stopping after coming in touch with the ground. Adding an enemy node To check collision in SceneKit, we can check for collision between the hero and the ground. But let us make it a little more interesting and also learn a new kind of body type: the kinematic body. For this, we will create a new box called enemy and make it move and collide with the hero. Create a new global SCNNode called enemyNode as follows: let scnView: SCNView! let _size:CGSize! var scene: SCNScene! var heroNode:SCNNode! var enemyNode:SCNNode! Also, create a new function called addEnemy to the class and add the following in it: func addEnemy(){           let geo = SCNBox(width: 4.0, height: 4.0, length: 4.0, chamferRadius: 0.0)           geo.firstMaterial?.diffuse.contents = UIColor.yellowColor()           enemyNode = SCNNode(geometry: geo)    enemyNode.position = SCNVector3Make(0, 20.0 , 60.0)    enemyNode.physicsBody = SCNPhysicsBody.kinematicBody()    scene.rootNode.addChildNode(enemyNode)           enemyNode.name = "enemy" } Nothing too fancy here! Just as when adding the groundNode, we have created a cube with all its sides four units long. We have also added a yellow color to its material. We then initialize enemyNode in the function. We position the node along the x, y, and z axes. Assign the body type as kinematic instead of static or dynamic. Then we add the body to the scene and finally name the enemyNode as enemy, which we will be needing while checking for collision. Before we forget, call the addEnemy function in the addColladaObjects function after where we called the addHero function. The difference between the kinematic body and other body types is that, like static, external forces cannot act on the body, but we can apply a force to a kinematic body to move it. In the case of a static body, we saw that it is not affected by gravity and even if we apply a force to it, the body just won't move. Here we won't be applying any force to move the enemy block but will simply move the object like we moved the enemy in the SpriteKit game. So, it is like making the same game, but in 3D instead of 2D, so that you can see that although we have a third dimension, the same principles of game development can be applied to both. For moving the enemy, we need an update function for the enemy. So, let us add it to the scene by creating an updateEnemy function and adding the following to it: func updateEnemy(){         enemyNode.position.z += -0.9             if((enemyNode.position.z - 5.0) < -40){                   var factor = arc4random_uniform(2) + 1                   if( factor == 1 ){            enemyNode.position = SCNVector3Make(0, 2.0 , 60.0)        }else{            enemyNode.position = SCNVector3Make(0, 15.0 , 60.0)        }    } } In the update function, similar to how we moved the enemy in the SpriteKit game, we increment the Z position of the enemy node by 0.9. The difference being that we are moving the z direction. Once the enemy has gone beyond –40 in the z direction, we reset the position of the enemy. To create an additional challenge to the player, when the enemy resets, a random number is chosen between 1 and 2. If it is 1, then the enemy is placed closer to the ground, otherwise it is placed at 15 units from the ground. Later, we will add a jump mechanic to the hero. So, when the enemy is closer to the ground, the hero has to jump over the enemy box, but when the enemy is spawned at a height, the hero shouldn't jump. If he jumps and hits the enemy box, then it is game over. Later we will also add a scoring mechanism to keep score. For updating the enemy, we actually need an update function to add the enemyUpdate function to so that the enemy moves and his position resets. So, create a function called update in the class and call the updateEnemy function in it as follows:    func update(){           updateEnemy()    } Summary This article has given insight on how to create a scene with SCNScene, how to add objects to a scene, how to import scenes from external 3D applications, how to adding physics to the scene, and how to add an enemy. Resources for Article: Further resources on this subject: Creating a Brick Breaking Game [article] iOS Security Overview [article] Code Sharing Between iOS and Android [article]

In this article by Dr. Edward Lavieri, the author of Getting Started with Unity 5, you will learn how to use Unity 5's new User Interface (UI) system. (For more resources related to this topic, see here.) An overview of graphical user interface Graphical User Interfaces or GUI (pronounced gooey) is a collection of visual components such as text, buttons, and images that facilitates a user's interaction with software. GUIs are also used to provide feedback to players. In the case of our game, the GUI allows players to interact with our game. Without a GUI, the user would have no visual indication of how to use the game. Imagine software without any on-screen indicators of how to use the software. The following image shows how early user interfaces were anything but intuitive: We use GUIs all the time and might not pay too close attention to them, unless they are poorly designed. If you've ever tried to figure out how to use an app on your Smartphone or could not figure out how to perform a specific action with desktop software, you've most likely encountered a poorly designed GUI. Functions of a GUI Our goal is to create a GUI for our game that both informs the user and allows for interaction between the game and the user. To that end, GUIs have two primary purposes: feedback and control. Feedback is generated by the game to the user and control is given to the user and managed by user input. Let's look at each of these more closely. Feedback Feedback can come in many forms. The most common forms of game feedback are visual and audio. Visual feedback can be something as simple as a text on a game screen. An example would be a game player's current score ever-present on the game screen. Games that include dialog systems where the player interacts with non-player characters (NPC) usually have text feedback on the screen that informs what the NPC's responses are. Visual feedback can also be non-textual, such as smoke, fire, explosions, or other graphic effect. Audio feedback can be as simple as a click sound when the user clicks or taps on a button or as complex as a radar ping when an enemy submarine is detected on long-distance sonar scans. You can probably think of all the audio feedback your favorite game provides. When you run your cart over a coin, an audio sound effect is played so there is no question that you earned the coin. If you take a moment to consider all of the audio feedback you are exposed to in games, you'll begin to appreciate the significance of them. Less typical feedback includes device vibration, which is sometimes used with smartphone applications and console games. Some attractions have taken feedback to another level through seat movement and vibration, dispensing liquid and vapor, and introducing chemicals that provide olfactory input. Control Giving players control of the game is the second function of GUIs. There is a wide gambit of types of control. The most simple is using buttons or menus in a game. A game might have a graphical icon of a backpack that, when clicked, gives the user access to the inventory management system of a game. Control seems like an easy concept and it is. Interestingly, most popular console games lack good GUI interfaces, especially when it comes to control. If you play console games, think about how many times you have to refer to the printed or in-game manual. Do you intuitively know all of the controller key mappings? How do you jump, switch weapons, crotch, throw a grenade, or go into stealth mode? In the defense of the game studios that publish these games, there is a lot of control and it can be difficult to make them intuitive. By extension, control is often physical in addition to graphical. Physical components of control include keyboards, mice, trackballs, console controllers, microphones, and other devices. Feedback and control Feedback and control GUI elements are often paired. When you click or tap a button, it usually has both visual and audio effects as well as executing the user's action. When you click (control) on a treasure chest, it opens (visual feedback) and you hear the creak of the old wooden hinges (audio feedback). This example shows the power of using adding feedback to control actions. Game Layers At a primitive level, there are three layers to every game. The core or base level is the Game Layer. The top layer is the User Layer; this is the actual person playing your game. So, it is the layer in between, the GUI Layer that serves as an intermediary between a game and its player. It becomes clear that designing and developing intuitive and well-functioning GUIs is important to a game's functionality, the user experience, and a game's success. Unity 5's UI system Unity's UI system has recently been re-engineered and is now more powerful than ever. Perhaps the most important concept to grasp is the Canvas object. All UI elements are contained in a canvas. Project and scenes can have more than one canvas. You can think of a canvas as a container for UI elements. Canvas To create a canvas, you simply navigate and select GameObject | UI | Canvas from the drop-down menu. You can see from the GameObject | UI menu pop-up that there are 11 different UI elements. Alternatively, you can create your first UI element, such as a button and Unity will automatically create a canvas for you and add it to your Hierarchy view. When you create subsequent UI elements, simply highlight the canvas in the Hierarchy view and then navigate to the GameObject | UI menu to select a new UI element. Here is a brief description of each of the UI elements: UI element Description Panel A frame object Button Standard button that can be clicked Text Text with standard text formatting Image Images can be simple, sliced, tiled, and filled Raw Image Texture file Slider Slider with min and max values Scrollbar Scrollbar with values between 0 and 1 Toggle Standard checkbox; can also be grouped Input Field Text input field Canvas The game object container for UI elements Event System Allows us to trigger scripts from UI elements. An Event System is automatically created when you create a canvas. You can have multiple canvases in your game. As you start building larger games, you'll likely find a use for more than one canvas. Render mode There are a few settings in the Inspector view that you should be aware of regarding your canvas game object. The first setting is the render mode. There are three settings: Screen Space – Overlay, Screen Space – Camera, and World Space: In this render mode, the canvas is automatically resized when the user changes the size or resolution of the game screen. The second render mode, Screen Space – Camera, has a plane distance property that determines how far the canvas is rendered from the camera. The third render mode is World Space. This mode gives you the most control and can be manipulated much like any other game object. I recommend experimenting with different render modes so you know which one you like best and when to use each one. Creating a GUI Creating a GUI in Unity is a relatively easy task. We first create a canvas, or have Unity create it for us when we create our first UI element. Next, we simply add the desired UI elements to our canvas. Once all the necessary elements are in our canvas, you can arrange and format them. It is often best to switch to 2D mode in the Scene view when placing the UI elements on the canvas. This simply makes the task a bit easier. If you have used earlier versions of Unity, you'll note that several things have changed regarding creating and referencing GUI elements. For example, you'll need to include the using UnityEngine.UI; statement before referencing UI components. Also, instead of referencing GUI text as public GUIText waterHeld; you now use public Text waterHeld;. Heads-up displays A game's heads-up display (HUD) is graphical and textual information available to the user at all times. No action should be required of the user other than to look at a specific region of the screen to read the displays. For example, if you are playing car-racing game, you might have an odometer, speedometer, compass, fuel tank level, air pressure, and other visual indicators always on the screen. Creating a HUD Here are the basic steps to create a HUD: Open the game project and load the scene. Navigate and select the GameObject | UI | Text option from the drop-down menu. This will result in a Canvas game object being added to the Hierarchy view, along with a text child item. Select the child item in the Hierarchy view. Then, in the Inspector view, change the text to what you want displayed on the screen. In the Inspector view, you can change the font size. If necessary, you can change the Horizontal Overflow option from Wrap to Overflow: Zoom out in the Scene view until you can see the GUI Canvas. Use the transform tools to place your new GUI element in the top-left corner of the screen. Depending on how you are viewing the scene in the Scene view, you might need to use the hand tool to rotate the scene. So, if your GUI text appears backwards, just rotate the scene until it is correct. Repeat steps 2 through 5 until you've created all the HUD elements you need for your game. Mini-maps Miniature-maps or mini-maps provide game players with a small visual aid that helps them maintain perspective and direction in a game. These mini-maps can be used for many different purposes, depending on the game. Some examples include the ability to view a mini-map that overlooks an enemy encampment; a zoomed out view of the game map with friendly and enemy force indicators; and a mini-map that has the overall tunnel map while the main game screen views the current section of tunnel. Creating a Mini-Map Here are the steps used to create a mini-map for our game: Navigate and select GameObject | Camera from the top menu. In the Hierarchy view, change the name from Camera to Mini-Map. With the mini-map camera selected, go to the Inspector view and click on the Layer button, then Add Layer in the pop-up menu. In the next available User Layer, add the name Mini-Map: Select the Mini-Map option in the Hierarchy view, and then select Layer | Mini-Map. Now the new mini-map camera is assigned to the Mini-Map layer: Next, we'll ensure the main camera is not rendering the Mini-Map camera. Select the Main Camera option in the Hierarchy view. In the Inspector view, select Culling Mask, and then deselect Mini-Map from the pop-up menu: Now we are ready to finish the configuration of our mini-map camera. Select the Mini-Map in the Hierarchy view. Using the transform tools in the Scene view, adjust the camera object so that it shows the area of the game environment you want visible via the mini-map. In the Inspector view, under Camera, make the settings match the following values: Setting Value Clear Flags Depth only Culling Mask Everything Projection Orthographic Size 25 Clipping Planes Near 0.3; Far 1000 Viewpoint Rect X 0.75; Y 0.75; W 1; H 1 Depth 1 Rendering Path User Player Settings Target Texture None Occlusion Culling Selected HDR Not Selected With the Mini-Map camera still selected, right-click on each of the Flare Layer, GUI Layer, and Audio Listener components in the Inspector view and select Remove Component. Save your scene and your project. You are ready to test your mini-map. Mini-maps can be very powerful game components. There are a couple of things to keep in mind if you are going to use mini-maps in your games: Make sure the mini-map size does not obstruct too much of the game environment. There is nothing worse than getting shot by an enemy that you could not see because a mini-map was in the way. The mini-map should have a purpose—we do not include them in games because they are cool. They take up screen real estate and should only be used if needed, such as helping the player make informed decisions. In our game, the player is able to keep an eye on Colt's farm animals while he is out gathering water and corn. Items should be clearly visible on the mini-map. Many games use red dots for enemies, yellow for neutral forces, and blue for friendlies. This type of color-coding provides users with a lot of information at a very quick glance. Ideally, the user should have the flexibility to move the mini-map to a corner of their choosing and toggle it on and off. In our game, we placed the mini-map in the top-right corner of the game screen so that the HUD objects would not be in the way. Summary In this article, you learned about the UI system in Unity 5. You gained an appreciation for the importance of GUIs in games we create. Resources for Article: Further resources on this subject: Bringing Your Game to Life with AI and Animations [article] Looking Back, Looking Forward [article] Introducing the Building Blocks for Unity Scripts [article]

In this Article by Jeremy Walker, author of the book Backbone.js Essentials, we will explore the following topics: The differences between JavaScript's class system and the class systems of traditional object-oriented languages How new, this, and prototype enable JavaScript's class system Extend, Backbone's much easier mechanism for creating subclasses (For more resources related to this topic, see here.) JavaScript's class system Programmers who use JavaScript can use classes to encapsulate units of logic in the same way as programmers of other languages. However, unlike those languages, JavaScript relies on a less popular form of inheritance known as prototype-based inheritance. Since Backbone classes are, at their core, just JavaScript classes, they too rely on the prototype system and can be subclassed in the same way as any other JavaScript class. For instance, let's say you wanted to create your own Book subclass of the Backbone Model class with additional logic that Model doesn't have, such as book-related properties and methods. Here's how you can create such a class using only JavaScript's native object-oriented capabilities: // Define Book's Initializervar Book = function() {// define Book's default propertiesthis.currentPage = 1;this.totalPages = 1;}// Define book's parent classBook.prototype = new Backbone.Model();// Define a method of BookBook.prototype.turnPage = function() {this.currentPage += 1;return this.currentPage;} If you've never worked with prototypes in JavaScript, the preceding code may look a little intimidating. Fortunately, Backbone provides a much easier and easier to read mechanism for creating subclasses. However, since that system is built on top of JavaScript's native system, it's important to first understand how the native system works. This understanding will be helpful later when you want to do more complex class-related tasks, such as calling a method defined on a parent class. The new keyword The new keyword is a relatively simple but extremely useful part of JavaScript's class system. The first thing that you need to understand about new is that it doesn't create objects in the same way as other languages. In JavaScript, every variable is either a function, object, or primitive, which means that when we refer to a class, what we're really referring to is a specially designed initialization function. Creating this class-like function is as simple as defining a function that modifies this and then using the new keyword to call that function. Normally, when you call a function, its this is obvious. For instance, when you call the turnPage method of a book object, the this method inside turnPage will be set to this book object, as shown here: var simpleBook = {currentPage: 3, pages: 60};simpleBook.turnPage = function() {this.currentPage += 1;return this.currentPage;}simpleBook.turnPage(); // == 4 Calling a function that isn't attached to an object (in other words, a function that is not a method) results in this being set to the global scope. In a web browser, this means the window object: var testGlobalThis = function() {alert(this);}testGlobalThis(); // alerts window When we use the new keyword before calling an initialization function, three things happen (well, actually four, but we'll wait to explain the fourth one until we explain prototypes): JavaScript creates a brand new object ({})for us JavaScript sets the this method inside the initialization function to the newly created object After the function finishes, JavaScript ignores the normal return value and instead returns the object that was created As you can see, although the new keyword is simple, it's nevertheless important because it allows you to treat initialization functions as if they really are actual classes. At the same time, it does so without violating the JavaScript principle that all variables must either be a function, object, or primitive. Prototypal inheritance That's all well and good, but if JavaScript has no true concept of classes, how can we create subclasses? As it turns out, every object in JavaScript has two special properties to solve this problem: prototype and __proto__ (hidden). These two properties are, perhaps, the most commonly misunderstood aspects of JavaScript, but once you learn how they work, they are actually quite simple to use. When you call a method on an object or try to retrieve a property JavaScript first checks whether the object has the method or property defined in the object itself. In other words if you define a method such as this one: book.turnPage = function()this.currentPage += 1;}; JavaScript will use that definition first when you call turnPage. In real-world code, however, you will almost never want to put methods directly in your objects for two reasons. First, doing that will result in duplicate copies of those methods, as each instance of your class will have its own separate copy. Second, adding methods in this way requires an extra step, and that step can be easily forgotten when you create new instances. If the object doesn't have a turnPage method defined in it, JavaScript will next check the object's hidden __proto__ property. If this __proto__ object doesn't have a turnPage method, then JavaScript will look at the __proto__ property on the object's __proto__. If that doesn't have the method, JavaScript continues to check the __proto__ of the __proto__ of the __proto__ and keeps checking each successive __proto__ until it has exhausted the chain. This is similar to single-class inheritance in more traditional object-oriented languages, except that instead of going through a class chain, JavaScript instead uses a prototype chain. Just as in an object-oriented language we wind up with only a single copy of each method, but instead of the method being defined on the class itself, it's defined on the class's prototype. In a future version of JavaScript (ES6), it will be possible to work with the __proto__ object directly, but for now, the only way to actually see the __proto__ property is to use your browser's debugging tool (for instance, the Chrome Developer Tools debugger):   This means that you can't use this line of code: book.__proto__.turnPage(); Also, you can't use the following code: book.__proto__ = {turnPage: function() {this.currentPage += 1;}}; But, if you can't manipulate __proto__ directly, how can you take advantage of it? Fortunately, it is possible to manipulate __proto__, but you can only do this indirectly by manipulating prototype. Do you remember I mentioned that the new keyword actually does four things? The fourth thing is that it sets the __proto__ property of the new object it creates to the prototype property of the initialization function. In other words, if you want to add a turnPage method to every new instance of Book that you create, you can assign this turnPage method to the prototype property of the Book initialization function, For example: var Book = function() {};Book.prototype.turnPage = function() {this.currentPage += 1;};var book = new Book();book.turnPage();// this works because book.__proto__ == Book.prototype Since these concepts often cause confusion, let's briefly recap: Every object has a prototype property and a hidden __proto__ property An object's __proto__ property is set to the prototype property of its constructor when it is first created and cannot be changed Whenever JavaScript can't find a property or method on an object, it "checks each step of the __proto__ chain until it finds one or until it runs "out of chain Extending Backbone classes With that explanation out of the way, we can finally get down to the workings of Backbone's subclassing system, which revolves around Backbone's extend method. To use extend, you simply call it from the class that your new subclass will be based on, and extend will return the new subclass. This new subclass will have its __proto__ property set to the prototype property of its parent class, allowing objects created with the new subclass to access all the properties and methods of the parent class. Take an example of the following code snippet: var Book = Backbone.Model.extend();// Book.prototype.__proto__ == Backbone.Model.prototype;var book = new Book();book.destroy(); In the preceding example, the last line works because JavaScript will look up the __proto__ chain, find the Model method destroy, and use it. In other words, all the functionality of our original class has been inherited by our new class. But of course, extend wouldn't be exciting if all it can do is make exact clones of the parent classes, which is why extend takes a properties object as its first argument. Any properties or methods on this object will be added to the new class's prototype. For instance, let's try making our Book class a little more interesting by adding a property and a method: var Book = Backbone.Model.extend({currentPage: 1,turnPage: function() {this.currentPage += 1;}});var book = new Book();book.currentPage; // == 1book.turnPage(); // increments book.currentPage by one The extend method also allows you to create static properties or methods, or in other words, properties or methods that live on the class rather than on objects created from that class. These static properties and methods are passed in as the second classProperties argument to extend. Here's a quick example of how to add a static method to our Book class: var Book = Backbone.Model.extend({}, {areBooksGreat: function() {alert("yes they are!");}});Book.areBooksGreat(); // alerts "yes they are!"var book = new Book();book.areBooksGreat(); // fails because static methods must becalled on a class As you can see, there are several advantages to Backbone's approach to inheritance over the native JavaScript approach. First, the word prototype did not appear even once in any of the previously mentioned code; while you still need to understand how prototype works, you don't have to think about it just to create a class. Another benefit is that the entire class definition is contained within a single extend call, keeping all of the class's parts together visually. Also, when we use extend, the various pieces of logic that make up the class are ordered the same way as in most other programming languages, defining the super class first and then the initializer and properties, instead of the other way around. Summary In this article, we explored how JavaScript's native class system works and how the new, this, and prototype keywords/properties form the basis of it. We also learned how Backbone's extend method makes creating new subclasses much more convenient as well as how to use apply and call to invoke parent methods (or when providing callback functions) to preserve the desired this method. Resources for Article: Further resources on this subject: Testing Backbone.js Application [Article] Building an app using Backbone.js [Article] Organizing Backbone Applications - Structure, Optimize, and Deploy [Article]

In this article by Stephen Radford, author of the book Learning Web Development with Bootstrap and AngularJS, we're going to use Bootstrap and AngularJS. We'll look at building a maintainable code base as well as exploring the full potential of both frameworks. (For more resources related to this topic, see here.) Working with directives Something we've been using already without knowing it is what Angular calls directives. These are essentially powerful functions that can be called from an attribute or even its own element, and Angular is full of them. Whether we want to loop data, handle clicks, or submit forms, Angular will speed everything up for us. We first used a directive to initialize Angular on the page using ng-app, and all of the directives we're going to look at in this article are used in the same way—by adding an attribute to an element. Before we take a look at some more of the built-in directives, we need to quickly make a controller. Create a new file and call it controller.js. Save this to your js directory within your project and open it up in your editor. Controllers are just standard JS constructor functions that we can inject Angular's services such as $scope into. These functions are instantiated when Angular detects the ng-controller attribute. As such, we can have multiple instances of the same controller within our application, allowing us to reuse a lot of code. This familiar function declaration is all we need for our controller. function AppCtrl(){} To let the framework know this is the controller we want to use, we need to include this on the page after Angular is loaded and also attach the ng-controller directive to our opening <html> tag: <html ng-controller="AppCtl">…<script type="text/javascript"src="assets/js/controller.js"></script> ng-click and ng-mouseover One of the most basic things you'll have ever done with JavaScript is listened for a click event. This could have been using the onclick attribute on an element, using jQuery, or even with an event listener. In Angular, we use a directive. To demonstrate this, we'll create a button that will launch an alert box—simple stuff. First, let's add the button to our content area we created earlier: <div class="col-sm-8"><button>Click Me</button></div> If you open this up in your browser, you'll see a standard HTML button created—no surprises there. Before we attach the directive to this element, we need to create a handler in our controller. This is just a function within our controller that is attached to the scope. It's very important we attach our function to the scope or we won't be able to access it from our view at all: function AppCtl($scope){$scope.clickHandler = function(){window.alert('Clicked!');};} As we already know, we can have multiple scopes on a page and these are just objects that Angular allows the view and the controller to have access to. In order for the controller to have access, we've injected the $scope service into our controller. This service provides us with the scope Angular creates on the element we added the ng-controller attribute to. Angular relies heavily on dependency injection, which you may or may not be familiar with. As we've seen, Angular is split into modules and services. Each of these modules and services depend upon one another and dependency injection provides referential transparency. When unit testing, we can also mock objects that will be injected to confirm our test results. DI allows us to tell Angular what services our controller depends upon, and the framework will resolve these for us. An in-depth explanation of AngularJS' dependency injection can be found in the official documentation at https://docs.angularjs.org/guide/di. Okay, so our handler is set up; now we just need to add our directive to the button. Just like before, we need to add it as an additional attribute. This time, we're going to pass through the name of the function we're looking to execute, which in this case is clickHandler. Angular will evaluate anything we put within our directive as an AngularJS expression, so we need to be sure to include two parentheses indicating that this is a function we're calling: <button ng-click="clickHandler()">Click Me</button> If you load this up in your browser, you'll be presented with an alert box when you click the button. You'll also notice that we don't need to include the $scope variable when calling the function in our view. Functions and variables that can be accessed from the view live within the current scope or any ancestor scope.   Should we wish to display our alert box on hover instead of click, it's just a case of changing the name of the directive to ng-mouseover, as they both function in the exact same way. ng-init The ng-init directive is designed to evaluate an expression on the current scope and can be used on its own or in conjunction with other directives. It's executed at a higher priority than other directives to ensure the expression is evaluated in time. Here's a basic example of ng-init in action: <div ng-init="test = 'Hello, World'"></div>{{test}} This will display Hello, World onscreen when the application is loaded in your browser. Above, we've set the value of the test model and then used the double curly-brace syntax to display it. ng-show and ng-hide There will be times when you'll need to control whether an element is displayed programmatically. Both ng-show and ng-hide can be controlled by the value returned from a function or a model. We can extend upon our clickHandler function we created to demonstrate the ng-click directive to toggle the visibility of our element. We'll do this by creating a new model and toggling the value between true or false. First of all, let's create the element we're going to be showing or hiding. Pop this below your button: <div ng-hide="isHidden">Click the button above to toggle.</div> The value within the ng-hide attribute is our model. Because this is within our scope, we can easily modify it within our controller: $scope.clickHandler = function(){$scope.isHidden = !$scope.isHidden;}; Here we're just reversing the value of our model, which in turn toggles the visibility of our <div>. If you open up your browser, you'll notice that the element isn't hidden by default. There are a few ways we could tackle this. Firstly, we could set the value of $scope.hidden to true within our controller. We could also set the value of hidden to true using the ng-init directive. Alternatively, we could switch to the ng-show directive, which functions in reverse to ng-hide and will only make an element visible if a model's value is set to true. Ensure Angular is loaded within your header or ng-hide and ng-show won't function correctly. This is because Angular uses its own classes to hide elements and these need to be loaded on page render. ng-if Angular also includes an ng-if directive that works in a similar fashion to ng-show and ng-hide. However, ng-if actually removes the element from the DOM whereas ng-show and ng-hide just toggles the elements' visibility. Let's take a quick look at how we'd use ng-if with the preceding code: <div ng-if="isHidden">Click the button above to toggle.</div> If we wanted to reverse the statement's meaning, we'd simply just need to add an exclamation point before our expression: <div ng-if="!isHidden">Click the button above to toggle.</div> ng-repeat Something you'll come across very quickly when building a web app is the need to render an array of items. For example, in our contacts manager, this would be a list of contacts, but it could be anything. Angular allows us to do this with the ng-repeat directive. Here's an example of some data we may come across. It's array of objects with multiple properties within it. To display the data, we're going to need to be able to access each of the properties. Thankfully, ng-repeat allows us to do just that. Here's our controller with an array of contact objects assigned to the contacts model: function AppCtrl($scope){$scope.contacts = [{name: 'John Doe',phone: '01234567890',email: 'john@example.com'},{name: 'Karan Bromwich',phone: '09876543210',email: 'karan@email.com'}];} We have just a couple of contacts here, but as you can imagine, this could be hundreds of contacts served from an API that just wouldn't be feasible to work with without ng-repeat. First, add an array of contacts to your controller and assign it to $scope.contacts. Next, open up your index.html file and create a <ul> tag. We're going to be repeating a list item within this unordered list so this is the element we need to add our directive to: <ul><li ng-repeat="contact in contacts"></li></ul> If you're familiar with how loops work in PHP or Ruby, then you'll feel right at home here. We create a variable that we can access within the current element being looped. The variable after the in keyword references the model we created on $scope within our controller. This now gives us the ability to access any of the properties set on that object with each iteration or item repeated gaining a new scope. We can display these on the page using Angular's double curly-brace syntax. <ul><li ng-repeat="contact in contacts">{{contact.name}}</li></ul> You'll notice that this outputs the name within our list item as expected, and we can easily access any property on our contact object by referencing it using the standard dot syntax. ng-class Often there are times where you'll want to change or add a class to an element programmatically. We can use the ng-class directive to achieve this. It will let us define a class to add or remove based on the value of a model. There are a couple of ways we can utilize ng-class. In its most simple form, Angular will apply the value of the model as a CSS class to the element: <div ng-class="exampleClass"></div> Should the model referenced be undefined or false, Angular won't apply a class. This is great for single classes, but what if you want a little more control or want to apply multiple classes to a single element? Try this: <div ng-class="{className: model, class2: model2}"></div> Here, the expression is a little different. We've got a map of class names with the model we wish to check against. If the model returns true, then the class will be added to the element. Let's take a look at this in action. We'll use checkboxes with the ng-model attribute, to apply some classes to a paragraph: <p ng-class="{'text-center': center, 'text-danger': error}">Lorem ipsum dolor sit amet</p> I've added two Bootstrap classes: text-center and text-danger. These observe a couple of models, which we can quickly change with some checkboxes: The single quotations around the class names within the expression are only required when using hyphens, or an error will be thrown by Angular. <label><input type="checkbox" ng-model="center"> textcenter</label><label><input type="checkbox" ng-model="error"> textdanger</label> When these checkboxes are checked, the relevant classes will be applied to our element. ng-style In a similar way to ng-class, this directive is designed to allow us to dynamically style an element with Angular. To demonstrate this, we'll create a third checkbox that will apply some additional styles to our paragraph element. The ng-style directive uses a standard JavaScript object, with the keys being the property we wish to change (for example, color and background). This can be applied from a model or a value returned from a function. Let's take a look at hooking it up to a function that will check a model. We can then add this to our checkbox to turn the styles off and on. First, open up your controller.js file and create a new function attached to the scope. I'm calling mine styleDemo: $scope.styleDemo = function(){if(!$scope.styler){return;}return {background: 'red',fontWeight: 'bold'};}; Inside the function, we need to check the value of a model; in this example, it's called styler. If it's false, we don't need to return anything, otherwise we're returning an object with our CSS properties. You'll notice that we used fontWeight rather than font-weight in our returned object. Either is fine, and Angular will automatically switch the CamelCase over to the correct CSS property. Just remember than when using hyphens in JavaScript object keys, you'll need to wrap them in quotation marks. This model is going to be attached to a checkbox, just like we did with ng-class: <label><input type="checkbox" ng-model="styler"> ng-style</label> The last thing we need to do is add the ng-style directive to our paragraph element: <p .. ng-style="styleDemo()">Lorem ipsum dolor sit amet</p> Angular is clever enough to recall this function every time the scope changes. This means that as soon as our model's value changes from false to true, our styles will be applied and vice versa. ng-cloak The final directive we're going to look at is ng-cloak. When using Angular's templates within our HTML page, the double curly braces are temporarily displayed before AngularJS has finished loading and compiling everything on our page. To get around this, we need to temporarily hide our template before it's finished rendering. Angular allows us to do this with the ng-cloak directive. This sets an additional style on our element whilst it's being loaded: display: none !important;. To ensure there's no flashing while content is being loaded, it's important that Angular is loaded in the head section of our HTML page. Summary We've covered a lot in this article, let's recap it all. Bootstrap allowed us to quickly create a responsive navigation. We needed to include the JavaScript file included with our Bootstrap download to enable the toggle on the mobile navigation. We also looked at the powerful responsive grid system included with Bootstrap and created a simple two-column layout. While we were doing this, we learnt about the four different column class prefixes as well as nesting our grid. To adapt our layout, we discovered some of the helper classes included with the framework to allow us to float, center, and hide elements. In this article, we saw in detail Angular's built-in directives: functions Angular allows us to use from within our view. Before we could look at them, we needed to create a controller, which is just a function that we can pass Angular's services into using dependency injection. Directives such as ng-click and ng-mouseover are essentially just new ways of handling events that you will have no doubt done using either jQuery or vanilla JavaScript. However, directives such as ng-repeat will probably be a completely new way of working. It brings some logic directly within our view to loop through data and display it on the page. We also looked at directives that observe models on our scope and perform different actions based on their values. Directives like ng-show and ng-hide will show or hide an element based on a model's value. We also saw this in action in ng-class, which allowed us to add some classes to our elements based on our models' values. Resources for Article: Further resources on this subject: AngularJS Performance [Article] AngularJS Web Application Development Cookbook [Article] Role of AngularJS [Article]

 In this article by Wagner Roberto dos Santos, author of the book Infinispan Data Grid Platform Definitive Guide, we will see the usage of Map/Reduce API and its introduction in Infinispan. Using the Map/Reduce API According to Gartner, from now on in-memory data grids and in-memory computing will be racing towards mainstream adoption and the market for this kind of technology is going to reach 1 billion by 2016. Thinking along these lines, Infinispan already provides a MapReduce API for distributed computing, which means that we can use Infinispan cache to process all the data stored in heap memory across all Infinispan instances in parallel. If you're new to MapReduce, don't worry, we're going to describe it in the next section in a way that gets you up to speed quickly. An introduction to Map/Reduce MapReduce is a programming model introduced by Google, which allows for massive scalability across hundreds or thousands of servers in a data grid. It's a simple concept to understand for those who are familiar with distributed computing and clustered environments for data processing solutions. You can find the paper about MapReduce in the following link:http://research.google.com/archive/mapreduce.html The MapReduce has two distinct computational phases; as the name states, the phases are map and reduce: In the map phase, a function called Map is executed, which is designed to take a set of data in a given cache and simultaneously perform filtering, sorting operations, and outputs another set of data on all nodes. In the reduce phase, a function called Reduce is executed, which is designed to reduce the final form of the results of the map phase in one output. The reduce function is always performed after the map phase. Map/Reduce in the Infinispan platform The Infinispan MapReduce model is an adaptation of the Google original MapReduce model. There are four main components in each map reduce task, they are as follows: MapReduceTask: This is a distributed task allowing a large-scale computation to be transparently parallelized across Infinispan cluster nodes. This class provides a constructor that takes a cache whose data will be used as the input for this task. The MapReduceTask orchestrates the execution of the Mapper and Reducer seamlessly across Infinispan nodes. Mapper: A Mapper is used to process each input cache entry K,V. A Mapper is invoked by MapReduceTask and is migrated to an Infinispan node, to transform the K,V input pair into intermediate keys before emitting them to a Collector. Reducer: A Reducer is used to process a set of intermediate key results from the map phase. Each execution node will invoke one instance of Reducer and each instance of the Reducer only reduces intermediate keys results that are locally stored on the execution node. Collator: This collates results from reducers executed on the Infinispan cluster and assembles a final result returned to an invoker of MapReduceTask. The following image shows that in a distributed environment, an Infinispan MapReduceTask is responsible for starting the process for a given cache, unless you specify an onKeys(Object...) filter, all available key/value pairs of the cache will be used as input data for the map reduce task:   In the preceding image, the Map/Reduce processes are performing the following steps: The MapReduceTask in the Master Task Node will start the Map Phase by hashing the task input keys and grouping them by the execution node they belong to and then, the Infinispan master node will send a map function and input keys to each node. In each destination, the map will be locally loaded with the corresponding value using the given keys. The map function is executed on each node, resulting in a map< KOut, VOut > object on each node. The Combine Phase is initiated when all results are collected, if a combiner is specified (via combineWith(Reducer<KOut, VOut> combiner) method), the combiner will extract the KOut keys and invoke the reduce phase on keys. Before starting the Reduce Phase, Infinispan will execute an intermediate migration phase, where all intermediate keys and values are grouped. At the end of the Combine Phase, a list of KOut keys are returned to the initial Master Task Node. At this stage, values (VOut) are not returned, because they are not needed in the master node. At this point, Infinispan is ready to start the Reduce Phase; the Master Task Node will group KOut keys by the execution node and send a reduce command to each node where keys are hashed. The reducer is invoked and for each KOut key, the reducer will grab a list of VOut values from a temporary cache belonging to MapReduceTask, wraps it with an iterator, and invokes the reduce method on it. Each reducer will return one map with the KOut/VOut result values. The reduce command will return to the Master Task Node, which in turn will combine all resulting maps into one single map and return it as a result of MapReduceTask. Sample application – find a destination Now that we have seen what map and reduce are, and how the Infinispan model works, let's create a Find Destination application that illustrates the concepts we have discussed. To demonstrate how CDI works, in the last section, we created a web service that provides weather information. Now, based on this same weather information service, let's create a map/reduce engine for the best destination based on simple business rules, such as destination type (sun destination, golf, skiing, and so on). So, the first step is to create the WeatherInfo cache object that will hold information about the weather: public class WeatherInfo implements Serializable {  private static final long serialVersionUID =     -3479816816724167384L;  private String country;  private String city;  private Date day;  private Double temp;  private Double tempMax;  private Double tempMin;  public WeatherInfo(String country, String city, Date day,     Double temp) {    this(country, city, day, temp, temp + 5, temp - 5);  }  public WeatherInfo(String country, String city, Date day,     Double temp,    Double tempMax, Double tempMin) {    super();    this.country = country;    this.city = city;    this.day = day;    this.temperature = temp;    this.temperatureMax = tempMax;    this.temperatureMin = tempMin;  }// Getters and Setters ommitted  @Override  public String toString() {    return "{WeatherInfo:{ country:" + country + ", city:" +       city + ", day:" + day + ", temperature:" + temperature + ",       temperatureMax:" + temperatureMax + ", temperatureMin:" +           temperatureMin + "}";  }} Now, let's create an enum object to define the type of destination a user can select and the rules associated with each destination. To keep it simple, we are going to have only two destinations, sun and skiing. The temperature value will be used to evaluate if the destination can be considered the corresponding type: public enum DestinationTypeEnum {SUN(18d, "Sun Destination"), SKIING(-5d, "Skiing Destination");private Double temperature;private String description;DestinationTypeEnum(Double temperature, String description) {this.temperature = temperature;this.description = description;}public Double getTemperature() {return temperature;}public String getDescription() {return description;} Now it's time to create the Mapper class—this class is going to be responsible for validating whether each cache entry fits the destination requirements. To define the DestinationMapper class, just extend the Mapper<KIn, VIn, KOut, VOut> interface and implement your algorithm in the map method; public class DestinationMapper implementsMapper<String, WeatherInfo, DestinationTypeEnum, WeatherInfo> {private static final long serialVersionUID =-3418976303227050166L;public void map(String key, WeatherInfo weather,Collector<DestinationTypeEnum, WeatherInfo> c) {if (weather.getTemperature() >= SUN.getTemperature()){c.emit(SUN, weather);}else if (weather.getTemperature() <=SKIING.getTemperature()) {c.emit(SKIING, weather);}}} The role of the Reducer class in our application is to return the best destination among all destinations, based on the highest temperature for sun destinations and the lowest temperature for skiing destinations, returned by the mapping phase. To implement the Reducer class, you'll need to implement the Reducer<KOut, VOut> interface: public class DestinationReducer implementsReducer<DestinationTypeEnum, WeatherInfo> {private static final long serialVersionUID = 7711240429951976280L;public WeatherInfo reduce(DestinationTypeEnum key,Iterator<WeatherInfo> it) {WeatherInfo bestPlace = null;if (key.equals(SUN)) {while (it.hasNext()) {WeatherInfo w = it.next();if (bestPlace == null || w.getTemp() >bestPlace.getTemp()) {bestPlace = w;}}} else { /// Best for skiingwhile (it.hasNext()) {WeatherInfo w = it.next();if (bestPlace == null || w.getTemp() <bestPlace.getTemp()) {bestPlace = w;}}}return bestPlace;}} Finally, to execute our sample application, we can create a JUnit test case with the MapReduceTask. But first, we have to create a couple of cache entries before executing the task, which we are doing in the setUp() method: public class WeatherInfoReduceTest {private static final Log logger =LogFactory.getLog(WeatherInfoReduceTest.class);private Cache<String, WeatherInfo> weatherCache;@Beforepublic void setUp() throws Exception {Date today = new Date();EmbeddedCacheManager manager = new DefaultCacheManager();Configuration config = new ConfigurationBuilder().clustering().cacheMode(CacheMode.LOCAL).build();manager.defineConfiguration("weatherCache", config);weatherCache = manager.getCache("weatherCache");WeatherInfoweatherCache.put("1", new WeatherInfo("Germany", "Berlin",today, 12d));weatherCache.put("2", new WeatherInfo("Germany","Stuttgart", today, 11d));weatherCache.put("3", new WeatherInfo("England", "London",today, 8d));weatherCache.put("4", new WeatherInfo("England","Manchester", today, 6d));weatherCache.put("5", new WeatherInfo("Italy", "Rome",today, 17d));weatherCache.put("6", new WeatherInfo("Italy", "Napoli",today, 18d));weatherCache.put("7", new WeatherInfo("Ireland", "Belfast",today, 9d));weatherCache.put("8", new WeatherInfo("Ireland", "Dublin",today, 7d));weatherCache.put("9", new WeatherInfo("Spain", "Madrid",today, 19d));weatherCache.put("10", new WeatherInfo("Spain", "Barcelona",today, 21d));weatherCache.put("11", new WeatherInfo("France", "Paris",today, 11d));weatherCache.put("12", new WeatherInfo("France","Marseille", today, -8d));weatherCache.put("13", new WeatherInfo("Netherlands","Amsterdam", today, 11d));weatherCache.put("14", new WeatherInfo("Portugal", "Lisbon",today, 13d));weatherCache.put("15", new WeatherInfo("Switzerland","Zurich", today, -12d));}@Testpublic void execute() {MapReduceTask<String, WeatherInfo, DestinationTypeEnum,WeatherInfo> task = new MapReduceTask<String, WeatherInfo,DestinationTypeEnum, WeatherInfo>(weatherCache);task.mappedWith(new DestinationMapper()).reducedWith(newDestinationReducer());Map<DestinationTypeEnum, WeatherInfo> destination =task.execute();assertNotNull(destination);assertEquals(destination.keySet().size(), 2);logger.info("********** PRINTING RESULTS FOR WEATHER CACHE*************");for (DestinationTypeEnum destinationType :destination.keySet()){logger.infof("%s - Best Place: %sn",destinationType.getDescription(),destination.get(destinationType));}}} When we execute the application, you should expect to see the following output: INFO: Skiing DestinationBest Place: {WeatherInfo:{ country:Switzerland, city:Zurich,day:Mon Jun 02 19:42:22 IST 2014, temp:-12.0, tempMax:-7.0,tempMin:-17.0}INFO: Sun DestinationBest Place: {WeatherInfo:{ country:Spain, city:Barcelona, day:MonJun 02 19:42:22 IST 2014, temp:21.0, tempMax:26.0, tempMin:16.0} Summary In this article, you learned how to work with applications in modern distributed server architecture, using the Map Reduce API, and how it can abstract parallel programming into two simple primitives, the map and reduce methods. We have seen a sample use case Find Destination that demonstrated how use map reduce almost in real time. Resources for Article: Further resources on this subject: MapReduce functions [Article] Hadoop and MapReduce [Article] Introduction to MapReduce [Article]

In this article by Ankur Goel, author of the book, Neo4j Cookbook, we will cover the following recipes: Installing the Neo4j Spatial extension Importing the Esri shapefiles Importing the OpenStreetMap files Importing data using the REST API Creating a point layer using the REST API Finding geometries within the bounding box Finding geometries within a distance Finding geometries within a distance using Cypher By definition, any database that is optimized to store and query the data that represents objects defined in a geometric space is called a spatial database. Although Neo4j is primarily a graph database, due to the importance of geospatial data in today's world, the spatial extension has been introduced in Neo4j as an unmanaged extension. It gives you most of the facilities, which are provided by common geospatial databases along with the power of connectivity through edges, which Neo4j, as a graph database, provides. In this article, we will take a look at some of the widely used use cases of Neo4j as a spatial database, and you will learn how typical geospatial operations can be performed on it. Before proceeding further, you need to install Neo4j on your system using one of the recipies that follow here. The installation will depend on your system type. (For more resources related to this topic, see here.) Single node installation of Neo4j over Linux Neo4j is a highly scalable graph database that runs over all the common platforms; it can be used as is or can be embedded inside applications as well. The following recipe will show you how to set up a single instance of Neo4j over the Linux operating system. Getting ready Perform the following steps to get started with this recipe: Download the community edition of Neo4j from http://www.neo4j.org/download for the Linux platform: $ wget http://dist.neo4j.org/neo4j-community-2.2.0-M02-unix.tar.gz Check whether JDK/JRE is installed for your operating system or not by typing this in the shell prompt: $ echo $JAVA_HOME If this command throws no output, install Java for your Linux distribution and also set the JAVA_HOME path How to do it... Now, let's install Neo4j over the Linux operating system, which is simple, as shown in the following steps: Extract the TAR file using the following command: $ tar –zxvf neo4j-community-2.2.0-M02-unix.tar.gz $ ls Go to the bin directory under the root folder: $ cd <neo4j-community-2.2.0-M02>/bin/ Start the Neo4j graph database server: $ ./neo4j start Check whether Neo4j is running or not using the following command: $ ./neo4j status Neo4j can also be monitored using the web console. Open http://<ip>:7474/webadmin, as shown in the following screenshot: The preceding diagram is a screenshot of the web console of Neo4j through which the server can be monitored and different Cypher queries can be run over the graph database. How it works... Neo4j comes with prebuilt binaries over the Linux operating system, which can be extracted and run over. Neo4j comes with both web-based and terminal-based consoles, over which the Neo4j graph database can be explored. See also During installation, you may face several kind of issues, such as max open files and so on. For more information, check out http://neo4j.com/docs/stable/server-installation.html#linux-install. Single node installation of Neo4j over Windows Neo4j is a highly scalable graph database that runs over all the common platforms; it can be used as is or can be embedded inside applications. The following recipe will show you how to set up a single instance of Neo4j over the Windows operating system. Getting ready Perform the following steps to get started with this recipe: Download the Windows installer from http://www.neo4j.org/download. This has both 32- and 64-bit prebuilt binaries Check whether Java is installed for the operating system or not by typing this in the cmd prompt: echo %JAVA_HOME% If this command throws no output, install Java for your Windows distribution and also set the JAVA_HOME path How to do it... Now, let's install Neo4j over the Windows operating system, which is simple, as shown here: Run the installer by clicking on the downloaded file: The preceding screenshot shows the Windows installer running. After the installation is complete, when you run the software, it will ask for the database location. Carefully choose the location as the entire graph database will be stored in this folder: The preceding screenshot shows the Windows installer asking for the graph database's location. The Neo4j browser can be opened by entering http://localhost:7474/ in the browser. The following screenshot depicts Neo4j started over the Windows platform: How it works... Neo4j comes with prebuilt binaries over the Windows operating system, which can be extracted and run over. Neo4j comes with both web-based and terminal-based consoles, over which the Neo4j graph database can be explored. See also During installation, you might face several kinds of issues such as max open files and so on. For more information, check out http://neo4j.com/docs/stable/server-installation.html#windows-install. Single node installation of Neo4j over Mac OS X Neo4j is a highly scalable graph database that runs over all the common platforms; it can be used as in a mode and can also be embedded inside applications. The following recipe will show you how to set up a single instance of Neo4j over the OS X operating system. Getting ready Perform the following steps to get started with this recipe: Download the binary version of Neo4j from http://www.neo4j.org/download for the Mac OS X platform and the community edition as shown in the following command: $ wget http://dist.neo4j.org/neo4j-community-2.2.0-M02-unix.tar.gz Check whether Java is installed for the operating system or not by typing this over the cmd prompt: $ echo $JAVA_HOME If this command throws no output, install Java for your Mac OS X distribution and also set the JAVA_HOME path How to do it... Now, let's install Neo4j over the OS X operating system, which is very simple, as shown in the following steps: Extract the TAR file using the following command: $ tar –zxvf neo4j-community-2.2.0-M02-unix.tar.gz $ ls Go to the bin directory under the root folder: $ cd <neo4j-community-2.2.0-M02>/bin/ Start the Neo4j graph database server: $ ./neo4j start Check whether Neo4j is running or not using the following command: $ ./neo4j status How it works... Neo4j comes with prebuilt binaries over the OS X operating system, which can be extracted and run over. Neo4j comes with both web-based and terminal-based consoles, over which the Neo4j graph database can be explored. There's more… Neo4j over Mac OS X can also be installed using brew, which has been explained here. Run the following commands over the shell: $ brew update $ brew install neo4j After this, Neo4j can be started using the start option with the Neo4j command: $ neo4j start This will start the Neo4j server, which can be accessed from the default URL (http://localhost:7474). The installation can be reached using the following commands: $ cd /usr/local/Cellar/neo4j/ $ cd {NEO4J_VERSION}/libexec/ You can learn more about OS X installation from http://neo4j.com/docs/stable/server-installation.html#osx-install. Due to the limitation of content that can provided in this article, we assume you would already know how to perform the basic operations using Neo4j such as creating a graph, importing data from different formats into Neo4j, the common configurations used for Neo4j. Installing the Neo4j Spatial extension Neo4j Spatial is a library of utilities for Neo4j that facilitates the enabling of spatial operations on the data. Even on the existing data, geospatial indexes can be added and many geospatial operations can be performed on it. In this recipe, you will learn how to install the Neo4j Spatial extension. Getting ready The following steps will get you started with this recipe: Install Neo4j using the earlier recipes in this article. Install the dependencies listed in the pom.xml file for this project from https://github.com/neo4j-contrib/spatial/blob/master/pom.xml. Install Maven using the following command for your operating system: For Debian systems: apt-get install maven For Redhat/Centos systems: yum install apache-maven To install on a Windows-based system, please refer to https://maven.apache.org/guides/getting-started/windows-prerequisites.html. How to do it... Now, let's install the Neo4j Spatial plugin, which is very simple to do, by following these steps: Clone the GitHub repository for spatial extension: git clone git://github.com/neo4j/spatial spatial Move into the spatial directory: cd spatial Build the code using Maven. This will download all the dependencies, compile the library, run the tests, and install the artifacts in the local repository: mvn clean install Move the built artifact to the Neo4j plugin directory: unzip target/neo4j/neo4j-spatial-0.11-SNAPSHOT-server-plugin.zip $NEO4J_ROOT_DIR/plugins/ Restart the Neo4j graph database server: $NEO4J_ROOT_DIR/bin/neo4j restart Check whether the Neo4j Spatial plugin is properly installed or not: curl –L http://<neo4j_server_ip>:<port>/db/data If you are using Neo4j 2.2 or higher, then use the following command: curl --user neo4j:<password> http://localhost:7474/db/data/ The output will look like what is shown in the following screenshot, which shows the Neo4j Spatial plugin installed: How it works... Neo4j Spatial is a library of utilities that helps perform geospatial operations on the dataset, which is present in the Neo4j graph database. You can add geospatial indexes on the existing data and perform operations, such as data within a specified region or within some distance of point of interest. Neo4j Spatial comes as an unmanaged extension, which can be easily installed as well as removed from Neo4j. The extension does not interfere with any of the core functionality. There's more… To read more about Neo4j Spatial extension, we encourage users to visit the GitHub repository at https://github.com/neo4j-contrib/spatial. Also, it will be good to read about the Neo4j unmanaged extension in general (http://neo4j.com/docs/stable/server-unmanaged-extensions.html). Importing the Esri shapefiles The shapefile format is a popular geospatial vector data format for the Geographic Information System (GIS) software. It is developed and regulated by Esri as an open specification for data interoperability among Esri. It is very popular among GIS products, and many times, the data format is in Esri shapefiles. The main file is the .shp file, which contains the geometry data. The binary data file consists of a single, fixed-length header followed by variable-length data records. In this recipe, you will learn how to import the Esri shapefiles in the Neo4j graph database. Getting ready Perform the following steps to get started with this recipe: Install Neo4j using the earlier recipies in this article. Install the Neo4j Spatial plugin using the recipe Installing the Neo4j Spatial extension, from this article. Restart the Neo4j graph database server using the following command: $NEO4J_ROOT_DIR/bin/neo4j restart How to do it... Since the Esri shapefile format is, by default, supported by the Neo4j Spatial extension, it is very easy to import data using the Java API from it using the following steps: Download a sample .shp file from http://www.statsilk.com/maps/download-free-shapefile-maps. Execute the following commands: wget http://biogeo.ucdavis.edu/data/diva/adm/AFG_adm.zip unzip AFG_adm.zip mv AFG_adm1.* /data The ShapefileImporter method lets you import data from the Esri shapefile using the following code: GraphDatabaseService esri_database = new GraphDatabaseFactory().newEmbeddedDatabase(storeDir); try {    ShapefileImporter importer = new ShapefileImporter(esri_database); importer.importFile("/data/AFG_adm1.shp", "layer_afganistan");        } finally {            esri_database.shutdown(); } Using similar code, we can import multiple SHP files into the same layer or different layers, as shown in the following code snippet: File dir = new File("/data");      FilenameFilter filter = new FilenameFilter() {          public boolean accept(File dir, String name) {      return name.endsWith(".shp"); }};   File[] listOfFiles = dir.listFiles(filter); for (final File fileEntry : listOfFiles) {      System.out.println("FileEntry Directory "+fileEntry);    try {    importer.importFile(fileEntry.toString(), "layer_afganistan"); } catch(Exception e){    esri_database.shutdown(); }} How it works... The Neo4j Spatial extension natively supports the import of data in the shapefile format. Using the ShapefileImporter method, any SHP file can be easily imported into Neo4j. The ShapefileImporter method takes two arguments: the first argument is the path to the SHP files and the second is the layer in which it should be imported. There's more… We will encourage you to read more about shapefiles and layers in general; for this, please visit the following URLs for more information: http://en.wikipedia.org/wiki/Shapefile http://wiki.openstreetmap.org/wiki/Shapefiles http://www.gdal.org/drv_shapefile.html Importing the OpenStreetMap files OpenStreetMap is a powerhouse of data when it comes to geospatial data. It is a collaborative project to create a free, editable map of the world. OpenStreetMap provides geospatial data in the .osm file format. To read more about .osm files in general, check out http://wiki.openstreetmap.org/wiki/.osm. In this recipe, you will learn how to import the .osm files in the Neo4j graph database. Getting ready Perform the following steps to get started with this recipe: Install Neo4j using the earlier recipies in this article. Install the Neo4j Spatial plugin using the recipe Installing the Neo4j Spatial extension, from this article. Restart the Neo4j graph database server: $NEO4J_ROOT_DIR/bin/neo4j restart How to do it... Since the OSM file format is, by default, supported by the Neo4j Spatial extension, it is very easy to import data from it using the following steps: Download one sample .osm file from http://wiki.openstreetmap.org/wiki/Planet.osm#Downloading. Execute the following commands: wget http://download.geofabrik.de/africa-latest.osm.bz2 bunzip2 africa-latest.osm.bz2 mv africa-latest.osm /data The importfile method lets you import data from the .osm file, as shown in the following code snippet: OSMImporter importer = new OSMImporter("africa"); try {    importer.importFile(osm_database, "/data/botswana-latest.osm", false, 5000, true); } catch(Exception e){    osm_database.shutdown(); } importer.reIndex(osm_database,10000); Using similar code, we can import multiple OSM files into the same layer or different layers, as shown here: File dir = new File("/data");      FilenameFilter filter = new FilenameFilter() {          public boolean accept(File dir, String name) {      return name.endsWith(".osm"); }}; File[] listOfFiles = dir.listFiles(filter); for (final File fileEntry : listOfFiles) {      System.out.println("FileEntry Directory "+fileEntry);    try {importer.importFile(osm_database, fileEntry.toString(), false, 5000, true); importer.reIndex(osm_database,10000); } catch(Exception e){    osm_database.shutdown(); } How it works... This is slightly more complex as it requires two phases: the first phase requires a batch inserter performing insertions into the database, and the second phase requires reindexing of the database with the spatial indexes. There's more… We will encourage you to read more about the OSM file and the batch inserter in general; for this, visit the following URLs: http://en.wikipedia.org/wiki/OpenStreetMap http://wiki.openstreetmap.org/wiki/OSM_file_formats http://neo4j.com/api_docs/2.0.2/org/neo4j/unsafe/batchinsert/BatchInserter.html Importing data using the REST API The recipes that you have learned until now consist of Java code, which is used to import spatial data into Neo4j. However, by using any other programming language, such as Python or Ruby, spatial data can be easily imported into Neo4j using the REST interface. In this recipe, you will learn how to import geospatial data using the REST interface. Getting ready Perform the following steps to get started with this recipe: Install Neo4j using the earlier recipies in this article. Install the Neo4j Spatial plugin using the recipe Installing the Neo4j Spatial extension, from this article. Restart the Neo4j graph database server: $NEO4J_ROOT_DIR/bin/neo4j restart How to do it... Using the REST interface is a very simple three-stage process to import the geospatial data into the Neo4j graph database server. For the sake of simplicity, the code of the Python language has been used to explain this recipe, although you can also use curl for this recipe: Create the spatial index, as shown in the following code: # Create geom index url = http://<neo4j_server_ip>:<port>/db/data/index/node/ payload= {    "name" : "geom",    "config" : {        "provider" : "spatial",        "geometry_type" : "point",        "lat" : "lat",        "lon" : "lon"    } } Create nodes as lat/lng and data as properties, as shown in the following code: url = "http://<neo4j_server_ip>:<port>/db/data/node" payload = {'lon': 38.6, 'lat': 67.88, 'name': 'abc'} req = urllib2.Request(url) req.add_header('Content-Type', 'application/json') response = urllib2.urlopen(req, json.dumps(payload)) node = json.loads(response.read())['self'] Add the preceding created node to the geospatial index, as shown in the following code snippet: #add node to geom index url = "http://<neo4j_server_ip>:<port>/db/data/index/node/geom" payload = {'value': 'dummy', 'key': 'dummy', 'uri': node} req = urllib2.Request(url) req.add_header('Content-Type', 'application/json') response = urllib2.urlopen(req, json.dumps(payload)) print response.read() The data will look like what is shown in the following screenshot after the addition of a few more nodes; this screenshot depicts the Neo4j Spatial data that has been imported: The following screenshot depicts the properties of a single node, which has been imported into Neo4j: How it works... Adding geospatial data using the REST API is a three-step process, listed as follows: Create a geospatial index using an endpoint, by following this URL as a template: http://<neo4j_server_ip>:<port>/db/data/index/node/ Add a node to the Neo4j graph database using an endpoint, by following this URL as a template: http://<neo4j_server_ip>:<port>/db/data/node Add the created node to the geospatial index using the endpoint, by following this URL as a template: http://<neo4j_server_ip>:<port>/db/data/index/node/geom There's more… We encourage you to read more about the spatial REST interfaces in general (http://neo4j-contrib.github.io/spatial/). Creating a point layer using the REST API In this recipe, you will learn how to create a point layer using the REST API interface. Getting ready Perform the following steps to get started with this recipe: Install Neo4j using the earlier recipies in this article. Install the Neo4j Spatial plugin using the recipe Installing the Neo4j Spatial extension, from this article. Restart the Neo4j graph database server using the following command: $NEO4J_ROOT_DIR/bin/neo4j restart How to do it... In this recipe, we will use the http://<neo4j_server_ip>:/db/data/ext/ SpatialPlugin/graphdb/addSimplePointlayer endpoint to create a simple point layer. Let's add a simple point layer, as shown in the following code: "layer" : "geom", "lat"   : "lat" , "lon"   : "lon",url = "http://<neo4j_server_ip>:<port>//db/data/ext/SpatialPlugin/graphdb/addSimplePointlayer payload= { "layer" : "geom", "lat"   : "lat" , "lon"   : "lon", } r = requests.post(url, data=json.dumps(payload), headers=headers) The data will look like what is shown in the following screenshot; this screenshot shows the output of the create point in layer query: How it works... Creating a point in the layer query is based on the REST interface, which the Neo4j Spatial plugin already provides with it. There's more… We will encourage you to read more about spatial REST interfaces in general; to do this, visit http://neo4j-contrib.github.io/spatial/. Finding geometries within the bounding box In this recipe, you will learn how to find all the geometries within the bounding box using the spatial REST interface. Getting ready Perform the following steps to get started with this recipe: Install Neo4j using the earlier recipies in this article. Install the Neo4j Spatial plugin using the recipe Installing the Neo4j Spatial extension, from this article. Restart the Neo4j graph database server using the following command: $NEO4J_ROOT_DIR/bin/neo4j restart How to do it... In this recipe, we will use the following endpoint to find all the geometries within the bounding box:http://<neo4j_server_ip>:<port>/db/data/ext/SpatialPlugin/graphdb/findGeometriesInBBox. Let's find the all the geometries, using the following information: "minx" : 0.0, "maxx" : 100.0, "miny" : 0.0, "maxy" : 100.0 url = "http://<neo4j_server_ip>:<port>//db/data/ext/SpatialPlugin/graphdb payload= { "layer" : "geom", "minx" : 0.0, "maxx" : 100.3, "miny" : 0.0, "maxy" : 100.0 } r = requests.post(url, data=json.dumps(payload), headers=headers) The data will look like what is shown in the following screenshot; this screenshot shows the output of the bounding box query: How it works... Finding geometries in the bounding box is based on the REST interface, which the Neo4j Spatial plugin provides. The output of the REST call contains an array of the nodes, containing the node's id, lat/lng, and its incoming/outgoing relationships. In the preceding output, you can see node id54 returned as the output. There's more… We will encourage you to read more about spatial REST interfaces in general; to do this, visit http://neo4j-contrib.github.io/spatial/. Finding geometries within a distance In this recipe, you will learn how to find all the geometries within a distance using the spatial REST interface. Getting ready Perform the following steps to get started with this recipe: Install Neo4j using the earlier recipies in this article. Install the Neo4j Spatial plugin using the recipe Installing the Neo4j Spatial extension, from this article. Restart the Neo4j graph database server using the following command: $NEO4J_ROOT_DIR/bin/neo4j restart How to do it... In this recipe, we will use the following endpoint to find all the geometries within a certain distance: http://<neo4j_server_ip>:<port>/db/data/ext/SpatialPlugin/graphdb/findGeometriesWithinDistance. Let's find all the geometries between the specified distance using the following information: "pointX" : -116.67, "pointY" : 46.89, "distanceinKm" : 500, url = "http://<neo4j_server_ip>:<port>//db/data/ext/SpatialPlugin/graphdb/findGeometriesWithinDistance payload= { "layer" : "geom", "pointY" : 46.8625, "pointX" : -114.0117, "distanceInKm" : 500, } r = requests.post(url, data=json.dumps(payload), headers=headers) The data will look like what is shown in the following screenshot; this screenshot shows the output of a withinDistance query: How it works... Finding geometries within a distance is based on the REST interface that the Neo4j Spatial plugin provides. The output of the REST call contains an array of the nodes, containing the node's id, lat/lng, and its incoming/outgoing relationships. In the preceding output, we can see node id71 returned as the output. There's more… We encourage you to read more about the spatial REST interfaces in general (http://neo4j-contrib.github.io/spatial/). Finding geometries within a distance using Cypher In this recipe, you will learn how to find all the geometries within a distance using the Cypher query. Getting ready Perform the following steps to get started with this recipe: Install Neo4j using the earlier recipies in this article. Install the Neo4j Spatial plugin using the recipe Installing the Neo4j Spatial extension, from this article. Restart the Neo4j graph database server: $NEO4J_ROOT_DIR/bin/neo4j restart How to do it... In this recipe, we will use the following endpoint to find all the geometries within a certain distance: http://<neo4j_server_ip>:<port>/db/data/cipher Let's find all the geometries within a distance using a Cypher query: "pointX" : -116.67, "pointY" : 46.89, "distanceinKm" : 500, url = "http://<neo4j_server_ip>:<port>//db/data/cypher payload= { "query" : "START n=node:geom('withinDistance:[46.9163, -114.0905, 500.0]') RETURN n" } r = requests.post(url, data=json.dumps(payload), headers=headers) The data will look like what is shown in the following screenshot; this screenshot shows the output of the withinDistance query that uses Cypher: The following is the Cypher output in the Neo4j console: How it works... Cypher comes with a withinDistance query, which takes three parameters: lat, lon, and search distance. There's more… We will encourage you to read more about the spatial REST interfaces in general (http://neo4j-contrib.github.io/spatial/). Summary Developing Location-based Services with Neo4j, teaches you the most important aspect of today's data, location, and how to deal with it in Neo4j. You have learnt how to import geospatial data into Neo4j and run queries, such as proximity searches, bounding boxes, and so on. Resources for Article:   Further resources on this subject: Recommender systems dissected Components [article] Working with a Neo4j Embedded Database [article] Differences in style between Java and Scala code [article]

In this article, Ashwin Pajankar, the author of the book, Raspberry PI Computer Vision Programming, takes us through basic image processing in OpenCV. We will do this with the help of the following topics: Image arithmetic operations—adding, subtracting, and blending images Splitting color channels in an image Negating an image Performing logical operations on an image This article is very short and easy to code with plenty of hands-on activities. (For more resources related to this topic, see here.) Arithmetic operations on images In this section, we will have a look at the various arithmetic operations that can be performed on images. Images are represented as matrices in OpenCV. So, arithmetic operations on images are similar to the arithmetic operations on matrices. Images must be of the same size for you to perform arithmetic operations on the images, and these operations are performed on individual pixels. cv2.add(): This function is used to add two images, where the images are passed as parameters. cv2.subtract(): This function is used to subtract an image from another. We know that the subtraction operation is not commutative. So, cv2.subtract(img1,img2) and cv2.(img2,img1) will yield different results, whereas cv2.add(img1,img2) and cv2.add(img2,img1) will yield the same result as the addition operation is commutative. Both the images have to be of same size and type, as explained before. Check out the following code: import cv2 img1 = cv2.imread('/home/pi/book/test_set/4.2.03.tiff',1) img2 = cv2.imread('/home/pi/book/test_set/4.2.04.tiff',1) cv2.imshow('Image1',img1) cv2.waitKey(0) cv2.imshow('Image2',img2) cv2.waitKey(0) cv2.imshow('Addition',cv2.add(img1,img2)) cv2.waitKey(0) cv2.imshow('Image1-Image2',cv2.subtract(img1,img2)) cv2.waitKey(0) cv2.imshow('Image2-Image1',cv2.subtract(img2,img1)) cv2.waitKey(0) cv2.destroyAllWindows() The preceding code demonstrates the usage of arithmetic functions on images. Here's the output window of Image1: Here is the output window of Addition: The output window of Image1-Image2 looks like this: Here is the output window of Image2-Image1: Blending and transitioning images The cv2.addWeighted() function calculates the weighted sum of two images. Because of the weight factor, it provides a blending effect to the images. Add the following lines of code before destroyAllWindows() in the previous code listing to see this function in action: cv2.addWeighted(img1,0.5,img2,0.5,0) cv2.waitKey(0) In the preceding code, we passed the following five arguments to the addWeighted() function: Img1: This is the first image. Alpha: This is the weight factor for the first image (0.5 in the example). Img2: This is the second image. Beta: This is the weight factor for the second image (0.5 in the example). Gamma: This is the scalar value (0 in the example). The output image value is calculated with the following formula: This operation is performed on every individual pixel. Here is the output of the preceding code: We can create a film-style transition effect on the two images by using the same function. Check out the output of the following code that creates a smooth image transition from an image to another image: import cv2 import numpy as np import time   img1 = cv2.imread('/home/pi/book/test_set/4.2.03.tiff',1) img2 = cv2.imread('/home/pi/book/test_set/4.2.04.tiff',1)   for i in np.linspace(0,1,40): alpha=i beta=1-alpha print 'ALPHA ='+ str(alpha)+' BETA ='+str (beta) cv2.imshow('Image Transition',    cv2.addWeighted(img1,alpha,img2,beta,0)) time.sleep(0.05) if cv2.waitKey(1) == 27 :    break   cv2.destroyAllWindows() Splitting and merging image colour channels On several occasions, we may be interested in working separately with the red, green, and blue channels. For example, we might want to build a histogram for every channel of an image. Here, cv2.split() is used to split an image into three different intensity arrays for each color channel, whereas cv2.merge() is used to merge different arrays into a single multi-channel array, that is, a color image. The following example demonstrates this: import cv2 img = cv2.imread('/home/pi/book/test_set/4.2.03.tiff',1) b,g,r = cv2.split (img) cv2.imshow('Blue Channel',b) cv2.imshow('Green Channel',g) cv2.imshow('Red Channel',r) img=cv2.merge((b,g,r)) cv2.imshow('Merged Output',img) cv2.waitKey(0) cv2.destroyAllWindows() The preceding program first splits the image into three channels (blue, green, and red) and then displays each one of them. The separate channels will only hold the intensity values of the particular color and the images will essentially be displayed as grayscale intensity images. Then, the program merges all the channels back into an image and displays it. Creating a negative of an image In mathematical terms, the negative of an image is the inversion of colors. For a grayscale image, it is even simpler! The negative of a grayscale image is just the intensity inversion, which can be achieved by finding the complement of the intensity from 255. A pixel value ranges from 0 to 255, and therefore, negation involves the subtracting of the pixel value from the maximum value, that is, 255. The code for the same is as follows: import cv2 img = cv2.imread('/home/pi/book/test_set/4.2.07.tiff') grayscale = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY) negative = abs(255-grayscale) cv2.imshow('Original',img) cv2.imshow('Grayscale',grayscale) cv2.imshow('Negative',negative) cv2.waitKey(0) cv2.destroyAllWindows() Here is the output window of Greyscale: Here's the output window of Negative: The negative of a negative will be the original grayscale image. Try this on your own by taking the image negative of the negative again Logical operations on images OpenCV provides bitwise logical operation functions for images. We will have a look at the functions that provide the bitwise logical AND, OR, XOR (exclusive OR), and NOT (inversion) functionality. These functions can be better demonstrated visually with grayscale images. I am going to use barcode images in horizontal and vertical orientation for demonstration. Let's have a look at the following code: import cv2 import matplotlib.pyplot as plt   img1 = cv2.imread('/home/pi/book/test_set/Barcode_Hor.png',0) img2 = cv2.imread('/home/pi/book/test_set/Barcode_Ver.png',0) not_out=cv2.bitwise_not(img1) and_out=cv2.bitwise_and(img1,img2) or_out=cv2.bitwise_or(img1,img2) xor_out=cv2.bitwise_xor(img1,img2)   titles = ['Image 1','Image 2','Image 1 NOT','AND','OR','XOR'] images = [img1,img2,not_out,and_out,or_out,xor_out]   for i in xrange(6):    plt.subplot(2,3,i+1)    plt.imshow(images[i],cmap='gray')    plt.title(titles[i])    plt.xticks([]),plt.yticks([]) plt.show() We first read the images in grayscale mode and calculated the NOT, AND, OR, and XOR, functionalities and then with matplotlib, we displayed those in a neat way. We leveraged the plt.subplot() function to display multiple images. Here in the preceding example, we created a grid with two rows and three columns for our images and displayed each image in every part of the grid. You can modify this line and change it to plt.subplot(3,2,i+1) to create a grid with three rows and two columns. Also, we can use the technique without a loop in the following way. For each image, you have to write the following statements. I will write the code for the first image only. Go ahead and write it for the rest of the five images: plt.subplot(2,3,1) , plt.imshow(img1,cmap='gray') , plt.title('Image 1') , plt.xticks([]),plt.yticks([]) Finally, use plt.show() to display. This technique is to avoid the loop when a very small number of images, usually 2 or 3 in number, have to be displayed. The output of this is as follows: Make a note of the fact that the logical NOT operation is the negative of the image. Exercise You may want to have a look at the functionality of cv2.copyMakeBorder(). This function is used to create the borders and paddings for images, and many of you will find it useful for your projects. The exploring of this function is left as an exercise for the readers. You can check the python OpenCV API documentation at the following location: http://docs.opencv.org/modules/refman.html Summary In this article, we learned how to perform arithmetic and logical operations on images and split images by their channels. We also learned how to display multiple images in a grid by using matplotlib. Resources for Article: Further resources on this subject: Raspberry Pi and 1-Wire [article] Raspberry Pi Gaming Operating Systems [article] The Raspberry Pi and Raspbian [article]

In this article by Lentin Joseph, author of the book Learning Robotics Using Python, we will see how to assemble this robot using these parts and also the final interfacing of sensors and other electronics components of this robot to Tiva C LaunchPad. We will also try to interface the necessary robotic components and sensors of ChefBot and program it in such a way that it will receive the values from all sensors and control the information from the PC. Launchpad will send all sensor values via a serial port to the PC and also receive control information (such as reset command, speed, and so on) from the PC. After receiving sensor values from the PC, a ROS Python node will receive the serial values and convert it to ROS Topics. There are Python nodes present in the PC that subscribe to the sensor's data and produces odometry. The data from the wheel encoders and IMU values are combined to calculate the odometry of the robot and detect obstacles by subscribing to the ultrasonic sensor and laser scan also, controlling the speed of the wheel motors by using the PID node. This node converts the linear velocity command to differential wheel velocity. After running these nodes, we can run SLAM to map the area and after running SLAM, we can run the AMCL nodes for localization and autonomous navigation. In the first section of this article, Building ChefBot hardware, we will see how to assemble the ChefBot hardware using its body parts and electronics components. (For more resources related to this topic, see here.) Building ChefBot hardware The first section of the robot that needs to be configured is the base plate. The base plate consists of two motors and its wheels, caster wheels, and base plate supports. The following image shows the top and bottom view of the base plate: Base plate with motors, wheels, and caster wheels The base plate has a radius of 15cm and motors with wheels are mounted on the opposite sides of the plate by cutting a section from the base plate. A rubber caster wheel is mounted on the opposite side of the base plate to give the robot good balance and support for the robot. We can either choose ball caster wheels or rubber caster wheels. The wires of the two motors are taken to the top of the base plate through a hole in the center of the base plate. To extend the layers of the robot, we will put base plate supports to connect the next layers. Now, we can see the next layer with the middle plate and connecting tubes. There are hollow tubes, which connect the base plate and the middle plate. A support is provided on the base plate for hollow tubes. The following figure shows the middle plate and connecting tubes: Middle plate with connecting tubes The connecting tubes will connect the base plate and the middle plate. There are four hollow tubes that connect the base plate to the middle plate. One end of these tubes is hollow, which can fit in the base plate support, and the other end is inserted with a hard plastic with an option to put a screw in the hole. The middle plate has no support except four holes: Fully assembled robot body The middle plate male connector helps to connect the middle plate and the top of the base plate tubes. At the top of the middle plate tubes, we can fit the top plate, which has four supports on the back. We can insert the top plate female connector into the top plate support and this is how we will get the fully assembled body of the robot. The bottom layer of the robot can be used to put the Printed Circuit Board (PCB) and battery. In the middle layer, we can put Kinect and Intel NUC. We can put a speaker and a mic if needed. We can use the top plate to carry food. The following figure shows the PCB prototype of robot; it consists of Tiva C LaunchPad, a motor driver, level shifters, and provisions to connect two motors, ultrasonic, and IMU: ChefBot PCB prototype The board is powered with a 12 V battery placed on the base plate. The two motors can be directly connected to the M1 and M2 male connectors. The NUC PC and Kinect are placed on the middle plate. The Launchpad board and Kinect should be connected to the NUC PC via USB. The PC and Kinect are powered using the same 12 V battery itself. We can use a lead-acid or lithium-polymer battery. Here, we are using a lead-acid cell for testing purposes. We will migrate to lithium-polymer for better performance and better backup. The following figure shows the complete assembled diagram of ChefBot: Fully assembled robot body After assembling all the parts of the robot, we will start working with the robot software. ChefBot's embedded code and ROS packages are available in GitHub. We can clone the code and start working with the software. Configuring ChefBot PC and setting ChefBot ROS packages In ChefBot, we are using Intel's NUC PC to handle the robot sensor data and its processing. After procuring the NUC PC, we have to install Ubuntu 14.04.2 or the latest updates of 14.04 LTS. After the installation of Ubuntu, install complete ROS and its packages. We can configure this PC separately, and after the completion of all the settings, we can put this in to the robot. The following are the procedures to install ChefBot packages on the NUC PC. Clone ChefBot's software packages from GitHub using the following command: $ git clone https://github.com/qboticslabs/Chefbot_ROS_pkg.git We can clone the code in our laptop and copy the chefbot folder to Intel's NUC PC. The chefbot folder consists of the ROS packages of ChefBot. In the NUC PC, create a ROS catkin workspace, copy the chefbot folder and move it inside the src directory of the catkin workspace. Build and install the source code of ChefBot by simply using the following command This should be executed inside the catkin workspace we created: $ catkin_make If all dependencies are properly installed in NUC, then the ChefBot packages will build and install in this system. After setting the ChefBot packages on the NUC PC, we can switch to the embedded code for ChefBot. Now, we can connect all the sensors in Launchpad. After uploading the code in Launchpad, we can again discuss ROS packages and how to run it. Interfacing ChefBot sensors with Tiva C LaunchPad We have discussed interfacing of individual sensors that we are going to use in ChefBot. In this section, we will discuss how to integrate sensors into the Launchpad board. The Energia code to program Tiva C LaunchPad is available on the cloned files at GitHub. The connection diagram of Tiva C LaunchPad with sensors is as follows. From this figure, we get to know how the sensors are interconnected with Launchpad: Sensor interfacing diagram of ChefBot M1 and M2 are two differential drive motors that we are using in this robot. The motors we are going to use here is DC Geared motor with an encoder from Pololu. The motor terminals are connected to the VNH2SP30 motor driver from Pololu. One of the motors is connected in reverse polarity because in differential steering, one motor rotates opposite to the other. If we send the same control signal to both the motors, each motor will rotate in the opposite direction. To avoid this condition, we will connect it in opposite polarities. The motor driver is connected to Tiva C LaunchPad through a 3.3 V-5 V bidirectional level shifter. One of the level shifter we will use here is available at: https://www.sparkfun.com/products/12009. The two channels of each encoder are connected to Launchpad via a level shifter. Currently, we are using one ultrasonic distance sensor for obstacle detection. In future, we could expand this number, if required. To get a good odometry estimate, we will put IMU sensor MPU 6050 through an I2C interface. The pins are directly connected to Launchpad because MPU6050 is 3.3 V compatible. To reset Launchpad from ROS nodes, we are allocating one pin as the output and connected to reset pin of Launchpad. When a specific character is sent to Launchpad, it will set the output pin to high and reset the device. In some situations, the error from the calculation may accumulate and it can affect the navigation of the robot. We are resetting Launchpad to clear this error. To monitor the battery level, we are allocating another pin to read the battery value. This feature is not currently implemented in the Energia code. The code you downloaded from GitHub consists of embedded code. We can see the main section of the code here and there is no need to explain all the sections because we already discussed it. Writing a ROS Python driver for ChefBot After uploading the embedded code to Launchpad, the next step is to handle the serial data from Launchpad and convert it to ROS Topics for further processing. The launchpad_node.py ROS Python driver node interfaces Tiva C LaunchPad to ROS. The launchpad_node.py file is on the script folder, which is inside the chefbot_bringup package. The following is the explanation of launchpad_node.py in important code sections: #ROS Python client import rospy import sys import time import math   #This python module helps to receive values from serial port which execute in a thread from SerialDataGateway import SerialDataGateway #Importing required ROS data types for the code from std_msgs.msg import Int16,Int32, Int64, Float32, String, Header, UInt64 #Importing ROS data type for IMU from sensor_msgs.msg import Imu The launchpad_node.py file imports the preceding modules. The main modules we can see is SerialDataGateway. This is a custom module written to receive serial data from the Launchpad board in a thread. We also need some data types of ROS to handle the sensor data. The main function of the node is given in the following code snippet: if __name__ =='__main__': rospy.init_node('launchpad_ros',anonymous=True) launchpad = Launchpad_Class() try:      launchpad.Start()    rospy.spin() except rospy.ROSInterruptException:    rospy.logwarn("Error in main function")   launchpad.Reset_Launchpad() launchpad.Stop() The main class of this node is called Launchpad_Class(). This class contains all the methods to start, stop, and convert serial data to ROS Topics. In the main function, we will create an object of Launchpad_Class(). After creating the object, we will call the Start() method, which will start the serial communication between Tiva C LaunchPad and PC. If we interrupt the driver node by pressing Ctrl + C, it will reset the Launchpad and stop the serial communication between the PC and Launchpad. The following code snippet is from the constructor function of Launchpad_Class(). In the following snippet, we will retrieve the port and baud rate of the Launchpad board from ROS parameters and initialize the SerialDateGateway object using these parameters. The SerialDataGateway object calls the _HandleReceivedLine() function inside this class when any incoming serial data arrives on the serial port. This function will process each line of serial data and extract, convert, and insert it to the appropriate headers of each ROS Topic data type: #Get serial port and baud rate of Tiva C Launchpad port = rospy.get_param("~port", "/dev/ttyACM0") baudRate = int(rospy.get_param("~baudRate", 115200))   ################################################################# rospy.loginfo("Starting with serial port: " + port + ", baud rate: " + str(baudRate))   #Initializing SerialDataGateway object with serial port, baud rate and callback function to handle incoming serial data self._SerialDataGateway = SerialDataGateway(port, baudRate, self._HandleReceivedLine) rospy.loginfo("Started serial communication")     ###################################################################Subscribers and Publishers   #Publisher for left and right wheel encoder values self._Left_Encoder = rospy.Publisher('lwheel',Int64,queue_size = 10) self._Right_Encoder = rospy.Publisher('rwheel',Int64,queue_size = 10)   #Publisher for Battery level(for upgrade purpose) self._Battery_Level = rospy.Publisher('battery_level',Float32,queue_size = 10) #Publisher for Ultrasonic distance sensor self._Ultrasonic_Value = rospy.Publisher('ultrasonic_distance',Float32,queue_size = 10)   #Publisher for IMU rotation quaternion values self._qx_ = rospy.Publisher('qx',Float32,queue_size = 10) self._qy_ = rospy.Publisher('qy',Float32,queue_size = 10) self._qz_ = rospy.Publisher('qz',Float32,queue_size = 10) self._qw_ = rospy.Publisher('qw',Float32,queue_size = 10)   #Publisher for entire serial data self._SerialPublisher = rospy.Publisher('serial', String,queue_size=10) We will create the ROS publisher object for sensors such as the encoder, IMU, and ultrasonic sensor as well as for the entire serial data for debugging purpose. We will also subscribe the speed commands for the left-hand side and the right-hand side wheel of the robot. When a speed command arrives on Topic, it calls the respective callbacks to send speed commands to the robot's Launchpad: self._left_motor_speed = rospy.Subscriber('left_wheel_speed',Float32,self._Update_Left_Speed) self._right_motor_speed = rospy.Subscriber('right_wheel_speed',Float32,self._Update_Right_Speed) After setting the ChefBot driver node, we need to interface the robot to a ROS navigation stack in order to perform autonomous navigation. The basic requirement for doing autonomous navigation is that the robot driver nodes, receive velocity command from ROS navigational stack. The robot can be controlled using teleoperation. In addition to these features, the robot must be able to compute its positional or odometry data and generate the tf data for sending into navigational stack. There must be a PID controller to control the robot motor velocity. The following ROS package helps to perform these functions. The differential_drive package contains nodes to perform the preceding operation. We are reusing these nodes in our package to implement these functionalities. The following is the link for the differential_drive package in ROS: http://wiki.ros.org/differential_drive The following figure shows how these nodes communicate with each other. We can also discuss the use of other nodes too: The purpose of each node in the chefbot_bringup package is as follows: twist_to_motors.py: This node will convert the ROS Twist command or linear and angular velocity to individual motor velocity target. The target velocities are published at a rate of the ~rate Hertz and the publish timeout_ticks times velocity after the Twist message stops. The following are the Topics and parameters that will be published and subscribed by this node: Publishing Topics: lwheel_vtarget (std_msgs/Float32): This is the the target velocity of the left wheel(m/s). rwheel_vtarget (std_msgs/Float32): This is the target velocity of the right wheel(m/s). Subscribing Topics: Twist (geometry_msgs/Twist): This is the target Twist command for the robot. The linear velocity in the x direction and angular velocity theta of the Twist messages are used in this robot. Important ROS parameters: ~base_width (float, default: 0.1): This is the distance between the robot's two wheels in meters. ~rate (int, default: 50): This is the rate at which velocity target is published(Hertz). ~timeout_ticks (int, default:2): This is the number of the velocity target message published after stopping the Twist messages. pid_velocity.py: This is a simple PID controller to control the speed of each motors by taking feedback from wheel encoders. In a differential drive system, we need one PID controller for each wheel. It will read the encoder data from each wheels and control the speed of each wheels. Publishing Topics: motor_cmd (Float32): This is the final output of the PID controller that goes to the motor. We can change the range of the PID output using the out_min and out_max ROS parameter. wheel_vel (Float32): This is the current velocity of the robot wheel in m/s. Subscribing Topics: wheel (Int16): This Topic is the output of a rotary encoder. There are individual Topics for each encoder of the robot. wheel_vtarget (Float32): This is the target velocity in m/s. Important parameters: ~Kp (float ,default: 10): This parameter is the proportional gain of the PID controller. ~Ki (float, default: 10): This parameter is the integral gain of the PID controller. ~Kd (float, default: 0.001): This parameter is the derivative gain of the PID controller. ~out_min (float, default: 255): This is the minimum limit of the velocity value to motor. This parameter limits the velocity value to motor called wheel_vel Topic. ~out_max (float, default: 255): This is the maximum limit of wheel_vel Topic(Hertz). ~rate (float, default: 20): This is the rate of publishing wheel_vel Topic. ticks_meter (float, default: 20): This is the number of wheel encoder ticks per meter. This is a global parameter because it's used in other nodes too. vel_threshold (float, default: 0.001): If the robot velocity drops below this parameter, we consider the wheel as stopped. If the velocity of the wheel is less than vel_threshold, we consider it as zero. encoder_min (int, default: 32768): This is the minimum value of encoder reading. encoder_max (int, default: 32768): This is the maximum value of encoder reading. wheel_low_wrap (int, default: 0.3 * (encoder_max - encoder_min) + encoder_min): These values decide whether the odometry is in negative or positive direction. wheel_high_wrap (int, default: 0.7 * (encoder_max - encoder_min) + encoder_min): These values decide whether the odometry is in the negative or positive direction. diff_tf.py: This node computes the transformation of odometry and broadcast between the odometry frame and the robot base frame. Publishing Topics: odom (nav_msgs/odometry): This publishes the odometry (current pose and twist of the robot. tf: This provides transformation between the odometry frame and the robot base link. Subscribing Topics: lwheel (std_msgs/Int16), rwheel (std_msgs/Int16): These are the output values from the left and right encoder of the robot. chefbot_keyboard_teleop.py: This node sends the Twist command using controls from the keyboard. Publishing Topics: cmd_vel_mux/input/teleop (geometry_msgs/Twist): This publishes the twist messages using keyboard commands. After discussing nodes in the chefbot_bringup package, we will look at the functions of launch files. Understanding ChefBot ROS launch files We will discuss the functions of each launch files of the chefbot_bringup package. robot_standalone.launch: The main function of this launch file is to start nodes such as launchpad_node, pid_velocity, diff_tf, and twist_to_motor to get sensor values from the robot and to send command velocity to the robot. keyboard_teleop.launch: This launch file will start the teleoperation by using the keyboard. This launch starts the chefbot_keyboard_teleop.py node to perform the keyboard teleoperation. 3dsensor.launch : This file will launch Kinect OpenNI drivers and start publishing RGB and depth stream. It will also start the depth stream to laser scanner node, which will convert point cloud to laser scan data. gmapping_demo.launch: This launch file will start SLAM gmapping nodes to map the area surrounding the robot. amcl_demo.launch: Using AMCL, the robot can localize and predict where it stands on the map. After localizing on the map, we can command the robot to move to a position on the map, then the robot can move autonomously from its current position to the goal position. view_robot.launch: This launch file displays the robot URDF model in RViz. view_navigation.launch: This launch file displays all the sensors necessary for the navigation of the robot. Summary This article was about assembling the hardware of ChefBot and integrating the embedded and ROS code into the robot to perform autonomous navigation. We assembled individual sections of the robot and connected the prototype PCB that we designed for the robot. This consists of the Launchpad board, motor driver, left shifter, ultrasonic, and IMU. The Launchpad board was flashed with the new embedded code, which can interface all sensors in the robot and can send or receive data from the PC. After discussing the embedded code, we wrote the ROS Python driver node to interface the serial data from the Launchpad board. After interfacing the Launchpad board, we computed the odometry data and differential drive controlling using nodes from the differential_drive package that existed in the ROS repository. We interfaced the robot to ROS navigation stack. This enables to perform SLAM and AMCL for autonomous navigation. We also discussed SLAM, AMCL, created map, and executed autonomous navigation on the robot. Resources for Article: Further resources on this subject: Learning Selenium Testing Tools with Python [article] Prototyping Arduino Projects using Python [article] Python functions – Avoid repeating code [article]

In this article by Rakhitha Nimesh Ratnayake, author of the book WordPress Web Application Development - Second Edition, we will see how to implement frontend registration and how to create a login form in the frontend. (For more resources related to this topic, see here.) Implementing frontend registration Fortunately, we can make use of the existing functionalities to implement registration from the frontend. We can use a regular HTTP request or AJAX-based technique to implement this feature. In this article, I will focus on a normal process instead of using AJAX. Our first task is to create the registration form in the frontend. There are various ways to implement such forms in the frontend. Let's look at some of the possibilities as described in the following section: Shortcode implementation Page template implementation Custom template implementation Now, let's look at the implementation of each of these techniques. Shortcode implementation Shortcodes are the quickest way to add dynamic content to your pages. In this situation, we need to create a page for registration. Therefore, we need to create a shortcode that generates the registration form, as shown in the following code: add_shortcode( "register_form", "display_register_form" );function display_register_form(){$html = "HTML for registration form";return $html;} Then, you can add the shortcode inside the created page using the following code snippet to display the registration form: [register_form] Pros and cons of using shortcodes Following are the pros and cons of using shortcodes: Shortcodes are easy to implement in any part of your application Its hard to manage the template code assigned using the PHP variables There is a possibility of the shortcode getting deleted from the page by mistake Page template implementation Page templates are a widely used technique in modern WordPress themes. We can create a page template to embed the registration form. Consider the following code for a sample page template: /** Template Name : Registration*/HTML code for registration form Next, we have to copy the template inside the theme folder. Finally, we can create a page and assign the page template to display the registration form. Now, let's look at the pros and cons of this technique. Pros and cons of page templates Following are the pros and cons of page templates: A page template is more stable than shortcode. Generally, page templates are associated with the look of the website rather than providing dynamic forms. The full width page, two-column page, and left sidebar page are some common implementations of page templates. A template is managed separately from logic, without using PHP variables. The page templates depend on the theme and need to be updated on theme switching. Custom template implementation Experienced web application developers will always look to separate business logic from view templates. This will be the perfect technique for such people. In this technique, we will create our own independent templates by intercepting the WordPress default routing process. An implementation of this technique starts from the next section on routing. Building a simple router for a user module Routing is one of the important aspects in advanced application development. We need to figure out ways of building custom routes for specific functionalities. In this scenario, we will create a custom router to handle all the user-related functionalities of our application. Let's list the requirements for building a router: All the user-related functionalities should go through a custom URL, such as http://www.example.com/user Registration should be implemented at http://www.example.com/user/register Login should be implemented at http://www.example.com/user/login Activation should be implemented at http://www.example.com/user/activate Make sure to set up your permalinks structure to post name for the examples in this article. If you prefer a different permalinks structure, you will have to update the URLs and routing rules accordingly. As you can see, the user section is common for all the functionalities. The second URL segment changes dynamically based on the functionality. In MVC terms, user acts as the controller and the next URL segment (register, login, and activate) acts as the action. Now, let's see how we can implement a custom router for the given requirements. Creating the routing rules There are various ways and action hooks used to create custom rewrite rules. We will choose the init action to define our custom routes for the user section, as shown in the following code: public function manage_user_routes() {add_rewrite_rule( '^user/([^/]+)/?','index.php?control_action=$matches[1]', 'top' );} Based on the discussed requirements, all the URLs for the user section will follow the /user/custom action pattern. Therefore, we will define the regular expression for matching all the routes in the user section. Redirection is made to the index.php file with a query variable called control_action. This variable will contain the URL segment after the /user segment. The third parameter of the add_rewrite_rule function will decide whether to check this rewrite rule before the existing rules or after them. The value of top will give a higher precedence, while the value of bottom will give a lower precedence. We need to complete two other tasks to get these rewriting rules to take effect: Add query variables to the WordPress query_vars Flush the rewriting rules Adding query variables WordPress doesn't allow you to use any type of variable in the query string. It will check for query variables within the existing list and all other variables will be ignored. Whenever we want to use a new query variable, make sure to add it to the existing list. First, we need to update our constructor with the following filter to customize query variables: add_filter( 'query_vars', array( $this, 'manage_user_routes_query_vars' ) ); This filter on query_vars will allow us to customize the list of existing variables by adding or removing entries from an array. Now, consider the implementation to add a new query variable: public function manage_user_routes_query_vars( $query_vars ) {$query_vars[] = 'control_action';return $query_vars;} As this is a filter, the existing query_vars variable will be passed as an array. We will modify the array by adding a new query variable called control_action and return the list. Now, we have the ability to access this variable from the URL. Flush the rewriting rules Once rewrite rules are modified, it's a must to flush the rules in order to prevent 404 page generation. Flushing existing rules is a time consuming task, which impacts the performance of the application and hence should be avoided in repetitive actions such as init. It's recommended that you perform such tasks in plugin activation or installation as we did earlier in user roles and capabilities. So, let's implement the function for flushing rewrite rules on plugin activation: public function flush_application_rewrite_rules() {flush_rewrite_rules();} As usual, we need to update the constructor to include the following action to call the flush_application_rewrite_rules function: register_activation_hook( __FILE__, array( $this,'flush_application_rewrite_rules' ) ); Now, go to the admin panel, deactivate the plugin, and activate the plugin again. Then, go to the URL http://www.example.com/user/login and check whether it works. Unfortunately, you will still get the 404 error for the request. You might be wondering what went wrong. Let's go back and think about the process in order to understand the issue. We flushed the rules on plugin activation. So, the new rules should persist successfully. However, we will define the rules on the init action, which is only executed after the plugin is activated. Therefore, new rules will not be available at the time of flushing. Consider the updated version of the flush_application_rewrite_rules function for a quick fix to our problem: public function flush_application_rewrite_rules() {$this->manage_user_routes();flush_rewrite_rules();} We call the manage_user_routes function on plugin activation, followed by the call to flush_rewrite_rules. So, the new rules are generated before flushing is executed. Now, follow the previous process once again; you won't get a 404 page since all the rules have taken effect. You can get 404 errors due to the modification in rewriting rules and not flushing it properly. In such situations, go to the Permalinks section on the Settings page and click on the Save Changes button to flush the rewrite rules manually. Now, we are ready with our routing rules for user functionalities. It's important to know the existing routing rules of your application. Even though we can have a look at the routing rules from the database, it's difficult to decode the serialized array, as we encountered in the previous section. So, I recommend that you use the free plugin called Rewrite Rules Inspector. You can grab a copy at http://wordpress.org/plugins/rewrite-rules-inspector/. Once installed, this plugin allows you to view all the existing routing rules as well as offers a button to flush the rules, as shown in the following screen: Controlling access to your functions We have a custom router, which handles the URLs of the user section of our application. Next, we need a controller to handle the requests and generate the template for the user. This works similar to the controllers in the MVC pattern. Even though we have changed the default routing, WordPress will look for an existing template to be sent back to the user. Therefore, we need to intercept this process and create our own templates. WordPress offers an action hook called template_redirect for intercepting requests. So, let's implement our frontend controller based on template_redirect. First, we need to update the constructor with the template_redirect action, as shown in the following code: add_action( 'template_redirect', array( $this, 'front_controller' ) ); Now, let's take a look at the implementation of the front_controller function using the following code: public function front_controller() {global $wp_query;$control_action = isset ( $wp_query->query_vars['control_action'] ) ? $wp_query->query_vars['control_action'] : ''; ;switch ( $control_action ) {case 'register':do_action( 'wpwa_register_user' );break;}} We will be handling custom routes based on the value of the control_action query variable assigned in the previous section. The value of this variable can be grabbed through the global query_vars array of the $wp_query object. Then, we can use a simple switch statement to handle the controlling based on the action. The first action to consider will be to register as we are in the registration process. Once the control_action query variable is matched with registration, we will call a handler function using do_action. You might be confused why we use do_action in this scenario. So, let's consider the same implementation in a normal PHP application, where we don't have the do_action hook: switch ( $control_action ) {case 'register':$this->register_user();break;} This is the typical scenario where we call a function within the class or in an external class to implement the registration. In the previous code, we called a function within the class, but with the do_action hook instead of the usual function call. The advantages of using the do_action function WordPress action hooks define specific points in the execution process, where we can develop custom functions to modify existing behavior. In this scenario, we are calling the wpwa_register_user function within the class using do_action. Unlike websites or blogs, web applications need to be extendable with future requirements. Think of a situation where we only allow Gmail addresses for user registration. This Gmail validation is not implemented in the original code. Therefore, we need to change the existing code to implement the necessary validations. Changing a working component is considered bad practice in application development. Let's see why it's considered as a bad practice by looking at the definition of the open/closed principle on Wikipedia. "Open/closed principle states "software entities (classes, modules, functions, and so on) should be open for extension, but closed for modification"; that is, such an entity can allow its behavior to be modified without altering its source code. This is especially valuable in a production environment, where changes to the source code may necessitate code reviews, unit tests, and other such procedures to qualify it for use in a product: the code obeying the principle doesn't change when it is extended, and therefore, needs no such effort." WordPress action hooks come to our rescue in this scenario. We can define an action for registration using the add_action function, as shown in the following code: add_action( 'wpwa_register_user', array( $this, 'register_user' ) ); Now, you can implement this action multiple times using different functions. In this scenario, register_user will be our primary registration handler. For Gmail validation, we can define another function using the following code: add_action( 'wpwa_register_user', array( $this, 'validate_gmail_registration') ); Inside this function, we can make the necessary validations, as shown in the following code: public function validate_user(){// Code to validate user// remove registration function if validation failsremove_action( 'wpwa_register_user', array( $this,'register_user' ) );} Now, the validate_user function is executed before the primary function. So, we can remove the primary registration function if something goes wrong in validation. With this technique, we have the capability of adding new functionalities as well as changing existing functionalities without affecting the already written code. We have implemented a simple controller, which can be quite effective in developing web application functionalities. In the following sections, we will continue the process of implementing registration on the frontend with custom templates. Creating custom templates Themes provide a default set of templates to cater to the existing behavior of WordPress. Here, we are trying to implement a custom template system to suit web applications. So, our first option is to include the template files directly inside the theme. Personally, I don't like this option due to two possible reasons: Whenever we switch the theme, we have to move the custom template files to a new theme. So, our templates become theme dependent. In general, all existing templates are related to CMS functionality. Mixing custom templates with the existing ones becomes hard to manage. As a solution to these concerns, we will implement the custom templates inside the plugin. First, create a folder inside the current plugin folder and name it as templates to get things started. Designing the registration form We need to design a custom form for frontend registration containing the default header and footer. The whole content area will be used for the registration and the default sidebar will be omitted for this screen. Create a PHP file called register-template.php inside the templates folder with the following code: <?php get_header(); ?><div id="wpwa_custom_panel"><?phpif( isset($errors) && count( $errors ) > 0) {foreach( $errors as $error ){echo '<p class="wpwa_frm_error">'. $error .'</p>';}}?>HTML Code for Form</div><?php get_footer(); ?> We can include the default header and footer using the get_header and get_footer functions, respectively. After the header, we will include a display area for the error messages generated in registration. Then, we have the HTML form, as shown in the following code: <form id='registration-form' method='post' action='<?php echoget_site_url() . '/user/register'; ?>'><ul><li><label class='wpwa_frm_label'><?php echo__('Username','wpwa'); ?></label><input class='wpwa_frm_field' type='text'id='wpwa_user' name='wpwa_user' value='' /></li><li><label class='wpwa_frm_label'><?php echo __('Email','wpwa'); ?></label><input class='wpwa_frm_field' type='text'id='wpwa_email' name='wpwa_email' value='' /></li><li><label class='wpwa_frm_label'><?php echo __('UserType','wpwa'); ?></label><select class='wpwa_frm_field' name='wpwa_user_type'><option <?php echo __('Follower','wpwa');?></option><option <?php echo __('Developer','wpwa');?></option><option <?php echo __('Member','wpwa');?></option></select></li><li><label class='wpwa_frm_label' for=''>&nbsp;</label><input type='submit' value='<?php echo__('Register','wpwa'); ?>' /></li></ul></form> As you can see, the form action is set to a custom route called user/register to be handled through the front controller. Also, we have added an extra field called user type to choose the preferred user type on registration. You might have noticed that we used wpwa as the prefix for form element names, element IDs, as well as CSS classes. Even though it's not a must to use a prefix, it can be highly effective when working with multiple third-party plugins. A unique plugin-specific prefix avoids or limits conflicts with other plugins and themes. We will get a screen similar to the following one, once we access the /user/register link in the browser: Once the form is submitted, we have to create the user based on the application requirements. Planning the registration process In this application, we have opted to build a complex registration process in order to understand the typical requirements of web applications. So, it's better to plan it upfront before moving into the implementation. Let's build a list of requirements for registration: The user should be able to register as any of the given user roles The activation code needs to be generated and sent to the user The default notification on successful registration needs to be customized to include the activation link Users should activate their account by clicking the link So, let's begin the task of registering users by displaying the registration form as given in the following code: public function register_user() {if ( !is_user_logged_in() ) {include dirname(__FILE__) . '/templates/registertemplate.php';exit;}} Once user requests /user/register, our controller will call the register_user function using the do_action call. In the initial request, we need to check whether a user is already logged in using the is_user_logged_in function. If not, we can directly include the registration template located inside the templates folder to display the registration form. WordPress templates can be included using the get_template_part function. However, it doesn't work like a typical template library, as we cannot pass data to the template. In this technique, we are including the template directly inside the function. Therefore, we have access to the data inside this function. Handling registration form submission Once the user fills the data and clicks the submit button, we have to execute quite a few tasks in order to register a user in WordPress database. Let's figure out the main tasks for registering a user: Validating form data Registering the user details Creating and saving activation code Sending e-mail notifications with an activate link In the registration form, we specified the action as /user/register, and hence the same register_user function will be used to handle form submission. Validating user data is one of the main tasks in form submission handling. So, let's take a look at the register_user function with the updated code: public function register_user() {if ( $_POST ) {$errors = array();$user_login = ( isset ( $_POST['wpwa_user'] ) ?$_POST['wpwa_user'] : '' );$user_email = ( isset ( $_POST['wpwa_email'] ) ?$_POST['wpwa_email'] : '' );$user_type = ( isset ( $_POST['wpwa_user_type'] ) ?$_POST['wpwa_user_type'] : '' );// Validating user dataif ( empty( $user_login ) )array_push($errors, __('Please enter a username.','wpwa') );if ( empty( $user_email ) )array_push( $errors, __('Please enter e-mail.','wpwa') );if ( empty( $user_type ) )array_push( $errors, __('Please enter user type.','wpwa') );}// Including the template} The following steps are to be performed: First, we will check whether the request is made as POST. Then, we get the form data from the POST array. Finally, we will check the passed values for empty conditions and push the error messages to the $errors variable created at the beginning of this function. Now, we can move into more advanced validations inside the register_user function, as shown in the following code: $sanitized_user_login = sanitize_user( $user_login );if ( !empty($user_email) && !is_email( $user_email ) )array_push( $errors, __('Please enter valid email.','wpwa'));elseif ( email_exists( $user_email ) )array_push( $errors, __('User with this email alreadyregistered.','wpwa'));if ( empty( $sanitized_user_login ) || !validate_username($user_login ) )array_push( $errors, __('Invalid username.','wpwa') );elseif ( username_exists( $sanitized_user_login ) )array_push( $errors, __('Username already exists.','wpwa') ); The steps to perform are as follows: First, we will use the existing sanitize_user function and remove unsafe characters from the username. Then, we will make validations on the e-mail to check whether it's valid and its existence status in the system. Both the email_exists and username_exists functions checks for the existence of an e-mail and username from the database. Once all the validations are completed, the errors array will be either empty or filled with error messages. In this scenario, we choose to go with the most essential validations for the registration form. You can add more advanced validation in your implementations in order to minimize potential security threats. In case we get validation errors in the form, we can directly print the contents of the error array on top of the form as it's visible to the registration template. Here is a preview of our registration screen with generated error messages: Also, it's important to repopulate the form values once errors are generated. We are using the same function for loading the registration form and handling form submission. Therefore, we can directly access the POST variables inside the template to echo the values, as shown in the updated registration form: <form id='registration-form' method='post' action='<?php echoget_site_url() . '/user/register'; ?>'><ul><li><label class='wpwa_frm_label'><?php echo__('Username','wpwa'); ?></label><input class='wpwa_frm_field' type='text'id='wpwa_user' name='wpwa_user' value='<?php echo isset($user_login ) ? $user_login : ''; ?>' /></li><li><label class='wpwa_frm_label'><?php echo __('Email','wpwa'); ?></label><input class='wpwa_frm_field' type='text'id='wpwa_email' name='wpwa_email' value='<?php echo isset($user_email ) ? $user_email : ''; ?>' /></li><li><label class='wpwa_frm_label'><?php echo __('User"Type','wpwa'); ?></label><select class='wpwa_frm_field' name='wpwa_user_type'><option <?php echo (isset( $user_type ) &&$user_type == 'follower') ? 'selected' : ''; ?> value='follower'><?phpecho __('Follower','wpwa'); ?></option><option <?php echo (isset( $user_type ) &&$user_type == 'developer') ? 'selected' : ''; ?>value='developer'><?php echo __('Developer','wpwa'); ?></option><option <?php echo (isset( $user_type ) && $user_type =='member') ? 'selected' : ''; ?> value='member'><?phpecho __('Member','wpwa'); ?></option></select></li><li><label class='wpwa_frm_label' for=''>&nbsp;</label><input type='submit' value='<?php echo__('Register','wpwa'); ?>' /></li></ul></form> Exploring the registration success path Now, let's look at the success path, where we don't have any errors by looking at the remaining sections of the register_user function: if ( empty( $errors ) ) {$user_pass = wp_generate_password();$user_id = wp_insert_user( array('user_login' =>$sanitized_user_login,'user_email' => $user_email,'role' => $user_type,'user_pass' => $user_pass));if ( !$user_id ) {array_push( $errors, __('Registration failed.','wpwa') );} else {$activation_code = $this->random_string();update_user_meta( $user_id, 'wpwa_activation_code',$activation_code );update_user_meta( $user_id, 'wpwa_activation_status', 'inactive');wp_new_user_notification( $user_id, $user_pass, $activation_code);$success_message = __('Registration completed successfully.Please check your email for activation link.','wpwa');}if ( !is_user_logged_in() ) {include dirname(__FILE__) . '/templates/login-template.php';exit;}} We can generate the default password using the wp_generate_password function. Then, we can use the wp_insert_user function with respective parameters generated from the form to save the user in the database. The wp_insert_user function will be used to update the current user or add new users to the application. Make sure you are not logged in while executing this function; otherwise, your admin will suddenly change into another user type after using this function. If the system fails to save the user, we can create a registration fail message and assign it to the $errors variable as we did earlier. Once the registration is successful, we will generate a random string as the activation code. You can use any function here to generate a random string. Then, we update the user with activation code and set the activation status as inactive for the moment. Finally, we will use the wp_new_user_notification function to send an e-mail containing the registration details. By default, this function takes the user ID and password and sends the login details. In this scenario, we have a problem as we need to send an activation link with the e-mail. This is a pluggable function and hence we can create our own implementation of this function to override the default behavior. Since this is a built-in WordPress function, we cannot declare it inside our plugin class. So, we will implement it as a standalone function inside our main plugin file. The full source code for this function will not be included here as it is quite extensive. I'll explain the modified code from the original function and you can have a look at the source code for the complete code: $activate_link = site_url() ."/user/activate/?wpwa_activation_code=$activate_code";$message = __('Hi there,') . 'rnrn';$message .= sprintf(__('Welcome to %s! Please activate youraccount using the link:','wpwa'), get_option('blogname')) .'rnrn';$message .= sprintf(__('<a href="%s">%s</a>','wpwa'),$activate_link, $activate_link) . 'rn';$message .= sprintf(__('Username: %s','wpwa'), $user_login) .'rn';$message .= sprintf(__('Password: %s','wpwa'), $plaintext_pass) .'rnrn'; We create a custom activation link using the third parameter passed to this function. Then, we modify the existing message to include the activation link. That's about all we need to change from the original function. Finally, we set the success message to be passed into the login screen. Now, let's move back to the register_user function. Once the notification is sent, the registration process is completed and the user will be redirected to the login screen. Once the user has the e-mail in their inbox, they can use the activation link to activate the account. Automatically log in the user after registration In general, most web applications uses e-mail confirmations before allowing users to log in to the system. However, there can be certain scenarios where we need to automatically authenticate the user into the application. A social network sign in is a great example for such a scenario. When using social network logins, the system checks whether the user is already registered. If not, the application automatically registers the user and authenticates them. We can easily modify our code to implement an automatic login after registration. Consider the following code: if ( !is_user_logged_in() ) {wp_set_auth_cookie($user_id, false, is_ssl());include dirname(__FILE__) . '/templates/login-template.php';exit;} The registration code is updated to use the wp_set_auth_cookie function. Once it's used, the user authentication cookie will be created and hence the user will be considered as automatically signed in. Then, we will redirect to the login page as usual. Since the user is already logged in using the authentication cookie, they will be redirected back to the home page with access to the backend. This is an easy way of automatically authenticating users into WordPress. Activating system users Once the user clicks on the activate link, redirection will be made to the /user/activate URL of the application. So, we need to modify our controller with a new case for activation, as shown in the following code: case 'activate':do_action( 'wpwa_activate_user' ); As usual, the definition of add_action goes in the constructor, as shown in the following code: add_action( 'wpwa_activate_user', array( $this,'activate_user') ); Next, we can have a look at the actual implementation of the activate_user function: public function activate_user() {$activation_code = isset( $_GET['wpwa_activation_code'] ) ?$_GET['wpwa_activation_code'] : '';$message = '';// Get activation record for the user$user_query = new WP_User_Query(array('meta_key' => ' wpwa_activation_code','meta_value' => $activation_code));$users = $user_query->get_results();// Check and update activation statusif ( !empty($users) ) {$user_id = $users[0]->ID;update_user_meta( $user_id, ' wpwa_activation_status','active' );$message = __('Account activated successfully.','wpwa');} else {$message = __('Invalid Activation Code','wpwa');}include dirname(__FILE__) . '/templates/info-template.php';exit;} We will get the activation code from the link and query the database for finding a matching entry. If no records are found, we set the message as activation failed or else, we can update the activation status of the matching user to activate the account. Upon activation, the user will be given a message using the info-template.php template, which consists of a very basic template like the following one: <?php get_header(); ?><div id='wpwa_info_message'><?php echo $message; ?></div><?php get_footer(); ?> Once the user visits the activation page on the /user/activation URL, information will be given to the user, as illustrated in the following screen: We successfully created and activated a new user. The final task of this process is to authenticate and log the user into the system. Let's see how we can create the login functionality. Creating a login form in the frontend The frontend login can be found in many WordPress websites, including small blogs. Usually, we place the login form in the sidebar of the website. In web applications, user interfaces are complex and different, compared to normal websites. Hence, we will implement a full page login screen as we did with registration. First, we need to update our controller with another case for login, as shown in the following code: switch ( $control_action ) {// Other casescase 'login':do_action( 'wpwa_login_user' );break;} This action will be executed once the user enters /user/login in the browser URL to display the login form. The design form for login will be located in the templates directory as a separate template called login-template.php. Here is the implementation of the login form design with the necessary error messages: <?php get_header(); ?><div id=' wpwa_custom_panel'><?phpif (isset($errors) && count($errors) > 0) {foreach ($errors as $error) {echo '<p class="wpwa_frm_error">' .$error. '</p>';}}if( isset( $success_message ) && $success_message != ""){echo '<p class="wpwa_frm_success">' .$success_message.'</p>';}?><form method='post' action='<?php echo site_url();?>/user/login' id='wpwa_login_form' name='wpwa_login_form'><ul><li><label class='wpwa_frm_label' for='username'><?phpecho __('Username','wpwa'); ?></label><input class='wpwa_frm_field' type='text'name='wpwa_username' value='<?php echo isset( $username ) ?$username : ''; ?>' /></li><li><label class='wpwa_frm_label' for='password'><?phpecho __('Password','wpwa'); ?></label><input class='wpwa_frm_field' type='password'name='wpwa_password' value="" /></li><li><label class='wpwa_frm_label' >&nbsp;</label><input type='submit' name='submit' value='<?php echo__('Login','wpwa'); ?>' /></li></ul></form></div><?php get_footer(); ?> Similar to the registration template, we have a header, error messages, the HTML form, and the footer in this template. We have to point the action of this form to /user/login. The remaining code is self-explanatory and hence I am not going to make detailed explanations. You can take a look at the preview of our login screen in the following screenshot: Next, we need to implement the form submission handler for the login functionality. Before this, we need to update our plugin constructor with the following code to define another custom action for login: add_action( 'wpwa_login_user', array( $this, 'login_user' ) ); Once the user requests /user/login from the browser, the controller will execute the do_action( 'wpwa_login_user' ) function to load the login form in the frontend. Displaying the login form We will use the same function to handle both template inclusion and form submission for login, as we did earlier with registration. So, let's look at the initial code of the login_user function for including the template: public function login_user() {if ( !is_user_logged_in() ) {include dirname(__FILE__) . '/templates/login-template.php';} else {wp_redirect(home_url());}exit;} First, we need to check whether the user has already logged in to the system. Based on the result, we will redirect the user to the login template or home page for the moment. Once the whole system is implemented, we will be redirecting the logged in users to their own admin area. Now, we can take a look at the implementation of the login to finalize our process. Let's take a look at the form submission handling part of the login_user function: if ( $_POST ) {$errors = array();$username = isset ( $_POST['wpwa_username'] ) ?$_POST['wpwa_username'] : '';$password = isset ( $_POST['wpwa_password'] ) ?$_POST['wpwa_password'] : '';if ( empty( $username ) )array_push( $errors, __('Please enter a username.','wpwa') );if ( empty( $password ) )array_push( $errors, __('Please enter password.','wpwa') );if(count($errors) > 0){include dirname(__FILE__) . '/templates/login-template.php';exit;}$credentials = array();$credentials['user_login'] = $username;$credentials['user_login'] = sanitize_user($credentials['user_login'] );$credentials['user_password'] = $password;$credentials['remember'] = false;// Rest of the code} As usual, we need to validate the post data and generate the necessary errors to be shown in the frontend. Once validations are successfully completed, we assign all the form data to an array after sanitizing the values. The username and password are contained in the credentials array with the user_login and user_password keys. The remember key defines whether to remember the password or not. Since we don't have a remember checkbox in our form, it will be set to false. Next, we need to execute the WordPress login function in order to log the user into the system, as shown in the following code: $user = wp_signon( $credentials, false );if ( is_wp_error( $user ) )array_push( $errors, $user->get_error_message() );elsewp_redirect( home_url() ); WordPress handles user authentication through the wp_signon function. We have to pass all the credentials generated in the previous code with an additional second parameter of true or false to define whether to use a secure cookie. We can set it to false for this example. The wp_signon function will return an object of the WP_User or the WP_Error class based on the result. Internally, this function sets an authentication cookie. Users will not be logged in if it is not set. If you are using any other process for authenticating users, you have to set this authentication cookie manually. Once a user is successfully authenticated, a redirection will be made to the home page of the site. Now, we should have the ability to authenticate users from the login form in the frontend. Checking whether we implemented the process properly Take a moment to think carefully about our requirements and try to figure out what we have missed. Actually, we didn't check the activation status on log in. Therefore, any user will be able to log in to the system without activating their account. Now, let's fix this issue by intercepting the authentication process with another built-in action called authenticate, as shown in the following code: public function authenticate_user( $user, $username, $password ) {if(! empty($username) && !is_wp_error($user)){$user = get_user_by('login', $username );if (!in_array( 'administrator', (array) $user->roles ) ) {$active_status = '';$active_status = get_user_meta( $user->data->ID, 'wpwa_activation_status', true );if ( 'inactive' == $active_status ) {$user = new WP_Error( 'denied', __('<strong>ERROR</strong>:Please activate your account.','wpwa') );}}}return $user;} This function will be called in the authentication action by passing the user, username, and password variables as default parameters. All the user types of our application need to be activated, except for the administrator accounts. Therefore, we check the roles of the authenticated user to figure out whether they are admin. Then, we can check the activation status of other user types before authenticating. If an authenticated user is in inactive status, we can return the WP_Error object and prevent authentication from being successful. Last but not least, we have to include the authenticate action in the controller, to make it work as shown in the following code: add_filter( 'authenticate', array( $this, 'authenticate_user' ), 30, 3 ); This filter is also executed when the user logs out of the application. Therefore, we need to consider the following validation to prevent any errors in the logout process: if(! empty($username) && !is_wp_error($user)) Now, we have a simple and useful user registration and login system, ready to be implemented in the frontend of web applications. Make sure to check login- and registration-related plugins from the official repository to gain knowledge of complex requirements in real-world scenarios. Time to practice In this article, we implemented a simple registration and login functionality from the frontend. Before we have a complete user creation and authentication system, there are plenty of other tasks to be completed. So, I would recommend you to try out the following tasks in order to be comfortable with implementing such functionalities for web applications: Create a frontend functionality for the lost password Block the default WordPress login page and redirect it to our custom page Include extra fields in the registration form Make sure to try out these exercises and validate your answers against the implementations provided on the website for this book. Summary In this article, we looked at how we can customize the built-in registration and login process in the frontend to cater to advanced requirements in web application development. By now, you should be capable of creating custom routers for common modules, implement custom controllers with custom template systems, and customize the existing user registration and authentication process. Resources for Article: Further resources on this subject: Web Application Testing [Article] Creating Blog Content in WordPress [Article] WordPress 3: Designing your Blog [Article]

In this article by Mert Çalışkan and Oleg Varaksin, author of PrimeFaces Cookbook - Second Edition, we will cover the following recipes: Setting up and configuring the PrimeFaces library AJAX basics with process and update PrimeFaces selectors Internationalization (i18n) and Localization (L10n) This article will provide details on the setup and configuration of PrimeFaces, along with the basics of the PrimeFaces AJAX mechanism. The goal of this article is to provide a sneak preview of some of the features of PrimeFaces, such as the AJAX processing mechanism and Internationalization, and Localization. (For more resources related to this topic, see here.) Setting up and configuring the PrimeFaces library PrimeFaces is a lightweight JSF component library with one JAR file, which needs no configuration and does not contain any required external dependencies. To start with the development of the library, all we need is the artifact for the library. Getting ready You can download the PrimeFaces library from http://primefaces.org/downloads.html, and you need to add the primefaces-{version}.jar file to your classpath. After that, all you need to do is import the namespace of the library that is necessary to add the PrimeFaces components to your pages to get started. If you are using Maven (for more information on installing Maven, please visit http://maven.apache.org/guides/getting-started/maven-in-five-minutes.html), you can retrieve the PrimeFaces library by defining the Maven repository in your Project Object Model XML file, pom.xml, as follows: <repository> <id>prime-repo</id> <name>PrimeFaces Maven Repository</name> <url>http://repository.primefaces.org</url> </repository> Add the dependency configuration as follows: <dependency> <groupId>org.primefaces</groupId> <artifactId>primefaces</artifactId> <version>5.2</version> </dependency> At the time of writing this article, the latest and most stable version of PrimeFaces was 5.2. To check whether this is the latest available version or not, please visit http://primefaces.org/downloads.html. The code in this article will work properly with PrimeFaces 5.2. In prior versions or the future versions, some methods, attributes, or components' behaviors may change. How to do it… In order to use PrimeFaces components, first we need to add the namespace declaration to our pages. The namespace for PrimeFaces components is as follows: That is all there is to it. Note that the p prefix is just a symbolic link, and any other character can be used to define the PrimeFaces components. Now you can create your first XHTML page with a PrimeFaces component, as shown in the following code snippet: <html > <f:view contentType="text/html"> <h:head /> <h:body> <h:form> <p:spinner /> </h:form> </h:body> </f:view> </html> This will render a spinner component with an empty value, as shown in the following screenshot: A link to the working example for the given page is given at the end of this recipe. How it works… When the page is requested, the p:spinner component is rendered with the SpinnerRenderer class implemented by the PrimeFaces library. Since the spinner component is an input component, the request-processing life cycle will get executed when the user inputs data and performs a post back on the page. For the first page, we also needed to provide the contentType parameter for f:view since WebKit-based browsers, such as Google Chrome and Safari, request for the content type application/xhtml+xml by default. This would overcome unexpected layout and styling issues that might occur. There's more… PrimeFaces only requires a Java 5+ runtime and a JSF 2.x implementation as mandatory dependencies. There are some optional libraries for certain features. All of these are listed in this table: Dependency Version Type Description JSF runtime 2.0, 2.1, or 2.2 Required Apache MyFaces or Oracle Mojarra itext 2.1.7 Optional DataExporter (PDF) apache-poi 3.7 Optional DataExporter (Excel) rome 1.0 Optional FeedReader commons-fileupload 1.3 Optional FileUpload commons-io 2.2 Optional FileUpload atmosphere 2.2.2 Optional PrimeFaces Push barcode4j-light 2.1 Optional Barcode Generation qrgen 1.4 Optional QR code support for barcode hazelcast 2.6.5+ Optional Integration of the <p:cache> component with hazelcast ehcache 2.7.4+ Optional Integration of the <p:cache> component with ehcache Please ensure that you have only one JAR file of PrimeFaces or a specific PrimeFaces theme in your classpath in order to avoid any issues regarding resource rendering. Currently, PrimeFaces fully supports nonlegacy web browsers with Internet Explorer 10, Safari, Firefox, Chrome, and Opera. The PrimeFaces Cookbook Showcase application This recipe is available in the demo web application on GitHub (https://github.com/ova2/primefaces-cookbook/tree/second-edition). Clone the project if you have not done it yet, explore the project structure, and build and deploy the WAR file on application servers compatible with Servlet 3.x, such as JBoss WildFly and Apache TomEE. The showcase for the recipe is available under http://localhost:8080/pf-cookbook/views/chapter1/yourFirstPage.jsf. AJAX basics with process and update PrimeFaces provides Partial Page Rendering (PPR) and the view-processing feature based on standard JSF 2 APIs to enable choosing what to process in the JSF life cycle and what to render in the end with AJAX. PrimeFaces AJAX Framework is based on standard server-side APIs of JSF 2. On the client side, rather than using the client-side API implementations of JSF, such as Mojarra or MyFaces, PrimeFaces scripts are based on the jQuery JavaScript library, which is well tested and widely adopted. How to do it... We can create a simple page with a command button to update a string property with the current time in milliseconds that is created on the server side and output text to show the value of that string property, as follows: <p:commandButton update="display" action="#{basicPPRBean.updateValue}" value="Update" /> <h:outputText id="display" value="#{basicPPRBean.value}"/> If we want to update multiple components with the same trigger mechanism, we can provide the ID's of the components to the update attribute by providing them with a space, comma, or both, as follows: <p:commandButton update="display1,display2" /> <p:commandButton update="display1 display2" /> <p:commandButton update="display1,display2 display3" /> In addition, there are reserved keywords that are used for a partial update. We can also make use of these keywords along with the ID's of the components, as described in the following table. Some of them come with the JSF standard, and PrimeFaces extends this list with custom keywords. Here's the table we talked about: Keyword JSF/PrimeFaces Description @this JSF The component that triggers the PPR is updated @form JSF The encapsulating form of the PPR trigger is updated @none JSF PPR does not change the DOM with an AJAX response @all JSF The whole document is updated as in non-AJAX requests @parent PrimeFaces The parent of the PPR trigger is updated @composite PrimeFaces This is the closest composite component ancestor @namingcontainer PrimeFaces This is the closest naming container ancestor of the current component @next PrimeFaces This is the next sibling @previous PrimeFaces This is the previous sibling @child(n) PrimeFaces This is the nth child @widgetVar(name) PrimeFaces This is a component stated with a given widget variable name The keywords are a server-side part of the PrimeFaces Search Expression Framework (SEF), which provides both server-side and client-side extensions to make it easier to reference components. We can also update a component that resides in a different naming container from the component that triggers the update. In order to achieve this, we need to specify the absolute component identifier of the component that needs to be updated. An example of this could be the following: <h:form id="form1"> <p:commandButton update=":form2:display" action="#{basicPPRBean.updateValue}" value="Update"/> </h:form> <h:form id="form2"> <h:outputText id="display" value="#{basicPPRBean.value}"/> </h:form> @Named @ViewScoped public class BasicPPRBean implements Serializable { private String value; public String updateValue() { value = String.valueOf(System.currentTimeMillis()); return null; } // getter / setter } PrimeFaces also provides partial processing, which executes the JSF life cycle phases—apply request values, process validations, update model, and invoke application—for determined components with the process attribute. This provides the ability to do group validation on the JSF pages easily. Mostly group validation needs arise in situations where different values need to be validated in the same form, depending on an action that gets executed. By grouping components for validation, errors that would arise from other components when the page has been submitted can be overcome easily. Components such as commandButton, commandLink, autoComplete, fileUpload, and many others provide this attribute to process partially instead of processing the whole view. Partial processing could become very handy in cases where a drop-down list needs to be populated upon a selection on another dropdown and where there is an input field on the page with the required attribute set to true. This approach also makes immediate subforms and regions obsolete. It will also prevent submission of the whole page; thus, this will result in lightweight requests. Without partially processing the view for the dropdowns, a selection on one of the dropdowns will result in a validation error on the required field. A working example for this is shown in the following code snippet: <h:outputText value="Country: " /> <h:selectOneMenu id="countries" value="#{partialProcessing Bean.country}"> <f:selectItems value="#{partialProcessingBean.countries}" /> <p:ajax listener= "#{partialProcessingBean.handleCountryChange}" event="change" update="cities" process="@this"/> </h:selectOneMenu> <h:outputText value="City: " /> <h:selectOneMenu id="cities" value="#{partialProcessingBean.city}"> <f:selectItems value="#{partialProcessingBean.cities}" /> </h:selectOneMenu> <h:outputText value="Email: " /> <h:inputText value="#{partialProcessingBean.email}" required="true" /> With this partial processing mechanism, when a user changes the country, the cities of that country will be populated in the dropdown regardless of whether any input exists for the email field or not. How it works… As illustrated in the partial processing example to update a component in a different naming container, <p:commandButton> is updating the <h:outputText> component that has the display ID and the :form2:display absolute client ID, which is the search expression for the findComponent method. An absolute client ID starts with the separator character of the naming container, which is : by default. The <h:form>, <h:dataTable>, and composite JSF components, along with <p:tabView>, <p:accordionPanel>, <p:dataTable>, <p:dataGrid>, <p:dataList>, <p:carousel>, <p:galleria>, <p:ring>, <p:sheet>, and <p:subTable> are the components that implement the NamingContainer interface. The findComponent method, which is described at http://docs.oracle.com/javaee/7/api/javax/faces/component/UIComponent.html, is used by both JSF core implementation and PrimeFaces. There's more… JSF uses : (colon) as the separator for the NamingContainer interface. The client IDs that will be rendered in the source page will be of the kind id1:id2:id3. If needed, the configuration of the separator can be changed for the web application to something other than the colon with a context parameter in the web.xml file of the web application, as follows: <context-param> <param-name>javax.faces.SEPARATOR_CHAR</param-name> <param-value>_</param-value> </context-param> It's also possible to escape the : character, if needed, in the CSS files with the character, as :. The problem that might occur with the colon is that it's a reserved keyword for the CSS and JavaScript frameworks, like jQuery, so it might need to be escaped. The PrimeFaces Cookbook Showcase application This recipe is available in the demo web application on GitHub (https://github.com/ova2/primefaces-cookbook/tree/second-edition). Clone the project if you have not done it yet, explore the project structure, and build and deploy the WAR file on application servers compatible with Servlet 3.x, such as JBoss WildFly and Apache TomEE. For the demos of this recipe, refer to the following: Basic Partial Page Rendering is available at http://localhost:8080/pf-cookbook/views/chapter1/basicPPR.jsf Updating Component in a Different Naming Container is available at http://localhost:8080/pf-cookbook/views/chapter1/componentInDifferentNamingContainer.jsf An example of Partial Processing is available at http://localhost:8080/pf-cookbook/views/chapter1/partialProcessing.jsf PrimeFaces selectors PrimeFaces integrates the jQuery Selector API (http://api.jquery.com/category/selectors) with the JSF component-referencing model. Partial processing and updating of the JSF components can be done using the jQuery Selector API instead of a regular server-side approach with findComponent(). This feature is called the PrimeFaces Selector (PFS) API. PFS provides an alternative, flexible approach to reference components to be processed or updated partially. PFS is a client-side part of the PrimeFaces SEF, which provides both server-side and client-side extensions to make it easier to reference components. In comparison with regular referencing, there is less CPU server load because the JSF component tree is not traversed on the server side in order to find client IDs. PFS is implemented on the client side by looking at the DOM tree. Another advantage is avoiding container limitations, and thus the cannot find component exception—since the component we were looking for was in a different naming container. The essential advantage of this feature, however, is speed. If we reference a component by an ID, jQuery uses document.getElementById(), a native browser call behind the scene. This is a very fast call, much faster than that on the server side with findComponent(). The second use case, where selectors are faster, is when we have a lot of components with the rendered attributes set to true or false. The JSF component tree is very big in this case, and the findComponent() call is time consuming. On the client side, only the visible part of the component tree is rendered as markup. The DOM is smaller than the component tree and its selectors work faster. In this recipe, we will learn PFS in detail. PFS is recognized when we use @(...) in the process or update attribute of AJAX-ified components. We will use this syntax in four command buttons to reference the parts of the page we are interested in. How to do it… The following code snippet contains two p:panel tags with the input, select, and checkbox components respectively. The first p:commandButton component processes/updates all components in the form(s). The second one processes / updates all panels. The third one processes input, but not select components, and updates all panels. The last button only processes the checkbox components in the second panel and updates the entire panel. <p:messages id="messages" autoUpdate="true"/> <p:panel id="panel1" header="First panel"> <h:panelGrid columns="2"> <p:outputLabel for="name" value="Name"/> <p:inputText id="name" required="true"/> <p:outputLabel for="food" value="Favorite food"/> <h:selectOneMenu id="food" required="true"> ... </h:selectOneMenu> <p:outputLabel for="married" value="Married?"/> <p:selectBooleanCheckbox id="married" required="true" label="Married?"> <f:validator validatorId="org.primefaces.cookbook. validator.RequiredCheckboxValidator"/> </p:selectBooleanCheckbox> </h:panelGrid> </p:panel> <p:panel id="panel2" header="Second panel"> <h:panelGrid columns="2"> <p:outputLabel for="address" value="Address"/> <p:inputText id="address" required="true"/> <p:outputLabel for="pet" value="Favorite pet"/> <h:selectOneMenu id="pet" required="true"> ... </h:selectOneMenu> <p:outputLabel for="gender" value="Male?"/> <p:selectBooleanCheckbox id="gender" required="true" label="Male?"> <f:validator validatorId="org.primefaces.cookbook. validator.RequiredCheckboxValidator"/> </p:selectBooleanCheckbox> </h:panelGrid> </p:panel> <h:panelGrid columns="5" style="margin-top:20px;"> <p:commandButton process="@(form)" update="@(form)" value="Process and update all in form"/> <p:commandButton process="@(.ui-panel)" update="@(.ui-panel)" value="Process and update all panels"/> <p:commandButton process="@(.ui-panel :input:not(select))" update="@(.ui-panel)" value="Process inputs except selects in all panels"/> <p:commandButton process="@(#panel2 :checkbox)" update="@(#panel2)" value="Process checkboxes in second panel"/> </h:panelGrid> In terms of jQuery selectors, regular input field, selection, and checkbox controls are all inputs. They can be selected by the :input selector. The following screenshot shows what happens when the third button is pushed. The p:inputText and p:selectBooleanCheckbox components are marked as invalid. The h:selectOneMenu component is not marked as invalid although no value was selected by the user. How it works… The first selector from the @(form) first button selects all forms on the page. The second selector, @(.ui-panel), selects all panels on the page as every main container of PrimeFaces' p:panel component has this style class. Component style classes are usually documented in the Skinning section in PrimeFaces User's Guide (http://www.primefaces.org/documentation.html). The third selector, @(.ui-panel :input:not(select)), only selects p:inputText and p:selectBooleanCheckbox within p:panel. This is why h:selectOneMenu was not marked as invalid in the preceding screenshot. The validation of this component was skipped because it renders itself as an HTML select element. The last selector variant, @(#panel2 :checkbox), intends to select p:selectBooleanCheckbox in the second panel only. In general, it is recommended that you use Firebug (https://getfirebug.com) or a similar browser add-on to explore the generated HTML structure when using jQuery selectors. A common use case is skipping validation for the hidden fields. Developers often hide some form components dynamically with JavaScript. Hidden components get validated anyway, and the form validation can fail if the fields are required or have other validation constraints. The first solution would be to disable the components (in addition to hiding them). The values of disabled fields are not sent to the server. The second solution would be to use jQuery's :visible selector in the process attribute of a command component that submits the form. There's more… PFS can be combined with regular component referencing as well, for example, update="compId1 :form:compId2 @(.ui-tabs :input)". The PrimeFaces Cookbook Showcase application This recipe is available in the demo web application on GitHub (https://github.com/ova2/primefaces-cookbook/tree/second-edition). Clone the project if you have not done it yet, explore the project structure, and build and deploy the WAR file on application servers compatible with Servlet 3.x, such as JBoss WildFly and Apache TomEE. The showcase for the recipe is available at http://localhost:8080/pf-cookbook/views/chapter1/pfs.jsf. Internationalization (i18n) and Localization (L10n) Internationalization (i18n) and Localization (L10n) are two important features that should be provided in the web application's world to make it accessible globally. With Internationalization, we are emphasizing that the web application should support multiple languages, and with Localization, we are stating that the text, date, or other fields should be presented in a form specific to a region. PrimeFaces only provides English translations. Translations for other languages should be provided explicitly. In the following sections, you will find details on how to achieve this. Getting ready For internationalization, first we need to specify the resource bundle definition under the application tag in faces-config.xml, as follows: <application> <locale-config> <default-locale>en</default-locale> <supported-locale>tr_TR</supported-locale> </locale-config> <resource-bundle> <base-name>messages</base-name> <var>msg</var> </resource-bundle> </application> A resource bundle is a text file with the .properties suffix that would contain locale-specific messages. So, the preceding definition states that the resource bundle messages_{localekey}.properties file will reside under classpath, and the default value of localekey is en, which stands for English, and the supported locale is tr_TR, which stands for Turkish. For projects structured by Maven, the messages_{localekey}.properties file can be created under the src/main/resources project path. The following image was made in the IntelliJ IDEA: How to do it… To showcase Internationalization, we will broadcast an information message via the FacesMessage mechanism that will be displayed in PrimeFaces' growl component. We need two components—growl itself and a command button—to broadcast the message: <p:growl id="growl" /> <p:commandButton action="#{localizationBean.addMessage}" value="Display Message" update="growl" /> The addMessage method of localizationBean is as follows: public String addMessage() { addInfoMessage("broadcast.message"); return null; } The preceding code uses the addInfoMessage method, which is defined in the static MessageUtil class as follows: public static void addInfoMessage(String str) { FacesContext context = FacesContext.getCurrentInstance(); ResourceBundle bundle = context.getApplication().getResourceBundle(context, "msg"); String message = bundle.getString(str); FacesContext.getCurrentInstance().addMessage(null, new FacesMessage(FacesMessage.SEVERITY_INFO, message, "")); } Localization of components, such as calendar and schedule, can be achieved by providing the locale attribute. By default, locale information is retrieved from the view's locale, and it can be overridden by a string locale key or with a java.util.Locale instance. Components such as calendar and schedule use a shared PrimeFaces.locales property to display labels. As stated before, PrimeFaces only provides English translations, so in order to localize the calendar, we need to put the corresponding locales into a JavaScript file and include the scripting file to the page. The content for the German locale of the Primefaces.locales property for calendar would be as shown in the following code snippet. For the sake of the recipe, only the German locale definition is given and the Turkish locale definition is omitted; you can find it in the showcase application Here's the code snippet we talked about: PrimeFaces.locales['de'] = { closeText: 'Schließen', prevText: 'Zurück', nextText: 'Weiter', monthNames: ['Januar', 'Februar', 'März', 'April', 'Mai', 'Juni', 'Juli', 'August', 'September', 'Oktober', 'November', 'Dezember'], monthNamesShort: ['Jan', 'Feb', 'Mär', 'Apr', 'Mai', 'Jun', 'Jul', 'Aug', 'Sep', 'Okt', 'Nov', 'Dez'], dayNames: ['Sonntag', 'Montag', 'Dienstag', 'Mittwoch', 'Donnerstag', 'Freitag', 'Samstag'], dayNamesShort: ['Son', 'Mon', 'Die', 'Mit', 'Don', 'Fre', 'Sam'], dayNamesMin: ['S', 'M', 'D', 'M ', 'D', 'F ', 'S'], weekHeader: 'Woche', FirstDay: 1, isRTL: false, showMonthAfterYear: false, yearSuffix: '', timeOnlyTitle: 'Nur Zeit', timeText: 'Zeit', hourText: 'Stunde', minuteText: 'Minute', secondText: 'Sekunde', currentText: 'Aktuelles Datum', ampm: false, month: 'Monat', week: 'Woche', day: 'Tag', allDayText: 'Ganzer Tag' }; The definition of the calendar components both with and without the locale attribute would be as follows: <p:calendar showButtonPanel="true" navigator="true" mode="inline" id="enCal"/> <p:calendar locale="tr" showButtonPanel="true" navigator="true" mode="inline" id="trCal"/> <p:calendar locale="de" showButtonPanel="true" navigator="true" mode="inline" id="deCal"/> They will be rendered as follows: How it works… For Internationalization of the PrimeFaces message, the addInfoMessage method retrieves the message bundle via the defined msg variable. It then gets the string from the bundle with the given key by invoking the bundle.getString(str) method. Finally, the message is added by creating a new PrimeFaces message with the FacesMessage.SEVERITY_INFO severity level. There's more… For some components, localization could be accomplished by providing labels to the components via attributes, such as with p:selectBooleanButton: <p:selectBooleanButton value="#{localizationBean.selectedValue}" onLabel="#{msg['booleanButton.onLabel']}" offLabel="#{msg['booleanButton.offLabel']}" /> The msg variable is the resource bundle variable that is defined in the resource bundle definition in the PrimeFaces configuration file. The English version of the bundle key definitions in the messages_en.properties file that resides under the classpath would be as follows: booleanButton.onLabel=Yes booleanButton.offLabel=No The PrimeFaces Cookbook Showcase application This recipe is available in the demo web application on GitHub (https://github.com/ova2/primefaces-cookbook/tree/second-edition). Clone the project if you have not done it yet, explore the project structure, and build and deploy the WAR file on application servers compatible with Servlet 3.x, such as JBoss WildFly and Apache TomEE. For the demos of this recipe, refer to the following: Internationalization is available at http://localhost:8080/pf-cookbook/views/chapter1/internationalization.jsf Localization of the calendar component is available at http://localhost:8080/pf-cookbook/views/chapter1/localization.jsf Localization with resources is available at http://localhost:8080/pf-cookbook/views/chapter1/localizationWithResources.jsf For already translated locales of the calendar, see http://code.google.com/p/primefaces/wiki/PrimeFacesLocales. Summary In this article, we learned about setting up and configuring the PrimeFaces library, AJAX basics with process and update, PrimeFaces selectors, and Internationalization (i18n) and Localization (L10n). Resources for Article: Further resources on this subject: Components of PrimeFaces Extensions [Article] JSF2 composite component with PrimeFaces [Article] Getting Started with PrimeFaces [Article]

This article written by Bill Pretty, Glenn Vander Veer, authors of the book Building Networks and Servers Using BeagleBone will serve as an installation guide for the software that will be used to monitor the traffic on your local network. These utilities can help determine which devices on your network are hogging the bandwidth, which slows down the network for other devices on your network. Here are the topics that we are going to cover: Installing traceroute and My Trace Route (MTR or Matt's Traceroute): These utilities will give you a real-time view of the connection between one node and another Installing Nmap: This utility is a network scanner that can list all the hosts on your network and all the services available on those hosts Installing iptraf-ng: This utility gathers various network traffic information and statistics (For more resources related to this topic, see here.) Installing Traceroute Traceroute is a tool that can show the path from one node on a network to another. This can help determine the ideal placement of a router to maximize wireless bandwidth in order to stream music and videos from the BeagleBone server to remote devices. Traceroute can be installed with the following command: apt-get install traceroute   Once Traceroute is installed, it can be run to find the path from the BeagleBone to any server anywhere in the world. For example, here's the route from my BeagelBone to the Canadian Google servers: Now, it is time to decipher all the information that is presented. This first command line tells traceroute the parameters that it must use: traceroute to google.ca (74.125.225.23), 30 hops max, 60 byte packets This gives the hostname, the IP address returned by the DNS server, the maximum number of hops to be taken, and the size of the data packet to be sent. The maximum number of hops can be changed with the –m flag and can be up to 255. In the context of this book, this will not have to be changed. After the first line, the next few lines show the trip from the BeagleBone, through the intermediate hosts (or hops), to the Google.ca server. Each line follows the following format: hop_number host_name (host IP_address) packet_round_trip_times From the command that was run previously (specifically hop number 4): 2 10.149.206.1 (10.149.206.1) 15.335 ms 17.319 ms 17.232 ms Here's a breakdown of the output: The hop number 2: This is a count of the number of hosts between this host and the originating host. The higher the number, the greater is the number of computers that the traffic has to go through to reach its destination. 10.149.206.1: This denotes the hostname. This is the result of a reverse DNS lookup on the IP address. If no information is returned from the DNS query (as in this case), the IP address of the host is given instead. (10.149.206.1): This is the actual host IP address. Various numbers: This is the round-trip time for a packet to go from the BeagleBone to the server and back again. These numbers will vary depending on network traffic, and lower is better. Sometimes, the traceroute will return some asterisks (*). This indicates that the packet has not been acknowledged by the host. If there are consecutive asterisks and the final destination is not reached, then there may be a routing problem. In a local network trace, it most likely is a firewall that is blocking the data packet. Installing My Traceroute My Traceroute (MTR) is an extension of traceroute, which probes the routers on the path from the packet source and destination, and keeps track of the response times of the hops. It does this repeatedly so that the response times can be averaged. Now, install mtr with the following command: sudo apt-get install mtr After it is run, mtr will provide quite a bit more information to look at, which would look like the following: While the output may look similar, the big advantage over traceroute is that the output is constantly updated. This allows you to accumulate trends and averages and also see how network performance varies over time. When using traceroute, there is a possibility that the packets that were sent to each hop happened to make the trip without incident, even in a situation where the route is suffering from intermittent packet loss. The mtr utility allows you to monitor this by gathering data over a wider range of time. Here's an mtr trace from my Beaglebone to my Android smartphone: Here's another trace, after I changed the orientation of the antennae of my router: As you can see, the original orientation was almost 100 milliseconds faster for ping traffic. Installing Nmap Nmap is designed to allow the scanning of networks in order to determine which hosts are up and what services are they offering. Nmap supports a large number of scanning options, which are overkill for what will be done in this book. Nmap is installed with the following command: sudo apt-get install nmap Answer Yes to install nmap and its dependent packages. Using Nmap After it is installed, run the following command to see all the hosts that are currently on the network: nmap –T4 –F <your_local_ip_range> The option -T4 sets the timing template to be used, and the -F option is for fast scanning. There are other options that can be used and found via the nmap manpage. Here, your_local_ip_range is within the range of addresses assigned by your router. Here's a node scan of my local network. If you have a lot of devices on your local network, this command may take a long time to complete. Now, I know that I have more nodes on my network, but they don't show up. This is because the command we ran didn't tell nmap to explicitly query each IP address to see whether the host responds but to query common ports that may be open to traffic. Instead, only use the -Pn option in the command to tell nmap to scan all the ports for every address in the range. This will scan more ports on each address to determine whether the host is active or not. Here, we can see that there are definitely more hosts registered in the router device table. This scan will attempt to scan a host IP address even if the device is powered off. Resetting the router and running the same scan will scan the same address range, but it will not return any device names for devices that are not powered at the time of the scan. You will notice that after scanning, nmap reports that some IP addresses' ports are closed and some are filtered. Closed ports are usually maintained on the addresses of devices that are locked down by their firewall. Filtered ports are on the addresses that will be handled by the router because there actually isn't a node assigned to these addresses. Here's a part of the output from an nmap scan of my Windows machine: Here's a part of the output of a scan of the BeagleBone: Installing iptraf-ng Iptraf-ng is a utility that monitors traffic on any of the interfaces or IP addresses on your network via custom filters. Because iptraf-ng is based on the ncurses libraries, we will have to install them first before downloading and compiling the actual iptraf-ng package. To install ncurses, run the following command: sudo apt-get install libncurses5-dev Here's how you will install ncurses and its dependent packages: Once ncurses is installed, download and extract the iptraf-ng tarball so that it can be built. At the time of writing this book, iptrf-ng's version 1.1.4 was available. This will change over time, and a quick search on Google will give you the latest and greatest version to download. You can download this version with the following command: wget https://fedorahosted.org/releases/i/p/iptraf-ng/iptraf-ng- <current_version_number>.tar.gz The following screenshot shows how to download the iptraf-ng tarball: After we have completed the downloading, extract the tarball using the following command: tar –xzf iptraf-ng-<current_version_number>.tar.gz Navigate to the iptraf-ng directory created by the tar command and issue the following commands: ./configure make sudo make install After these commands are complete, iptraf-ng is ready to run, using the following command: sudo iptraf-ng When the program starts, you will be presented with the following screen: Configuring iptraf-ng As an example, we are going to monitor all incoming traffic to the BeagleBone. In order to do this, iptraf-ng should be configured. Selecting the Configure... menu item will show you the following screen: Here, settings can be changed by highlighting an option in the left-hand side window and pressing Enter to select a new value, which will be shown in the Current Settings window. In this case, I have enabled all the options except Logging. Exit the configuration screen and enter the Filter Status screen. This is where we will set up the filter to only monitor traffic coming to the BeagleBone and from it. Then, the following screen will be presented: Selecting IP... will create an IP filter, and the following subscreen will pop up: Selecting Define new filter... will allow the creation and saving of a filter that will only display traffic for the IP address and the IP protocols that are selected, as shown in the following screenshot: Here, I have put in the BeagleBone's IP address, and to match all IP protocols. Once saved, return to the main menu and select IP traffic monitor. Here, you will be able to select the network interfaces to be monitored. Because my BeagleBone is connected to my wired network, I have selected eth0. The following screenshot should shows us the options: If all went well with your filter, you should see traffic to your BeagleBone and from it. Here are the entries for my PuTTy session; 192.168.17.2 is my Windows 8 machine, and 192.168.17.15 is my BeagleBone: Here's an image of the traffic generated by browsing the DLNA server from the Windows Explorer: Moreover, here's the traffic from my Android smartphone running a DLNA player, browsing the shared directories that were set up: Summary In this article, you saw how to install and configure the software that will be used to monitor the traffic on your local network. With these programs and a bit of experience, you can determine which devices on your network are hogging the bandwidth and find out whether you have any unauthorized users. Resources for Article: Further resources on this subject: Learning BeagleBone [article] Protecting GPG Keys in BeagleBone [article] Home Security by BeagleBone [article]

This is an article written by Rodolfo Giometti, the author of the book BeagleBone Essentials. The pulse-width modulation (PWM) is a technique used to encode a message in a pulsing signal, to generate an analog signal using a digital source, to allow the control of the power supplied to electrical devices, such as electrical motors, or to set the position of a servo motor. In this article, we're going to discuss how an embedded developer can use the BeagleBone Black's PWM generator to control a servo motor with a few Bash commands. (For more resources related to this topic, see here.) What is a PWM generator? A PWM generator is a device that can generate a PWM signal, according to its internal settings. The output of a PWM generator is just a sequence of pulse signals as square waveforms with well-defined characteristics: By referring to the preceding diagram, where we have a simple PWM waveform, we can define the following parameters: Amplitude (A): This is the difference between the maximum output value (ymax) and the minimum one (ymin) Period (T): This is the duration of one cycle of the output square waveform Duty cycle (dc): This is the ratio(in percentage) between the high state time (thigh) and the period (T) In the preceding diagram, the amplitude is 5 V (ymax=5 V and ymin=0 V), the period is 1 ms (the wave is periodic and it repeats itself every 0.001 seconds), and the duty cycle is 25 percent (thigh=0.25 ms and T=1 ms). You can find the details about the PWM at https://en.wikipedia.org/wiki/Pulse-width_modulation. The electrical lines The PWM generator lines are reported in the following table: Name Description PWM output The PWM output signal GND Common ground The PWM in Linux Our BeagleBone Black has eight PWM generators, and even if some of them may have their output lines multiplexed with another device, they cannot be used without disabling the conflicting device. The complete list is reported at the BeagleBone Black's support page (http://beagleboard.org/support/bone101) and summarized in the following table: Name PWM output pwm0 P9.22 or P9.31 pwm1 P9.21 or P9.29 pwm2 P9.42 pwm3 P8.36 or P9.14 pwm4 P8.34 or P9.16 pwm5 P8.19 or P8.45 pwm6 P8.13 or P8.46 pwm7 P9.28 In the preceding table, the notation P9.22 means that pin 22 is on the expansion connector P9. We can directly get these values from the BeagleBone Black firmware settings, using the following command: root@BeagleBone:~# dtc -I dtb -O dts <dtbo> | grep exclusive-use Here, <dtbo> is one of the firmware files available in the /lib/firmware/ directory: root@beaglebone:~# ls /lib/firmware/bone_pwm_*.dtbo/lib/firmware/bone_pwm_P8_13-00A0.dtbo    /lib/firmware/bone_pwm_P9_16-00A0.dtbo/lib/firmware/bone_pwm_P8_19-00A0.dtbo    /lib/firmware/bone_pwm_P9_21-00A0.dtbo/lib/firmware/bone_pwm_P8_34-00A0.dtbo    /lib/firmware/bone_pwm_P9_22-00A0.dtbo/lib/firmware/bone_pwm_P8_36-00A0.dtbo    /lib/firmware/bone_pwm_P9_28-00A0.dtbo/lib/firmware/bone_pwm_P8_45-00A0.dtbo    /lib/firmware/bone_pwm_P9_29-00A0.dtbo/lib/firmware/bone_pwm_P8_46-00A0.dtbo    /lib/firmware/bone_pwm_P9_31-00A0.dtbo/lib/firmware/bone_pwm_P9_14-00A0.dtbo    /lib/firmware/bone_pwm_P9_42-00A0.dtbo To enable a PWM generator, we have to use one of the preceding dtbo files in conjunction with the /lib/firmware/am33xx_pwm-00A0.dtbo file, as shown in the following code: root@beaglebone:~# echo am33xx_pwm > /sys/devices/bone_capemgr.9/slotsroot@beaglebone:~# echo bone_pwm_P9_22 > /sys/devices/bone_capemgr.9/slots This should cause the following kernel messages activity: [   31.350494] bone-capemgr bone_capemgr.9: slot #7: Applied #8 overlays.[   46.144068] bone-capemgr bone_capemgr.9: part_number 'bone_pwm_P9_22', version'N/A'[   46.144266] bone-capemgr bone_capemgr.9: slot #8: generic override[   46.144319] bone-capemgr bone_capemgr.9: bone: Using override eeprom dataat slot 8[   46.144374] bone-capemgr bone_capemgr.9: slot #8: 'Override BoardName,00A0,Override Manuf,bone_pwm_P9_22'[   46.144640] bone-capemgr bone_capemgr.9: slot #8: Requesting part number/version based 'bone_pwm_P9_22-00A0.dtbo[   46.144698] bone-capemgr bone_capemgr.9: slot #8: Requesting firmware'bone_pwm_P9_22-00A0.dtbo' for board-name 'Override Board Name',version '00A0'[   46.144762] bone-capemgr bone_capemgr.9: slot #8: dtbo 'bone_pwm_P9_22-00A0.dtbo' loaded; converting to live tree[   46.148901] bone-capemgr bone_capemgr.9: slot #8: #2 overlays[   46.155642] bone-capemgr bone_capemgr.9: slot #8: Applied #2 overlays Now, in the /sys/devices/ocp.3/ directory, we should see a new directory named pwm_test_P9_22.12, where we have the following files: root@beaglebone:~# ls /sys/devices/ocp.3/pwm_test_P9_22.12/driver  duty  modalias  period  polarity  power  run  subsystem  uevent Here, the important files are period, duty, and polarity. In the period file, we can store the period (T) of the desired PWM waveform in nanoseconds (ns), while in duty, we can store the high state time (thigh) in nanoseconds (of course, with this parameter, we can set the duty cycle too). In the end, with the polarity file, we can invert the waveform polarity (that is, by swapping the high state and low state). For example, the waveform of the preceding figure can be obtained using the following commands: root@beaglebone:~# echo 1000000 > /sys/devices/ocp.3/pwm_test_P9_22.12/periodroot@beaglebone:~# echo 250000 > /sys/devices/ocp.3/pwm_test_P9_22.12/ Managing a servo motor To show you how to use a PWM generator in order to manage a peripheral, we can use a servo motor. This is a really simple motor where we can set a specific gear position by setting a proper duty cycle of the PWM signal. The following image shows the servo motor used in this example: The device can be purchased at (or by surfing the Internet) http://www.cosino.io/product/nano-servo-motor. First of all, we've to set up the electrical connections. In the following table, we have reported the correspondence between the BeagleBone Black's pins and the servo motor's cables: BeagleBone Black pins – Label Servo motor's cables – Color P9.4 – Vcc Red P9.22 Yellow P9.2 – GND Black By taking a look at the datasheet available at http://hitecrcd.com/files/Servomanual.pdf, we discover that the servo can be managed using a periodic square waveform of 20 ms period (T) and a high state time (thigh) between 0.9 ms and 2.1 ms with 1.5 ms as (more or less) the center. So, once connected, we can set the center position using the following settings: root@beaglebone:~# echo 0 > /sys/devices/ocp.3/pwm_test_P9_22.12/polarityroot@beaglebone:~# echo 20000000  > /sys/devices/ocp.3/pwm_test_P9_22.12/periodroot@beaglebone:~# echo 1500000  > /sys/devices/ocp.3/pwm_test_P9_22.12/duty Then, we can move the gear totally clockwise using the following command: root@beaglebone:~# echo 2100000 > /sys/devices/ocp.3/pwm_test_P9_22.12/duty We can move the gear totally anticlockwise using the following command: root@beaglebone:~# echo 900000 > /sys/devices/ocp.3/pwm_test_P9_22.12/duty Summary In this article, we discovered that managing a BeagleBone Black's PWM generator is really as simple as controlling a servo motor! Resources for Article: Further resources on this subject: Home Security by BeagleBone [Article] Protecting GPG Keys in BeagleBone [Article] Learning BeagleBone [Article]

In this article, written by Cedric Rajendran, author of the book Getting Started with VMware Virtual SAN, we will discuss one of the key characteristics of Virtual SAN called Storage Policy Based Management (SPBM). Traditionally, storage capabilities are tiered and provisioned. Some of the key attributes for tiering are performance, capacity, and availability. The actual implementation of tiers is performed at the hardware level, governed by physical disk capabilities and RAID configuration. VSAN, however, establishes the capabilities at the software layer through policies. Here we will closely review: Why is SPBM used? Attributes that are configurable through SPBM Understand how SPBM works Overall, we will discuss the various permutations and combinations of the policy-based management of storage, and how this method modernizes storage provisioning and paves the way for being truly software-defined. (For more resources related to this topic, see here.) Why do we need policies? Back in the '90s, Gartner discussed tiered storage architecture with traditional storage arrays. Devices were tiered based on their cost and data on certain factors such as criticality, age, performance, and a few others. This meant that some data made their way to the fastest and most reliable tier and other data into slower and less expensive ones. This tiering was done at the device level, that is, the storage administrator segmented devices based on cost or there was heterogeneous storage presented to servers varying between high-end, mid-end, and low-end arrays. An administrator would then manually provision data on the respective tiers. There have been several advancements with storage arrays automating tiering at the array level. With virtualization however, data seldom static and the ability to move data around through features such as Storage vMotion gave the right level of agility to the vSphere administrators. The flip side of this is that it became very error prone and difficult to maintain compliance. For example, during maintenance tasks, a high I/O intensive virtual machine may be migrated to a low IOPS capable datastore; this would silently lead to a performance issue for the application and overall user experience. Hence, there was a need for a very high degree of control, automation, and a VM-centric approach to satisfy each virtual machine's storage requirements. The solution for this problem is SPBM. With SPBM, we are able to build very granular policies to each VMDK associated to a virtual machine and these policies follow the virtual machine wherever they go. Understanding SPBM An SPBM policy can be thought of as a blueprint or plan that outlines the storage performance, capacity and availability requirement of a virtual machine. The policy is then associated with individual objects (VMDK). These policies are then applied by replicating, distributing and caching the objects. At this juncture, it suffices to understand that objects are parts of a virtual machine; a virtual machine disk (VMDK) and the snapshot delta files are examples of objects. Let's discuss this with an example of RAID 0 concept. In RAID 0, data is striped, that is, data is broken down into blocks and each block is written on a different disk drives/controller in the RAID group so that, cumulative IOPS of all disks in the RAID group are efficiently used, and this in turn increases the performance. Similarly, we can define a policy with SPBM for an object (VMDK) that will stripe the object across a VSAN datastore. It is mandatory for each virtual machine that is to be deployed on a VSAN datastore to be associated with a policy. If one has not been defined, a default, predefined policy will be applied. In a nutshell, the capabilities of VSAN datastore will be abstracted and presented in such a way that an object can distinctly be placed adhering to very specific needs of the specific object. All this, while another virtual machines' objects resid on the same VSAN datastore, can have a totally different set of capabilities. An important component that enables this abstraction is vStorage APIs for Storage Awareness (VASA); more details on VASA are discussed at the end of this article. The communication workflow is as follows: Define the capabilities required for a VM in a storage policy in vCenter Policy information is cascaded to VSAN through VASA VASA assesses whether VSAN can accommodate the capability requirement and reports compliance on a per-storage object basis Let's understand this concept with a simple example. Consider a fileserver virtual machine that comprises of two VMDKs or objects, one of which is for the OS and the other where the actual data is being read from or written to by several users. The OS VMDK requires lower IOPS capability, while the other VMDK is very I/O intensive and requires a significantly faster disk. The application team that maintains this server demands this workload to be placed in a tier 1 datastore, which in turn translates to a LUN from a mid-range or high-end array, the cost of which obviously is rather high. A vSphere administrator can argue that the OS VMDK can be part of a tier 2 or tier 3 VMFS datastore that is less expensive, whereas the database VMDK can be placed on a Tier 1 datastore to meet the business SLAs for storage optimization. While this is theoretically achievable, in reality it possesses significant administrative overheads and a serious sore-point if there are any failures in the datastore where the files reside. Troubleshooting and restoring the VM to the running state will be quite a cumbersome and time-consuming task. Now imagine if a policy is able to cater to the storage requirements of this VM, an administrator carves out a policy as per the requirements and associates it to the VM's objects residing on the VSAN datastore. After this one-time effort, the policy ensures that the virtual machine is compliant with the demands of the application team throughout its lifecycle. Another interesting and useful feature of SPBM is that during the lifecycle of the virtual machine, the administrator can amend the policies and reapply without disruption or downtime. To summarize, with Storage Policy Based Management, the virtual machine deployment is tied to the Virtual SAN capabilities and thereby removes the administrative overhead and complication associated with manually building this setup. VSAN datastore capabilities VSAN datastore capabilities help define the performance, availability, reliability, and the capabilities indirectly governing the capacity consumed by an object. Let's dive into the specific capabilities that can be abstracted and managed. The following is a list of capabilities that can be defined on a VSAN datastore: Number of disk stripes per object Number of failures to tolerate Flash read cache reservation Force provisioning Object space reservation Accessing the VSAN datastore capabilities We can access these capabilities through the vSphere web client as described in the following steps and screenshots: Connect to the vCenter server through the vSphere web client. Navigate to Home | VM Storage Policies, as shown here: Choose VSAN from the dropdown for Rules based on vendor specific capabilities. Create (or edit) a VM storage policy, as shown in the following screenshot: Define Rule Set of the policy describing the storage requirements of an object. Review the configuration settings and click on Finish. Number of disk stripes per object This capability simulates the traditional RAID 0 concept by defining the number of physical disks across which each replica of a storage object is striped. In a typical RAID 0, this means that there is concurrent and parallel I/O running into multiple disks. However, in the context of VSAN, this raises a few questions. Consider this typical scenario: A disk group can have a maximum of one SSD All I/O read cache and write buffer are routed first to SSD I/O is then destaged from SSD to magnetic disks How will having more stripes improve performance if SSD intercepts all I/O? The answer to this question is that it depends, and cannot be administratively controlled. However, at a high level, performance improvement can be witnessed. If the structure of the object is spread across magnetic disks from different hosts in the cluster, then multiple SSDs and magnetic disks will be used. This is very similar to the traditional RAID 0. Another influencing factor is how I/O moves from SSD to magnetic disks. Number of disk stripes per object is by default 1. There can be a maximum of 12 stripes per object. Number of failures to tolerate This capability defines the availability requirements of the object. In this context, the nature of failure can be at host, network, and disk level in the cluster. Based on the value defined for the number of failures to tolerate (n), there are n+1 replicas that are built to sustain n failures. It is important to understand that the object can sustain n concurrent failures, that is, all permutations and combinations of host, network, and/or disk-level failures can be sustained until n failures. This is similar to a RAID 1 mirroring concept, albeit replicas are placed on different hosts. Number of failures to tolerate is by default set to 1. We can have a maximum value of 3. Scenario based examples Outlined here are three scenarios demonstrating the placements of components of an object. Note that objects are of four types. For easier understanding, we will discuss scenarios based on the VMDK object. We'll sample VMDK since these are the most sensitive and relevant in the context of objects on the VSAN datastore. In addition, these are some illustrations of how VSAN may place the objects by adhering to the policies defined, and this may vary depending on resource availability and layout specific to each deployment. Scenario 1 Number of failures to tolerate is equal to 1. In the first scenario, we have crafted a simple policy to tolerate one failure. The virtual machine objects are expected to have a mirrored copy and the objective is to eliminate a single point of failure. The typical use for this policy is an operating system VMDK: Scenario 2 Number of failures to tolerate is equal to 1. Number of disk stripes per object is equal to 2. In this scenario, we increase the stripe width of the object, while keeping the failure tolerance left at 1. The objective here is to improve the performance as well as ensure that there is no single point of failure. The expected layout is as shown here; the object is mirrored and striped: Scenario 3 Number of failures to tolerate is equal to 2. Number of disk stripes per object is equal to 2. Extending from the preceding scenario, we increase the failure tolerance level to 2. Effectively, two mirrors can fail, so the layout will expand as illustrated in the following diagram. Note that to facilitate n failures, you would need 2n+1 nodes. An administrator can validate the actual physical disk placement of the components, that is, the parts that make up the object from the virtual machines' Manage tab from the vSphere web client. Navigate to VM | Manage | VM Storage Policies: Flash read cache reservation By default, all virtual machine objects based on demand share the read cache available from the flash device that is part of each disk group. However, there may be scenarios wherein specific objects require reserved read cache, typically for a read intensive workload that needs to have the maximum amount of its reads to be serviced by a flash device. In such cases, an administrator can explicitly define a percentage of flash cache to be reserved for the object. The flash read cache reservation capability defines the amount of flash capacity that will be reserved/blocked for the storage object to be used as read cache. The reservation is displayed as a percentage of the object. You can have a minimum of 0 percent and can go up to 100 percent, that is, you can reserve the entire object size on the flash disk for read cache. For purposes of granularity, since the flash device may run into terabytes of capacity, the value for flash cache can be specified up to 4 decimal places; for example, it can be set to 0.0001 percent. As with any reservation concept, blocking resources for one object implies the resource is unavailable for another object. Therefore, unless there is a specific need, this should be left at default and Virtual SAN should be allowed to have control over the allocation. This will ensure adequate capacity distribution between objects. The default value is 0 percent and the maximum value is 100 percent. Force provisioning We create policies to ensure that the storage requirements of a virtual machine object is strictly adhered to. In the event that the VSAN datastore cannot satisfy the storage requirements specified by the policy, the virtual machine will not be provisioned. This capability allows for a strict compliance check. However, it may also become an obstacle when you need to urgently deploy virtual machines but the datastore does not satisfy the storage requirements of the virtual machine. The force provisioning capability allows an administrator to override this behavior. By default, Force Provisioning is set to No. By toggling this setting to Yes, virtual machines can be forcefully provisioned. It is important to understand that an administrator should remediate the constraints that lead to provisioning failing in the first place. It has a boolean value, which is set to No by default. Object space reservation Virtual machines provisioned on Virtual SAN are, by default, provisioned as thin disks. The Object Space Reservation parameter defines the logical size of the storage object or, in other words, whether the specific object should remain thin, partially, or fully allocated. While this is not entirely new and is similar to the traditional practice of either thin provisioning or thick provisioning a VMDK, VSAN provides a greater degree of control by letting the vSphere administrators choose the percentage of disk that should be thick provisioned. The default value is 0 percent and maximum value is 100 percent. Under the hood – SBPM It is important to understand how the abstraction works under the hood in order to surface the Virtual SAN capabilities, which in turn help to create and associate policies to virtual machines. The following section about VASA and managing storage providers is informative, and for better understanding; you may not run into a situation where you need to make any configuration changes to storage providers. vSphere APIs for Storage Awareness To understand VASA better, let's consider a scenario wherein an administrator is deploying a virtual machine on a traditional SAN array. He would need to choose the appropriate datastore to suit the capabilities and requirements of the virtual machine or certain business requirements. For instance, there could be workloads that need to be deployed in a tier 1 LUN. The existing practice is to ensure that the right virtual machine gets deployed on the right datastore; there were rather archaic styles of labelling, or simply asking the administrator the capability of the LUN. Now, replace this methodology with a mechanism to identify the storage capabilities through API. VASA provides such a capability and aids in identifying the specific attributes of the array and passes on these capabilities to vCenter. This implies that a vSphere administrator can have end-to-end visibility through a single management plane of vCenter. Storage DRS, storage health, and capacity monitoring, to name a few, are very useful and effective features implemented through VASA. To facilitate VASA, storage array vendors create plugins called vendor/storage providers. These plugins allow storage vendors to publish the capabilities to vCenter, which in turn surfaces it in the UI. For VMware Virtual SAN, the VSAN storage provider is developed by VMware and built into ESXi hypervisors. By enabling VSAN on a cluster, the plugins get automatically registered with vCenter. The VSAN storage provider surfaces the VSAN datastores' capabilities which in turn is used to create appropriate policies. Managing Virtual SAN storage providers Once Virtual SAN is enabled and storage provider registration is complete, an administrator can verify this through the vSphere web client: Navigate to the vCenter server in the vSphere web client. Click on the Manage tab, and click on Storage Providers. The expected outcome would be to have one VSAN provider online and the remaining storage providers on standby mode. The following screenshot shows a three-node cluster: If the host that currently has the online storage provider fails, another host will bring its provider online. Summary In this article, we discussed the significance of Storage Policy Based Management in detail and how it plays a key factor in defining the storage provisioning at the software layer. We further discussed the VSAN datastore capabilities with scenarios and how it operates under the hood.

In this article by David Shaw, author of the book Nmap Essentials, we will see that although being able to conduct port scans is an integral part of using the Nmap suite of tools, the developers of Nmap created a very powerful engine that's built into the tool: the Nmap Scripting Engine (NSE). This article introduces the NSE, and covers all the topics needed to use reliably-written scripts in the Nmap script repository, in order to conduct reconnaissance scans that include much more than just what ports are open and which services are listening. In this article, we will cover: The history of the NSE How the NSE works (For more resources related to this topic, see here.) The history of the NSE By the mid-2000s, Nmap had established itself as the clear leader in port scanning tools—and security tools in general—whether open source or not. Although it's a constant battle to continually innovate and optimize, Nmap can only be considered as an extremely successful project. Due to its popularity, and the fact that it's an open source project with a relatively high profile, Nmap was selected to participate in Google Summer of Code several times. Google Summer of Code is a software development internship/association project, during which students are selected and put on open source software teams to build new features into existing projects. In May 2006—when the currently released version of Nmap was only 4.0—Nmap was selected for its second Summer of Code season. The previous year, in 2005, several improvements had been made through the students' coding for the Nmap project: the students had written a contemporary implementation of Netcat (called Ncat), upgraded the OS detection for Nmap to its second (and much better) generation, and created a small, simplified GUI that would later become Zenmap. For this second run through, after an extremely successful first summer, the participant developers were even more ambitious. Since Nmap clearly had an excellent set of features, why not make those features extendable by the greater community? New vulnerabilities and scanning techniques were being pioneered on a very frequent basis, and full Nmap releases couldn't keep up with the things that security professionals needed to assess. Every time a new vulnerability came out, security professionals (and malicious hackers!) would scan for vulnerable services with Nmap, but could only test whether software versions were vulnerable by using manual analysis: clearly, not a very efficient use of time. Because of the new resources granted by Google Summer of Code developers, an arbitrary scripting framework was created that allows users to trigger additional checks based on certain open ports or services. This means, for example, that if you're looking for a specific file on all web servers—robots.txt, for example—you can easily create a script that can check for it on all HTTP and HTTPS services. The NSE (and the inclusion of Nmap scripts in default installations of Nmap) truly revolutionized the versatility of the tool suite. After months of hard work, the NSE was released in December 2006, packaged with Nmap release 4.21ALPHA1. The scripts that come packaged with the NSE have continued to grow in complexity and usability, and are excellent resources to turn Nmap into a fully-featured security tool suite. The inner working of the NSE The NSE is a framework that runs code written in the programming language Lua with specific flags that the engine can parse. Lua is a lightweight, fast, and interpreted programming language—one that has the most fame for scripting user interfaces for computer games such as World of Warcraft—that has a similar syntax to other contemporary interpreted languages. If you've ever seen code written in Python or Ruby, Lua won't seem too alien to you. The preceding screenshot shows an Nmap script that identifies information about Bitcoins (written by Patrik Karlsson). Don't worry if you don't understand it yet but you can see that the code used to generate a relatively complex Nmap script looks very simple. This is the whole point of the NSE! Where security engineers and system administrators used to have to export Nmap results, find the information they are looking for and then use third-party tools to assist them; they are now able to either find a script that serves their purposes, or write a simple one themselves. Many penetration testers can leverage the Nmap scripting language to even weaponize the tool for security exploits. Summary This article introduced the NSE, which can be one of the most useful, versatile, and engaging features of the Nmap tool suite. We should now be able to launch scans that do more than just port and service versions—Nmap scripts can actually interact with the services listening, and in some cases can even exploit vulnerabilities! In this article, we covered the history of the NSE, and how NSE works. Resources for Article: Further resources on this subject: Target Exploitation [article] Enabling and configuring SNMP on Windows [article] Gathering all rejects prior to killing a job [article]

In this article by Ivan Morgillo, the author of RxJava Essentials, we will approach Observable filtering with RxJava filter(). We will manipulate a list on installed app to show only a subset of this list, according to our criteria. (For more resources related to this topic, see here.) Filtering a sequence with RxJava RxJava lets us use filter() to keep certain values that we don't want, out of the sequence that we are observing. In this example, we will use a list, but we will filter it, passing to the filter() function the proper predicate to include only the values we want. We are using loadList() to create an Observable sequence, filter it, and populate our adapter: private void loadList(List<AppInfo> apps) {    mRecyclerView.setVisibility(View.VISIBLE);      Observable.from(apps)           .filter((appInfo) ->                appInfo.getName().startsWith("C"))            .subscribe(new Observer<AppInfo>() {                @Override                public void onCompleted() {                    mSwipeRefreshLayout.setRefreshing(false);                }                  @Override                public void onError(Throwable e) {                    Toast.makeText(getActivity(), "Something went                      south!", Toast.LENGTH_SHORT).show();                    mSwipeRefreshLayout.setRefreshing(false);                }                  @Override                public void onNext(AppInfo appInfo) {                    mAddedApps.add(appInfo);                    mAdapter.addApplication(mAddedApps.size() - 1,                   appInfo);                }            }); } We have added the following line to the loadList() function: .filter((appInfo) -> appInfo.getName().startsWith("C")) After the creation of the Observable, we are filtering out every emitted element that has a name starting with a letter that is not a C. Let's have it in Java 7 syntax too, to clarify the types here: .filter(new Func1<AppInfo, Boolean>() {    @Override    public Boolean call(AppInfo appInfo) {        return appInfo.getName().startsWith("C");    } }) We are passing a new Func1 object to filter(), that is, a function having just one parameter. The Func1 object has an AppInfo object as parameter type and it returns a Boolean object. The filter() function will return true only if the condition will be verified. At that point, the value will be emitted and received by all the Observers. As you can imagine, filter() is critically useful to create the perfect sequence that we need from the Observable sequence we get. We don't need to know the source of the Observable sequence or why it's emitting tons of different elements. We just want a useful subset of those elements to create a new sequence we can use in our app. This mindset enforces the separation and the abstraction skills of our coding day life. One of the most common use of filter() is filtering null objects: .filter(new Func1<AppInfo, Boolean>() {    @Override    public Boolean call(AppInfo appInfo) {        return appInfo != null;    } }) This seems to be trivial and there is a lot of boilerplate code for something that trivial, but this will save us from checking for null values in the onNext() call, letting us focus on the actual app logic. As result of our filtering, the next figure shows the installed apps list, filtered by name starting with C: Summary In this article, we introduced RxJava filter() function and we used it in a real-world example in an Android app. RxJava offers a lot more functions allowing you to filter and manipulate Observable sequences. A comprehensive list of methods, scenarios and example are available in RxJava that will drive you in a step-by-step journey, from the basic of the Observer pattern to composing Observables and querying REST API using RxJava. Resources for Article: Further resources on this subject: Android Native Application API [article] Android Virtual Device Manager [article] Putting It All Together – Community Radio [article]