Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Save more on your purchases! discount-offer-chevron-icon
Savings automatically calculated. No voucher code required.
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Events
Videos
Audiobooks
Packt Hub
Free Learning
Arrow right icon
timer SALE ENDS IN
0 Days
:
00 Hours
:
00 Minutes
:
00 Seconds

How-To Tutorials

7018 Articles
Packt
02 Nov 2015
28 min read
Save for later

Let's Get Physical – Using GameMaker's Physics System

Packt
02 Nov 2015
28 min read
 In this article by Brandon Gardiner and Julián Rojas Millán, author of the book GameMaker Cookbook we'll cover the following topics: Creating objects that use physics Alternating the gravity Applying a force via magnets Creating a moving platform Making a rope (For more resources related to this topic, see here.) The majority of video games are ruled by physics in one way or another. 2D platformers require coded movement and jump physics. Shooters, both 2D and 3D, use ballistic calculators that vary in sophistication to calculate whether you shot that guy or missed him and he's still coming to get you. Even Pong used rudimentary physics to calculate the ball's trajectory after bouncing off of a paddle or wall. The next time you play a 3D shooter or action-adventure game, check whether or not you see the logo for Havok, a physics engine used in over 500 games since it was introduced in 2000. The point is that physics, however complex, is important in video games. GameMaker comes with its own engine that can be used to recreate physics-based sandbox games, such as The Incredible Machine, or even puzzle games, such as Cut the Rope or Angry Birds. Let's take a look at how elements of these games can be accomplished using GameMaker's built-in physics engine. Physics engine 101 In order to use GameMaker's physics engine, we first need to set it up. Let's create and test some basic physics before moving on to something more complicated. Gravity and force One of the things that we learned with regards to GameMaker physics was to create our own simplistic gravity. Now that we've set up gravity using the physics engine, let's see how we can bend it according to our requirements. Physics in the environment GameMaker's physics engine allows you to choose not only the objects that are affected by external forces but also allows you to see how they are affected. Let's take a look at how this can be applied to create environmental objects in your game. Advanced physics-based objects Many platforming games, going all the way back to Pitfall!, have used objects, such as a rope as a gameplay feature. Pitfall!, mind you, uses static rope objects to help the player avoid crocodiles, but many modern games use dynamic ropes and chains, among other things, to create a more immersive and challenging experience. Creating objects that use physics There's a trend in video games where developers create products that have less games than play areas; worlds and simulators in which a player may or may not be given an objective and it wouldn't matter either way. These games can take on a life of their own; Minecraft is essentially a virtual game of building blocks and yet has become a genre of its own, literally making its creator, Markus Persson (also known as Notch), a billionaire in the process. While it is difficult to create, the fun in games such as Minecraft is designed by the player. If you give a player a set of tools or objects to play with, you may end up seeing an outcome you hadn't initially thought of and that's a good thing. The reason why I have mentioned all of this is to show you how it binds to GameMaker and what we can do with it. In a sense, GameMaker is a lot like Minecraft. It is a set of tools, such as the physics engine we're about to use, that the user can employ if he/she desires (of course, within limits), in order to create something funny or amazing or both. What you do with these tools is up to you, but you have to start somewhere. Let's take a look at how to build a simple physics simulator. Getting ready The first thing you'll need is a room. Seems simple enough, right? Well, it is. One difference, however, is that you'll need to enable physics before we begin. With the room open, click on the Physics tab and make sure that the box marked Room is Physics World is checked. After this, we'll need some sprites and objects. For sprites, you'll need a circle, triangle, and two squares, each of a different color. The circle is for obj_ball. The triangle is for obj_poly. One of the squares is for obj_box, while the other is for obj_ground. You'll also need four objects without sprites: obj_staticParent, obj_dynamicParent, obj_button, and obj_control. How to do it Open obj_staticParent and add two collision events: one with itself and one with obj_dynamicParent. In each of the collision events, drag and drop a comment from the Control tab to the Actions box. In each comment, write Collision. Close obj_staticParent and repeat steps 1-3 for obj_dynamicParent. In obj_dynamicParent, click on Add Event, and then click on Other and select Outside Room. From the Main1 tab, drag and drop Destroy Instance in the Actions box. Select Applies to Self. Open obj_ground and set the parent to obj_staticParent. Add a create event with a code block containing the following code: var fixture = physics_fixture_create(); physics_fixture_set_box_shape(fixture, sprite_width / 2, sprite_height / 2); physics_fixture_set_density(fixture, 0); physics_fixture_set_restitution(fixture, 0.2); physics_fixture_set_friction(fixture, 0.5); physics_fixture_bind(fixture, id); physics_fixture_delete(fixture); Open the room that you created and start placing instances of obj_ground around it to create platforms, stairs, and so on. This is how mine looked like: Open obj_ball and set the parent to obj_dynamicParent. Add a create event and enter the following code: var fixture = physics_fixture_create(); physics_fixture_set_circle_shape(fixture, sprite_get_width(spr_ball) / 2); physics_fixture_set_density(fixture, 0.25); physics_fixture_set_restitution(fixture, 1); physics_fixture_set_friction(fixture, 0.5); physics_fixture_bind(fixture, id); physics_fixture_delete(fixture); Repeat steps 10 and 11 for obj_box, but use this code: var fixture = physics_fixture_create(); physics_fixture_set_box_shape(fixture, sprite_width / 2, sprite_height / 2); physics_fixture_set_density(fixture, 0.5); physics_fixture_set_restitution(fixture, 0.2); physics_fixture_set_friction(fixture, 0.01); physics_fixture_bind(fixture, id); physics_fixture_delete(fixture); Repeat steps 10 and 11 for obj_poly, but use this code: var fixture = physics_fixture_create(); physics_fixture_set_polygon_shape(fixture); physics_fixture_add_point(fixture, 0, -(sprite_height / 2)); physics_fixture_add_point(fixture, sprite_width / 2, sprite_height / 2); physics_fixture_add_point(fixture, -(sprite_width / 2), sprite_height / 2); physics_fixture_set_density(fixture, 0.01); physics_fixture_set_restitution(fixture, 0.1); physics_fixture_set_linear_damping(fixture, 0.5); physics_fixture_set_angular_damping(fixture, 0.01); physics_fixture_set_friction(fixture, 0.5); physics_fixture_bind(fixture, id); physics_fixture_delete(fixture); Open obj_control and add a create event using the following code: globalvar shape_select; globalvar shape_output; shape_select = 0; Add a Step and add the following code to a code block: if mouse_check_button(mb_left) && alarm[0] < 0 && !place_meeting(x, y, obj_button) { instance_create(mouse_x, mouse_y, shape_output); alarm[0] = 5; } if mouse_check_button_pressed(mb_right) { shape_select += 1; } Now, add an event to alarm[0] and give it a comment stating Set Timer. Place an instance of obj_control in the room that you created, but make sure that it is placed in the coordinates (0, 0). Open obj_button and add a step event. Drag a code block to the Actions tab and input the following code: if shape_select > 2 { shape_select = 0; } if shape_select = 0 { sprite_index = spr_ball; shape_output = obj_ball; } if shape_select = 1 { sprite_index = spr_box; shape_output = obj_box; } if shape_select = 2 { sprite_index = spr_poly; shape_output = obj_poly; } Once these steps are completed, you can test your physics environment. Use the right mouse button to select the shape you would like to create, and use the left mouse button to create it. Have fun! How it works While not overly complicated, there is a fair amount of activity in this recipe. Let's take a quick look at the room itself. When you created this room, you checked the box for Room is Physics World. This does exactly what it says it does; it enables physics in the room. If you have any physics-enabled objects in a room that is not a physics world, errors will occur. In the same menu, you have the gravity settings (which are vector-based) and pixels to meters, which sets the scale of objects in the room. This setting is important as it controls how each object is affected by the coded physics. YoYo Games based GameMaker's physics on the real world (as they should) and so GameMaker needs to know how many meters are represented by each pixel. The higher the number, the larger the world in the room. If you place an object in two different rooms with different pixel to meter settings, even though the objects have the same settings, GameMaker will apply physics to them differently because it views them as being of differing size and weight. Let's take a look at the objects in this simulation. Firstly, you have two parent objects: one static and the other dynamic. The static object is the only parent to one object: obj_ground. The reason for this is that static objects are not affected by outside forces in a physics world, that is, the room you built. Because of this, the ground pieces are able to ignore gravity and forces applied by other objects that collide with them. Now, neither obj_staticParent nor obj_dynamicParent contain any physics code; we saved this for our other objects. We use our parent objects to govern our collision groups using two objects instead of coding collisions in each object. So, we use drag and drop collision blocks to ensure that any children can collide with instances of one another and with themselves. Why did you drag comment blocks into these collision events? We did this so that GameMaker doesn't ignore them; the contents of each comment block are irrelevant. Also, the dynamic parent has an event that destroys any instance of its children that end up outside the room. The reason for this is simply to save memory. Otherwise, each object, even those off-screen, will be accounted for calculations at every step and this will slow everything down and eventually crash the program. Now, as we're using physics-enabled objects, let's see how each one differs from the others. When working with the object editor, you may have noticed the checkbox labelled Uses Physics. This checkbox will automatically set up the basic physics code within the selected object, but only after assuming that you're using the drag and drop method of programming. If you click on it, you'll see a new menu with basic collision options as well as several values and associated options: Density: Density in GameMaker works exactly as it does in real life. An object with a high density will be much heavier and harder to move via force than a low-density object of the same size. Think of how far you can kick an empty cardboard box versus how far you can kick a cardboard box full of bricks, assuming that you don't break your foot. Restitution: Restitution essentially governs an object's bounciness. A higher restitution will cause an object to bounce like a rubber ball, whereas a lower restitution will cause an object to bounce like a box of bricks, as mentioned in the previous example. Collision group: Collision grouping tells GameMaker how certain objects react with one another. By default, all physics objects are set to collision group 0. This means that they will not collide with other objects without a specific collision event. Assigning a positive number to this setting will cause the object in question to collide with all other objects in the same collision group, regardless of collision events. Assigning a negative number will prevent the object from colliding with any objects in that group. I don't recommend that you use collision groups unless absolutely necessary, as it takes a great deal of memory to work properly. Linear damping: Linear damping works a lot like air friction in real life. This setting affects the velocity (momentum) of objects in motion over time. Imagine a military shooter where thrown grenades don't arc, they just keep soaring through the air. We don't need this. This is what rockets are for. Angular damping: Angular damping is similar to linear damping. It only affects an object's rotation. This setting keeps objects from spinning forever. Have you ever ridden the Teacup ride at Disneyland? If so, you will know that angular damping is a good thing. Friction: Friction also works in a similar way to linear damping, but it affects an object's momentum as it collides with another object or surface. If you want to create icy surfaces in a platformer, friction is your friend. We didn't use this menu in this recipe but we did set and modify these settings through code. First, in each of the objects, we set them to use physics and then declared their shapes and collision masks. We started with declaring the fixture variable because, as you can see, it is part of each of the functions we used and typing fixture is easier than typing physics_fixture_create() every time. The fixture variable that we bind to the object is what is actually being affected by forces and other physics objects, so we must set its shape and properties in order to tell GameMaker how it should react. In order to set the fixture's shape, we use physics_set_circle_shape, physics_set_box_shape, and physics_set_polygon_shape. These functions define the collision mask associated with the object in question. In the case of the circle, we got the radius from half the width of the sprite, whereas for the box, we found the outer edges used via half the width and half the height. GameMaker then uses this information to create a collision mask to match the sprite from which the information was gathered. When creating a fixture from a more complex sprite, you can either use the aforementioned methods to approximate a mask, or you can create a more complex shape using a polygon like we did for the triangle. You'll notice that the code to create the triangle fixture had extra lines. This is because polygons require you to map each point on the shape you're trying to create. You can map three to eight points by telling GameMaker where each one is situated in relation to the center of the image (0, 0). One very important detail is that you cannot create a concave shape; this will result in an error. Every fixture you create must have a convex shape. The only way to create a concave fixture is to actually create multiple fixtures in the same object. If you were to take the code for the triangle, duplicate all of it in the same code block and alter the coordinates for each point in the duplicated code; you can create concave shapes. For example, you can use two rectangles to make an L shape. This can only be done using a polygon fixture, as it is the only fixture that allows you to code the position of individual points. Once you've coded the shape of your fixture, you can begin to code its attributes. I've described what each physics option does, and you've coded and tested them using the instructions mentioned earlier. Now, take a look at the values for each setting. The ball object has a higher restitution than the rest; did you notice how it bounced? The box object has a very low friction; it slides around on platforms as though it is made of ice. The triangle has very low density and angular damping; it is easily knocked around by the other objects and spins like crazy. You can change how objects react to forces and collisions by changing one or more of these values. I definitely recommend that you play around with these settings to see what you can come up with. Remember how the ground objects are static? Notice how we still had to code them? Well, that's because they still interact with other objects but in an almost opposite fashion. Since we set the object's density to 0, GameMaker more or less views this as an object that is infinitely dense; it cannot be moved by outside forces or collisions. It can, however, affect other objects. We don't have to set the angular and linear damping values simply because the ground doesn't move. We do, however, have to set the restitution and friction levels because we need to tell GameMaker how other objects should react when they come in contact with the ground. Do you want to make a rubber wall to bounce a player off? Set the restitution to a higher level. Do you want to make that icy patch we talked about? Then, you need to lower the friction. These are some fun settings to play around with, so try it out. Alternating gravity Gravity can be a harsh mistress; if you've ever fallen from a height, you will understand what I mean. I often think it would be great if we could somehow lessen gravity's hold on us, but then I wonder what it would be like if we could just reverse it all together! Imagine flipping a switch and then walking on the ceiling! I, for one, think that it would be great. However, since we don't have the technology to do it in real life, I'll have to settle for doing it in video games. Getting ready For this recipe, let's simplify things and use the physics environment that we created in the previous recipe. How to do it In obj_control, open the code block in the create event. Add the following code: physics_world_gravity(0, -10); That's it! Test the environment and see what happens when you create your physics objects. How it works GameMaker's physics world of gravity is vector-based. This means that you simply need to change the values of x and y in order to change how gravity works in a particular room. If you take a look at the Physics tab in the room editor, you'll see that there are values under x and y. The default value is 0 for x and 10 for y. When we added this code to the control object's create event, we changed the value of y to -10, which means that it will flow in the opposite direction. You can change the direction to 360 degrees by altering both x and y, and you can change the gravity's strength by raising and lowering the values. There's more Alternating the gravity's flow can be a lot of fun in a platformer. Several games have explored this in different ways. Your character can change the gravity by hitting a switch in a game, the player can change it by pressing a button, or you can just give specific areas different gravity settings. Play around with this and see what you can create. Applying force via magnets Remember playing with magnets in a science class when you were a kid. It was fun back then, right? Well, it's still fun; powerful magnets make a great gift for your favorite office worker. What about virtual magnets, though? Are they still fun? The answer is yes. Yes, they are. Getting ready Once again, we're simply going to modify our existing physics environment in order to add some new functionality.  How to do it In obj_control, open the code block in the step event. Add the following code: if keyboard_check(vk_space) { with (obj_dynamicParent) { var dir = point_direction(x,y,mouse_x,mouse_y); physics_apply_force(x, y, lengthdir_x(30, dir), lengthdir_y(30, dir)); } } Once you close the code block, you can test your new magnet. Add some objects, hold down the spacebar, and see what happens. How it works Applying a force to a physics-enabled object in GameMaker will add a given value to the direction, rotation, and speed of the said object. Force can be used to gradually propel an object in a given direction, or through a little math, as in this case, draw objects nearer. What we're doing here is that while the Spacebar is held down, any objects in the vicinity are drawn to the magnet (in this case, your mouse). In order to accomplish this, we first declare that the following code needs to act on obj_dynamicParent, as opposed to acting on the control object where the code resides. We then set the value of a dir variable to the point_direction of the mouse, as it relates to any child of obj_dynamicParent. From there, we can begin to apply force. With physics_apply_force, the first two values represent the x and y coordinates of the object to which the force is being applied. Since the object(s) in question is/are not static, we simply set the coordinates to whatever value they have at the time. The other two values are used in tandem to calculate the direction in which the object will travel and the force propelling it in Newtons. We get these values, in this instance, by calculating the lengthdir for both x and y. The lengthdir finds the x or y value of a point at a given length (we used 30) at a given angle (we used dir, which represents point_direction, that finds the angle where the mouse's coordinates lie). If you want to increase the length value, then you need to increase the power of the magnet. Creating a moving platform We've now seen both static and dynamic physics objects in GameMaker, but what happens when we want the best of both the worlds? Let's take a look at how to create a platform that can move and affect other objects via collisions but is immune to said collisions. Getting ready Again, we'll be using our existing physics environment, but this time, we'll need a new object. Create a sprite that is128 px wide by 32 px high and assign it to an object called obj_platform. Also, create another object called obj_kinematicParent but don't give it a sprite. Add collision events to obj_staticParent, obj_dynamicParent, and itself. Make sure that there is a comment in each event. How to do it In obj_platform, add a create event. Drag a code block to the actions box and add the following code: var fixture = physics_fixture_create(); physics_fixture_set_box_shape(fixture, sprite_width / 2, sprite_height / 2); physics_fixture_set_density(fixture, 0); physics_fixture_set_restitution(fixture, 0.2); physics_fixture_set_friction(fixture, 0.5); physics_fixture_bind(fixture, id); physics_fixture_delete(fixture); phy_speed_x = 5; Add a Step event with a code block containing the following code: if (x <64) or (x > room_width-64) { phy_speed_x = phy_speed_x * -1; } Place an instance of obj_platform in the room, which is slightly higher than the highest instance of obj_ground. Once this is done, you can go ahead and test it. Try dropping various objects on the platform and see what happens! How it works Kinematic objects in GameMaker's physics world are essentially static objects that can move. While the platform has a density of 0, it also has a speed of 5 along the x axis. You'll notice that we didn't just use speed equal to 5, as this would not have the desired effect in a physics world. The code in the step simply causes the platform to remain within a set boundary by multiplying its current horizontal speed by -1. Any static object to which a movement is applied automatically becomes a kinematic object. Making a rope Is there anything more useful than a rope? I mean besides your computer, your phone or even this book. Probably, a lot of things, but that doesn't make a rope any less useful. Ropes and chains are also useful in games. Some games, such as Cut the Rope, have based their entire gameplay structure around them. Let's see how we can create ropes and chains in GameMaker. Getting ready For this recipe, you can either continue using the physics environment that we've been working with, or you can simply start from scratch. If you've gone through the rest of this chapter, you should be fairly comfortable with setting up physics objects. I completed this recipe with a fresh .gmx file. Before we begin, go ahead and set up obj_dynamicParent and obj_staticParent with collision events for one another. Next, you'll need to create the obj_ropeHome, obj_rope, obj_block, and obj_ropeControl objects. The sprite for obj_rope can simply be a 4 px wide by 16 px high box, while obj_ropeHome and obj_block can be 32 px squares. Obj_ropeControl needs to use the same sprite as obj_rope, but with the y origin set to 0. Obj_ropeControl should also be invisible. As for parenting, obj_rope should be a child of obj_dynamicParent and obj_ropeHome, and obj_block should be children of obj_staticParent, and obj_ropeControl does not require any parent at all. As always, you'll also need a room in which you need to place your objects. How to do it Open obj_ropeHome and add a create event. Place a code block in the actions box and add the following code: var fixture = physics_fixture_create(); physics_fixture_set_box_shape(fixture, sprite_width / 2, sprite_height / 2); physics_fixture_set_density(fixture, 0); physics_fixture_set_restitution(fixture, 0.2); physics_fixture_set_friction(fixture, 0.5); physics_fixture_bind(fixture, id); physics_fixture_delete(fixture); In obj_rope, add a create event with a code block. Enter the following code: var fixture = physics_fixture_create(); physics_fixture_set_box_shape(fixture, sprite_width / 2, sprite_height / 2); physics_fixture_set_density(fixture, 0.25); physics_fixture_set_restitution(fixture, 0.01); physics_fixture_set_linear_damping(fixture, 0.5); physics_fixture_set_angular_damping(fixture, 1); physics_fixture_set_friction(fixture, 0.5); physics_fixture_bind(fixture, id); physics_fixture_delete(fixture); Open obj_ropeControl and add a create event. Drag a code block to the actions box and enter the following code: setLength = image_yscale-1; ropeLength = 16; rope1 = instance_create(x,y,obj_ropeHome2); rope2 = instance_create(x,y,obj_rope2); physics_joint_revolute_create(rope1, rope2, rope1.x, rope1.y, 0,0,0,0,0,0,0); repeat (setLength) { ropeLength += 16; rope1 = rope2; rope2 = instance_create(x, y+ropeLength, obj_rope2); physics_joint_revolute_create(rope1, rope2, rope1.x, rope1.y, 0,0,0,0,0,0,0); } In obj_block, add a create event. Place a code block in the actions box and add the following code: var fixture = physics_fixture_create(); physics_fixture_set_circle_shape(fixture, sprite_get_width(spr_ropeHome)/2); physics_fixture_set_density(fixture, 0); physics_fixture_set_restitution(fixture, 0.01); physics_fixture_set_friction(fixture, 0.5); physics_fixture_bind(fixture, id); physics_fixture_delete(fixture); Now, add a step event with the following code in a code block: phy_position_x = mouse_x; phy_position_y = mouse_y; Place an instance of obj_ropeControl anywhere in the room. This will be the starting point of the rope. You can place multiple instances of the object if you wish. For every instance of obj_ropeControl you place in the room, use the bounding box to stretch it to however long you wish. This will determine the length of your rope. Place a single instance of obj_block in the room. Once you've completed these steps, you can go ahead and test them. How it works This recipe may seem somewhat complicated but it's really not. What you're doing here is that we are taking multiple instances of the same physics-enabled object and stringing them together. Since you're using instances of the same object, you only have to code one and the rest will follow. Once again, our collisions are handled by our parent objects. This way, you don't have to set collisions for each object. Also, setting the physical properties of each object is done exactly as we have done in previous recipes. By setting the density of obj_ropeHome and obj_block to 0, we're ensuring that they are not affected by gravity or collisions, but they can still collide with other objects and affect them. In this case, we set the physics coordinates of obj_block to those of the mouse so that, when testing, you can use them to collide with the rope, moving it. The most complex code takes place in the create event for obj_ropeControl. Here, we not only define how many sections of a rope or chain will be used, but we also define how they are connected. To begin, the y scale of the control object is measured in order to determine how many instances of obj_rope are required. Based on how long you stretched obj_ropeControl in the room, the rope will be longer (more instances) or shorter (fewer instances). We then set a variable (ropeLength) to the size of the sprite used for obj_rope. This will be used later to tell GameMaker where each instance of obj_rope should be so that we can connect them in a line. Next, we create the object that will hold the obj_ropeHome rope. This is a static object that will not move, no matter how much the rope moves. This is connected to the first instance of obj_rope via a revolute joint. In GameMaker, a revolute joint is used in several ways: it can act as part of a motor, moving pistons; it can act as a joint on a ragdoll body; in this case, it acts as the connection between instances of obj_rope. A revolute joint allows the programmer to code its angle and torque; but for our purposes, this isn't necessary. We declared the objects that are connected via the joint as well as the anchor location, but the other values remain null. Once the rope holder (obj_ropeHome) and initial joint are set up, we can automate the creation of the rest. Using the repeat function, we can tell GameMaker to repeat a block of code a set number of times. In this case, this number is derived from how many instances of obj_rope can fit within the distance between the y origin of obj_ropeControl and the point to which you stretched it. We subtract 1 from this number as GameMaker will calculate too many in order to cover the distance in its entirety. The code that will be repeated does a few things at once. First, it increases the value of the ropeLength variable by 16 for each instance that is calculated. Then, GameMaker changes the value of rope1 (which creates an instance of obj_ropeHome) to that of rope2 (which creates an instance of obj_rope). The rope2 variable is then reestablished to create an instance of obj_rope, but also adds the new value of ropeLength so as to move its coordinates directly below those of the previous instance, thus creating a chain. This process is repeated until the set length of the overall rope is reached. There's more Each section of a rope is a physics object and acts in the physics world. By changing the physics settings, when initially creating the rope sections, you can see how they react to collisions. How far and how quickly the rope moves when pushed by another object is very much related to the difference between their densities. If you make the rope denser than the object colliding with it, the rope will move very little. If you reverse these values, you can cause the rope to flail about, wildly. Play around with the settings and see what happens, but when placing a rope or chain in a game, you really must consider what the rope and other objects are made of. It wouldn't seem right for a lead chain to be sent flailing about by a collision with a pillow; now would it? Summary This article introduces the physics system and demonstrates how GameMaker handles gravity, friction, and so on. Learn how to implement this system to make more realistic games. Resources for Article: Further resources on this subject: Getting to Know LibGDX [article] HTML5 Game Development – A Ball-shooting Machine with Physics Engine [article] Introducing GameMaker [article]
Read more
  • 0
  • 0
  • 21186

article-image-privacy-experts-discuss-gdpr-its-impact-and-its-future-on-beth-kindigs-tech-lightning-rounds-podcast
Savia Lobo
28 May 2019
9 min read
Save for later

Privacy Experts discuss GDPR, its impact, and its future on Beth Kindig’s Tech Lightning Rounds Podcast

Savia Lobo
28 May 2019
9 min read
User’s data was being compromised even before the huge Cambridge Analytica scandal was brought to light. On May 25th, 2018, when the GDPR first came into existence in the European Union for data protection and privacy, it brought in much power to individuals over their personal data and to simplify the regulatory environment for international businesses. GDPR recently completed one year and since its inception, these have highly helped in better data privacy regulation. These privacy regulations divided companies into data processors and data controllers. Any company who has customers in the EU must comply regardless of where the company is located. In episode 6 of Tech Lightning Rounds, Beth Kindig of Intertrust speaks to experts from three companies who have implemented GDPR. Robin Andruss, the Director of Privacy at Twilio, a leader in global communications that is uniquely positioned to handle data from text messaging sent inside its applications. Tomas Sander of Intertrust, the company that invented digital rights management and has been advocating for privacy for nearly 30 years. Katryna Dow, CEO of Meeco, a startup that introduces the concept of data control for digital life. Robin Andruss’ on Twilio’s stance on privacy Twilio provides messaging, voice, and video inside mobile and web applications for nearly 40,000 companies including Uber, Lyft, Yelp, Airbnb, Salesforce and many more. “Twilio is one of the leaders in the communications platform as a service space, where we power APIs to help telecommunication services like SMS and texting, for example. A good example is when you order a Lyft or an Uber and you’ll text with a Uber driver and you’ll notice that’s not really their phone number. So that’s an example of one of our services”, Andruss explains. Twilio includes “binding corporate rules”, the global framework around privacy. He says, for anyone who’s been in the privacy space for a long time, they know that it’s actually very challenging to reach this standard. Organizations need to work with a law firm or consultancy to make sure they are meeting a bar of privacy and actually have their privacy regulations and obligations agreed to and approved by their lead DPA, Data Protection Authority in the EU, which in Twilio’s case is the Irish DPC. “We treat everyone who uses Twilio services across the board the same, our corporate rules. One rule, we don’t have a different one for the US or the EU. So I’d say that they are getting GDPR level of privacy standards when you use Twilio”, Andruss said. Talking about the California Consumer Privacy Act (CCPA), Andruss said that it’s mostly more or less targeted towards advertising companies and companies that might sell data about individuals and make money off of it, like Intelius or Spokeo or those sort of services. Beth asked Andruss on “how concerned the rest of us should be about data and what companies can do internally to improve privacy measures” to which he said, “just think about, really, what you’re putting out there, and why, and this third party you’re giving your information to when you are giving it away”. Twilio’s “no-shenanigans” and “Wear your customers’ shoes” approach to privacy Twilio’s “No-shenigans” approach to privacy encourages employees to do the right thing for their end-users and customers. Andruss explained this with an example, “You might be in a meeting, and you can say, “Is that the right thing? Do we really wanna do that? Is that the right thing to do for our customers or is that shenanigany does it not feel right?”. The “Wear your customers’ shoes.” approach is, when Twilio builds a product or thinks about something, they think about how to do the right thing for their customers. This builds trust within the customers that the organization really cares about privacy and wants to do the right thing while customers use Twilio’s tools and services. Tomas Sander on privacy pre-GDPR and post-GDPR Tomas Sander started off by explaining the basics of GDPR, what it does, and how it can help users, and so on. He also cleared a common doubt that most people have about the reach of EU’s GDPR. He said, “One of the main things that the GDPR has done is that it has an extraterritorial reach. So GDPR not only applies to European companies, but to companies worldwide if they provide goods and services to European citizens”. GDPR has “made privacy a much more important issue for many organizations” due to which GDPR has huge fines for non-compliance and that has contributed for it to be taken seriously by companies globally. Because of data breaches, “security has become a boardroom issue for many companies. Now, privacy has also become a boardroom issue”, Sander adds. He said that GDPR has been extremely effective in setting the privacy debate worldwide. Although it’s a regulation in Europe, it’s been extremely effective through its global impact on organizations and on thinking of policymakers, what they wanna do about privacy in their countries. However, talking about positive impact, Sander said that data behemoths such as Google and Facebook are still collecting data from many, many different sources, aggregating it about users, and creating detailed profiles for the purpose of selling advertising, usually, so for profit. This is why the jury is still out! “And this practice of taking all this different data, from location data to smart home data, to their social media data and so on and using them for sophisticated user profiling, that practice hasn’t recognizably changed yet”, he added. Sander said he “recently heard data protection commissioners speak at a privacy conference in Washington, and they believe that we’re going to see some of these investigations conclude this summer. And hopefully then there’ll be some enforcement, and some of the commissioners certainly believe that there will be fines”. Sander’s suggestion for users who are not much into tech is,  “I think people should be deeply concerned about privacy.” He said they can access your web browsing activities, your searches, location data, the data shared on social media, facial recognition from images, and also these days IoT and smart home data that give people intimate insights into what’s happening in your home. With this data, the company can keep a tab on what you do and perhaps create a user profile. “A next step they could take is that they don’t only observe what you do and predict what the next step is you’re going to do, but they may also try to manipulate and influence what you do. And they would usually do that for profit motives, and that is certainly a major concern. So people may not even know, may not even realize, that they’re being influenced”. This is a major concern because it really questions “our individual freedom about… It really becomes about democracy”. Sander also talked about an incident that took place in Germany where its far-right party, “Alternative For Germany”, “Alternative für Deutschland” were able to use a Facebook feature that has been created for advertisers to help it achieve the best result in the federal election for any far right-wing party in Germany after World War 2. The feature that was being used here was a feature of “look-alike” audiences. Facebook helped this party to analyze the characteristics of the 300,000 users who had liked the “Alternative For Germany”, who had liked this party. Further, from these users, it created a “look-alike” audience of another 300,000 users that were similar in characteristics to those who had already liked this party, and then they were specifically targeting ads to this group. Katrina Dow on getting people digitally aware Dow thinks, “the biggest challenge right now is that people just don’t understand what goes on under the surface”. She explains how by a simple picture sharing of a child playing in a park can impact the child’s credit rating in the future.  She says, “People don’t understand the consequences of something that I do right now, that’s digital, and what it might impact some time in the future”. She also goes on explaining how to help people make a more informed choice around the services they wanna use or argue for better rights in terms of those services, so those consequences don’t happen. Dow also discusses one of the principles of the GDPR, which is designing privacy into the applications or websites as the foundation of the design, rather than adding privacy as an afterthought. Beth asked if GDPR, which introduces some level of control, is effective. To which Dow replied, “It’s early days. It’s not working as intended right now.” Dow further explained, “the biggest problem right now is the UX level is just not working. And organizations that have been smart in terms of creating enormous amounts of friction are using that to their advantage.” “They’re legally compliant, but they have created that compliance burden to be so overwhelming, that I agree or just anything to get this screen out of the way is driving the behavior”, Dow added. She says that a part of GDPR is privacy by design, but what we haven’t seen the surface to the UX level. “And I think right now, it’s just so overwhelming for people to even work out, “What’s the choice?” What are they saying yes to? What are they saying no to? So I think, the underlying components are there and from a legal framework. Now, how do we move that to what we know is the everyday use case, which is how you interact with those frameworks”, Dow further added. To listen to this podcast and know more about this in detail, visit Beth Kindig’s official website. Github Sponsors: Could corporate strategy eat FOSS culture for dinner? Mozilla and Google Chrome refuse to support Gab’s Dissenter extension for violating acceptable use policy SnapLion: An internal tool Snapchat employees abused to spy on user data
Read more
  • 0
  • 0
  • 21178

article-image-internationalization-and-localization
Packt
03 Mar 2018
16 min read
Save for later

Internationalization and localization

Packt
03 Mar 2018
16 min read
In this article by Dmitry Sheiko, the author of the book, Cross Platform Desktop Application Development: Electron, Node, NW.js and React, will cover the concept of Internationalization and localization and will be also covering context menu and system clipboard in detail. Internationalization, often abbreviated as i18n, implies a particular software design capable of adapting to the requirements of target local markets. In other words if we want to distribute our application to the markets other than USA we need to take care of translations, formatting of datetime, numbers, addresses, and such. (For more resources related to this topic, see here.) Date format by country Internationalization is a cross-cutting concern. When you are changing the locale it usually affects multiple modules. So I suggest going with the observer pattern that we already examined while working on DirService'. The ./js/Service/I18n.js file contains the following code: const EventEmitter = require( "events" ); class I18nService extends EventEmitter { constructor(){ super(); this.locale = "en-US"; } Internationalization and localization [ 2 ] notify(){ this.emit( "update" ); } } As you see, we can change the locale by setting a new value to locale property. As soon as we call notify method, then all the subscribed modules immediately respond. But locale is a public property and therefore we have no control on its access and mutation. We can fix it by using overloading. The ./js/Service/I18n.js file contains the following code: //... constructor(){ super(); this._locale = "en-US"; } get locale(){ return this._locale; } set locale( locale ){ // validate locale... this._locale = locale; } //... Now if we access locale property of I18n instance it gets delivered by the getter (get locale). When setting it a value, it goes through the setter (set locale). Thus we can add extra functionality such as validation and logging on property access and mutation. Remember we have in the HTML, a combobox for selecting language. Why not give it a view? The ./js/View/LangSelector.j file contains the following code: class LangSelectorView { constructor( boundingEl, i18n ){ boundingEl.addEventListener( "change", this.onChanged.bind( this ), false ); this.i18n = i18n; } onChanged( e ){ const selectEl = e.target; this.i18n.locale = selectEl.value; this.i18n.notify(); } } Internationalization and localization [ 3 ] exports.LangSelectorView = LangSelectorView; In the preceding code, we listen for change events on the combobox. When the event occurs we change locale property of the passed in I18n instance and call notify to inform the subscribers. The ./js/app.js file contains the following code: const i18nService = new I18nService(), { LangSelectorView } = require( "./js/View/LangSelector" ); new LangSelectorView( document.querySelector( "[data-bind=langSelector]" ), i18nService ); Well, we can change the locale and trigger the event. What about consuming modules? In FileList view we have static method formatTime that formats the passed in timeString for printing. We can make it formated in accordance with currently chosen locale. The ./js/View/FileList.js file contains the following code: constructor( boundingEl, dirService, i18nService ){ //... this.i18n = i18nService; // Subscribe on i18nService updates i18nService.on( "update", () => this.update( dirService.getFileList() ) ); } static formatTime( timeString, locale ){ const date = new Date( Date.parse( timeString ) ), options = { year: "numeric", month: "numeric", day: "numeric", hour: "numeric", minute: "numeric", second: "numeric", hour12: false }; return date.toLocaleString( locale, options ); } update( collection ) { //... this.el.insertAdjacentHTML( "beforeend", `<li class="file-list__li" data-file="${fInfo.fileName}"> <span class="file-list__li__name">${fInfo.fileName}</span> <span class="filelist__li__size">${filesize(fInfo.stats.size)}</span> <span class="file-list__li__time">${FileListView.formatTime( fInfo.stats.mtime, this.i18n.locale )}</span> </li>` ); //... } //... In the constructor, we subscribe for I18n update event and update the file list every time the locale changes. Static method formatTime converts passed in string into a Date object and uses Date.prototype.toLocaleString() method to format the datetime according to a given locale. This method belongs to so called The ECMAScript Internationalization API (http://norbertlindenberg.com/2012/12/ecmascript-internationalization-api/index .html). The API describes methods of built-in object String, Date and Number designed to format and compare localized data. But what it really does is formatting a Date instance with toLocaleString for the English (United States) locale ("en-US") and it returns the date as follows: 3/17/2017, 13:42:23 However if we feed to the method German locale ("de-DE") we get quite a different result: 17.3.2017, 13:42:23 To put it into action we set an identifier to the combobox. The ./index.html file contains the following code: .. <select class="footer__select" data-bind="langSelector"> .. And of course, we have to create an instance of I18n service and pass it in LangSelectorView and FileListView: ./js/app.js // ... const { I18nService } = require( "./js/Service/I18n" ), { LangSelectorView } = require( "./js/View/LangSelector" ), i18nService = new I18nService(); new LangSelectorView( document.querySelector( "[data-bind=langSelector]" ), i18nService ); // ... new FileListView( document.querySelector( "[data-bind=fileList]" ), dirService, i18nService ); Now we start the application. Yeah! As we change the language in the combobox the file modification dates adjust accordingly: Multilingual support Localization dates and number is a good thing, but it would be more exciting to provide translation to multiple languages. We have a number of terms across the application, namely the column titles of the file list and tooltips (via title attribute) on windowing action buttons. What we need is a dictionary. Normally it implies sets of token translation pairs mapped to language codes or locales. Thus when you request from the translation service a term, it can correlate to a matching translation according to currently used language/locale. Here I have suggested making the dictionary as a static module that can be loaded with the required function. The ./js/Data/dictionary.js file contains the following code: exports.dictionary = { "en-US": { NAME: "Name", SIZE: "Size", MODIFIED: "Modified", MINIMIZE_WIN: "Minimize window", Internationalization and localization [ 6 ] RESTORE_WIN: "Restore window", MAXIMIZE_WIN: "Maximize window", CLOSE_WIN: "Close window" }, "de-DE": { NAME: "Dateiname", SIZE: "Grösse", MODIFIED: "Geändert am", MINIMIZE_WIN: "Fenster minimieren", RESTORE_WIN: "Fenster wiederherstellen", MAXIMIZE_WIN: "Fenster maximieren", CLOSE_WIN: "Fenster schliessen" } }; So we have two locales with translations per term. We are going to inject the dictionary as a dependency into our I18n service. The ./js/Service/I18n.js file contains the following code: //... constructor( dictionary ){ super(); this.dictionary = dictionary; this._locale = "en-US"; } translate( token, defaultValue ) { const dictionary = this.dictionary[ this._locale ]; return dictionary[ token ] || defaultValue; } //... We also added a new method translate that accepts two parameters: token and default translation. The first parameter can be one of the keys from the dictionary like NAME. The second one is guarding value for the case when requested token does not yet exist in the dictionary. Thus we still get a meaningful text at least in English. Let's see how we can use this new method. The ./js/View/FileList.js file contains the following code: //... update( collection ) { this.el.innerHTML = `<li class="file-list__li file-list__head"> <span class="file-list__li__name">${this.i18n.translate( "NAME", "Name" )}</span> <span class="file-list__li__size">${this.i18n.translate( "SIZE", Internationalization and localization [ 7 ] "Size" )}</span> <span class="file-list__li__time">${this.i18n.translate( "MODIFIED", "Modified" )}</span> </li>`; //... We change in FileList view hardcoded column titles with calls for translate method of I18n instance, meaning that every time view updates it receives the actual translations. We shall not forget about TitleBarActions view where we have windowing action buttons. The ./js/View/TitleBarActions.js file contains the following code: constructor( boundingEl, i18nService ){ this.i18n = i18nService; //... // Subscribe on i18nService updates i18nService.on( "update", () => this.translate() ); } translate(){ this.unmaximizeEl.title = this.i18n.translate( "RESTORE_WIN", "Restore window" ); this.maximizeEl.title = this.i18n.translate( "MAXIMIZE_WIN", "Maximize window" ); this.minimizeEl.title = this.i18n.translate( "MINIMIZE_WIN", "Minimize window" ); this.closeEl.title = this.i18n.translate( "CLOSE_WIN", "Close window" ); } Here we add method translate, which updates button title attributes with actual translations. We subscribe for i18n update event to call the method every time user changes locale:   Context menu Well, with our application we can already navigate through the file system and open files. Yet, one might expect more of a File Explorer. We can add some file related actions like delete, copy/paste. Usually these tasks are available via the context menu, what gives us a good opportunity to examine how to make it with NW.js. With the environment integration API we can create an instance of system menu (http://docs.nwjs.io/en/latest/References/Menu/). Then we compose objects representing menu items and attach them to the menu instance (http://docs.nwjs.io/en/latest/References/MenuItem/). This menu can be shown in an arbitrary position: const menu = new nw.Menu(), menutItem = new nw.MenuItem({ label: "Say hello", click: () => console.log( "hello!" ) }); menu.append( menu ); menu.popup( 10, 10 ); Yet our task is more specific. We have to display the menu on the right mouse click in the position of the cursor. That is, we achieve by subscribing a handler to contextmenu DOM event: document.addEventListener( "contextmenu", ( e ) => { console.log( `Show menu in position ${e.x}, ${e.y}` ); }); Now whenever we right-click within the application window the menu shows up. It's not exactly what we want, isn't it? We need it only when the cursor resides within a particular region. For an instance, when it hovers a file name. That means we have to test if the target element matches our conditions: document.addEventListener( "contextmenu", ( e ) => { const el = e.target; if ( el instanceof HTMLElement && el.parentNode.dataset.file ) { console.log( `Show menu in position ${e.x}, ${e.y}` ); } }); Here we ignore the event until the cursor hovers any cell of file table row, given every row is a list item generated by FileList view and therefore provided with a value for data file attribute. This passage explains pretty much how to build a system menu and how to attach it to the file list. But before starting on a module capable of creating menu, we need a service to handle file operations. The ./js/Service/File.js file contains the following code: const fs = require( "fs" ), path = require( "path" ), // Copy file helper cp = ( from, toDir, done ) => { const basename = path.basename( from ), to = path.join( toDir, basename ), write = fs.createWriteStream( to ) ; fs.createReadStream( from ) .pipe( write ); write .on( "finish", done ); }; class FileService { Internationalization and localization [ 10 ] constructor( dirService ){ this.dir = dirService; this.copiedFile = null; } remove( file ){ fs.unlinkSync( this.dir.getFile( file ) ); this.dir.notify(); } paste(){ const file = this.copiedFile; if ( fs.lstatSync( file ).isFile() ){ cp( file, this.dir.getDir(), () => this.dir.notify() ); } } copy( file ){ this.copiedFile = this.dir.getFile( file ); } open( file ){ nw.Shell.openItem( this.dir.getFile( file ) ); } showInFolder( file ){ nw.Shell.showItemInFolder( this.dir.getFile( file ) ); } }; exports.FileService = FileService; What's going on here? FileService receives an instance of DirService as a constructor argument. It uses the instance to obtain the full path to a file by name ( this.dir.getFile( file ) ). It also exploits notify method of the instance to request all the views subscribed to DirService to update. Method showInFolder calls the corresponding method of nw.Shell to show the file in the parent folder with the system file manager. As you can recon method remove deletes the file. As for copy/paste we do the following trick. When user clicks copy we store the target file path in property copiedFile. So when user next time clicks paste we can use it to copy that file to the supposedly changed current location. Method open evidently opens file with the default associated program. That is what we do in FileList view directly. Actually this action belongs to FileService. So we rather refactor the view to use the service. The ./js/View/FileList.js file contains the following code: constructor( boundingEl, dirService, i18nService, fileService ){ this.file = fileService; //... } Internationalization and localization [ 11 ] bindUi(){ //... this.file.open( el.dataset.file ); //... } Now we have a module to handle context menu for a selected file. The module will subscribe for contextmenu DOM event and build a menu when user right clicks on a file. This menu will contain items Show Item in the Folder, Copy, Paste, and Delete. Whereas copy and paste are separated from other items with delimiters. Besides, Paste will be disabled until we store a file with copy. Further goes the source code. The ./js/View/ContextMenu.js file contains the following code: class ConextMenuView { constructor( fileService, i18nService ){ this.file = fileService; this.i18n = i18nService; this.attach(); } getItems( fileName ){ const file = this.file, isCopied = Boolean( file.copiedFile ); return [ { label: this.i18n.translate( "SHOW_FILE_IN_FOLDER", "Show Item in the Folder" ), enabled: Boolean( fileName ), click: () => file.showInFolder( fileName ) }, { type: "separator" }, { label: this.i18n.translate( "COPY", "Copy" ), enabled: Boolean( fileName ), click: () => file.copy( fileName ) }, { label: this.i18n.translate( "PASTE", "Paste" ), enabled: isCopied, click: () => file.paste() }, { type: "separator" }, { Internationalization and localization [ 12 ] label: this.i18n.translate( "DELETE", "Delete" ), enabled: Boolean( fileName ), click: () => file.remove( fileName ) } ]; } render( fileName ){ const menu = new nw.Menu(); this.getItems( fileName ).forEach(( item ) => menu.append( new nw.MenuItem( item ))); return menu; } attach(){ document.addEventListener( "contextmenu", ( e ) => { const el = e.target; if ( !( el instanceof HTMLElement ) ) { return; } if ( el.classList.contains( "file-list" ) ) { e.preventDefault(); this.render() .popup( e.x, e.y ); } // If a child of an element matching [data-file] if ( el.parentNode.dataset.file ) { e.preventDefault(); this.render( el.parentNode.dataset.file ) .popup( e.x, e.y ); } }); } } exports.ConextMenuView = ConextMenuView; So in ConextMenuView constructor, we receive instances of FileService and I18nService. During the construction we also call attach method that subscribes for contextmenu DOM event, creates the menu and shows it in the position of the mouse cursor. The event gets ignored unless the cursor hovers a file or resides in empty area of the file list component. When user right clicks the file list, the menu still appears, but with all items disable except paste (in case a file was copied before). Method render create an instance of menu and populates it with nw.MenuItems created by getItems method. The method creates an array representing menu items. Elements of the array are object literals. Internationalization and localization [ 13 ] Property label accepts translation for item caption. Property enabled defines the state of item depending on our cases (whether we have copied file or not, whether the cursor on a file or not). Finally property click expects the handler for click event. Now we need to enable our new components in the main module. The ./js/app.js file contains the following code: const { FileService } = require( "./js/Service/File" ), { ConextMenuView } = require( "./js/View/ConextMenu" ), fileService = new FileService( dirService ); new FileListView( document.querySelector( "[data-bind=fileList]" ), dirService, i18nService, fileService ); new ConextMenuView( fileService, i18nService ); Let's now run the application, right-click on a file and voilà! We have the context menu and new file actions. System clipboard Usually Copy/Paste functionality involves system clipboard. NW.js provides an API to control it (http://docs.nwjs.io/en/latest/References/Clipboard/). Unfortunately it's quite limited, we cannot transfer an arbitrary file between applications, what you may expect of a file manager. Yet some things we are still available to us. Transferring text In order to examine text transferring with the clipboard we modify the method copy of FileService: copy( file ){ this.copiedFile = this.dir.getFile( file ); const clipboard = nw.Clipboard.get(); clipboard.set( this.copiedFile, "text" ); } What does it do? As soon as we obtained file full path, we create an instance of nw.Clipboard and save the file path there as a text. So now, after copying a file within the File Explorer we can switch to an external program (for example, a text editor) and paste the copied path from the clipboard. Transferring graphics It doesn't look very handy, does it? It would be more interesting if we could copy/paste a file. Unfortunately NW.js doesn't give us many options when it comes to file exchange. Yet we can transfer between NW.js application and external programs PNG and JPEG images. The ./js/Service/File.js file contains the following code: //... copyImage( file, type ){ const clip = nw.Clipboard.get(), // load file content as Base64 data = fs.readFileSync( file ).toString( "base64" ), // image as HTML html = `<img src="file:///${encodeURI( data.replace( /^//, "" ) )}">`; // write both options (raw image and HTML) to the clipboard clip.set([ Internationalization and localization [ 16 ] { type, data: data, raw: true }, { type: "html", data: html } ]); } copy( file ){ this.copiedFile = this.dir.getFile( file ); const ext = path.parse( this.copiedFile ).ext.substr( 1 ); switch ( ext ){ case "jpg": case "jpeg": return this.copyImage( this.copiedFile, "jpeg" ); case "png": return this.copyImage( this.copiedFile, "png" ); } } //... We extended our FileService with private method copyImage. It reads a given file, converts its contents in Base64 and passes the resulting code in a clipboard instance. In addition, it creates HTML with image tag with Base64-encoded image in data Uniform Resource Identifier (URI). Now after copying an image (PNG or JPEG) in the File Explorer, we can paste it in an external program such as graphical editor or text processor. Receiving text and graphics We've learned how to pass a text and graphics from our NW.js application to external programs. But how can we receive data from outside? As you can guess it is accessible through get method of nw.Clipboard. Text can be retrieved that simple: const clip = nw.Clipboard.get(); console.log( clip.get( "text" ) ); When graphic is put in the clipboard we can get it with NW.js only as Base64-encoded content or as HTML. To see it in practice we add a few methods to FileService. The ./js/Service/File.js file contains the following code: //... hasImageInClipboard(){ const clip = nw.Clipboard.get(); return clip.readAvailableTypes().indexOf( "png" ) !== -1; } pasteFromClipboard(){ const clip = nw.Clipboard.get(); if ( this.hasImageInClipboard() ) { Internationalization and localization [ 17 ] const base64 = clip.get( "png", true ), binary = Buffer.from( base64, "base64" ), filename = Date.now() + "--img.png"; fs.writeFileSync( this.dir.getFile( filename ), binary ); this.dir.notify(); } } //... Method hasImageInClipboard checks if the clipboard keeps any graphics. Method pasteFromClipboard takes graphical content from the clipboard as Base64-encoded PNG. It converts the content into binary code, writes into a file and requests DirService subscribers to update. To make use of these methods we need to edit ContextMenu view. The ./js/View/ContextMenu.js file contains the following code: getItems( fileName ){ const file = this.file, isCopied = Boolean( file.copiedFile ); return [ //... { label: this.i18n.translate( "PASTE_FROM_CLIPBOARD", "Paste image from clipboard" ), enabled: file.hasImageInClipboard(), click: () => file.pasteFromClipboard() }, //... ]; } We add to the menu a new item Paste image from clipboard, which is enabled only when there is any graphic in the clipboard. Summary In this article, we have covered concept of internationalization and localization and also covered context menu and system clipboard in detail. Resources for Article:   Further resources on this subject: [article] [article] [article]
Read more
  • 0
  • 0
  • 21172

article-image-what-the-us-china-tech-and-ai-arms-race-means-for-the-world-frederick-kempe-at-davos-2019
Sugandha Lahoti
24 Jan 2019
6 min read
Save for later

What the US-China tech and AI arms race means for the world - Frederick Kempe at Davos 2019

Sugandha Lahoti
24 Jan 2019
6 min read
Atlantic Council CEO, Frederick Kempe spoke in the World Economic Forum (WEF) in Davos, Switzerland. He talked about the Cold war between the US and China and why the countries need to co-operate and not compete in the tech arms race, in his presentation Future Frontiers of Technology Control. He began his presentation by posing a question set forth by Former US Foreign National Security Advisor Stephen Hadley, “Can the incumbent US and insurgent China become strategic collaborators and strategic competitors in this tech space at the same time?” Read also: The New AI Cold War Between China and the USA Kempe’s three framing arguments Geopolitical Competition This fusion of tech breakthroughs blurring lines of the physical, digital, and biological space is reaching an inflection point that makes it already clear that they will usher in a revolution that will determine the shape of the global economy. It will also determine which nations and political constructs may assume the commanding heights of global politics in the coming decade. Technological superiority Over the course of history, societies that dominated economic innovation and progress have dominated in international relations — from military superiority to societal progress and prosperity. On balance, technological progress has contributed to higher standards of living in most parts of the world; however, the disproportionate benefit goes to first movers. Commanding Heights The technological arms race for supremacy in the fourth industrial revolution has essentially become a two-horse contest between the United States and China. We are in the early stages of this race, but how it unfolds and is conducted will do much to shape global human relations. The shift in 2018 in US-China relations from a period of strategic engagement to greater strategic competition has also significantly accelerated the Tech arms race. China vs the US: Why China has the edge? It was Vladimir Putin, President of the Russian Federation who said that “The one who becomes the leader in Artificial Intelligence, will rule the world.” In 2017, DeepMind’s AlphaGo defeated a Chinese master in Go, a traditional Chinese game. Following this defeat, China launched an ambitious roadmap, called the next generation AI plan. The goal was to become the Global leader in AI by 2030 in theory, technology, and application. On current trajectories, in the four primary areas of AI over the next 5 years, China will emerge the winner of this new technology race. Kempe also quotes, author of the book, AI superpowers, Kai-fu Lee who argues that harnessing of the power of AI today- the electricity of the 21st century- requires abundant data, hungry entrepreneurs, AI scientists, and an AI friendly policy. He believes that China has the edge in all of these. The current AI has translated from out of the box research, where the US has expertise in, to actual implementation, where China has the edge. Per, Kai-fu Lee China already has the edge in entrepreneurship, data, and government support, and is rapidly catching up to the U.S. in expertise. The world has translated from the age of world-leading expertise (US department) to the age of data, where China wins hands down. Economists call China the Saudi Arabia of Data and with that as the fuel for AI, it has an enormous advantage. The Chinese government without privacy restrictions can gain and use data in a manner that is out of reach of any democracy. Kemper concludes that the nature of this technological arms contest may favor insurgent China rather than the incumbent US. What are the societal implications of this tech cold war He also touched upon the societal implications of AI and the cold war between the US and China. A number of jobs will be lost by 2030. Quoting from Kai-fu Lee’s book, Kempe says that Job displacement caused by artificial intelligence and advanced robotics could possibly displace up to 54 million US workers which comprise 30% of the US labor force. It could also displace up to 100 million Chinese workers which are 12% of the Chinese labor force. What is the way forward with these huge societal implications of a bi-lateral race underway? Kempe sees three possibilities. A sloppy Status Quo A status quo where China and the US will continue to cooperate but increasingly view each other with suspicion. They will manage their rising differences and distrust imperfectly, never bridging them entirely, but also not burning bridges, either between researchers, cooperations, or others. Techno Cold War China and the US turn the global tech contest into more of a zero-sum battle for global domination. They organize themselves in a manner that separates their tech sectors from each other and ultimately divides up the world. Collaborative Future - the one we hope for Nicholas Thompson and Ian Bremmer argued in a wired interview that despite the two countries’ societal difference, the US should wrap China in a tech embrace. The two countries should work together to establish international standards to ensure that the algorithms governing people’s lives and livelihoods are transparent and accountable. They should recognize that while the geopolitics of technological change is significant, even more important will be the challenges AI poses to all societies across the world in terms of job automation and the social disruptions that may come with it. It may sound utopian to expect US and China to cooperate in this manner, but this is what we should hope for. To do otherwise would be self-defeating and at the cost of others in the global community which needs our best thinking to navigate the challenges of the fourth industrial revolution. Kempe concludes his presentation with a quote by Henry Kissinger, Former US Secretary of State and National Security Advisor, “We’re in a position in which the peace and prosperity of the world depend on whether China and the US can find a method to work together, not always in agreement, but to handle our disagreements...This is the key problem of our time.” Note: All images in this article are taken from Frederick Kempe’s presentation. We must change how we think about AI, urge AI founding fathers Does AI deserve to be so Overhyped? Alarming ways governments are using surveillance tech to watch you
Read more
  • 0
  • 0
  • 21166

article-image-ai-distilled-31-evolving-boundaries-and-opportunities
Merlyn Shelley
08 Jan 2024
14 min read
Save for later

AI_Distilled #31: Evolving Boundaries and Opportunities

Merlyn Shelley
08 Jan 2024
14 min read
Dive deeper into the world of AI innovation and stay ahead of the AI curve! Subscribe to our AI_Distilled newsletter for the latest insights. Don't miss out – sign up today!👋 Hello ,🎉 Joyous 2024! Wishing you a year as delightful as your dreams!  Dive into the new year with our outstanding edition, filled with essential features to boost your AI practice. “The speed at which people will be able to come up with an idea, to test the idea, to make something, it’s going to be so accelerated…You don’t need to have a degree in computer science to do that.” - Matthew Candy, IBM’s global managing partner for generative AI Coding without coding is a revolutionary idea indeed. What might have been previously perceived as unbelievable is a living reality and new features like Github’s Copilot Chat make it all the more seamless.  The real possibilities of AI expand far beyond computing, with the technology making waves in healthcare, finance, and supply chain management. Starting from this edition, we’ll bring you fresh updates from each of these sectors, so stay tuned! Let's kick things off by tapping into the latest news and developments. AI Launches & Industry Updates:  Microsoft Copilot Integrates with GenAI Music App Suno for Song Composition Google Plans Potential Layoffs Amidst AI Integration in Ad Sales GitHub Expands Copilot Chat Availability for Developers AI in Healthcare: AI Streamlining Health Insurance Shopping Process Revolutionizing Healthcare with AI Stethoscope on Smartphones Generative AI's Impact on Mental Health Counseling AI in Finance: Next-Gen Banks to Leverage AI for Financial Influence and Support Invest Qatar Introduces Cutting-Edge Azure OpenAI GPT-Powered Chatbot AI in Supply Chain Management: AI Safeguards Supply Chains Amidst Holiday Challenges AI-SaaS Integration Revolutionizes E-commerce Analytics Here are some handpicked GPT and LLM resources, tutorials, and secret knowledge that’ll come in handy for your next project: Understanding the Prompt Development Life Cycle Building Platforms with LLMs: Overcoming Challenges in Summarization as a Service Understanding the Risks of Prompt Injection in LLM Applications Creating an Open Source LLM Recommender System: Mastering Prompt Iteration and Optimization Looking for hands-on tips and strategies straight from the developer community? We’ve got you covered: Exploring Google's Gemini Pro Vision LLM with Javascript: A Practical Guide Accelerating AI Application Productionization: A Guide with SageMaker JumpStart, Amazon Bedrock, and TruEra Quantizing LLMs with Activation-aware Weight Quantization (AWQ) Unlocking Your MacBook's AI Potential: Running 70B LLM Models Without Quantization Check out our curated list of smoking hot GitHub repositories: Giskard-AI/giskard CopilotKit/CopilotKit chengzeyi/stable-fast ml-explore/mlx  📥 Feedback on the Weekly EditionQ: How can we foster effective collaboration between humans and AI systems, ensuring that AI complements human skills and enhances productivity without causing job displacement or widening societal gaps?Share your valued opinions discreetly! Your insights could shine in our next issue for the 39K-strong AI community. Join the conversation! 🗨️✨ As a big thanks, get our bestselling "Interactive Data Visualization with Python - Second Edition" in PDF. Let's make AI_Distilled even more awesome! 🚀 Jump on in! Share your thoughts and opinions here! Writer’s Credit: Special shout-out to Vidhu Jain for their valuable contribution to this week’s newsletter content!  Cheers,  Merlyn Shelley  Editor-in-Chief, Packt  SignUp | Advertise | Archives⚡ TechWave: AI/GPT News & AnalysisNew Launches & Industry Updates: ⭐ Microsoft Copilot Integrates with GenAI Music App Suno for Song Composition: Microsoft Copilot has partnered with GenAI music app Suno, enabling users to create complete songs including lyrics, instrumentals, and singing voices. Accessible via Microsoft Edge, the integration aims to make music creation inclusive and enjoyable. However, ethical and legal concerns persist, with some artists uncomfortable with AI algorithms learning from their work without consent or compensation. Suno attempts to address such issues by blocking certain prompts and preventing the generation of covers using existing lyrics. Read Microsoft’s official blog here. ⭐ Google Plans Potential Layoffs Amidst AI Integration in Ad Sales: Google is reportedly considering laying off around 30,000 employees within its ad sales division due to the implementation of internal AI, aiming for improved operational efficiency. The restructuring primarily targets the ad sales team, reflecting Google's exploration of AI benefits in operational processes. Earlier in 2023, Google had already laid off 12,000 employees, emphasizing the need for organizational adaptation amidst evolving global dynamics. Read about other significant 2023 layoffs here. ⭐ GitHub Expands Copilot Chat Availability for Developers: GitHub is extending the availability of Copilot Chat, a programming-centric chatbot powered by GPT-4, to all users. The tool was initially launched for Copilot for Business subscribers and later in beta for $10 per month users. Integrated into Microsoft's IDEs, Visual Studio Code and Visual Studio, it's included in GitHub Copilot's paid tiers and free for verified teachers, students, and maintainers of specific open-source projects. Developers can prompt Copilot Chat in natural language, seeking real-time guidance on code-related tasks. Know more about Copilot Chat here. AI in Healthcare: ⭐ AI Streamlining Health Insurance Shopping Process: Companies are utilizing AI to simplify the often complex and tedious task of shopping for health insurance, aiming to guide consumers to better and more affordable options. With many Americans sticking to their health plans due to the difficulty of predicting their future healthcare needs, AI-powered tools gather individual information and predict the most suitable health plans. Alight, a cloud-based HR services provider, reports that 95% of its served employers use AI technology, including a virtual assistant, for employee health benefits selection.  ⭐ Revolutionizing Healthcare with AI Stethoscope on Smartphones: A startup AI Health Highway is addressing the challenge of limited access to specialists in healthcare by introducing an innovative solution, AI Steth, which combines traditional stethoscope use with cutting-edge signal processing and AI. Targeting the early detection and prediction of heart and lung disorders, the device transforms sound patterns into visual representations on smartphones, allowing non-specialists like family physicians and nurses to examine patients effectively. AI Steth has shown exceptional accuracy in murmur detection, paving the way for more objective and efficient diagnoses. Discover AI Health Highway’s work here. ⭐ Generative AI's Impact on Mental Health Counseling: Generative AI is finding use in mental health counseling, sparking discussions about its potential to assist or even replace human therapists. Recent research testing ChatGPT on mental health counseling questions has raised questions about the technology's role in therapy. AI therapy has evolved from basic chatbots to sophisticated entities capable of nuanced emotional responses, offering accessible mental health support 24/7. While the benefits are evident, challenges such as risk, coverage, and ethical considerations must be addressed for responsible implementation.  AI in Finance: ⭐ Next-Gen Banks to Leverage AI for Financial Influence and Support: Experts predict that next-generation banks will harness generative AI to impact various aspects of financial services, ranging from influencing customer decisions to identifying vulnerable clients. Tom Merry, Head of Banking Strategy at Accenture, suggests that generative AI could significantly influence banking operations, touching nearly every aspect. While the UK banking industry has been utilizing AI for fraud detection and risk analysis, the introduction of generative AI, capable of creating novel solutions based on extensive data, is gaining traction. ⭐ Invest Qatar Introduces Cutting-Edge Azure OpenAI GPT-Powered Chatbot: Invest Qatar, in collaboration with Microsoft, has launched Ai.SHA, an innovative AI-powered chatbot utilizing GPT capabilities through the Azure OpenAI service. This move positions Invest Qatar as a pioneer among investment promotion agencies globally, embracing advanced technology to transform interactions between investors and businesses in Qatar. Ai.SHA acts as a comprehensive resource, providing information on business opportunities, the investment ecosystem, and business setup in Qatar.  AI in Supply Chain Management: ⭐ AI Safeguards Supply Chains Amidst Holiday Challenges: Businesses face unique challenges in managing complex supply chains amid the holiday season, from counterfeit airplane parts to recalls affecting festive foods. The reliance on suppliers underscores the need for transparency and visibility to prevent disruptions caused by supplier misconduct. Leveraging AI in contracts offers a solution, allowing businesses to streamline due diligence, enhance visibility, conduct predictive analytics, and align with environmental, social, and governance (ESG) regulations. AI-powered contracts emerge as vital tools to proactively address supply chain challenges and ensure customer trust during the holiday season and beyond. ⭐ AI-SaaS Integration Revolutionizes E-commerce Analytics: In the logistics sector, where precision and speed are critical, SaaS coupled with AI is transforming traditional approaches. This integration allows for real-time data processing and learning from it, offering unprecedented insights and optimization capabilities. Learn how AI-SaaS integration streamlines inventory, boosts operational efficiency, and fortifies against fraud, becoming the recipe for e-commerce success in a hypercompetitive landscape.  🔮 Expert Insights from Packt Community Architectural Patterns and Techniques for Developing IoT Solutions - By Jasbir Singh Dhaliwal Unique requirements of IoT use cases IoT use cases tend to have very unique requirements concerning power consumption, bandwidth, analytics, and more. Additionally, the inherent complexity of IoT implementations (computationally challenged field devices on one end of the spectrum vis-à-vis almost infinite capacity of the cloud on the other) forces architects to make difficult architectural decisions and implementation choices. Before presenting the various IoT patterns, it is worth mentioning the unique expectations from IoT architectures that are different from non-IoT architectures: Sensing events and actuation commands have a wide range of latency expectations – from real-time to fire and forget. Data analysis results need to be reported/visualized/consumed on a variety of consumer devices – mobiles, desktops, tablets, and more. Similarly, data consumers have diverse backgrounds, data needs, and application roles (personas). One is often forced to integrate with legacy as well as cutting-edge devices and/or external systems – very few trivial use cases have isolated/standalone architectures. There is a considerable difference in the way the data is extracted from legacy versus non-legacy systems – legacy systems may internally collate the data and then push it to the external port (file transfer), whereas newer systems may push the data in a continuous stream (time-series data). This variability is one of the critical considerations when choosing a particular IoT architectural pattern. Varied deployment requirements – edge, on-premise, hybrid, the cloud, and more. Adherence to strict regulatory compliances, especially in medical and aeronautical domains. There are expectations considering immediate payback, return on investment (ROI), business outcomes, and new service business models. Continuous innovation, which results in new services or offerings (especially by cloud vendors), forcing IoT architectures to be in continuous sync mode with these new offerings or services. This is an excerpt from the book Architectural Patterns and Techniques for Developing IoT Solutions written by Jasbir Singh Dhaliwal and published in Sep ‘23. To see what's inside the book, read the entire chapter here or try a 7-day free trial to access the full Packt digital library. To discover more, click the button below.   Read through the Chapter 1 unlocked here...  🌟 Secret Knowledge: AI/LLM Resources⭐ Understanding the Prompt Development Life Cycle: Explore PDLC and gain insights into how prompt engineering mirrors software development. The primer unfolds a step-by-step guide, beginning with the Initial Build phase where an imperfect prompt is crafted, incorporating techniques like zero-shot and few-shot. The Optimization stage strategically refines prompts based on historical data. Finally, the Fine-tune phase demonstrates the refinement of models, emphasizing the importance of continuous tracking. ⭐ Building Platforms with LLMs: Overcoming Challenges in Summarization as a Service: Get to know more about Summarization as a Service, a platform designed by a Microsoft team for Viva Engage. Learn about the complexities of prompt design, ensuring accuracy and grounding, addressing privacy and compliance concerns, managing performance, cost, and availability of LLM services, and integrating outputs seamlessly with the Copilot and other Viva Engage features.  ⭐ Understanding the Risks of Prompt Injection in LLM Applications: Explore the intricacies of prompt injection in LLM applications. The author emphasizes the critical security implications and potential impacts, citing the OWASP Top 10 for LLM Applications. Drawing parallels to injection vulnerabilities like A03 in traditional security, the article illustrates potential risks through a thought experiment involving a robotic server.  ⭐ Creating an Open Source LLM Recommender System: Mastering Prompt Iteration and Optimization: Open Recommender is an open-source YouTube video recommendation system adept at tailoring content to your interests based on Twitter feed analysis. Discover its data pipeline, utilizing GPT-4, and the transition towards cost-effective open-source models using OpenPipe. Explore the challenges faced during prompt iteration, with a focus on better prompt engineering tools, including the introduction of a TypeScript library, Prompt Iteration Assistant.    🔛 Masterclass: AI/LLM Tutorials⭐ Exploring Google's Gemini Pro Vision LLM with Javascript: A Practical Guide: The blog introduces the concept of multi-modal LLMs capable of interpreting various data modes, including images. Learn how to utilize Google's multi-modal Gemini Pro Vision LLM with Javascript. The tutorial guides you through creating an AI-powered nutrition-fact explainer app using the newly released LLM. The tutorial covers prerequisites, such as installing node.js and obtaining a Gemini LLM API key.  ⭐ Accelerating AI Application Productionization: A Guide with SageMaker JumpStart, Amazon Bedrock, and TruEra: The post emphasizes the importance of observability in LLM applications and provides insights into evaluating responses for honesty, harmlessness, and helpfulness. You'll learn how to deploy, fine-tune, and iterate on foundation models for LLM applications using Amazon SageMaker JumpStart, Amazon Bedrock, and TruEra.  ⭐ Quantizing LLMs with Activation-aware Weight Quantization (AWQ): Explore the application of Activation-aware Weight Quantization (AWQ) to democratize LLMs like Llama-2, making them more accessible for deployment on regular CPUs or less powerful GPUs. The process involves setting up a GPU instance, installing necessary packages like AutoAWQ and transformers, and saving the quantized model. The tutorial further covers the model upload to the Hugging Face Model Hub and concludes with the successful reduction of the Llama-2 model from ~27GB to ~4GB, enhancing its efficiency for diverse applications. ⭐ Unlocking Your MacBook's AI Potential: Running 70B LLM Models Without Quantization: Discover how to unleash the hidden AI power of your 8GB MacBook as this post explores the latest 2.8 version of AirLLM. Without the need for quantization or model compression, an ordinary MacBook can now efficiently run top-tier 70 billion parameter models. Explore the MacBook's AI capabilities, understanding Apple's role in AI evolution through its M1, M2, and M3 series GPUs, which offer competitive performance in the era of generative AI. Gain insights into GPU capabilities, memory advantages, and the open-source MLX platform. 🚀 HackHub: Trending AI Tools⭐ Giskard-AI/giskard: Specialized testing framework for ML models, covering a range from tabular to LLMs. Developers can efficiently scan AI models using just four lines of code. ⭐ CopilotKit/CopilotKit: Build in-app AI chatbots that seamlessly interact with the app state, execute actions within the app, and communicate with both frontend, backend, and third-party services via plugins, serving as an AI "second brain" for users. ⭐ chengzeyi/stable-fast:  Leverage stable-fast for efficient and high-performance inference on various diffuser models while enjoying fast model compilation and out-of-the-box support for dynamic shape, LoRA, and ControlNet. ⭐ ml-explore/mlx: Array framework for machine learning on Apple silicon by Apple's ML research, offering familiar Python and C++ APIs closely aligned with NumPy and PyTorch. 
Read more
  • 0
  • 0
  • 21145

article-image-cloud-native-architectures-microservices-containers-serverless-part-1
Guest Contributor
13 Aug 2018
9 min read
Save for later

Modern Cloud Native architectures: Microservices, Containers, and Serverless - Part 1

Guest Contributor
13 Aug 2018
9 min read
This whitepaper is written by Mina Andrawos, an experienced engineer who has developed deep experience in the Go language, and modern software architectures. He regularly writes articles and tutorials about the Go language, and also shares open source projects. Mina Andrawos has authored the book Cloud Native programming with Golang, which provides practical techniques, code examples, and architectural patterns required to build cloud native microservices in the Go language.He is also the author of the Mastering Go Programming, and the Modern Golang Programming video courses. This paper sheds some light and provides practical exposure on some key topics in the modern software industry, namely cloud native applications.This includes microservices, containers , and serverless applications. The paper will cover the practical advantages, and disadvantages of the technologies covered. Microservices The microservices architecture has gained reputation as a powerful approach to architect modern software applications. So what are microservices? Microservices can be described as simply the idea of separating the functionality required from a software application into multiple independent small software services or “microservices.” Each microservice is responsible for an individual focused task. In order for microservices to collaborate together to form a large scalable application, they communicate and exchange data. Microservices were born out of the need to tame the complexity, and inflexibility of “monolithic” applications. A monolithic application is a type of application, where all required functionality is coded together into the same service. For example, here is a diagram representing a monolithic events (like concerts, shows..etc) booking application that takes care of the booking payment processing and event reservation: The application can be used by a customer to book a concert or a show. A user interface will be needed. Furthermore, we will also need a search functionality to look for events, a bookings handler to process the user booking then save it, and an events handler to help find the event, ensure it has seats available, then link it to the booking. In a production level application, more tasks will be needed like payment processing for example, but for now let’s focus on the four tasks outlined in the above figure. This monolithic application will work well with small to medium load. It will run on a single server, connect to a single database and will be written probably in the same programming language. Now, what will happen if the business grows exponentially and hundreds of thousands or millions of users need to be handled and processed? Initially, the short term solution would be to ensure that the server where the application runs, has powerful hardware specifications to withstand higher loads, and if not then add more memory, storage, and processing power to the server. This is called vertical scaling, which is the act of increasing the power of the hardware  like RAM and hard drive capacity to run heavy applications.However, this is typically not  sustainable in the long run as the load on the application continues to grow. Another challenge with monolithic applications is the inflexibility caused by being limited to only one or two programming languages. This inflexibility can affect the overall quality, and efficiency of the application. For example, node.js is a popular JavaScript framework for building web applications, whereas R is popular for data science applications. A monolithic application will make it difficult to utilize both technologies, whereas in a microservices application, we can simply build a data science service written in R and a web service written in Node.js. The microservices version of the events application will take the below form: This application will be capable of scaling among multiple servers, a practice known as horizontal scaling. Each service can be deployed on a different server with dedicated resources or in separate containers (more on that later). The different services can be written in different programming languages enabling greater flexibility, and different dedicated teams can focus on different services achieving more overall quality for the application. Another notable advantage of using microservices is the ease of continuous delivery, which is the ability to deploy software often, and at any time. The reason why microservices make continuous delivery easier is because a new feature deployed to one microservices is less likely to affect other microservices compared to monolithic applications. Issues with Microservices One notable drawback of relying heavily on microservices is the fact that they can become too complicated to manage in the long run as they grow in numbers and scope. There are approaches to mitigate this by utilizing monitoring tools such as Prometheus to detect problems, container technologies such as Docker to avoid pollutions of the host environments and avoiding over designing the services. However, these approaches take effort and time. Cloud native applications Microservices architectures are a natural fit for cloud native applications. A cloud native application is simply defined as an application built from the ground up for cloud computing architectures. This simply means that our application is cloud native, if we design it as if it is expected to be deployed on a distributed, and scalable infrastructure. For example, building an application with a redundant microservices architecture -we’ll see an example shortly- makes the application cloud native, since this architecture allows our application to be deployed in a distributed manner that allows it to be scalable and almost always available. A cloud native application does not need to always be deployed to a public cloud like AWS, we can deploy it to our own distributed cloud-like infrastructure instead if we have one. In fact, what makes an application fully cloud native is beyond just using microservices. Your application  should employ continuous delivery, which is your ability to continuously deliver updates to your production applications without  disruptions. Your application should also make use of services like message queues and technologies like containers, and serverless (containers and serverless are important topics for modern software architectures, so we’ll be discussing them in the next few sections). Cloud native applications assume access to numerous server nodes, having access to pre-deployed software services like message queues or load balancers, ease of integration with continuous delivery services, among other things. If you deploy your cloud native application to a commercial cloud like AWS or Azure, your application gets the option to utilize cloud only software services. For example, DynamoDB is a powerful database engine that can only be used on Amazon Web Services for production applications. Another example is the DocumentDB database in Azure. There are also cloud only message queues such as Amazon Simple Queue Service (SQS), which can be used to allow communication between microservices in the Amazon Web Services cloud. As mentioned earlier, cloud native microservices should be designed to allow redundancy between services. If we take the events booking application as an example, the application will look like this: Multiple server nodes would be allocated per microservice, allowing a redundant microservices architecture to be deployed. If the primary node or service fails for any reason, the secondary can take over ensuring lasting reliability and availability for cloud native applications. This availability is vital for fault intolerant applications such as e-commerce platforms, where downtime translates into large amounts of lost revenue. Cloud native applications provide great value for developers, enterprises, and startups. A notable tool worth mentioning in the world of microservices and cloud computing is Prometheus. Prometheus is an open source system monitoring and alerting tool that can be used to monitor complex microservices architectures and alert when an action needs to be taken. Prometheus was originally created by SoundCloud to monitor their systems, but then grew to become an independent project. The project is now a part of the cloud native computing foundation, which is a foundation tasked with building a sustainable ecosystem for cloud native applications. Cloud native limitations For cloud native applications, you will face some challenges if the need arises to migrate some or all of the applications. That is due to multiple reasons, depending on where your application is deployed. For example,if your cloud native application is deployed on a public cloud like AWS, cloud native APIs are not cross cloud platform. So, a DynamoDB database API utilized in an application will only work on AWS but not on Azure, since DynamoDB belongs exclusively to AWS. The API will also never work in a local environment because DynamoDB can only be utilized in AWS in production. Another reason is because there are some assumptions made when some cloud native applications are built, like the fact that there will be virtually unlimited number of server nodes to utilize when needed and that a new server node can be made available very quickly. These assumptions are sometimes hard to guarantee in a local data center environment, where real servers, networking hardware, and wiring need to be purchased. This brings us to the end of Part 1 of this whitepaper. Check out Part 2 tomorrow to learn about Containers and Serverless applications along with their practical advantages and limitations. About Author: Mina Andrawos Mina Andrawos is an experienced engineer who has developed deep experience in Go from using it personally and professionally. He regularly authors articles and tutorials about the language, and also shares Go's open source projects. He has written numerous Go applications with varying degrees of complexity. Other than Go, he has skills in Java, C#, Python, and C++. He has worked with various databases and software architectures. He is also skilled with the agile methodology for software development. Besides software development, he has working experience of scrum mastering, sales engineering, and software product management. Building microservices from a monolith Java EE app [Tutorial] 6 Ways to blow up your Microservices! Have Microservices killed the monolithic architecture? Maybe not!
Read more
  • 0
  • 0
  • 21139
Unlock access to the largest independent learning library in Tech for FREE!
Get unlimited access to 7500+ expert-authored eBooks and video courses covering every tech area you can think of.
Renews at $19.99/month. Cancel anytime
article-image-whats-new-in-intellij-idea-2018-2
Sugandha Lahoti
26 Jul 2018
4 min read
Save for later

What’s new in IntelliJ IDEA 2018.2

Sugandha Lahoti
26 Jul 2018
4 min read
JetBrains has released the second version of their popular IDE for this year. IntelliJ IDEA 2018.2 is full of changes including support for Java 11, updates to version editor, user interface, JVM debugger, Gradle and more. Let’s have a quick look at all the different features and updates. Updates to Java IntelliJ IDEA 2018.2 brings support for the upcoming Java 11. The IDE now supports local-variable syntax for lambda parameters according to JEP 323. The Dataflow information can now be viewed in the editor. Quick Documentation can now be configured to pop-up together with autocompletion. Extract Method has a new preview panel to check the results of refactoring before actual changes are made. The @Contract annotation adds new return values: new, this, and paramX. The IDE also has updated its inspections and intention actions including smarter Join Line action and Stream API support, among many others. Improvements to Editor IntelliJ IDEA now underlines reassigned local variables and reassigned parameters, by default. While typing, users can use Tab to navigate outside the closing brackets or closing quotes. For or while keywords are highlighted when the caret is placed on the corresponding break or continue keyword. Changes to User Interface IntelliJ IDEA 2018.2 comes with support for the MacBook Touch Bar. Users can now use dark window headers in MacOS There are new cleaner and simpler icons on the IDE toolbar and tool windows for better readability. The IntelliJ theme on Linux has been updated to look more modern. Updates to the Version Control System The updated Files Merged with Conflicts dialog displays Git branch names and adds a new Group files by directory option. Users can now open several Log tabs in the Version Control toolwindow. The IDE now displays the Favorites branches in the Branch filter on the Log tab. While using the Commit and Push action users can either skip the Push dialog completely or only show this dialog when pushing to protected branches. The  IDE also adds support for configuring multiple GitHub accounts. Improvements in the JVM debugger IntelliJ IDEA 2018.2 includes several new breakpoint intentions actions for debugging Java projects. Users now have the ability to filter a breakpoint hit by the caller method. Changes to Gradle The included buildSrc Gradle projects are now discovered automatically. Users can now debug Gradle DSL blocks. Updates to Kotlin The Kotlin plugin bundled with the IDE has been updated to v1.2.51. Users can now run Kotlin Script scratch files and see the results right inside the editor. An intention to convert end-of-line comments into the block comments and vice versa has been added. New coroutine inspections and intentions added. Improvements in the Scala plugin The Scala plugin can show implicits right in the editor and can even show places where implicits are not found. The Scalafmt formatter has been integrated. Added updates to Semantic highlighting and improved auto-completion for pattern matching. JavaScript & TypeScript changes The newExtract React component can be used for refactoring to break a component into two. A new intention to Convert React class components into functional components added. New features can be added to an Angular app using the integration with ng add. New JavaScript and TypeScript intentions: Implement interface, Create derived class, Implement members of an interface or abstract class, Generate cases for ‘switch’, and Iterate with ‘for..of’. A new Code Coverage feature for finding the unused code in client-side apps. These are just a select few updates from the IntelliJ IDEA 2018.2 release. A complete list of all the changes can be found in the release notes. You can also read the JetBrains blog for a concise version. How to set up the Scala Plugin in IntelliJ IDE [Tutorial] Eclipse IDE’s Photon release will support Rust GitLab open sources its Web IDE in GitLab 10.7
Read more
  • 0
  • 0
  • 21118

article-image-gadgets-jira
Packt
29 Dec 2011
14 min read
Save for later

Gadgets in JIRA

Packt
29 Dec 2011
14 min read
(For more resources on JIRA, see here.) Writing JIRA 4 gadgets Gadgets are a big leap in JIRA's reporting features! The fact that JIRA is now an OpenSocial container lets its user add useful gadgets (both JIRA's own and third-party) into its dashboard. At the same time, gadgets written for JIRA can be added in other containers like iGoogle, Gmail, and so on! In this recipe, we will have a look at writing a very simple gadget, one that says 'Hello from JTricks'. By keeping the content simple, it will let us concentrate more on writing the gadget! Before we start writing the gadget, it is probably worth understanding the key components of a JIRA gadget: Gadget XML is the most important part of a JIRA Gadget. It holds the specification of the gadget and includes the following: Gadget Characteristics. It includes title, description, author's name, and so on Screenshot and a thumbnail image. Please note that the screenshot is not used within Atlassian containers such as JIRA or Confluence. We can optionally add it if we want them to be used in other OpenSocial containers Required features that the gadget container must provide for the gadget User preferences which will be configured by the gadget users The Gadget content created using HTML and JavaScript A screenshot and thumbnail image will be used during preview and while selecting the gadget from the container. An i18n property file used for internationalization in the gadget. Optional CSS and JavaScript file used to render the display in the Content section of the gadget. We will see each of them in the recipe. Getting ready Create a skeleton plugin using Atlassian Plugin SDK. How to do it... The following are the steps to write our first gadget, one that shows the greetings from JTricks! Modify the plugin descriptor with the gadget module and the resources required for our gadget: Add the Gadget module in the plugin descriptor: <gadget key="hello-gadget" name="Hello Gadget" location="hello-gadget.xml"> <description>Hello Gadget! </description></gadget> As you can see, this has a unique key and points to the location of the gadget XML! You can have as many gadget definitions as you want in your atlassian-plugin.xml file, but in our example, we stick with the preceding one. Include the thumbnail and screenshot images and downloadable resources in the plugin descriptor. More can be learned at http://confluence.atlassian.com/display/JIRADEV/Downloadable+Plugin+Resources. In our example, the resources are added on to the plugin descriptor as: <resource type="download" name="screenshot.png" location="/images/screenshot.png"/><resource type="download" name="thumbnail.png" location="/images/thumbnail.png"/> The location is relative to the src/main/resources folder in the plugin. As mentioned before, the screenshot is optional. Add the i18n properties file that will be used in the gadget also as a downloadable resource: <resource type="download" name="i18n/messages.xml" location="i18n/messages.xml"> <param name="content-type" value="text/xml; charset=UTF-8"/></resource> The atlassian-plugin.xml will now look like this: <atlassian-plugin key="com.jtricks.gadgets" name="Gadgets Plugin" plugins-version="2"> <plugin-info> <description>Gadgets Example</description> <version>2.0</version> <vendor name="JTricks" url="http://www.j-tricks.com/" /> </plugin-info> <gadget key="hello-gadget" name="Hello Gadget" location="hello-gadget.xml"> <description>Hello Gadget!</description> </gadget> <resource type="download" name="screenshot.png" location="/images/screenshot.png"/> <resource type="download" name="thumbnail.png" location="/images/thumbnail.png"/> <resource type="download" name="i18n/messages.xml" location="i18n/messages.xml"> <param name="content-type" value="text/xml; charset=UTF-8"/> </resource> </atlassian-plugin> Add the screenshot and thumbnail images under the src/main/resources/ images folder. The thumbnail image should be of the size 120 x 60 pixels. Add the i18n properties file under the src/main/resources/i18n folder. The name of the filer we defined in messages.xml. This file is an XML file wrapped within the messagebundle tag. Each property in the file is entered as an XML tag, as shown next: <msg name="gadget.title">Hello Gadget</msg> The msg tag has a name attribute, which is the property, and the corresponding Value is enclosed in the msg tag. We use three properties in our example and the entire file in our example looks like the following: <messagebundle> <msg name="gadget.title">Hello Gadget</msg> <msg name="gadget.title.url">http://www.j-tricks.com</msg> <msg name="gadget.description">Example Gadget from J-Tricks</msg></messagebundle> Write the Gadget XML. The Gadget XML has a Module element at the root of the XML. It has mainly three elements underneath – ModulePrefs, UserPref, and Content. We will write of each of them in this example. The entire set of attributes and elements and other details of the gadget specification can be read at http://confluence.atlassian. com/display/GADGETDEV/Creating+your+Gadget+XML+Specification. Write the ModulePrefs element. This element holds the information about the gadget. It also has two child elements – Require and Optional, that are used to define the required or optional features for the gadget. The following is how the ModulePrefs element looks in our example after it is populated with all the attributes: <ModulePrefs title="__MSG_gadget.title__" title_url="__MSG_gadget.title.url__" description="__MSG_gadget.description__" author="Jobin Kuruvilla" author_email=jobinkk@gmail.com screenshot='#staticResourceUrl("com.jtricks.gadgets:hello-gadget", "screenshot.png")' thumbnail='#staticResourceUrl("com.jtricks.gadgets:hello-gadget", "thumbnail.png")' height="150" ></ModulePrefs> As you can see, it holds information like title, title URL (to which the gadget title will link to), description, author name and email, height of the gadget, and URLs to screenshot and thumbnail images. Anything that starts with __MSG_ and ends with __ is a property that is referred from the i18n properties file. The height of the gadget is optional and 200, by default. The images are referenced using #staticResourceUrl where the first argument is the fully qualified gadget module key which is of the form ${atlassianplugin- key}:${module-key}. In our example, the plugin key is com. jtricks.gadgets and the module key is hello-gadget. Add the optional gadget directory feature inside ModulePrefs. This is currently supported only in JIRA: <Optional feature="gadget-directory"> <Param name="categories"> Other </Param></Optional> In the example, we add the category as Other! Other values supported for category are: JIRA, Confluence, FishEye, Crucible, Crowd, Clover, Bamboo, Admin, Charts, and External Content. You can add the gadget to more than one category by adding the categories within the Param element, each in a new line. Include Required features if there are any under the XML tag require. A full list of supported features can be found at http://confluence.atlassian.com/display/GADGETDEV/Including+Features+into+your+Gadget. Add the Locale element to point to the i18n properties file: <Locale messages="__ATLASSIAN_BASE_URL__/download/resources/com.jtricks.gadgets/i18n/messages.xml"/> Here the property __ATLASSIAN_BASE_URL__ will be automatically substituted with JIRA's configured base URL when the gadget is rendered. The path to the property file here is __ATLASSIAN_BASE_URL__/download/ resources/com.jtricks.gadgets, where com.jtricks.gadgets is the Atlassian plugin key. The path to the XML file /i18n/messages.xml is what is defined in the resource module earlier. Add User Preferences if required, using the UserPref element. We will omit the same in this example as the 'Hello Gadget' doesn't take any inputs from the user. Add the Content for the gadget. This is where the gadget is rendered using HTML and JavaScript. In our example, we just need to provide the static text 'Hello From JTricks' and it is fairly easy. The entire content is wrapped within the <![CDATA[ and ]]>, so that they won't be treated as XML tags. The following is how it looks in our example: <Content type="html" view="profile"> <![CDATA[ Hello From JTricks ]]></Content> Our gadget's XML is now ready and looks like the following block of code: <?xml version="1.0" encoding="UTF-8" ?><Module> <ModulePrefs title="__MSG_gadget.title__" title_url="__MSG_gadget.title.url__" description="__MSG_gadget.description__" author="Jobin Kuruvilla" author_email=jobinkk@gmail.com screenshot='#staticResourceUrl("com.jtricks.gadgets:hello-gadget", "screenshot.png")' thumbnail='#staticResourceUrl("com.jtricks.gadgets:hello-gadget", "thumbnail.png")' height="150" > <Optional feature="gadget-directory"> <Param name="categories"> Other </Param> </Optional> <Locale messages="__ATLASSIAN_BASE_URL__/download/resources/com.jtricks.gadgets/i18n/messages.xml"/> </ModulePrefs> <Content type="html" view="profile"> <![CDATA[ Hello From JTricks ]]> </Content></Module> Package the plugin, deploy it, and test it. How it works... Once the plugin is deployed, we need to add the gadget in the JIRA dashboard. The following is how it appears in the Add Gadget screen. Note the thumbnail is the one we have in the plugin and also note that it appears in the Other section: Once it is added, it appears as follows in the Dashboards section:   (Move the mouse over the image to enlarge.)   There's more... We can modify the look-and-feel of the gadgets by adding more HTML or gadget preferences! For example, <font color="red">Hello From JTricks</font> will make it appear in red. We can adjust the size of the gadget using the dynamic-height feature. We should add the following under the ModulePrefs element: <Require feature="dynamic-height"/> We should then invoke gadgets.window.adjustHeight(); whenever the content is reloaded. For example, we can do it in a window onload event, as shown next: <script type="text/javascript" charset="utf-8"> function resize() { gadgets.window.adjustHeight(); } window.onload=resize;</script> The gadget xml file, in this case, will look like this: <?xml version="1.0" encoding="UTF-8" ?><Module> <ModulePrefs title="__MSG_gadget.title__" title_url="__MSG_gadget.title.url__" description="__MSG_gadget.description__" author="Jobin Kuruvilla" author_email="jobinkk@gmail.com" screenshot='#staticResourceUrl("com.jtricks.gadgets:hello-gadget", "screenshot.png")' thumbnail='#staticResourceUrl("com.jtricks.gadgets:hello-gadget", "thumbnail.png")' height="150" > <Optional feature="gadget-directory"> <Param name="categories"> Other </Param> </Optional> <Require feature="dynamic-height"/> <Locale messages="__ATLASSIAN_BASE_URL__/download/resources/com.jtricks.gadgets/i18n/messages.xml"/> </ModulePrefs> <Content type="html" view="profile"> <![CDATA[ <script type="text/javascript" charset="utf-8"> function resize() { gadgets.window.adjustHeight(); } window.onload=resize; </script> Hello From JTricks ]]> </Content></Module> The gadget should now appear as follows: Note that the size is adjusted to just fit the text! Invoking REST services from gadgets In the previous recipe, we saw how to write a gadget with static content. In this recipe, we will have a look at creating a gadget with dynamic content or the data that is coming from the JIRA server. JIRA uses REST services to communicate between the gadgets and the server. In this recipe, we will use an existing REST service. Getting ready Create the Hello Gadget, as described in the previous recipe. How to do it... Let us consider a simple modification to the existing Hello Gadget to understand the basics of invoking REST services from gadgets. We will try to greet the current user by retrieving the user details from the server instead of displaying the static text: Hello From JTricks. JIRA ships with some inbuilt REST methods, one of which is to retrieve the details of the current user. The method can be reached in the URL: /rest/gadget/1.0/currentUser. We will use this method to retrieve the current user's full name and then display it in the gadget greeting. If the user's name is Jobin Kuruvilla, the gadget will display the message as Hello, Jobin Kuruvilla. As we are only changing the content of the gadget, the only modification is required in the gadget XML, which is hello-gadget.xml in our example. Only the Content element needs to be modified, which will now invoke the REST service and render the content. The following are the steps: Include the common Atlassian gadget resources: #requireResource("com.atlassian.jira.gadgets:common")#includeResources() #requireResource will bring in the JIRA gadget JavaScript framework into the gadget's context. #includeResources will write out the HTML tags for the resource in place. Check out http://confluence.atlassian.com/display/GADGETDEV/Using+Web+Resources+in+your+Gadget for more details. Construct a gadget object as follows: var gadget = AJS.Gadget The gadget object has four top-level options: baseUrl: An option to pass the base URL. It is a mandatory option, and we use __ATLASSIAN_BASE_URL__ here which will be rendered as JIRA's base URL. useOauth: An optional parameter. Used to configure the type of authentication which must be a URL. /rest/gadget/1.0/currentUser is commonly used. config: Another optional parameter. Only used if there are any configuration options for the gadget. view: Used to define the gadget's view. In our example, we don't use authentication or any configuration options. We will just go with the baseUrl and view options. The following is how the Gadget is created using JavaScript: <script type="text/javascript"> (function () { var gadget = AJS.Gadget({ baseUrl: "__ATLASSIAN_BASE_URL__", view: { ................ } }); })();</script> Populate the gadget view. The view object has the following properties: enableReload: Optional. Used to reload the gadget at regular intervals. onResizeReload: Optional. Used to reload the gadget when the browser is resized. onResizeAdjustHeight: Optional and used along with the dynamicheight feature. This will adjust the gadget height when the browser is resized. template: Created the actual view. args: An array of objects or function that returns an array of objects. It has two attributes. Key –used to access the data from within the template and ajaxOptions – set of request options used to connect to the server and retrieve data. In our example, we will use the template and args properties to render the view. First, let us see args because we use the data retrieved here in the template. args will look like the following: args: [{ key: "user", ajaxOptions: function() { return { url: "/rest/gadget/1.0/currentUser" }; } }] As you can see, we invoke the /rest/gadget/1.0/currentUser method and use the key user to refer the data we retrieved while rendering the view. ajaxOptions uses the jQuery Ajax Options, details of which can be found at http://api.jquery.com/jQuery.ajax#options. The key user will now hold the user details from the REST method, as follows: {"username":"jobinkk","fullName":"Jobin Kuruvilla","email":"jobinkk@gmail.com"} The template function will now use this args object (defined earlier) and its key, user to render the view as follows: template: function(args) { var gadget = this; var userDetails = AJS.$("<h1/>").text("Hello, "+args.user["fullName"]); gadget.getView().html(userDetails); } Here, args.user["fullName"] will retrieve the user's fullName from the REST output. Username or e-mail can be retrieved in a similar fashion. AJS.$ will construct the view as <h1>Hello, Jobin Kuruvilla</h1>, where Jobin Kuruvilla is the fullName retrieved. The entire Content section will look as shown in the following lines of code: <Content type="html" view="profile"> <![CDATA[ #requireResource("com.atlassian.jira.gadgets:common") #includeResources() <script type="text/javascript"> (function () { var gadget = AJS.Gadget({ baseUrl: "__ATLASSIAN_BASE_URL__", view: { template: function(args) { var gadget = this; var userDetails = AJS.$("<h1/>").text("Hello, "+args.user["fullName"]); gadget.getView().html(userDetails); }, args: [{ key: "user", ajaxOptions: function() { return { url: "/rest/gadget/1.0/currentUser" }; } }] } }); })(); </script> ]]> </Content> Package the gadget and deploy it. How it works... After the modification to the gadget XML, the gadget will now display the method as follows:
Read more
  • 0
  • 0
  • 21115

article-image-lights-and-effects
Packt
29 Sep 2015
27 min read
Save for later

Lights and Effects

Packt
29 Sep 2015
27 min read
 In this article by Matt Smith and Chico Queiroz, authors of Unity 5.x Cookbook, we will cover the following topics: Using lights and cookie textures to simulate a cloudy day Adding a custom Reflection map to a scene Creating a laser aim with Projector and Line Renderer Reflecting surrounding objects with Reflection Probes Setting up an environment with Procedural Skybox and Directional Light (For more resources related to this topic, see here.) Introduction Whether you're willing to make a better-looking game, or add interesting features, lights and effects can boost your project and help you deliver a higher quality product. In this article, we will look at the creative ways of using lights and effects, and also take a look at some of Unity's new features, such as Procedural Skyboxes, Reflection Probes, Light Probes, and custom Reflection Sources. Lighting is certainly an area that has received a lot of attention from Unity, which now features real-time Global Illumination technology provided by Enlighten. This new technology provides better and more realistic results for both real-time and baked lighting. For more information on Unity's Global Illumination system, check out its documentation at http://docs.unity3d.com/Manual/GIIntro.html. The big picture There are many ways of creating light sources in Unity. Here's a quick overview of the most common methods. Lights Lights are placed into the scene as game objects, featuring a Light component. They can function in Realtime, Baked, or Mixed modes. Among the other properties, they can have their Range, Color, Intensity, and Shadow Type set by the user. There are four types of lights: Directional Light: This is normally used to simulate the sunlight Spot Light: This works like a cone-shaped spot light Point Light: This is a bulb lamp-like, omnidirectional light Area Light: This baked-only light type is emitted in all directions from a rectangle-shaped entity, allowing for a smooth, realistic shading For an overview of the light types, check Unity's documentation at http://docs.unity3d.com/Manual/Lighting.html. Different types of lights Environment Lighting Unity's Environment Lighting is often achieved through the combination of a Skybox material and sunlight defined by the scene's Directional Light. Such a combination creates an ambient light that is integrated into the scene's environment, and which can be set as Realtime or Baked into Lightmaps. Emissive materials When applied to static objects, materials featuring the Emission colors or maps will cast light over surfaces nearby, in both real-time and baked modes, as shown in the following screenshot: Projector As its name suggests, a Projector can be used to simulate projected lights and shadows, basically by projecting a material and its texture map onto the other objects. Lightmaps and Light Probes Lightmaps are basically texture maps generated from the scene's lighting information and applied to the scene's static objects in order to avoid the use of processing-intensive real-time lighting. Light Probes are a way of sampling the scene's illumination at specific points in order to have it applied onto dynamic objects without the use of real-time lighting. The Lighting window The Lighting window, which can be found through navigating to the Window | Lighting menu, is the hub for setting and adjusting the scene's illumination features, such as Lightmaps, Global Illumination, Fog, and much more. It's strongly recommended that you take a look at Unity's documentation on the subject, which can be found at http://docs.unity3d.com/Manual/GlobalIllumination.html. Using lights and cookie textures to simulate a cloudy day As it can be seen in many first-person shooters and survival horror games, lights and shadows can add a great deal of realism to a scene, helping immensely to create the right atmosphere for the game. In this recipe, we will create a cloudy outdoor environment using cookie textures. Cookie textures work as masks for lights. It functions by adjusting the intensity of the light projection to the cookie texture's alpha channel. This allows for a silhouette effect (just think of the bat-signal) or, as in this particular case, subtle variations that give a filtered quality to the lighting. Getting ready If you don't have access to an image editor, or prefer to skip the texture map elaboration in order to focus on the implementation, please use the image file called cloudCookie.tga, which is provided inside the 1362_06_01 folder. How to do it... To simulate a cloudy outdoor environment, follow these steps: In your image editor, create a new 512 x 512 pixel image. Using black as the foreground color and white as the background color, apply the Clouds filter (in Photoshop, this is done by navigating to the Filter | Render | Clouds menu). Learning about the Alpha channel is useful, but you could get the same result without it. Skip steps 3 to 7, save your image as cloudCookie.png and, when changing texture type in step 9, leave Alpha from Greyscale checked. Select your entire image and copy it. Open the Channels window (in Photoshop, this can be done by navigating to the Window | Channels menu). There should be three channels: Red, Green, and Blue. Create a new channel. This will be the Alpha channel. In the Channels window, select the Alpha 1 channel and paste your image into it. Save your image file as cloudCookie.PSD or TGA. Import your image file to Unity and select it in the Project view. From the Inspector view, change its Texture Type to Cookie and its Light Type to Directional. Then, click on Apply, as shown: We will need a surface to actually see the lighting effect. You can either add a plane to your scene (via navigating to the GameObject | 3D Object | Plane menu), or create a Terrain (menu option GameObject | 3D Object | Terrain) and edit it, if you so you wish. Let's add a light to our scene. Since we want to simulate sunlight, the best option is to create a Directional Light. You can do this through the drop-down menu named Create | Light | Directional Light in the Hierarchy view. Using the Transform component of the Inspector view, reset the light's Position to X: 0, Y: 0, Z: 0 and its Rotation to X: 90; Y: 0; Z: 0. In the Cookie field, select the cloudCookie texture that you imported earlier. Change the Cookie Size field to 80, or a value that you feel is more appropriate for the scene's dimension. Please leave Shadow Type as No Shadows. Now, we need a script to translate our light and, consequently, the Cookie projection. Using the Create drop-down menu in the Project view, create a new C# Script named MovingShadows.cs. Open your script and replace everything with the following code: using UnityEngine; using System.Collections; public class MovingShadows : MonoBehaviour{ public float windSpeedX; public float windSpeedZ; private float lightCookieSize; private Vector3 initPos; void Start(){ initPos = transform.position; lightCookieSize = GetComponent<Light>().cookieSize; } void Update(){ Vector3 pos = transform.position; float xPos= Mathf.Abs (pos.x); float zPos= Mathf.Abs (pos.z); float xLimit = Mathf.Abs(initPos.x) + lightCookieSize; float zLimit = Mathf.Abs(initPos.z) + lightCookieSize; if (xPos >= xLimit) pos.x = initPos.x; if (zPos >= zLimit) pos.z = initPos.z; transform.position = pos; float windX = Time.deltaTime * windSpeedX; float windZ = Time.deltaTime * windSpeedZ; transform.Translate(windX, 0, windZ, Space.World); } } Save your script and apply it to the Directional Light. Select the Directional Light. In the Inspector view, change the parameters Wind Speed X and Wind Speed Z to 20 (you can change these values as you wish, as shown). Play your scene. The shadows will be moving. How it works... With our script, we are telling the Directional Light to move across the X and Z axis, causing the Light Cookie texture to be displaced as well. Also, we reset the light object to its original position whenever it traveled a distance that was either equal to or greater than the Light Cookie Size. The light position must be reset to prevent it from traveling too far, causing problems in real-time render and lighting. The Light Cookie Size parameter is used to ensure a smooth transition. The reason we are not enabling shadows is because the light angle for the X axis must be 90 degrees (or there will be a noticeable gap when the light resets to the original position). If you want dynamic shadows in your scene, please add a second Directional Light. There's more... In this recipe, we have applied a cookie texture to a Directional Light. But what if we were using the Spot or Point Lights? Creating Spot Light cookies Unity documentation has an excellent tutorial on how to make the Spot Light cookies. This is great to simulate shadows coming from projectors, windows, and so on. You can check it out at http://docs.unity3d.com/Manual/HOWTO-LightCookie.html. Creating Point Light Cookies If you want to use a cookie texture with a Point Light, you'll need to change the Light Type in the Texture Importer section of the Inspector. Adding a custom Reflection map to a scene Whereas Unity Legacy Shaders use individual Reflection Cubemaps per material, the new Standard Shader gets its reflection from the scene's Reflection Source, as configured in the Scene section of the Lighting window. The level of reflectiveness for each material is now given by its Metallic value or Specular value (for materials using Specular setup). This new method can be a real time saver, allowing you to quickly assign the same reflection map to every object in the scene. Also, as you can imagine, it helps keep the overall look of the scene coherent and cohesive. In this recipe, we will learn how to take advantage of the Reflection Source feature. Getting ready For this recipe, we will prepare a Reflection Cubemap, which is basically the environment to be projected as a reflection onto the material. It can be made from either six or, as shown in this recipe, a single image file. To help us with this recipe, it's been provided a Unity package, containing a prefab made of a 3D object and a basic Material (using a TIFF as Diffuse map), and also a JPG file to be used as the reflection map. All these files are inside the 1362_06_02 folder. How to do it... To add Reflectiveness and Specularity to a material, follow these steps: Import batteryPrefab.unitypackage to a new project. Then, select battery_prefab object from the Assets folder, in the Project view. From the Inspector view, expand the Material component and observe the asset preview window. Thanks to the Specular map, the material already features a reflective look. However, it looks as if it is reflecting the scene's default Skybox, as shown: Import the CustomReflection.jpg image file. From the Inspector view, change its Texture Type to Cubemap, its Mapping to Latitude - Longitude Layout (Cylindrical), and check the boxes for Glossy Reflection and Fixup Edge Seams. Finally, change its Filter Mode to Trilinear and click on the Apply button, shown as follows: Let's replace the Scene's Skybox with our newly created Cubemap, as the Reflection map for our scene. In order to do this, open the Lighting window by navigating to the Window | Lighting menu. Select the Scene section and use the drop-down menu to change the Reflection Source to Custom. Finally, assign the newly created CustomReflection texture as the Cubemap, shown as follows: Check out for the new reflections on the battery_prefab object. How it works... While it is the material's specular map that allows for a reflective look, including the intensity and smoothness of the reflection, the refection itself (that is, the image you see on the reflection) is given by the Cubemap that we have created from the image file. There's more... Reflection Cubemaps can be achieved in many ways and have different mapping properties. Mapping coordinates The Cylindrical mapping that we applied was well-suited for the photograph that we used. However, depending on how the reflection image is generated, a Cubic or Spheremap-based mapping can be more appropriate. Also, note that the Fixup Edge Seams option will try to make the image seamless. Sharp reflections You might have noticed that the reflection is somewhat blurry compared to the original image; this is because we have ticked the Glossy Reflections box. To get a sharper-looking reflection, deselect this option; in which case, you can also leave the Filter Mode option as default (Bilinear). Maximum size At 512 x 512 pixels, our reflection map will probably run fine on the lower-end machines. However, if the quality of the reflection map is not so important in your game's context, and the original image dimensions are big (say, 4096 x 4096), you might want to change the texture's Max Size at the Import Settings to a lower number. Creating a laser aim with Projector and Line Renderer Although using GUI elements, such as a cross-hair, is a valid way to allow players to aim, replacing (or combining) it with a projected laser dot might be a more interesting approach. In this recipe, we will use the Projector and Line components to implement this concept. Getting ready To help us with this recipe, it's been provided with a Unity package containing a sample scene featuring a character holding a laser pointer, and also a texture map named LineTexture. All files are inside the 1362_06_03 folder. Also, we'll make use of the Effects assets package provided by Unity (which you should have installed when installing Unity). How to do it... To create a laser dot aim with a Projector, follow these steps: Import BasicScene.unitypackage to a new project. Then, open the scene named BasicScene. This is a basic scene, featuring a player character whose aim is controlled via mouse. Import the Effects package by navigating to the Assets | Import Package | Effects menu. If you want to import only the necessary files within the package, deselect everything in the Importing package window by clicking on the None button, and then check the Projectors folder only. Then, click on Import, as shown: From the Inspector view, locate the ProjectorLight shader (inside the Assets | Standard Assets | Effects | Projectors | Shaders folder). Duplicate the file and name the new copy as ProjectorLaser. Open ProjectorLaser. From the first line of the code, change Shader "Projector/Light" to Shader "Projector/Laser". Then, locate the line of code – Blend DstColor One and change it to Blend One One. Save and close the file. The reason for editing the shader for the laser was to make it stronger by changing its blend type to Additive. However, if you want to learn more about it, check out Unity's documentation on the subject, which is available at http://docs.unity3d.com/Manual/SL-Reference.html. Now that we have fixed the shader, we need a material. From the Project view, use the Create drop-down menu to create a new Material. Name it LaserMaterial. Then, select it from the Project view and, from the Inspector view, change its Shader to Projector/Laser. From the Project view, locate the Falloff texture. Open it in your image editor and, except for the first and last columns column of pixels that should be black, paint everything white. Save the file and go back to Unity. Change the LaserMaterial's Main Color to red (RGB: 255, 0, 0). Then, from the texture slots, select the Light texture as Cookie and the Falloff texture as Falloff. From the Hierarchy view, find and select the pointerPrefab object (MsLaser | mixamorig:Hips | mixamorig:Spine | mixamorig:Spine1 | mixamorig:Spine2 | mixamorig:RightShoulder | mixamorig:RightArm | mixamorig:RightForeArm | mixamorig:RightHand | pointerPrefab). Then, from the Create drop-down menu, select Create Empty Child. Rename the new child of pointerPrefab as LaserProjector. Select the LaserProjector object. Then, from the Inspector view, click the Add Component button and navigate to Effects | Projector. Then, from the Projector component, set the Orthographic option as true and set Orthographic Size as 0.1. Finally, select LaserMaterial from the Material slot. Test the scene. You will be able to see the laser aim dot, as shown: Now, let's create a material for the Line Renderer component that we are about to add. From the Project view, use the Create drop-down menu to add a new Material. Name it as Line_Mat. From the Inspector view, change the shader of the Line_Mat to Particles/Additive. Then, set its Tint Color to red (RGB: 255;0;0). Import the LineTexture image file. Then, set it as the Particle Texture for the Line_Mat, as shown: Use the Create drop-down menu from Project view to add a C# script named LaserAim. Then, open it in your editor. Replace everything with the following code: using UnityEngine; using System.Collections; public class LaserAim : MonoBehaviour { public float lineWidth = 0.2f; public Color regularColor = new Color (0.15f, 0, 0, 1); public Color firingColor = new Color (0.31f, 0, 0, 1); public Material lineMat; private Vector3 lineEnd; private Projector proj; private LineRenderer line; void Start () { line = gameObject.AddComponent<LineRenderer>(); line.material = lineMat; line.material.SetColor("_TintColor", regularColor); line.SetVertexCount(2); line.SetWidth(lineWidth, lineWidth); proj = GetComponent<Projector> (); } void Update () { RaycastHit hit; Vector3 fwd = transform.TransformDirection(Vector3.forward); if (Physics.Raycast (transform.position, fwd, out hit)) { lineEnd = hit.point; float margin = 0.5f; proj.farClipPlane = hit.distance + margin; } else { lineEnd = transform.position + fwd * 10f; } line.SetPosition(0, transform.position); line.SetPosition(1, lineEnd); if(Input.GetButton("Fire1")){ float lerpSpeed = Mathf.Sin (Time.time * 10f); lerpSpeed = Mathf.Abs(lerpSpeed); Color lerpColor = Color.Lerp(regularColor, firingColor, lerpSpeed); line.material.SetColor("_TintColor", lerpColor); } if(Input.GetButtonUp("Fire1")){ line.material.SetColor("_TintColor", regularColor); } } } Save your script and attach it to the LaserProjector game object. Select the LaserProjector GameObject. From the Inspector view, find the Laser Aim component and fill the Line Material slot with the Line_Mat material, as shown: Play the scene. The laser aim is ready, and looks as shown: In this recipe, the width of the laser beam and its aim dot have been exaggerated. Should you need a more realistic thickness for your beam, change the Line Width field of the Laser Aim component to 0.05, and the Orthographic Size of the Projector component to 0.025. Also, remember to make the beam more opaque by setting the Regular Color of the Laser Aim component brighter. How it works... The laser aim effect was achieved by combining two different effects: a Projector and Line Renderer. A Projector, which can be used to simulate light, shadows, and more, is a component that projects a material (and its texture) onto other game objects. By attaching a projector to the Laser Pointer object, we have ensured that it will face the right direction at all times. To get the right, vibrant look, we have edited the projector material's Shader, making it brighter. Also, we have scripted a way to prevent projections from going through objects, by setting its Far Clip Plane on approximately the same level of the first object that is receiving the projection. The line of code that is responsible for this action is—proj.farClipPlane = hit.distance + margin;. Regarding the Line Renderer, we have opted to create it dynamically, via code, instead of manually adding the component to the game object. The code is also responsible for setting up its appearance, updating the line vertices position, and changing its color whenever the fire button is pressed, giving it a glowing/pulsing look. For more details on how the script works, don't forget to check out the commented code, available within the 1362_06_03 | End folder. Reflecting surrounding objects with Reflection Probes If you want your scene's environment to be reflected by game objects, featuring reflective materials (such as the ones with high Metallic or Specular levels), then you can achieve such effect using Reflection Probes. They allow for real-time, baked, or even custom reflections through the use of Cubemaps. Real-time reflections can be expensive in terms of processing; in which case, you should favor baked reflections, unless it's really necessary to display dynamic objects being reflected (mirror-like objects, for instance). Still, there are some ways real-time reflections can be optimized. In this recipe, we will test three different configurations for reflection probes: Real-time reflections (constantly updated) Real-time reflections (updated on-demand) via script Baked reflections (from the Editor) Getting ready For this recipe, we have prepared a basic scene, featuring three sets of reflective objects: one is constantly moving, one is static, and one moves whenever it is interacted with. The Probes.unitypackage package that is containing the scene can be found inside the 1362_06_04 folder. How to do it... To reflect the surrounding objects using the Reflection probes, follow these steps: Import Probes.unitypackage to a new project. Then, open the scene named Probes. This is a basic scene featuring three sets of reflective objects. Play the scene. Observe that one of the systems is dynamic, one is static, and one rotates randomly, whenever a key is pressed. Stop the scene. First, let's create a constantly updated real-time reflection probe. From the Create drop-down button of the Hierarchy view, add a Reflection Probe to the scene (Create | Light | Reflection Probe). Name it as RealtimeProbe and make it a child of the System 1 Realtime | MainSphere game object. Then, from the Inspector view, the Transform component, change its Position to X: 0; Y: 0; Z: 0, as shown: Now, go to the Reflection Probe component. Set Type as Realtime; Refresh Mode as Every Frame and Time Slicing as No time slicing, shown as follows: Play the scene. The reflections will be now be updated in real time. Stop the scene. Observe that the only object displaying the real-time reflections is System 1 Realtime | MainSphere. The reason for this is the Size of the Reflection Probe. From the Reflection Probe component, change its Size to X: 25; Y: 10; Z: 25. Note that the small red spheres are now affected as well. However, it is important to notice that all objects display the same reflection. Since our reflection probe's origin is placed at the same location as the MainSphere, all reflective objects will display reflections from that point of view. If you want to eliminate the reflection from the reflective objects within the reflection probe, such as the small red spheres, select the objects and, from the Mesh Renderer component, set Reflection Probes as Off, as shown in the following screenshot: Add a new Reflection Probe to the scene. This time, name it OnDemandProbe and make it a child of the System 2 On Demand | MainSphere game object. Then, from the Inspector view, Transform component, change its Position to X: 0; Y: 0; Z: 0. Now, go to the Reflection Probe component. Set Type as Realtime, Refresh Mode as Via scripting, and Time Slicing as Individual faces, as shown in the following screenshot: Using the Create drop-down menu in the Project view, create a new C# Script named UpdateProbe. Open your script and replace everything with the following code: using UnityEngine; using System.Collections; public class UpdateProbe : MonoBehaviour { private ReflectionProbe probe; void Awake () { probe = GetComponent<ReflectionProbe> (); probe.RenderProbe(); } public void RefreshProbe(){ probe.RenderProbe(); } } Save your script and attach it to the OnDemandProbe. Now, find the script named RandomRotation, which is attached to the System 2 On Demand | Spheres object, and open it in the code editor. Right before the Update() function, add the following lines: private GameObject probe; private UpdateProbe up; void Awake(){ probe = GameObject.Find("OnDemandProbe"); up = probe.GetComponent<UpdateProbe>(); } Now, locate the line of code called transform.eulerAngles = newRotation; and, immediately after it, add the following line: up.RefreshProbe(); Save the script and test your scene. Observe how the Reflection Probe is updated whenever a key is pressed. Stop the scene. Add a third Reflection Probe to the scene. Name it as CustomProbe and make it a child of the System 3 On Custom | MainSphere game object. Then, from the Inspector view, the Transform component, change its Position to X: 0; Y: 0; Z: 0. Go to the Reflection Probe component. Set Type as Custom and click on the Bake button, as shown: A Save File dialog window will show up. Save the file as CustomProbe-reflectionHDR.exr. Observe that the reflection map does not include the reflection of red spheres on it. To change this, you have two options: set the System 3 On Custom | Spheres GameObject (and all its children) as Reflection Probe Static or, from the Reflection Probe component of the CustomProbe GameObject, check the Dynamic Objects option, as shown, and bake the map again (by clicking on the Bake button). If you want your reflection Cubemap to be dynamically baked while you edit your scene, you can set the Reflection Probe Type to Baked, open the Lighting window (the Assets | Lighting menu), access the Scene section, and check the Continuous Baking option as shown. Please note that this mode won't include dynamic objects in the reflection, so be sure to set System 3 Custom | Spheres and System 3 Custom | MainSphere as Reflection Probe Static. How it works... The Reflection Probes element act like omnidirectional cameras that render Cubemaps and apply them onto the objects within their constraints. When creating Reflection Probes, it's important to be aware of how the different types work: Real-time Reflection Probes: Cubemaps are updated at runtime. The real-time Reflection Probes have three different Refresh Modes: On Awake (Cubemap is baked once, right before the scene starts); Every frame (Cubemap is constantly updated); Via scripting (Cubemap is updated whenever the RenderProbe function is used).Since Cubemaps feature six sides, the Reflection Probes features Time Slicing, so each side can be updated independently. There are three different types of Time Slicing: All Faces at Once (renders all faces at once and calculates mipmaps over 6 frames. Updates the probe in 9 frames); Individual Faces (each face is rendered over a number of frames. It updates the probe in 14 frames. The results can be a bit inaccurate, but it is the least expensive solution in terms of frame-rate impact); No Time Slicing (The Probe is rendered and mipmaps are calculated in one frame. It provides high accuracy, but it also the most expensive in terms of frame-rate). Baked: Cubemaps are baked during editing the screen. Cubemaps can be either manually or automatically updated, depending whether the Continuous Baking option is checked (it can be found at the Scene section of the Lighting window). Custom: The Custom Reflection Probes can be either manually baked from the scene (and even include Dynamic objects), or created from a premade Cubemap. There's more... There are a number of additional settings that can be tweaked, such as Importance, Intensity, Box Projection, Resolution, HDR, and so on. For a complete view on each of these settings, we strongly recommend that you read Unity's documentation on the subject, which is available at http://docs.unity3d.com/Manual/class-ReflectionProbe.html. Setting up an environment with Procedural Skybox and Directional Light Besides the traditional 6 Sided and Cubemap, Unity now features a third type of skybox: the Procedural Skybox. Easy to create and setup, the Procedural Skybox can be used in conjunction with a Directional Light to provide Environment Lighting to your scene. In this recipe, we will learn about different parameters of the Procedural Skybox. Getting ready For this recipe, you will need to import Unity's Standard Assets Effects package, which you should have installed when installing Unity. How to do it... To set up an Environment Lighting using the Procedural Skybox and Directional Light, follow these steps: Create a new scene inside a Unity project. Observe that a new scene already includes two objects: the Main Camera and a Directional Light. Add some cubes to your scene, including one at Position X: 0; Y: 0; Z: 0 scaled to X: 20; Y: 1; Z: 20, which is to be used as the ground, as shown: Using the Create drop-down menu from the Project view, create a new Material and name it MySkybox. From the Inspector view, use the appropriate drop-down menu to change the Shader of MySkybox from Standard to Skybox/Procedural. Open the Lighting window (menu Window | Lighting), access the Scene section. At the Environment Lighting subsection, populate the Skybox slot with the MySkybox material, and the Sun slot with the Directional Light from the Scene. From the Project view, select MySkybox. Then, from the Inspector view, set Sun size as 0.05 and Atmosphere Thickness as 1.4. Experiment by changing the Sky Tint color to RGB: 148; 128; 128, and the Ground color to a value that resembles the scene cube floor's color (such as RGB: 202; 202; 202). If you feel the scene is too bright, try bringing the Exposure level down to 0.85, shown as follows: Select the Directional Light and change its Rotation to X: 5; Y: 170; Z: 0. Note that the scene should resemble a dawning environment, something like the following scene: Let's make things even more interesting. Using the Create drop-down menu in the Project view, create a new C# Script named RotateLight. Open your script and replace everything with the following code: using UnityEngine; using System.Collections; public class RotateLight : MonoBehaviour { public float speed = -1.0f; void Update () { transform.Rotate(Vector3.right * speed * Time.deltaTime); } } Save it and add it as a component to the Directional Light. Import the Effects Assets package into your project (via the Assets | Import Package | Effects menu). Select the Directional Light. Then, from Inspector view, Light component, populate the Flare slot with the Sun flare. From the Scene section of the Lighting window, find the Other Settings subsection. Then, set Flare Fade Speed as 3 and Flare Strength as 0.5, shown as follows: Play the scene. You will see the sun rising and the Skybox colors changing accordingly. How it works... Ultimately, the appearance of Unity's native Procedural Skyboxes depends on the five parameters that make them up: Sun size: The size of the bright yellow sun that is drawn onto the skybox is located according to the Directional Light's Rotation on the X and Y axes. Atmosphere Thickness: This simulates how dense the atmosphere is for this skybox. Lower values (less than 1.0) are good for simulating the outer space settings. Moderate values (around 1.0) are suitable for the earth-based environments. Values that are slightly above 1.0 can be useful when simulating air pollution and other dramatic settings. Exaggerated values (like more than 2.0) can help to illustrate extreme conditions or even alien settings. Sky Tint: It is the color that is used to tint the skybox. It is useful for fine-tuning or creating stylized environments. Ground: This is the color of the ground. It can really affect the Global Illumination of the scene. So, choose a value that is close to the level's terrain and/or geometry (or a neutral one). Exposure: This determines the amount of light that gets in the skybox. The higher levels simulate overexposure, while the lower values simulate underexposure. It is important to notice that the Skybox appearance will respond to the scene's Directional Light, playing the role of the Sun. In this case, rotating the light around its X axis can create dawn and sunset scenarios, whereas rotating it around its Y axis will change the position of the sun, changing the cardinal points of the scene. Also, regarding the Environment Lighting, note that although we have used the Skybox as the Ambient Source, we could have chosen a Gradient or a single Color instead—in which case, the scene's illumination wouldn't be attached to the Skybox appearance. Finally, also regarding the Environment Lighting, please note that we have set the Ambient GI to Realtime. The reason for this was to allow the real-time changes in the GI, promoted by the rotating Directional Light. In case we didn't need these changes at runtime, we could have chosen the Baked alternative. Summary In this article you have learned and had hands-on approach to a number Unity's lighting system features, such as cookie textures, Reflection maps, Lightmaps, Light and Reflection probes, and Procedural Skyboxes. The article also demonstrated the use of Projectors. Resources for Article: Further resources on this subject: Animation features in Unity 5[article] Scripting Strategies[article] Editor Tool, Prefabs, and Main Menu [article]
Read more
  • 0
  • 0
  • 21105

article-image-converting-xml-pdf
Packt
23 Oct 2009
5 min read
Save for later

Converting XML to PDF

Packt
23 Oct 2009
5 min read
XML is the most suitable format for data exchange, but not for data presentation. Adobe's PDF and Microsoft Excel's spreadsheet are the commonly used formats for data presentation. If you receive an XML file containing data that needs to be included in a PDF or Excel spreadsheet file, you need to convert the XML file to the relevant format. Some of the commonly used XML-to-PDF conversion tools/APIs are discussed in the following table: Tool/API Description iText iText is a Java library for generating a PDF document. iText may be downloaded from http://www.lowagie.com/iText/. Stylus Studio XML Publisher XML Publisher is a report designer, which supports many data sources, including XML, to generate PDF reports. Stylus Studio XML Editor XML Editor supports XML conversion to PDF. Apache FOP The Apache project provides an open source FO processor called Apache FOP to render an XSL-FO document as a PDF document. We will discuss the Apache FOP processor in this chapter. XMLMill for Java XMLMill may be used to generate PDF documents from XML data combined with XSL and XSLT. XMLMill may be downloaded from http://www.xmlmill.com/. RenderX XEP RenderX provides an XSL-FO processor called XEP that may be used to generate a PDF document.   We can convert an XML file data to a PDF document using any of these tools/APIs in JDeveloper 11g. Here, we will use Apache FOP API. The XSL specification consists of two components: a language for transforming XML documents (XSLT), and XML syntax for specifying formatting objects (XSL-FO). Using XSL-FO, the layout, fonts, and representations of the data may be formatted. Apache FOP (Formatting Objects Processor) is a print formatter for converting XSL formatting objects (XSL-FO) to an output format such as PDF, PCL, PS, SVG, XML, Print, AWT, MIF, or TXT. In this  article, we will convert an XML document to PDF using XSL-FO and the FOP processor in Oracle JDeveloper 11g.The procedure to create a PDF document from an XML file using the Apache FOP processor in JDeveloper is as follows: Create an XML document. Create an XSL stylesheet. Convert the XML document to an XSL-FO document. Convert the XSL-FO document to a PDF file. Setting the environment We need to download the FOP JAR file fop-0.20.5-bin.zip (or a later version) from http://archive.apache.org/dist/xmlgraphics/fop/binaries/ and extract the ZIP file to a directory. To develop an XML-to-PDF conversion application, we need to create an application (ApacheFOP, for example) and a project (ApacheFOP for example) in JDeveloper. In the project add an XML document, catalog.xml, with File | New. In the New Gallery window select Categories | General | XML and Items | XML Document. Click on OK. In the Create XML File window specify a File Name, catalog.xml, and click on OK. A catalog.xml file gets added to the ApacheFOP project. Copy the following catalog.xml listing to catalog.xml: <?xml version="1.0" encoding="UTF-8"?><catalog title="Oracle Magazine" publisher="Oracle Publishing"> <journal edition="September-October 2008"> <article> <title>Share 2.0</title> <author>Alan Joch</author> </article> <article> <title>Restrictions Apply</title> <author>Alan Joch</author> </article> </journal> <journal edition="March-April 2008"> <article> <title>Oracle Database 11g Redux</title> <author>Tom Kyte</author> </article> <article> <title>Declarative Data Filtering</title> <author>Steve Muench</author> </article> </journal></catalog> We also need to add an XSL stylesheet to convert the XML document to an XSL-FO document. Create an XSL stylesheet with File | New. In the New Gallery window, select Categories | General | XML and Items | XSL Stylesheet. Click on OK. In the Create XSL File window specify a File Name (catalog.xsl) and click on OK. A catalog.xsl file gets added to the ApacheFOP project. To convert the XML document to an XSL-FO document and subsequently create a PDF file from the XSL-FO file, we need a Java application. Add a Java class,  XMLToPDF.java, with File | New. In the New Gallery window select Categories | General and Items | Java Class. Click on OK. In the Create Java Class window specify a class Name (XMLToPDF for example) and click on OK. A Java class gets added to the ApacheFOP project. The directory structure of the FOP application is shown in the following illustration: Next, add the FOP JAR files to the project. Select the project node (ApacheFOP node) and then Tools | Project Properties. In the Project Properties window, select Libraries and Classpath. Add the Oracle XML Parser v2 library with the  Add Library button. The JAR files required to develop an FOP application are listed in the following table: JAR File Description <FOP>/fop-0.20.5/build/fop.jar Apache FOP API <FOP>/fop-0.20.5/lib/batik.jar Graphics classes <FOP>/fop-0.20.5/lib/ avalon-framework-cvs-20020806.jar Logger classes <FOP>/fop-0.20.5/lib/ xercesImpl-2.2.1.jar The DOMParser and the SAXParser classes
Read more
  • 0
  • 0
  • 21099
article-image-how-to-build-a-real-time-data-pipeline-for-web-developers-part-1-tutorial
Sugandha Lahoti
29 Aug 2018
12 min read
Save for later

How to build a real-time data pipeline for web developers - Part 1 [Tutorial]

Sugandha Lahoti
29 Aug 2018
12 min read
There are many differences between the idealized usage of ML algorithms and real-world usage. This post gives advice related to using ML in the real world, in real applications, and in production environments. Specifically, we will talk about how to build a real-time data pipeline. The article aims to answer the following questions How do you collect, store, and process gigabytes or terabytes of training data? How and where do you store and distribute serialized models to clients? How do you collect new training examples from millions of users? This post is extracted from the book Hands-on Machine Learning with JavaScript by Burak Kanber. The book is a  definitive guide to creating an intelligent web application with the best of machine learning and JavaScript. What are Data pipelines? When developing a production ML system, it's not likely that you will have the training data handed to you in a ready-to-process format. Production ML systems are typically part of larger application systems, and the data that you use will probably originate from several different sources. The training set for an ML algorithm may be a subset of your larger database, combined with images hosted on a Content Delivery Network (CDN) and event data from an Elasticsearch server. The process of ushering data through various stages of a life cycle is called data pipelining. Data pipelining may include data selectors that run SQL or Elasticsearch queries for objects, event subscriptions which allow data to flow in from event-or log-based data, aggregations, joins, combining data with data from third-party APIs, sanitization, normalization, and storage. In an ideal implementation, the data pipeline acts as an abstraction layer between the larger application environment and the ML process. The ML algorithm should be able to read the output of the data pipeline without any knowledge of the original source of the data, similar to our examples. As there are many possible data sources and infinite ways to architect an application, there is no one-size-fits-all data pipeline. However, most data pipelines will contain these components, which we will discuss in the following sections: Data querying and event subscription Data joining or aggregation Transformation and normalization Storage and delivery This article is a two-part post. In the first part, we will talk about Data Querying and event subscription and Data joining. Data querying Imagine an application such as Disqus, which is an embeddable comment form that website owners can use to add comment functionality to blog posts or other pages. The primary functionality of Disqus is to allow users to like or leave comments on posts, however, as an additional feature and revenue stream, Disqus can make content recommendations and display them alongside sponsored content. The content recommendation system is an example of an ML system that is only one feature of a larger application. A content recommendation system in an application such as Disqus does not necessarily need to interact with the comment data, but might use the user's likes history to generate recommendations similar to the current page. Such a system would also need to analyze the text content of the liked pages and compare that to the text content of all pages in the network in order to make recommendations. Disqus does not need the post's content in order to provide comment functionality, but does need to store metadata about the page (like its URL and title) in its database. The post content may therefore not reside in the application's main database, though the likes and page metadata would likely be stored there. A data pipeline built around Disqus's recommendation system needs first to query the main database for pages the user has liked—or pages that were liked by users who liked the current page—and return their metadata. In order to find similar content, however, the system will need to use the text content of each liked post. This data might be stored in a separate system, perhaps a secondary database such as MongoDB or Elasticsearch, or in Amazon S3 or some other data warehouse. The pipeline will need to retrieve the text content based on the metadata returned by the main database, and associate the content with the metadata. This is an example of multiple data selectors or data sources in the early stages of a data pipeline. One data source is the primary application data, which stores post and likes metadata. The other data source is a secondary server which stores the post's text content. The next step in this pipeline might involve finding a number of candidate posts similar to the ones the user has liked, perhaps through a request to Elasticsearch or some other service that can find similar content. Similar content is not necessarily the correct content to serve, however, so these candidate articles will ultimately be ranked by an (hypothetical) ANN in order to determine the best content to display. In this example, the input to the data pipeline is the current page and the output from the data pipeline is a list of, say, 200 similar pages that the ANN will then rank. If all the necessary data resides in the primary database, the entire pipeline can be achieved with an SQL statement and some JOINs. Even in this case, care should be taken to develop a degree of abstraction between the ML algorithm and the data pipeline, as you may decide to update the application's architecture in the future. In other cases, however, the data will reside in separate locations and a more considered pipeline should be developed. There are many ways to build this data pipeline. You could develop a JavaScript module that performs all the pipeline tasks, and in some cases, you could even write a bash script using standard Unix tools to accomplish the task. On the other end of the complexity spectrum, there are purpose-built tools for data pipelining such as Apache Kafka and AWS Pipeline. These systems are designed modularly and allow you to define a specific data source, query, transformation, and aggregation modules as well as the workflows that connect them. In AWS Pipeline, for instance, you define data nodes that understand how to interact with the various data sources in your application. The earliest stage of a pipeline is typically some sort of data query operation. Training examples must be extracted from a larger database, keeping in mind that not every record in a database is necessarily a training example. In the case of a spam filter, for instance, you should only select messages that have been marked as spam or not spam by a user. Messages that were automatically marked as spam by a spam filter should probably not be used for training, as that might cause a positive feedback loop that ultimately causes an unacceptable false positive rate. Similarly, you may want to prevent users that have been blocked or banned by your system from influencing your model training. A bad actor could intentionally mislead an ML model by taking inappropriate actions on their own data, so you should disqualify these data points as training examples. Alternatively, if your application is such that recent data points should take precedence over older training points, your data query operation might set a time-based limit on the data to use for training, or select a fixed limit ordered reverse chronologically. No matter the situation, make sure you carefully consider your data queries as they are an essential first step in your data pipeline. Not all data needs to come from database queries, however. Many applications use a pub/sub or event subscription architecture to capture streaming data. This data could be activity logs aggregated from a number of servers, or live transaction data from a number of sources. In these cases, an event subscriber will be an early part of your data pipeline. Note that event subscription and data querying are not mutually exclusive operations. Events that come in through a pub/sub system can still be filtered based on various criteria; this is still a form of data querying. One potential issue with an event subscription model arises when it's combined with a batch-training scheme. If you require 5,000 data points but receive only 100 per second, your pipeline will need to maintain a buffer of data points until the target size is reached. There are various message-queuing systems that can assist with this, such as RabbitMQ or Redis. A pipeline requiring this type of functionality might hold messages in a queue until the target of 5,000 messages is achieved, and only then release the messages for batch processing through the rest of the pipeline. In the case that data is collected from multiple sources, it most likely will need to be joined or aggregated in some manner. Let's now take a look at a situation where data needs to be joined to data from an external API. Data joining and aggregation Let's return to our example of the Disqus content recommendation system. Imagine that the data pipeline is able to query likes and post metadata directly from the primary database, but that no system in the applications stores the post's text content. Instead, a microservice was developed in the form of an API that accepts a post ID or URL and returns the page's sanitized text content. In this case, the data pipeline will need to interact with the microservice API in order to get the text content for each post. This approach is perfectly valid, though if the frequency of post content requests is high, some caching or storage should probably be implemented. The data pipeline will need to employ an approach similar to the buffering of messages in the event subscription model. The pipeline can use a message queue to queue posts that still require content, and make requests to the content microservice for each post in the queue until the queue is depleted. As each post's content is retrieved it is added to the post metadata and stored in a separate queue for completed requests. Only when the source queue is depleted and the sink queue is full should the pipeline move on to the next step. Data joining does not necessarily need to involve a microservice API. If the pipeline collects data from two separate sources that need to be combined, a similar approach can be employed. The pipeline is the only component that needs to understand the relationship between the two data sources and formats, leaving both the data sources and the ML algorithm to operate independently of those details. The queue approach also works well when a data aggregation is required. An example of this situation is a pipeline in which the input is streaming input data and the output is token counts or value aggregations. Using a message queue is desirable in these situations as most message queues ensure that a message can be consumed only once, therefore preventing any duplication by the aggregator. This is especially valuable when the event stream is very high frequency, such that tokenizing each event as it comes in would lead to backups or server overload. Because message queues ensure that each message is consumed only once, high-frequency event data can stream directly into a queue where messages are consumed by multiple workers in parallel. Each worker might be responsible for tokenizing the event data and then pushing the token stream to a different message queue. The message queue software ensures that no two workers process the same event, and each worker can operate as an independent unit that is only concerned with tokenization. As the tokenizers push their results onto a new message queue, another worker can consume those messages and aggregate token counts, delivering its own results to the next step in the pipeline every second or minute or 1,000 events, whatever is appropriate for the application. The output of this style of pipeline might be fed into a continually updating Bayesian model, for example. One benefit of a data pipeline designed in this manner is performance. If you were to attempt to subscribe to high-frequency event data, tokenize each message, aggregate token counts, and update a model all in one system, you might be forced to use a very powerful (and expensive) single server. The server would simultaneously need a high-performance CPU, lots of RAM, and a high-throughput network connection. By breaking up the pipeline into stages, however, you can optimize each stage of the pipeline for its specific task and load condition. The message queue that receives the source event stream needs only to receive the event stream but does not need to process it. The tokenizer workers do not necessarily need to be high-performance servers, as they can be run in parallel. The aggregating queue and worker will process a large volume of data but will not need to retain data for longer than a few seconds and therefore may not need much RAM. The final model, which is a compressed version of the source data, can be stored on a more modest machine. Many components of the data pipeline can be built of commodity hardware simply because a data pipeline encourages modular design. In many cases, you will need to transform your data from format to format throughout the pipeline. That could mean converting from native data structures to JSON, transposing or interpolating values, or hashing values. We talked about two data pipelines components. In the next post, we will discuss several types of data transformations that may occur in the data pipeline. We will also discuss a few considerations to make when transporting and storing training data or serialized models. Create machine learning pipelines using unsupervised AutoML [Tutorial] Top AutoML libraries for building your ML pipelines
Read more
  • 0
  • 0
  • 21096

article-image-google-cloud-next19-day-1-open-source-partnerships-hybrid-cloud-platform-cloud-run-and-more
Bhagyashree R
10 Apr 2019
6 min read
Save for later

Google Cloud Next’19 day 1: open-source partnerships, hybrid-cloud platform, Cloud Run, and more

Bhagyashree R
10 Apr 2019
6 min read
Google Cloud Next’ 19 kick started yesterday in San Francisco. On day 1 of the event, Google showcased its new tools for application developers, its partnership with open-source companies, and outlined its strategy to make a mark in the Cloud industry, which is currently dominated by Amazon and Microsoft. Here’s the rundown of the announcements Google made yesterday: Google Cloud’s new CEO is set to expand its sales team Cloud Next’19 is the first event where the newly-appointed Google Cloud CEO, Thomas Kurian took on stage to share his plans for Google Cloud. He plans to make Google Cloud "the best strategic partner" for organizations modernizing their IT infrastructure. To step up its game in the Cloud industry, Google needs to put more focus on understanding its customers, providing them better support, and making it easier for them to conduct business. This is why Kurian is planning to expand the sales team and add more technical specialists. Kurian, who joined Google after working at Oracle for 22 years, also shared that the team is rolling out new contracts to make contracting easier and also promised simplified pricing. Anthos, Google’s hybrid cloud platform is coming to AWS and Azure During the opening keynote, Sundar Pichai, Google’s CEO confirmed the rebranding of Cloud Services Platform, a platform for building and managing hybrid applications, as it enters general availability. This rebranded version named Anthos provides customers a single managed service, which is not limited to just Google-based environments and comes with extended support for Amazon Web Services (AWS) and Azure. With this extended support, Google aims to provide organizations that have multi-cloud sourcing strategy a more consistent experience across all three clouds. Urs Hölzle, Google’s Senior Vice President for Technical Infrastructure, shared in a press conference, “I can’t really stress how big a change that is in the industry, because this is really the stack for the next 20 years, meaning that it’s not really about the three different clouds that are all randomly different in small ways. This is the way that makes these three clouds — and actually on-premise environments, too — look the same.” Along with this extended support, another plus point of Anthos is that it is hardware agnostic, which means customers can run the service on top of their current hardware without having to immediately invest in new servers. It is a subscription-based service, with prices starting at $10,000/month per 100 vCPU block. Google also announced the first beta release of Anthos Migrate, a service that auto-migrates VMs from on-premises, or other clouds, directly into containers in Google Kubernetes Environment (GKE) with minimum effort. Explaining the advantage of this tool, Google wrote in a blog post, “Through this transformation, your IT team is free from managing infrastructure tasks like VM maintenance and OS patching, so it can focus on managing and developing applications.” Google Cloud partners with top open-source projects challenging AWS Google has partnered with several top open-source data management and analytics companies including Confluent, DataStax, Elastic, InfluxData, MongoDB, Neo4j and Redis Labs. The services and products provided by these companies will be deeply integrated into the Google Cloud Platform. With this integration, Google aims to provide customers a seamless experience by allowing them to use these open source technologies at a single place, Google Cloud. These will be managed services and the invoicing and billing of these services will be handled by Google Cloud. Customer support will also be the responsibility of Google so that users manage and log tickets across all of these services via a single platform. Google’s approach of partnering with these open source companies is quite different from that of other cloud providers. Over the past few years, we have come across cases where cloud providers sell open-source projects as service, often without giving any credits to the original project. This led to companies revisiting their open-source licenses to stop such behavior. For instance, Redis adopted the Common Clause license for its Redis Modules and later dropped its revised license in February. Similarly, MongoDB, Neo4j, and Confluent also embraced a similar strategy. Kurian said, “In order to sustain the company behind the open-source technology, they need a monetization vehicle. If the cloud provider attacks them and takes that away, then they are not viable and it deteriorates the open-source community.” Cloud Run for running stateless containers serverlessly Google has combined serverless computing and containerization into a single product called Cloud Run. Yesterday, Oren Teich, Director Product Management for Serverless, announced the beta release of Cloud Run and also explained how it works. Cloud Run is a managed compute platform for running stateless containers that can be invoked via HTTP requests. It is built on top of Knative, a Kubernetes-based platform for building, deploying, and managing serverless workloads. You get two options to choose from, either you can run your containers fully-managed with Cloud Run or in your Google Kubernetes Engine cluster with Cloud Run on GKE. Announcing the release of Cloud Run, Teich wrote in a blog post, “Cloud Run is introducing a brand new product that takes Docker containers and instantly gives you a URL. This is completely unique in the industry. We’re taking care of everything from the top end of SSL provisioning and routing, all the way down to actually running the container for you. You pay only by the hundred milliseconds of what you need to use, and it’s end-to-end managed.” Google releases closed source VS Code plugin Google announced the beta release of “Cloud Code for VS Code” as a closed source library. It allows you to extend the VS Code to bring the convenience of IDEs to developing cloud-native Kubernetes applications. This extension aims to speed up the builds, deployment, and debugging cycles. You can deploy your applications to either local clusters or across multiple cloud providers. Under the hood, Cloud Code for VS Code uses Google’s popular command-line tools such as skaffold and kubectl, to provide users continuous feedback as they build their projects. It also supports deployment profiles that lets you define different environments to make testing and debugging easier on your workstation or in the cloud. Cloud SQL now supports PostgreSQL 11.1 Beta Cloud SQL is Google’s fully-managed database service that makes it easier to set up, maintain, manage, and administer your relational databases on GCP. It now comes with support for PostgreSQL 11.1 Beta. Along with that, it supports the following relational databases: MySQL 5.5, 5.6, and 5.7 PostgreSQL 9.6 Google’s Cloud Healthcare API is now available in beta Ian Goodfellow quits Google and joins Apple as a director of machine learning Google Podcasts is transcribing full podcast episodes for improving search results  
Read more
  • 0
  • 0
  • 21072

Packt
06 Feb 2014
6 min read
Save for later

Bitcoins – Pools and Mining

Packt
06 Feb 2014
6 min read
(For more resources related to this topic, see here.) Installing Bitcoind Bitcoind is the software that connects you to other nodes in the network. There is no single point of failure, and the more nodes there are the faster and more secure the network becomes. Peers are rewarded with a transaction fee for the first validation of a transaction and are assigned randomly. After installing Bitcoind by using the following command, we have to synchronize it with the network, which means downloading millions of transactions exceeding a gigabyte in size to your Pi. sudo apt-get install bitcoind Before running the background daemon, you need to consider changing the data directory as the database files can take up many gigabytes of space. Mount a new storage place and always start Bitcoind with the following line: bitcoind –datadir /mnt/HDD/bitcoin –daemon After a few minutes, you can type in the following basic commands. You will need to wait until the entire block chain data is downloaded before you can do anything useful. During this time, the Pi might become less responsive, use a lot of bandwidth, and create a large amount of data in the data directory. This can take up to 12 hours. bitcoind getinfobitcoind getblockcount If you already have most of the block chain data downloaded on another computer, you can just copy all the data to the Pi. You can even have the same wallet and addresses in both locations. It is not recommended though, as that means it's twice as easy for hackers to get your wallet. Always encrypt your wallet with a very strong password and keep the password and wallet backup in a safe, offline place—like CDs or USB drives. If you lose the password, you lose access to the wallet, forever. Bitcoin wallet You need to generate addresses where other people can send you funds. Theoretically, you can generate an unlimited number of addresses, but practically managing all those addresses would become difficult. Bitcoin, by default, allows you to have a hundred addresses. You always get a default wallet address, which counts as the first address, and this address will receive network fees if you process any transactions for the first time. Cryptocurrency transactions cannot be reversed. Sending coins to the incorrect address means your coins will be lost forever. On the other hand, it also protects you if you are receiving coins from somebody else, as they cannot reverse the transaction. These addresses are globally unique, and can never be generated by anybody else in the universe or beyond it. It can get stolen by hackers or viruses, though. Imagine losing your wallet in real life. Unless a kind person returns it to you, the contents of that wallet will be lost forever. You can back up your wallet file, but you should store it on any two USB flash drives or CDs, and store it in a safety deposit box at home or in the bank. You should never be tempted to copy your wallet to any kind of online storage or backup services, especially if it's unencrypted. Creating a Bitcoin address These examples will work with the command line, as you can re-use them to display data on your web server. All online wallets use the Bitcoind API to generate addresses and manage your wallet. As a word of warning, you should never use online wallet services as almost all of them have been hacked, resulting in massive numbers of coins getting stolen. You can download the Bitcoin-QT client and run it in X if you prefer an easy way to manage your wallet. The following command lines will create a new receiving address and then list all your addresses: bitcoind getnewaddressbitcoind listaccounts Receiving Bitcoins As long as you know your address, people can send you Bitcoins. You do not need to have Bitcoind running all the time as the transactions are processes and are stored in the network. It is just very important to keep a backup of your wallet.dat file if you want to send the Bitcoins somewhere else, like an online exchange. Sending Bitcoins As long as you have coins in your wallet, you can easily send coins to another address by using the sendtoaddress command followed by the address and the amount. Here is an example sending 0.01 Bitcoins to the author's tip jar: bitcoind sendtoaddress 126HA8L1dP7fr7ECu2oEMtz6JpDRnzqp9N 0.01 You will get a response, which is a transaction ID. If you do not, the transaction has failed. You need to have at least 0.0001 Bitcoins reserved for the transaction fees. If you are sending large amounts of coins, this transaction fee will also become more expensive. The value of Bitcoins In July 2010, the value of Bitcoin was below 1 USD. Investing in Bitcoins was very controversial and a huge risk, since it was difficult to predict if the currency would be accepted or rejected by the community. As of writing this article, the Bitcoin value has exceeded 1100 USD. You can search the Internet and find interesting articles on early adopters mining Bitcoins, or buying a few hundred dollars and leaving their wallets lying around. Just like one person in the UK who threw away his hard drive with 7500 BTC stored on it. After going public, thousands of people flocked to the public dump ground to search for the hard drive. Other stories include a student who invested a few hundred dollars in 2010, and sold them for hundreds of thousands of dollars. With such large amounts, it is important to consult your local TAX authority or council. The trend seems to be that the more people find out about Bitcoins and the more media publicity it gets, the higher the value of the currency rises. This also looks like it applies to alternative currencies such as Litecoin. Summary We have seen what Bitcoin is. We have seen what are the uses of Bitcoins and also it's value. We have also seen how to send and receive Bitcoins. I hope this has given you a better idea about what Bitcoins really are. Resources for Article: Further resources on this subject: Clusters, Parallel Computing, and Raspberry Pi – A Brief Background [Article] Creating a file server (Samba) [Article] Our First Project – A Basic Thermometer [Article]
Read more
  • 0
  • 0
  • 21068
article-image-central-air-and-heating-thermostat
Packt
03 Mar 2015
15 min read
Save for later

Central Air and Heating Thermostat

Packt
03 Mar 2015
15 min read
In this article by Andrew K. Dennis, author of the book Raspberry Pi Home Automation with Arduino Second Edition, you will learn how to build a thermostat device using an Arduino. You will also learn how to use the temperature data to switch relays on and off. Relays are the main components that you can use for interaction between your Arduino and high-voltage electronic devices. The thermostat will also provide a web interface so that you can connect to it and check out the temperature. (For more resources related to this topic, see here.) Introducing the thermostat A thermostat is a control device that is used to manipulate other devices based on a temperature setting. This temperature setting is known as the setpoint. When the temperature changes in relation to the setpoint, a device can be switched on or off. For example, let's imagine a system where a simple thermostat is set to switch an electric heater on when the temperature drops below 25 degrees Celsius. Within our thermostat, we have a temperature-sensing device such as a thermistor that returns a temperature reading every few seconds. When the thermistor reads a temperature below the setpoint (25 degrees Celsius), the thermostat will switch a relay on, completing the circuit between the wall plug and our electric heater and providing it with power. Thus, we can see that a simple electronic thermostat can be used to switch on a variety of devices. Warren S. Johnson, a college professor in Wisconsin, is credited with inventing the electric room thermostat in the 1880s. Johnson was known throughout his lifetime as a prolific inventor who worked in a variety of fields, including electricity. These electric room thermostats became a common feature in homes across the course of the twentieth century as larger parts of the world were hooked up the electricity grid. Now, with open hardware electronic tools such as the Arduino available, we can build custom thermostats for a variety of home projects. They can be used to control baseboard heaters, heat lamps, and air conditioner units. They can also be used for the following: Fish tank heaters Indoor gardens Electric heaters Fans Now that we have explored the uses of thermostats, let's take a look at our project. Setting up our hardware In the following examples, we will list the pins to which you need to connect your hardware. However, we recommend that when you purchase any device such as the Ethernet shield, you check whether certain pins are available or not. Due to the sheer range of hardware available, it is not possible to list every potential hardware combination. Therefore, if the pin in the example is not free, you can update the circuit and source code to use a different pin. When building the example, we also recommend using a breadboard. This will allow you to experiment with building your circuit without having to solder any components. Our first task will be to set up our thermostat device so that it has Ethernet access. Adding the Ethernet shield The Arduino Uno does not contain an Ethernet port. Therefore, you will need a way for your thermostat to be accessible on your home network. One simple solution is to purchase an Ethernet shield and connect it to your microcontroller. There are several shields in the market, including the Arduino Ethernet shield (http://arduino.cc/en/Main/ArduinoEthernetShield) and Seeed Ethernet shield (http://www.seeedstudio.com/wiki/Ethernet_Shield_V1.0). These shields are plugged into the GPIO pins on the Arduino. If you purchase one of these shields, then we would also recommend buying some extra GPIO headers. These are plugged into the existing headers attached to the Ethernet shield. Their purpose is to provide some extra clearance above the Ethernet port on the board so that you can connect other shields in future if you decide to purchase them. Take a board of your choice and attach it to the Arduino Uno. When you plug the USB cable into your microcontroller and into your computer, the lights on both the Uno and Ethernet shield should light up. Now our device has a medium to send and receive data over a LAN. Let's take a look at setting up our thermostat relays. Relays A relay is a type of switch controlled by an electromagnet. It allows us to use a small amount of power to control a much larger amount, for example, using a 9V power supply to switch 220V wall power. Relays are rated to work with different voltages and currents. A relay has three contact points: Normally Open, Common Connection, and Normally Closed. Two of these points will be wired up to our fan. In the context of an Arduino project, the relay will also have a pin for ground, 5V power and a data pin that is used to switch the relay on and off. A popular choice for a relay is the Pololu Basic SPDT Relay Carrier. This can be purchased from http://www.pololu.com/category/135/relay-modules. This relay has featured in some other Packt Publishing books on the Arduino, so it is a good investment. Once you have the relay, you need to wire it up to the microcontroller. Connect a wire from the relay to digital pin 5 on the Arduino, another wire to the GRD pin, and the final wire to the 5V pin. This completes the relay setup. In order to control relays though, we need some data to trigger switching them between on and off. Our thermistor device handles the task of collecting this data. Connecting the thermistor A thermistor is an electronic component that, when included in a circuit, can be used to measure temperature. The device is a type of resistor that has the property whereby its resistance varies as the temperature changes. It can be found in a variety of devices, including thermostats and electronic thermometers. There are two categories of thermistors available: Negative Thermistor Coefficient (NTC) and Positive Thermistor Coefficient (PTC). The difference between them is that as the temperature increases, the resistance decreases in the case of an NTC, and on the other hand, it increases in the case of a PTC. We are going to use a prebuilt digital device with the model number AM2303. This can be purchased at https://www.adafruit.com/products/393. This device reads both temperature and humidity. It also comes with a software library that you can use in your Arduino sketches. One of the benefits of this library is that many functions that precompute values, such as temperature in Celsius, are available and thus don't require you to write a lot of code. Take your AM203 and connect it to the GRD pin, 5V pin and digital pin 4. The following diagram shows how it should be set up: You are now ready to move on to creating the software to test for temperature readings. Setting up our software We now need to write an application in the Arduino IDE to control our new thermostat device. Our software will contain the following: The code responsible for collecting the temperature data Methods to switch relays on and off based on this data Code to handle accepting incoming HTTP requests so that we can view our thermostat's current temperature reading and change the setpoint A method to send our temperature readings to the Raspberry Pi The next step is to hook up our Arduino thermostat with the USB port of the device we installed the IDE on. You may need to temporarily disconnect your relay from the Arduino. This will prevent your thermostat device from drawing too much power from your computer's USB port, which may result in the port being disabled. We now need to download the DHT library that interacts with our AM2303. This can be found on GitHub, at https://github.com/adafruit/DHT-sensor-library. Click on the Download ZIP link and unzip the file to a location on your hard drive. Next, we need to install the library to make it accessible from our sketch: Open the Arduino IDE. Navigate to Sketch | Import Library. Next, click on Add library. Choose the folder on your hard drive. You can now use the library. With the library installed, we can include it in our sketch and access a number of useful functions. Let's now start creating our software. Thermostat software We can start adding some code to the Arduino to control our thermostat. Open a new sketch in the Arduino IDE and perform the following steps: Inside the sketch, we are going to start by adding the code to include the libraries we need to use. At the top of the sketch, add the following code: #include "DHT.h" // Include this if using the AM2302 #include <SPI.h> #include <Ethernet.h> Next, we will declare some variables to be used by our application. These will be responsible for defining:     The pin the AM2303 thermistor is located on     The relay pin     The IP address we want our Arduino to use, which should be unique     The Mac address of the Arduino, which should also be unique     The name of the room the thermostat is located in     The variables responsible for Ethernet communication The IP address will depend on your own home network. Check out your wireless router to see what range of IP addresses is available. Select an address that isn't in use and update the IPAddress variable as follows: #define DHTPIN 4 // The digital pin to read from #define DHTTYPE DHT22 // DHT 22 (AM2302)   unsigned char relay = 5; //The relay pins String room = "library"; byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED }; IPAddress ip(192,168,3,5); DHT dht(DHTPIN, DHTTYPE); EthernetServer server(80); EthernetClient client; We can now include the setup() function. This is responsible for initializing some variables with their default values, and setting the pin to which our relay is connected to output mode: void setup() {   Serial.begin(9600);   Ethernet.begin(mac, ip);   server.begin();   dht.begin();   pinMode(relay, OUTPUT); } The next block of code we will add is the loop() function. This contains the main body of our program to be executed. Here, we will assign a value to the setpoint and grab our temperature readings: void loop() {   int setpoint = 25;   float h = dht.readHumidity();   float t = dht.readTemperature(); Following this, we check whether the temperature is above or below the setpoint and switch the relay on or off as needed. Paste this code below the variables you just added: if(t <setpoint) {   digitalWrite(relay,HIGH); } else {   digitalWrite(relay,LOW); } Next, we need to handle the HTTP requests to the thermostat. We start by collecting all of the incoming data. The following code also goes inside the loop() function: client = server.available(); if (client) {   // an http request ends with a blank line   booleancurrentLineIsBlank = true;   String result;   while (client.connected()) {     if (client.available()) {       char c = client.read();       result= result + c;     } With the incoming request stored in the result variable, we can examine the HTTP header to know whether we are requesting an HTML page or a JSON object. You'll learn more about JavaScript Object Notation (JSON) shortly. If we request an HTML page, this is displayed in the browser. Next, add the following code to your sketch: if(result.indexOf("text/html") > -1) {   client.println("HTTP/1.1 200 OK");   client.println("Content-Type: text/html");   client.println();   if (isnan(h) || isnan(t)) {     client.println("Failed to read from DHT sensor!");     return;   }   client.print("<b>Thermostat</b> set to: ");   client.print(setpoint);    client.print("degrees C <br />Humidity: ");   client.print(h);   client.print(" %t");   client.print("<br />Temperature: ");   client.print(t);   client.println(" degrees C ");   break; } The following code handles a request for the data to be returned in JSON format. Our Raspberry Pi will make HTTP requests to the Arduino, and then process the data returned to it. At the bottom of this last block of code is a statement adding a short delay to allow the Arduino to process the request and close the client connection. Paste this final section of code in your sketch: if( result.indexOf("application/json") > -1 ) { client.println("HTTP/1.1 200 OK"); client.println("Content-Type: application/json;charset=utf-8"); client.println("Server: Arduino"); client.println("Connnection: close"); client.println(); client.print("{"thermostat":[{"location":""); client.print(room); client.print(""},"); client.print("{"temperature":""); client.print(t); client.print(""},"); client.print("{"humidity":""); client.print(h); client.print(""},"); client.print("{"setpoint":""); client.print(setpoint); client.print(""}"); client.print("]}"); client.println(); break;           }     } delay(1); client.stop();   }  } This completes our program. We can now save it and run the Verify process. Click on the small check mark in a circle located in the top-left corner of the sketch. If you have added all of the code correctly, you should see Binary sketch size: 16,962 bytes (of a 32,256 byte maximum). Now that our code is verified and saved, we can look at uploading it to the Arduino, attaching the fan, and testing our thermostat. Testing our thermostat and fan We have our hardware set up and the code ready. Now we can test the thermostat and see it in action with a device connected to the mains electricity. We will first attach a fan and then run the sketch to switch it on and off. Attaching the fan Ensure that your Arduino is powered down and that the fan is not plugged into the wall. Using a wire stripper and cutters, cut one side of the cable that connects the plug to the fan body. Take the end of the cable attached to the plug, and attach it to the NO point on the relay. Use a screwdriver to ensure that it is fastened correctly. Now, take the other portion of the cut cable that is attached to the fan body, and attach this to the COM point. Once again, use a screwdriver to ensure that it is fastened securely to the relay. Your connection should look as follows: You can now reattach your Arduino to the computer via its USB cable. However, do not plug the fan into the wall yet. Starting your thermostat application With the fan connected to our relay, we can upload our sketch and test it: From the Arudino IDE, select the upload icon. Once the code has been uploaded, disconnect your Arduino board. Next, connect an Ethernet cable to your Arduino. Following this, plug the Arduino into the wall to get mains power. Finally, connect the fan to the wall outlet. You should hear the clicking sound of the relay as it switches on or off depending on the room temperature. When the relay switch is on (or off), the fan will follow suit. Using a separate laptop if you have it, or from your Raspberry Pi, access the IP address you specified in the application via a web browser, for example, http://192.168.3.5/. You should see something similar to this: Thermostat set to: 25degrees C  Humidity: 35.70 % Temperature: 14.90 degrees C You can now stimulate the thermistor using an ice cube and hair dryer, to switch the relay on and off, and the fan will follow suit. If you refresh your connection to the IP address, you should see the change in the temperature output on the screen. You can use the F5 key to do this. Let's now test the JSON response. Testing the JSON response A format useful in transferring data between applications is JavaScript Object Notation (JSON). You can read more about this on the official JSON website, at http://www.json.org/. The purpose of us generating data in JSON format is to allow the Raspberry Pi control device we are building to query the thermostat periodically and collect the data being generated. We can verify that we are getting JSON data back from the sketch by making an HTTP request using the application/json header. Load a web browser such as Google Chrome or FireFox. We are going to make an XML HTTP request directly from the browser to our thermostat. This type of request is commonly known as an Asynchronous JavaScript and XML (AJAX) request. It can be used to refresh data on a page without having to actually reload it. In your web browser, locate and open the developer tools. The following link lists the location and shortcut keys in major browsers: http://webmasters.stackexchange.com/questions/8525/how-to-open-the-javascript-console-in-different-browsers In the JavaScript console portion of the developer tools, type the following JavaScript code: var xmlhttp; xmlhttp=new XMLHttpRequest(); xmlhttp.open("POST","192.168.3.5",true); xmlhttp.setRequestHeader("Content-type","application/json"); xmlhttp.onreadystatechange = function() {//Call a function when the state changes.    if(xmlhttp.readyState == 4 &&xmlhttp.status == 200) {          console.log(xmlhttp);    } }; xmlhttp.send() Press the return key or run option to execute the code. This will fire an HTTP request, and you should see a JSON object return: "{"thermostat":     [      {"location":"library"},      {"temperature":"14.90"},      {"humidity":"29.90"},      {"setpoint":"25"}   ] }" This confirms that our application can return data to the Raspberry Pi. We have tested our software and hardware and seen that they are working. Summary In this article, we built a thermostat device. We looked at thermistors, and we learned how to set up an Ethernet connection. To control our thermostat, we wrote an Arduino sketch, uploaded it to the microcontroller, and then tested it with a fan plugged into the mains electricity. Resources for Article: Further resources on this subject: The Raspberry Pi and Raspbian? [article] Clusters Parallel Computing and Raspberry Pi Brief Background [article] The Arduino Mobile Robot [article]
Read more
  • 0
  • 0
  • 21062

article-image-getting-started-nwjs
Packt
21 May 2015
19 min read
Save for later

Getting Started with NW.js

Packt
21 May 2015
19 min read
In this article by Alessandro Benoit, author of the book NW.js Essentials, we will learn that until a while ago, developing a desktop application that was compatible with the most common operating systems required an enormous amount of expertise, different programming languages, and logics for each platform. (For more resources related to this topic, see here.) Yet, for a while now, the evolution of web technologies has brought to our browsers many web applications that have nothing to envy from their desktop alternative. Just think of Google apps such as Gmail and Calendar, which, for many, have definitely replaced the need for a local mail client. All of this has been made possible thanks to the amazing potential of the latest implementations of the Browser Web API combined with the incredible flexibility and speed of the latest server technologies. Although we live in a world increasingly interconnected and dependent on the Internet, there is still the need for developing desktop applications for a number of reasons: To overcome the lack of vertical applications based on web technologies To implement software solutions where data security is essential and cannot be compromised by exposing data on the Internet To make up for any lack of connectivity, even temporary Simply because operating systems are still locally installed Once it's established that we cannot completely get rid of desktop applications and that their implementation on different platforms requires an often prohibitive learning curve, it comes naturally to ask: why not make desktop applications out of the very same technologies used in web development? The answer, or at least one of the answers, is NW.js! NW.js doesn't need any introduction. With more than 20,000 stars on GitHub (in the top four hottest C++ projects of the repository-hosting service) NW.js is definitely one of the most promising projects to create desktop applications with web technologies. Paraphrasing the description on GitHub, NW.js is a web app runtime that allows the browser DOM to access Node.js modules directly. Node.js is responsible for hardware and operating system interaction, while the browser serves the graphic interface and implements all the functionalities typical of web applications. Clearly, the use of the two technologies may overlap; for example, if we were to make an asynchronous call to the API of an online service, we could use either a Node.js HTTP client or an XMLHttpRequest Ajax call inside the browser. Without going into technical details, in order to create desktop applications with NW.js, all you need is a decent understanding of Node.js and some expertise in developing HTML5 web apps. In this article, we are going to dissect the topic dwelling on these points: A brief technical digression on how NW.js works An analysis of the pros and cons in order to determine use scenarios Downloading and installing NW.js Development tools Making your first, simple "Hello World" application Important notes about NW.js (also known as Node-Webkit) and io.js Before January 2015, since the project was born, NW.js was known as Node-Webkit. Moreover, with Node.js getting a little sluggish, much to the concern of V8 JavaScript engine updates, from version 0.12.0, NW.js is not based on Node.js but on io.js, an npm-compatible platform originally based on Node.js. For the sake of simplicity, we will keep referring to Node.js even when talking about io.js as long as this does not affect a proper comprehension of the subject. NW.js under the hood As we stated in the introduction, NW.js, made by Roger Wang of Intel's Open Source Technology Center (Shanghai office) in 2011, is a web app runtime based on Node.js and the Chromium open source browser project. To understand how it works, we must first analyze its two components: Node.js is an efficient JavaScript runtime written in C++ and based on theV8 JavaScript engine developed by Google. Residing in the operating system's application layer, Node.js can access hardware, filesystems, and networking functionalities, enabling its use in a wide range of fields, from the implementation of web servers to the creation of control software for robots. (As we stated in the introduction, NW.js has replaced Node.js with io.js from version 0.12.0.) WebKit is a layout engine that allows the rendering of web pages starting from the DOM, a tree of objects representing the web page. NW.js is actually not directly based on WebKit but on Blink, a fork of WebKit developed specifically for the Chromium open source browser project and based on the V8 JavaScript engine as is the case with Node.js. Since the browser, for security reasons, cannot access the application layer and since Node.js lacks a graphical interface, Roger Wang had the insight of combining the two technologies by creating NW.js. The following is a simple diagram that shows how Node.js has been combined with WebKit in order to give NW.js applications access to both the GUI and the operating system: In order to integrate the two systems, which, despite speaking the same language, are very different, a couple of tricks have been adopted. In the first place, since they are both event-driven (following a logic of action/reaction rather than a stream of operations), the event processing has been unified. Secondly, the Node context was injected into WebKit so that it can access it. The amazing thing about it is that you'll be able to program all of your applications' logic in JavaScript with no concerns about where Node.js ends and WebKit begins. Today, NW.js has reached version 0.12.0 and, although still young, is one of the most promising web app runtimes to develop desktop applications adopting web technologies. Features and drawbacks of NW.js Let's check some of the features that characterize NW.js: NW.js allows us to realize modern desktop applications using HTML5, CSS3, JS, WebGL, and the full potential of Node.js, including the use of third-party modules The Native UI API allows you to implement native lookalike applications with the support of menus, clipboards, tray icons, and file binding Since Node.js and WebKit run within the same thread, NW.js has excellent performance With NW.js, it is incredibly easy to port existing web applications to desktop applications Thanks to the CLI and the presence of third-party tools, it's really easy to debug, package, and deploy applications on Microsoft Windows, Mac OS, and Linux However, all that glitters is not gold. There are some cons to consider when developing an application with NW.js: Size of the application: Since a copy of NW.js (70-90 MB) must be distributed along with each application, the size of the application makes it quite expensive compared to native applications. Anyway, if you're concerned about download times, compressing NW.js for distribution will save you about half the size. Difficulties in distributing your application through Mac App Store: In this article, it will not be discussed (just do a search on Google), but even if the procedure is rather complex, you can distribute your NW.js application through Mac App Store. At the moment, it is not possible to deploy a NW.js application on Windows Store due to the different architecture of .appx applications. Missing support for iOS or Android: Unlike other SDKs and libraries, at the moment, it is not possible to deploy an NW.js application on iOS or Android, and it does not seem to be possible to do so in the near future. However, the portability of the HTML, JavaScript, and CSS code that can be distributed on other platforms with tools such as PhoneGap or TideSDK should be considered. Unfortunately, this is not true for all of the features implemented using Node.js. Stability: Finally, the platform is still quite young and not bug-free. NW.js – usage scenarios The flexibility and good performance of NW.js allows its use in countless scenarios, but, for convenience, I'm going to report only a few notable ones: Development tools Implementation of the GUI around existing CLI tools Multimedia applications Web services clients Video games The choice of development platform for a new project clearly depends only on the developer; for the overall aim of confronting facts, it may be useful to consider some specific scenarios where the use of NW.js might not be recommended: When developing for a specific platform, graphic coherence is essential, and, perhaps, it is necessary to distribute the application through a store If the performance factor limits the use of the preceding technologies If the application does a massive use of the features provided by the application layer via Node.js and it has to be distributed to mobile devices Popular NW.js applications After summarizing the pros and cons of NW.js, let's not forget the real strength of the platform—the many applications built on top of NW.js that have already been distributed. We list a few that are worth noting: Wunderlist for Windows 7: This is a to-do list / schedule management app used by millions Cellist: This is an HTTP debugging proxy available on Mac App Store Game Dev Tycoon: This is one of the first NW.js games that puts you in the shoes of a 1980s game developer Intel® XDK: This is an HTML5 cross-platform solution that enables developers to write web and hybrid apps Downloading and installing NW.js Installing NW.js is pretty simple, but there are many ways to do it. One of the easiest ways is probably to run npm install nw from your terminal, but for the educational purposes, we're going to manually download and install it in order to properly understand how it works. You can find all the download links on the project website at http://nwjs.io/ or in the Downloads section on the GitHub project page at https://github.com/nwjs/nw.js/; from here, download the package that fits your operating system. For example, as I'm writing this article, Node-Webkit is at version 0.12.0, and my operating system is Mac OS X Yosemite 10.10 running on a 64-bit MacBook Pro; so, I'm going to download the nwjs-v0.12.0-osx-x64.zip file. Packages for Mac and Windows are zipped, while those for Linux are in the tar.gz format. Decompress the files and proceed, depending on your operating system, as follows. Installing NW.js on Mac OS X Inside the archive, we're going to find three files: Credits.html: This contains credits and licenses of all the dependencies of NW.js nwjs.app: This is the actual NW.js executable nwjc: This is a CLI tool used to compile your source code in order to protect it Before v0.12.0, the filename of nwjc was nwsnapshot. Currently, the only file that interests us is nwjs.app (the extension might not be displayed depending on the OS configuration). All we have to do is copy this file in the /Applications folder—your main applications folder. If you'd rather install NW.js using Homebrew Cask, you can simply enter the following command in your terminal: $ brew cask install nw If you are using Homebrew Cask to install NW.js, keep in mind that the Cask repository might not be updated and that the nwjs.app file will be copied in ~/Applications, while a symlink will be created in the /Applications folder. Installing NW.js on Microsoft Windows Inside the Microsoft Windows NW.js package, we will find the following files: credits.html: This contains the credits and licenses of all NW.js dependencies d3dcompiler_47.dll: This is the Direct3D library ffmpegsumo.dll: This is a media library to be included in order to use the <video> and <audio> tags icudtl.dat: This is an important network library libEGL.dll: This is the WebGL and GPU acceleration libGLESv2.dll: This is the WebGL and GPU acceleration locales/: This is the languages folder nw.exe: This is the actual NW.js executable nw.pak: This is an important JS library pdf.dll: This library is used by the web engine for printing nwjc.exe: This is a CLI tool to compile your source code in order to protect it Some of the files in the folder will be omitted during the final distribution of our application, but for development purposes, we are simply going to copy the whole content of the folder to C:/Tools/nwjs. Installing NW.js on Linux On Linux, the procedure can be more complex depending on the distribution you use. First, copy the downloaded archive into your home folder if you have not already done so, and then open the terminal and type the following command to unpack the archive (change the version accordingly to the one downloaded): $ gzip -dc nwjs-v0.12.0-linux-x64.tar.gz | tar xf - Now, rename the newly created folder in nwjs with the following command: $ mv ~/nwjs-v0.12.0-linux-x64 ~/nwjs Inside the nwjs folder, we will find the following files: credits.html: This contains the credits and licenses of all the dependencies of NW.js icudtl.dat This is an important network library libffmpegsumo.so: This is a media library to be included in order to use the <video> and <audio> tags locales/: This is a languages folder nw: This is the actual NW.js executable nw.pak: This is an important JS library nwjc: This is a CLI tool to compile your source code in order to protect it Open the folder inside the terminal and try to run NW.js by typing the following: $ cd nwjs$ ./nw If you get the following error, you are probably using a version of Ubuntu later than 13.04, Fedora later than 18, or another Linux distribution that uses libudev.so.1 instead of libudev.so.0: otherwise, you're good to go to the next step: error while loading shared libraries: libudev.so.0: cannot open shared object file: No such file or directory Until NW.js is updated to support libudev.so.1, there are several solutions to solve the problem. For me, the easiest solution is to type the following terminal command inside the directory containing nw: $ sed -i 's/udev.so.0/udev.so.1/g' nw This will replace the string related to libudev, within the application code, with the new version. The process may take a while, so wait for the terminal to return the cursor before attempting to enter the following: $ ./nw Eventually, the NW.js window should open properly. Development tools As you'll make use of third-party modules of Node.js, you're going to need npm in order to download and install all the dependencies; so, Node.js (http://nodejs.org/) or io.js (https://iojs.org/) must be obviously installed in your development environment. I know you cannot wait to write your first application, but before you start, I would like to introduce you to Sublime Text 2. It is a simple but sophisticated IDE, which, thanks to the support for custom build scripts, allows you to run (and debug) NW.js applications from inside the editor itself. If I wasn't convincing and you'd rather keep using your favorite IDE, you can skip to the next section; otherwise, follow these steps to install and configure Sublime Text 2: Download and install Sublime Text 2 for your platform from http://www.sublimetext.com/. Open it and from the top menu, navigate to Tools | Build System | New Build System. A new edit screen will open; paste the following code depending on your platform: On Mac OS X: {"cmd": ["nwjs", "--enable-logging",     "${project_path:${file_path}}"],"working_dir": "${project_path:${file_path}}","path": "/Applications/nwjs.app/Contents/MacOS/"} On Microsoft Windows: {"cmd": ["nw.exe", "--enable-logging",     "${project_path:${file_path}}"],"working_dir": "${project_path:${file_path}}","path": "C:/Tools/nwjs/","shell": true} On Linux: {"cmd": ["nw", "--enable-logging",     "${project_path:${file_path}}"],"working_dir": "${project_path:${file_path}}","path": "/home/userName/nwjs/"} Type Ctrl + S (Cmd + S on Mac) and save the file as nw-js.sublime-build. Perfect! Now you are ready to run your applications directly from the IDE. There are a lot of packages, such as SublimeLinter, LiveReload, and Node.js code completion, available to Sublime Text 2. In order to install them, you have to install Package Control first. Just open https://sublime.wbond.net/installation and follow the instructions. Writing and running your first "Hello World" app Finally, we are ready to write our first simple application. We're going to revisit the usual "Hello World" application by making use of a Node.js module for markdown parsing. "Markdown is a plain text formatting syntax designed so that it can be converted to HTML and many other formats using a tool by the same name."                                                                                                              – Wikipedia Let's create a Hello World folder and open it in Sublime Text 2 or in your favorite IDE. Now open a new package.json file and type in the following JSON code: {"name": "nw-hello-world","main": "index.html","dependencies": {   "markdown": "0.5.x"}} The package.json manifest file is essential for distribution as it determines many of the window properties and primary information about the application. Moreover, during the development process, you'll be able to declare all of the dependencies. In this specific case, we are going to assign the application name, the main file, and obviously our dependency, the markdown module, written by Dominic Baggott. If you so wish, you can create the package.json manifest file using the npm init command from the terminal as you're probably used to already when creating npm packages. Once you've saved the package.json file, create an index.html file that will be used as the main application file and type in the following code: <!DOCTYPE html><html><head>   <title>Hello World!</title></head><body>   <script>   <!--Here goes your code-->   </script></body></html> As you can see, it's a very common HTML5 boilerplate. Inside the script tag, let's add the following: var markdown = require("markdown").markdown,   div = document.createElement("div"),   content = "#Hello World!n" +   "We are using **io.js** " +   "version *" + process.version + "*"; div.innerHTML = markdown.toHTML(content);document.body.appendChild(div); What we do here is require the markdown module and then parse the content variable through it. To keep it as simple as possible, I've been using Vanilla JavaScript to output the parsed HTML to the screen. In the highlighted line of code, you may have noticed that we are using process.version, a property that is a part of the Node.js context. If you try to open index.html in a browser, you'd get the Reference Error: require is not defined error as Node.js has not been injected into the WebKit process. Once you have saved the index.html file, all that is left is to install the dependencies by running the following command from the terminal inside the project folder: $ npm install And we are ready to run our first application! Running NW.js applications on Sublime Text 2 If you opted for Sublime Text 2 and followed the procedure in the development tools section, simply navigate to Project | Save Project As and save the hello-world.sublime-project file inside the project folder. Now, in the top menu, navigate to Tools | Build System and select nw-js. Finally, press Ctrl + B (or Cmd + B on Mac) to run the program. If you have opted for a different IDE, just follow the upcoming steps depending on your operating system. Running NW.js applications on Microsoft Windows Open the command prompt and type: C:> c:Toolsnwjsnw.exe c:pathtotheproject On Microsoft Windows, you can also drag the folder containing package.json to nw.exe in order to open it. Running NW.js applications on Mac OS Open the terminal and type: $ /Applications/nwjs.app/Contents/MacOS/nwjs /path/to/the/project/ Or, if running NW.js applications inside the directory containing package.json, type: $ /Applications/nwjs.app/Contents/MacOS/nwjs. As you can see in Mac OS X, the NW.js kit's executable binary is in a hidden directory within the .app file. Running NW.js applications on Linux Open the terminal and type: $ ~/nwjs/nw /path/to/the/project/ Or, if running NW.js applications inside the directory containing package.json, type: $ ~/nwjs/nw . Running the application, you may notice that a few errors are thrown depending on your platform. As I stated in the pros and cons section, NW.js is still young, so that's quite normal, and probably we're talking about minor issues. However, you can search in the NW.js GitHub issues page in order to check whether they've already been reported; otherwise, open a new issue—your help would be much appreciated. Now, regardless of the operating system, a window similar to the following one should appear: As illustrated, the process.version object variable has been printed properly as Node.js has correctly been injected and can be accessed from the DOM. Perhaps, the result is a little different than what you expected since the top navigation bar of Chromium is visible. Do not worry! You can get rid of the navigation bar at any time simply by adding the window.toolbar = false parameter to the manifest file, but for now, it's important that the bar is visible in order to debug the application. Summary In this article, you discovered how NW.js works under the hood, the recommended tools for development, a few usage scenarios of the library, and eventually, how to run your first, simple application using third-party modules of Node.js. I really hope I haven't bored you too much with the theoretical concepts underlying the functioning of NW.js; I really did my best to keep it short.
Read more
  • 0
  • 0
  • 21053
Modal Close icon
Modal Close icon