Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Save more on your purchases! discount-offer-chevron-icon
Savings automatically calculated. No voucher code required.
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Events
Videos
Audiobooks
Packt Hub
Free Learning
Arrow right icon
timer SALE ENDS IN
0 Days
:
00 Hours
:
00 Minutes
:
00 Seconds

How-To Tutorials

7018 Articles
article-image-building-grid-system-susy
Packt
09 Aug 2016
14 min read
Save for later

Building a Grid System with Susy

Packt
09 Aug 2016
14 min read
In this article by Luke Watts, author of the book Mastering Sass, we will build a responsive grid system using the Susy library and a few custom mixins and functions. We will set a configuration map with our breakpoints which we will then loop over to automatically create our entire grid, using interpolation to create our class names. (For more resources related to this topic, see here.) Detailing the project requirements For this example, we will need bower to download Susy. After Susy has been downloaded we will only need two files. We'll place them all in the same directory for simplicity. These files will be style.scss and _helpers.scss. We'll place the majority of our SCSS code in style.scss. First, we'll import susy and our _helpers.scss at the beginning of this file. After that we will place our variables and finally our code which will create our grid system. Bower and Susy To check if you have bower installed open your command line (Terminal on Unix or CMD on Windows) and run: bower -v If you see a number like "1.7.9" you have bower. If not you will need to install bower using npm, a package manager for NodeJS. If you don't already have NodeJS installed, you can download it from: https://nodejs.org/en/. To install bower from your command line using npm you will need to run: npm install -g bower Once bower is installed cd into the root of your project and run: bower install susy This will create a directory called bower_components. Inside that you will find a folder called susy. The full path to file we will be importing in style.scss is bower_components/susy/sass/_susy.scss. However we can leave off the underscore (_) and also the extension (.scss). Sass will still load import the file just fine. In style.scss add the following at the beginning of our file: // style.scss @import 'bower_components/susy/sass/susy'; Helpers (mixins and functions) Next, we'll need to import our _helpers.scss file in style.scss. Our _helpers.scss file will contain any custom mixins or functions we'll create to help us in building our grid. In style.scss import _helpers.scss just below where we imported Susy: // style.scss @import 'bower_components/susy/sass/susy'; @import 'helpers'; Mixin: bp (breakpoint) I don't know about you, but writing media queries always seems like bit of a chore to me. I just don't like to write (min-width: 768px) all the time. So for that reason I'm going to include the bp mixin, which means instead of writing: @media(min-width: 768px) { // ... } We can simply use: @include bp(md) { // ... } First we are going to create a map of our breakpoints. Add the $breakpoints map to style.scss just below our imports: // style.scss @import 'bower_components/susy/sass/susy'; @import 'helpers'; $breakpoints: ( sm: 480px, md: 768px, lg: 980px ); Then, inside _helpers.scss we're going to create our bp mixin which will handle creating our media queries from the $breakpoints map. Here's the breakpoint (bp) mixin: @mixin bp($size: md) { @media (min-width: map-get($breakpoints, $size)) { @content; } } Here we are setting the default breakpoint to be md (768px). We then use the built in Sass function map-get to get the relevant value using the key ($size). Inside our @media rule we use the @content directive which will allows us pass any Sass or CSS directly into our bp mixin to our @media rule. The container mixin The container mixin sets the max-width of the containing element, which will be the .container element for now. However, it is best to use the container mixin to semantically restrict certain parts of the design to your max width instead of using presentational classes like container or row. The container mixin takes a width argument, which will be the max-width. It also automatically applies the micro-clearfix hack. This prevents the containers height from collapsing when the elements inside it are floated. I prefer the overflow: hidden method myself, but they do the same thing essentially. By default, the container will be set to max-width: 100%. However, you can set it to be any valid unit of dimension, such as 60em, 1160px, 50%, 90vw, or whatever. As long as it's a valid CSS unit it will work. In style.scss let's create our .container element using the container mixin: // style.scss .container { @include container(1160px); } The preceding code will give the following CSS output: .container { max-width: 1160px; margin-left: auto; margin-right: auto; } .container:after { content: " "; display: block; clear: both; } Due to the fact the container uses a max-width we don't need to specify different dimensions for various screen sizes. It will be 100% until the screen is above 1160px and then the max-width value will kick in. The .container:after rule is the micro-clearfix hack. The span mixin To create columns in Susy we use the span mixin. The span mixin sets the width of that element and applies a padding or margin depending on how Susy is set up. By default, Susy will apply a margin to the right of each column, but you can set it to be on the left, or to be padding on the left or right or padding or margin on both sides. Susy will do the necessary work to make everything work behind the scenes. To create a half width column in a 12 column grid you would use: .col-6 { @include span(6 of 12); } The of 12 let's Susy know this is a 12 column grid. When we define our $susy map later we can tell Susy how many columns we are using via the columns property. This means we can drop the of 12 part and simply use span(6) instead. Susy will then know we are using 12 columns unless we explicitly pass another value. The preceding SCSS will output: .col-6 { width: 49.15254%; float: left; margin-right: 1.69492%; } Notice the width and margin together would actually be 50.84746%, not 50% as you might expect. Therefor two of these column would actually be 101.69492%. That will cause the last column to wrap to the next row. To prevent this, you would need to remove the margin from the last column. The last keyword To address this, Susy uses the last keyword. When you pass this to the span mixin it lets Susy know this is the last column in a row. This removes the margin right and also floats the element in question to the right to ensure it's at the very end of the row. Let's take the previous example where we would have two col-6 elements. We could create a class of col-6-last and apply the last keyword to that span mixin: .col-6 { @include span(6 of 12); &-last { @include span(last 6 of 12) } } The preceding SCSS will output: .col-6 { width: 49.15254%; float: left; margin-right: 1.69492%; } .col-6-last { width: 49.15254%; float: right; margin-right: 0; } You can also place the last keyword at the end. This will also work: .col-6 { @include span(6 of 12); &-last { @include span(6 of 12 last) } } The $susy configuration map Susy allows for a lot of configuration through its configuration map which is defined as $susy. The settings in the $susy map allow us to set how wide the container should be, how many columns our grid should have, how wide the gutters are, whether those gutters should be margins or padding, and whether the gutters should be on the left, right or both sides of each column. Actually, there are even more settings available depending what type of grid you'd like to build. Let's, define our $susy map with the container set to 1160px just after our $breakpoints map: // style.scss $susy: ( container: 1160px, columns: 12, gutters: 1/3 ); Here we've set our containers max-width to be 1160px. This is used when we use the container mixin without entering a value. We've also set our grid to be 12 columns with the gutters, (padding or margin) to be 1/3 the width of a column. That's about all we need to set for our purposes, however, Susy has a lot more to offer. In fact, to cover everything in Susy would need an entirely book of its own. If you want to explore more of what Susy can do you should read the documentation at http://susydocs.oddbird.net/en/latest/. Setting up a grid system We've all used a 12 column grid which has various sizes (small, medium, large) or a set breakpoint (or breakpoints). These are the most popular methods for two reasons...it works, and it's easy to understand. Furthermore, with the help of Susy we can achieve this with less than 30 lines of Sass! Don't believe me? Let's begin. The concept of our grid system Our grid system will be similar to that of Foundation and Bootstrap. It will have 3 breakpoints and will be mobile-first. It will have a container, which will act as both .container and .row, therefore removing the need for a .row class. The breakpoints Earlier we defined three sizes in our $breakpoints map. These were: $breakpoints: ( sm: 480px, md: 768px, lg: 980px ); So our grid will have small, medium and large breakpoints. The columns naming convention Our columns will use a similar naming convention to that of Bootstrap. There will be four available sets of columns. The first will start from 0px up to the 399px (example: .col-12) The next will start from 480px up to 767px (example: .col-12-sm) The medium will start from 768px up to 979px (example: .col-12-md) The large will start from 980px (example: .col-12-lg) Having four options will give us the most flexibility. Building the grid From here we can use an @for loop and our bp mixin to create our four sets of classes. Each will go from 1 through 12 (or whatever our Susy columns property is set to) and will use the breakpoints we defined for small (sm), medium (md) and large (lg). In style.scss add the following: // style.scss @for $i from 1 through map-get($susy, columns) { .col-#{$i} { @include span($i); &-last { @include span($i last); } } } These 9 lines of code are responsible for our mobile-first set of column classes. This loops from one through 12 (which is currently the value of the $susy columns property) and creates a class for each. It also adds a class which handles removing the final columns right margin so our last column doesn't wrap onto a new line. Having control of when this happens will give us the most control. The preceding code would create: .col-1 { width: 6.38298%; float: left; margin-right: 2.12766%; } .col-1-last { width: 6.38298%; float: right; margin-right: 0; } /* 2, 3, 4, and so on up to col-12 */ That means our loop which is only 9 lines of Sass will generate 144 lines of CSS! Now let's create our 3 breakpoints. We'll use an @each loop to get the sizes from our $breakpoints map. This will mean if we add another breakpoint, such as extra-large (xl) it will automatically create the correct set of classes for that size. @each $size, $value in $breakpoints { // Breakpoint will go here and will use $size } Here we're looping over the $breakpoints map and setting a $size variable and a $value variable. The $value variable will not be used, however the $size variable will be set to small, medium and large for each respective loop. We can then use that to set our bp mixin accordingly: @each $size, $value in $breakpoints { @include bp($size) { // The @for loop will go here similar to the above @for loop... } } Now, each loop will set a breakpoint for small, medium and large, and any additional sizes we might add in the future will be generated automatically. Now we can use the same @for loop inside the bp mixin with one small change, we'll add a size to the class name: @each $size, $value in $breakpoints { @include bp($size) { @for $i from 1 through map-get($susy, columns) { .col-#{$i}-#{$size} { @include span($i); &-last { @include span($i last); } } } } } That's everything we need for our grid system. Here's the full stye.scss file: / /style.scss @import 'bower_components/susy/sass/susy'; @import 'helpers'; $breakpoints: ( sm: 480px, md: 768px, lg: 980px ); $susy: ( container: 1160px, columns: 12, gutters: 1/3 ); .container { @include container; } @for $i from 1 through map-get($susy, columns) { .col-#{$i} { @include span($i); &-last { @include span($i last); } } } @each $size, $value in $breakpoints { @include bp($size) { @for $i from 1 through map-get($susy, columns) { .col-#{$i}-#{$size} { @include span($i); &-last { @include span($i last); } } } } } With our bp mixin that's 45 lines of SCSS. And how many lines of CSS does that generate? Nearly 600 lines of CSS! Also, like I've said, if we wanted to create another breakpoint it would only require a change to the $breakpoint map. Then, if we wanted to have 16 columns instead we would only need to the $susy columns property. The above code would then automatically loop over each and create the correct amount of columns for each breakpoint. Testing our grid Next we need to check our grid works. We mainly want to check a few column sizes for each breakpoint and we want to be sure our last keyword is doing what we expect. I've created a simple piece of HTML to do this. I've also add a small bit of CSS to the file to correct box-sizing issues which will happen because of the additional 1px border. I've also restricted the height so text which wraps to a second line won't affect the heights. This is simply so everything remains in line so it's easy to see our widths are working. I don't recommend setting heights on elements. EVER. Instead using padding or line-height if you can to give an element more height and let the content dictate the size of the element. Create a file called index.html in the root of the project and inside add the following: <!doctype html> <html lang="en-GB"> <head> <meta charset="UTF-8"> <title>Susy Grid Test</title> <link rel="stylesheet" type="text/css" href="style.css" /> <style type="text/css"> *, *::before, *::after { box-sizing: border-box; } [class^="col"] { height: 1.5em; background-color: grey; border: 1px solid black; } </style> </head> <body> <div class="container"> <h1>Grid</h1> <div class="col-12 col-10-sm col-2-md col-10-lg">.col-sm-10.col-2-md.col-10-lg</div> <div class="col-12 col-2-sm-last col-10-md-last col-2-lg-last">.col-sm-2-last.col-10-md-last.col-2-lg-last</div> <div class="col-12 col-9-sm col-3-md col-9-lg">.col-sm-9.col-3-md.col-9-lg</div> <div class="col-12 col-3-sm-last col-9-md-last col-3-lg-last">.col-sm-3-last.col-9-md-last.col-3-lg-last</div> <div class="col-12 col-8-sm col-4-md col-8-lg">.col-sm-8.col-4-md.col-8-lg</div> <div class="col-12 col-4-sm-last col-8-md-last col-4-lg-last">.col-sm-4-last.col-8-md-last.col-4-lg-last</div> <div class="col-12 col-7-sm col-md-5 col-7-lg">.col-sm-7.col-md-5.col-7-lg</div> <div class="col-12 col-5-sm-last col-7-md-last col-5-lg-last">.col-sm-5-last.col-7-md-last.col-5-lg-last</div> <div class="col-12 col-6-sm col-6-md col-6-lg">.col-sm-6.col-6-md.col-6-lg</div> <div class="col-12 col-6-sm-last col-6-md-last col-6-lg-last">.col-sm-6-last.col-6-md-last.col-6-lg-last</div> </div> </body> </html> Use your dev tools responsive tools or simply resize the browser from full size down to around 320px and you'll see our grid works as expected. Summary In this article we used Susy grids as well as a simple breakpoint mixin (bp) to create a solid, flexible grid system. With just under 50 lines of Sass we generated our grid system which consists of almost 600 lines of CSS.  Resources for Article: Further resources on this subject: Implementation of SASS [article] Use of Stylesheets for Report Designing using BIRT [article] CSS Grids for RWD [article]
Read more
  • 0
  • 0
  • 14613

article-image-optimizing-games-android
Packt
16 Aug 2016
13 min read
Save for later

Optimizing Games for Android

Packt
16 Aug 2016
13 min read
In this article by Avisekhar Roy, the author of The Android Game Developer's Handbook, we will focus on the need for optimization and types of optimization with respect to games for the Android OS. We will also look at some common game development mistakes in this article. (For more resources related to this topic, see here.) The rendering pipeline in Android Let's now have a look at the types of rendering pipelines in Android. The 2D rendering pipeline In thecase of the 2D Android drawing system through Canvas, all the assets are first drawn on the canvas, and the canvas is rendered on screen. The graphic engine maps all the assets within the finite Canvas according to the given position. Many times, developers use small assets separately that cause a mapping instruction to execute for each asset. It is always recommended that you use sprite sheets to merge as many small assets as possible. A single draw call can then be applied to draw every object on the Canvas. Now, the question is how to create the sprite and what are the other consequences. Previously, Android could not support images or sprites of a size more than 1024x1024 pixels. Since Android 2.3, the developer can use a sprite of size 4096x4096. However, using such sprites can cause permanent memory occupancy during the scopes of all the small assets. Many low-configuration Android devices do not support such large images to be loaded during an application. It is a best practice that developers limit themselves to 2048x2048 pixels. This will reduce memory peak as well as significant amount of draw calls to the canvas. The 3D rendering pipeline Android uses OpenGL to render assets on the screen. So,the rendering pipeline for Android 3D is basically theOpenGL pipeline. Let's have look at the OpenGL rendering system: Now, let's have a detailed look at each step of thepreceding rendering flow diagram: The vertex shader processes individual vertices with vertex data. The control shader is responsible for controlling vertex data and patches for the tessellation. The polygon arrangement system arranges the polygon with each pair of intersecting lines created by vertices. Thus, it creates the edges without repeating vertices. Tessellation is the process of tiling the polygons in a shape without overlap or any gap. The geometry shader is responsible for optimizing the primitive shape. Thus triangles are generated. After constructing the polygons and shapes, the model is clipped for optimization. Vertex post processing is used to filter out unnecessary data. The mesh is then rasterized. The fragment shader is used to process fragments generated from rasterization. All the pixels are mapped after fragmentation and processed with the processed data. The mesh is added to the frame buffer for final rendering. Optimizing 2D assets Any digital game cannot be made without 2D art assets. There must be 2D assets in some form inside the game. So, as far as game component optimization is concerned, every 2D asset should also be optimized. Optimization of 2D assets means these three main things. Size optimization Each asset frame should contain only the effective pixels to be used in games. Unnecessary pixels increase the asset size and memory use during runtime. Data optimization Not all images require full data information for pixels. A significant amount of data might be stored in each pixel, depending on the image format. For example, fullscreen opaque images should never contain transparency data. Similarly, depending on the color set, images must be formatted in 8-bit, 16-bit, or 24-bit format. Image optimization tools can be used to perform such optimizations. Process optimization The larger the amount of data compressed during optimization, the more time it takes to decompress it and load it to memory. So, image optimization has a direct effect on the processing speed. From another point of view, creating an image atlas or sprite sheet is another way to reduce the processing time of images. Optimizing 3D assets A 3D art asset has two parts to be optimized. A 2D texture part is to be optimized in the same 2D optimization style. The only thing the developer needs to consider is after optimization, the shader should have the same effect on the structure. Rest of the 3D asset optimization entirely depends on the number of vertices and the model polygon. Limiting polygon count It is very obvious that a large number of polygons used to create the mesh can create more details. However, we all know that Android is a mobile OS, and it always has hardware limitations. The developer should count the number of polygons used in the mesh and the total number of polygons rendered on the screen in a single draw cycle. There is always a limitation depending on the hardware configuration. So, limiting polygon and vertex count per mesh is always an advantage in order to achieve a certain frame rate or performance. Model optimization Models are created with more than one mesh. Using a separate mesh in the final model always results in heavy processing. This is a major effort for the game artist. Multiple overlaps can occur if multiple meshes are used. This increases vertex processing. Rigging is another essential part of finalizing the model. A good rigger defines the skeleton with minimum possible joints for minimum processing. Common game development mistakes It is not always possible to look into each and every performance aspect at every development stage. It is a very common practice to use assets and write code in a temporary mode and use it in the final game. This affects the overall performance and future maintenance procedure. Here are few of the most common mistakes made during game development. Use of non-optimized images An artist creates art assets, and the developer directly integrates those into the game for debug build. However, most of the time, those assets are never optimized even for the release candidate. This is the reason there may be plenty of high-bit images where the asset contains limited information. Alpha information may be found in opaque images. Use of full utility third-party libraries The modernday development style does not require each and every development module to be written from scratch. Most of the developers use a predefined third-party library for common utility mechanisms. Most of the time, these packages come with most of the possible methods, and among them, very few are actually used in games. Developers, most of the time, use these packages without any filtration. A lot of unused data occupies memory during runtime in such cases. Many times, a third-party library comes without any editing facility. In this case, the developer should choose such packages very carefully depending on specific requirements. Use of unmanaged networking connections In modern Android games, use of Internet connectivity is very common. Many games use server-based gameplay. In such cases, the entire game runs on the server with frequent data transfers between the server and the client device.Each data transfer process takes time, and the connectivity drains battery charge significantly. Badly managed networking states often freeze the application. Especially for real-time multiplayer games, a significant amount of data is handled. In this case, a request and response queue should be created and managed properly. However, the developer often skips this part to save development time. Another aspect of unmanaged connections is unnecessary packet data transferred between the server and client. So, there is an extra parsing process involved each time data is transferred. Using substandard programming We have already discussed programming styles and standards. The modular programming approach may increase a few extra processes, but the longer management of programming demandsmodular programming. Otherwise, developers end up repeating code, and this increases process overhead. Memory management also demands a good programming style. In few cases, the developer allocates memory but often forgets to free the memory. This causes a lot of memory leakage. At times,the application crashes due to insufficient memory. Substandard programming includes the following mistakes: Declaring the same variables multiple times Creating many static instances Writing non-modular coding Improper singleton class creation Loading objects at runtime Taking the shortcut This is the funniest fact among ill-practiced development styles. Taking a shortcut during development is very common among game developers. Making games is mostly about logical development. There may be multiple ways of solving a logical problem. Very often, the developer chooses the most convenient way to solve such problems.For example, the developer mostly uses the bubble sorting method for most of the sorting requirements, despite knowing that it is the most inefficient sorting process. Using such shortcuts multiple times in a game may cause a visible process delay, which directly affects frame rate. 2D/3D performance comparison Android game development in 2D and 3D is different. It is a fact that 3D game processing is heavier than 2D games. However, game scale is always the deciding factor. Different look and feel 3D look and feel is way different than 2D. Use of a particle system in 3D games is very common to provide visual effects. In the case of 2D games, sprite animation and other transformations are used to show such effects. Another difference between 2D and 3D look and feel is dynamic light and shadow. Dynamic light is always a factor for greater visual quality. Nowadays, most 3D games use dynamic lighting, which has a significant effect on game performance. In the case of 2D games, light management is done through assets. So, there is no extra processing in 2D games for light and shadow. In 2D games, the game screen is rendered on a Canvas. There is only one fixed point of view. So, the concept of camera is limited to a fixed camera. However, in 3D games, it is a different case. Multiple types of cameras can be implemented. Multiple cameras can be used together for a better feel of the game. Rendering objects through multiple cameras causes more process overhead. Hence, it decreases the frame rate of the game. There is a significant performance difference in using 2D physics and 3D physics. A 3D physics engine is far more process heavy than a 2D physics engine. 3D processing is way heavier than 2D processing It is a common practice in the gaming industry to accept less FPS of 3D games in comparison to 2D games. In Android, the standard accepted FPS for 2D games is around 60FPS, whereas a 3D game is acceptable even if it runs at as low as 40FPS. The logical reason behind that is 3D games are way heavier than 2D games in terms of process. The main reasons are as follows: Vertex processing: In 3D games, each vertex is processed on the OpenGL layer during rendering. So, increasing the number of vertices leads to heavier processing. Mesh rendering: A mesh consists of multiple vertices and many polygons. Processing a mesh increasesthe rendering overhead as well. 3D collision system: A 3D dynamic collisiondetection system demands each vertex of the collider to be calculated for collision. This calculation is usually done by the GPU. 3D physics implementation: 3D transformation calculation completely depends on matrix manipulation, which is always heavy. Multiple camera use: Use of multiple cameras and dynamically setting up the rendering pipeline takes more memory and clock cycles. Device configuration Android has a wide range of device configuration options supported by the platform. Running the same game on different configurations does not produce the same result. Performance depends on the following factors. Processor There are many processors used for Android devices in terms of the number of cores and speed of each core. Speed decides the number of instructions that can be executed in a single cycle.There was a time when Android used to have a single core CPU with speed less than 500MHz. Now.we have multicore CPUs with more than 2GHz speed on each core. RAM Availability of RAM is another factor to decide performance. Heavy games require a greater amount of RAM during runtime. If RAM is limited, then frequent loading/unloading processesaffect performance. GPU GPU decides the rendering speed. It acts as the processing unit for graphical objects. A more powerful processor can process more rendering instructions, resulting in better performance. Display quality Display quality is actually inversely proportional to the performance. Better display quality has to be backed by better GPU, CPU, and RAM, because better displays always consist of bigger resolution with better dpi and more color support. We can see various devices with different display quality. Android itself has divided the assets by this feature: LDPI: Lowest dpi display for Android (~120 dpi) MDPI: Medium dpi display for Android (~160 dpi) HDPI: High dpi display for Android (~240 dpi) XHDPI: Extra high dpi display for Android (~320 dpi) XXHDPI: Extra extra high dpi display for Android (~480 dpi) XXXHDPI: Extra extraextra high dpi display for Android (~640 dpi) It can be easily predicted that the list will include more options in the near future, with the advancement of hardware technology. Battery capacity It is an odd factor in the performance of the application. More powerful CPU, GPU, and RAM demand more power. If the battery is incapable of delivering power, then processing units cannot run at their peak efficiency. To summarize these factors, we can easily make a few relational equations with performance: CPU is directly proportional to performance GPU is directly proportional to performance RAM is directly proportional to performance Display quality is inversely proportional to performance Battery capacity is directly proportional to performance Summary There are many technical differences between 2D and 3D games in terms of rendering, processing, and assets. The developer should always use an optimized approach to create assets and write code. One more way of gaining performance is to port the games for different hardware systems for both 2D and 3D games. We can see a revolutionary upgrade in hardware platforms since the last decade. Accordingly, the nature of games has also changed. However, the scope of 2D games is still there with a large set of possibilities. Gaining performance is more of a logical task than technical. There are a few tools available to do the job, but it is the developer's decision to choose them. So, selecting the right tool for the right purpose is necessary, and there should be a different approach for making 2D and 3D games. Resources for Article: Further resources on this subject: Drawing and Drawables in Android Canvas [article] Hacking Android Apps Using the Xposed Framework [article] Getting started with Android Development [article]
Read more
  • 0
  • 0
  • 14607

article-image-configuring-manage-out-directaccess-clients
Packt
14 Oct 2013
14 min read
Save for later

Configuring Manage Out to DirectAccess Clients

Packt
14 Oct 2013
14 min read
In this article by Jordan Krause, the author of the book Microsoft DirectAccess Best Practices and Troubleshooting, we will have a look at how Manage Out is configured to DirectAccess clients. DirectAccess is obviously a wonderful technology from the user's perspective. There is literally nothing that they have to do to connect to company resources; it just happens automatically whenever they have Internet access. What isn't talked about nearly as often is the fact that DirectAccess is possibly of even greater benefit to the IT department. Because DirectAccess is so seamless and automatic, your Group Policy settings, patches, scripts, and everything that you want to use to manage and manipulate those client machines is always able to run. You no longer have to wait for the user to launch a VPN or come into the office for their computer to be secured with the latest policies. You no longer have to worry about laptops being off the network for weeks at a time, and coming back into the network after having been connected to dozens of public hotspots while someone was on a vacation with it. While many of these management functions work right out of the box with a standard DirectAccess configuration, there are some functions that will need a couple of extra steps to get them working properly. That is our topic of discussion for this article. We are going to cover the following topics: Pulls versus pushes What does Manage Out have to do with IPv6 Creating a selective ISATAP environment Setting up client-side firewall rules RDP to a DirectAccess client No ISATAP with multisite DirectAccess (For more resources related to this topic, see here.) Pulls versus pushes Often when thinking about management functions, we think of them as the software or settings that are being pushed out to the client computers. This is actually not true in many cases. A lot of management tools are initiated on the client side, and so their method of distributing these settings and patches are actually client pulls. A pull is a request that has been initiated by the client, and in this case, the server is simply responding to that request. In the DirectAccess world, this kind of request is handled very differently than an actual push, which would be any case where the internal server or resource is creating the initial outbound communication with the client, a true outbound initiation of packets. Pulls typically work just fine over DirectAccess right away. For example, Group Policy processing is initiated by the client. When a laptop decides that it's time for a Group Policy refresh, it reaches out to Active Directory and says "Hey AD, give me my latest stuff". The Domain Controllers then simply reply to that request, and the settings are pulled down successfully. This works all day, every day over DirectAccess. Pushes, on the other hand, require some special considerations. This scenario is what we commonly refer to as DirectAccess Manage Out, and this does not work by default in a stock DirectAccess implementation. What does Manage Out have to do with IPv6? IPv6 is essentially the reason why Manage Out does not work until we make some additions to your network. "But DirectAccess in Server 2012 handles all of my IPv6 to IPv4 translations so that my internal network can be all IPv4, right?" The answer to that question is yes, but those translations only work in one direction. For example, in our previous Group Policy processing scenario, the client computer calls out for Active Directory, and those packets traverse the DirectAccess tunnels using IPv6. When the packets get to the DirectAccess server, it sees that the Domain Controller is IPv4, so it translates those IPv6 packets into IPv4 and sends them on their merry way. Domain Controller receives the said IPv4 packets, and responds in kind back to the DirectAccess server. Since there is now an active thread and translation running on the DirectAccess server for that communication, it knows that it has to take those IPv4 packets coming back (as a response) from the Domain Controller and spin them back into IPv6 before sending across the IPsec tunnels back to the client. This all works wonderfully and there is absolutely no configuration that you need to do to accomplish this behavior. However, what if you wanted to send packets out to a DirectAccess client computer from inside the network? One of the best examples of this is a Helpdesk computer trying to RDP into a DirectAccess-connected computer. To accomplish this, the Helpdesk computer needs to have IPv6 routability to the DirectAccess client computer. Let's walk through the flow of packets to see why this is necessary. First of all, if you have been using DirectAccess for any duration of time, you might have realized by now that the client computers register themselves in DNS with their DirectAccess IP addresses when they connect. This is a normal behavior, but what may not look "normal" to you is that those records they are registering are AAAA (IPv6) records. Remember, all DirectAccess traffic across the internet is IPv6, using one of these three transition technologies to carry the packets: 6to4, Teredo, or IP-HTTPS. Therefore, when the clients connect, whatever transition tunnel is established has an IPv6 address on the adapter (you can see it inside ipconfig /all on the client), and those addresses will register themselves in DNS, assuming your DNS is configured to allow it. When the Helpdesk personnel types CLIENT1 in their RDP client software and clicks on connect, it is going to reach out to DNS and ask, "What is the IP address of CLIENT1?" One of two things is going to happen. If that Helpdesk computer is connected to an IPv4-only network, it is obviously only capable of transmitting IPv4 packets, and DNS will hand them the DirectAccess client computer's A (IPv4) record, from the last time the client was connected inside the office. Routing will fail, of course, because CLIENT1 is no longer sitting on the physical network. The following screenshot is an example of pinging a DirectAccess connected client computer from an IPv4 network: How to resolve this behavior? We need to give that Helpdesk computer some form of IPv6 connectivity on the network. If you have a real, native IPv6 network running already, you can simply tap into it. Each of your internal machines that need this outbound routing, as well as the DirectAccess server or servers, all need to be connected to this network for it to work. However, I find out in the field that almost nobody is running any kind of IPv6 on their internal networks, and they really aren't interested in starting now. This is where the Intra-Site Automatic Tunnel Addressing Protocol, more commonly referred to as ISATAP, comes into play. You can think of ISATAP as a virtual IPv6 cloud that runs on top of your existing IPv4 network. It enables computers inside your network, like that Helpdesk machine, to be able to establish an IPv6 connection with an ISATAP router. When this happens, that Helpdesk machine will get a new network adapter, visible via ipconfig / all, named ISATAP, and it will have an IPv6 address. Yes, this does mean that the Helpdesk computer, or any machine that needs outbound communications to the DirectAccess clients, has to be capable of talking IPv6, so this typically means that those machines must be Windows 7, Windows 8, Server 2008, or Server 2012. What if your switches and routers are not capable of IPv6? No problem. Similar to the way that 6to4, Teredo, and IP-HTTPS take IPv6 packets and wrap them inside IPv4 so they can make their way across the IPv4 internet, ISATAP also takes IPv6 packets and encapsulates them inside IPv4 before sending them across the physical network. This means that you can establish this ISATAP IPv6 network on top of your IPv4 network, without needing to make any infrastructure changes at all. So, now I need to go purchase an ISATAP router to make this happen? No, this is the best part. Your DirectAccess server is already an ISATAP router; you simply need to point those internal machines at it. Creating a selective ISATAP environment All of the Windows operating systems over the past few years have ISATAP client functionality built right in. This has been the case since Vista, I believe, but I have yet to encounter anyone using Vista in a corporate environment, so for the sake of our discussion, we are generally talking about Windows 7, Windows 8, Server 2008, and Server 2012. For any of these operating systems, out of the box all you have to do is give it somewhere to resolve the name ISATAP, and it will go ahead and set itself up with a connection to that ISATAP router. So, if you wanted to immediately enable all of your internal machines that were ISATAP capable to suddenly be ISATAP connected, all you would have to do is create a single host record in DNS named ISATAP and point it at the internal IP address of your DirectAccess server. To get that to work properly, you would also have to tweak DNS so that ISATAP was no longer part of the global query block list, but I'm not even going to detail that process because my emphasis here is that you should not set up your environment this way. Unfortunately, some of the step-by-step guides that are available on the web for setting up DirectAccess include this step. Even more unfortunately, if you have ever worked with UAG DirectAccess, you'll remember on the IP address configuration screen that the GUI actually told you to go ahead and set ISATAP up this way. Please do not create a DNS host record named ISATAP! If you have already done so, please consider this article to be a guide on your way out of danger. The primary reason why you should stay away from doing this is because Windows prefers IPv6 over IPv4. Once a computer is setup with connection to an ISATAP router, it receives an IPv6 address which registers itself in DNS, and from that point onward whenever two ISATAP machines communicate with each other, they are using IPv6 over the ISATAP tunnel. This is potentially problematic for a couple of reasons. First, all ISATAP traffic default routes through the ISATAP router for which it is configured, so your DirectAccess server is now essentially the default gateway for all of these internal computers. This can cause performance problems and even network flooding. The second reason is that because these packets are now IPv6, even though they are wrapped up inside IPv4, the tools you have inside the network that you might be using to monitor internal traffic are not going to be able to see this traffic, at least not in the same capacity as it would do for normal IPv4 traffic. It is in your best interests that you do not follow this global approach for implementing ISATAP, and instead take the slightly longer road and create what I call a "Selective ISATAP environment", where you have complete control over which machines are connected to the ISATAP network, and which ones are not. Many DirectAccess installs don't require ISATAP at all. Remember, this is only used for those instances where you need true outbound reach to the DirectAccess clients. I recommend installing DirectAccess without ISATAP first, and test all of your management tools. If they work without ISATAP, great! If they don't, then you can create your selective ISATAP environment. To set ourselves up for success, we need to create a simple Active Directory security group, and a GPO. The combination of these things is going to allow us to decisively grant or deny access to the ISATAP routing features on the DirectAccess server. Creating a security group and DNS record First, let's create a new security group in Active Directory. This is just a normal group like any other, typically a global or universal, whichever you prefer. This group is going to contain the computer accounts of the internal computers to which we want to give that outbound reaching capability. So, typically the computer accounts that we will eventually add into this group are Helpdesk computers, SCCM servers, maybe a management "jump box" terminal server, that kind of thing. To keep things intuitive, let's name the group DirectAccess – ISATAP computers or something similar. We will also need a DNS host record created. For obvious reasons we don't want to call this ISATAP, but perhaps something such as Contoso_ISATAP, swapping your name in, of course. This is just a regular DNS A record, and you want to point it at the internal IPv4 address of the DirectAccess server. If you are running a clustered array of DirectAccess servers that are configured for load balancing, then you will need multiple DNS records. All of the records have the same name,Contoso_ISATAP, and you point them at each internal IP address being used by the cluster. So, one gets pointed at the internal Virtual IP (VIP), and one gets pointed at each of the internal Dedicated IPs (DIP). In a two-node cluster, you will have three DNS records for Contoso_ISATAP. Creating the GPO Now go ahead and follow these steps to create a new GPO that is going to contain the ISATAP connection settings: Create a new GPO, name it something such as DirectAccess – ISATAP settings. Place the GPO wherever it makes sense to you, keeping in mind that these settings should only apply to the ISATAP group that we created. One way to manage the distribution of these settings is a strategically placed link. Probably the better way to handle distribution of these settings is to reflect the same approach that the DirectAccess GPOs themselves take. This would mean linking this new GPO at a high level, like the top of the domain, and then using security filtering inside the GPO settings so that it only applies to the group that we just created. This way you ensure that in the end the only computers to receive these settings are the ones that you add to your ISATAP group. I typically take the Security Filtering approach, because it closely reflects what DirectAccess itself does with GPO filtering. So, create and link the GPO at a high level, and then inside the GPO properties, go ahead and add the group (and only the group, remove everything else) to the Security Filtering section, like what is shown in the following screenshot: Then move over to the Details tab and set the GPO Status to User configuration settings disabled. Configuring the GPO Now that we have a GPO which is being applied only to our special ISATAP group that we created, let's give it some settings to apply. What we are doing with this GPO is configuring those computers which we want to be ISATAP-connected with the ISATAP server address with which they need to communicate , which is the DNS name that we created for ISATAP. First, edit the GPO and set the ISATAP Router Name by configuring the following setting: Computer Configuration | Policies | Administrative Templates | Network | TCPIP Settings | IPv6 Transition Technologies | ISATAP Router Name = Enabled (and populate your DNS record). Second, in the same location within the GPO, we want to enable your ISATAP state with the following configuration: Computer Configuration | Policies | Administrative Templates | Network | TCPIP Settings | IPv6 Transition Technologies | ISATAP State = Enabled State. Adding machines to the group All of our settings are now squared away, time to test! Start by taking a single computer account of a computer inside your network from which you want to be able to reach out to DirectAccess client computers, and add the computer account to the group that we created. Perhaps pause for a few minutes to ensure Active Directory has a chance to replicate, and then simply restart that computer. The next time it boots, it'll grab those settings from the GPO, and reach out to the DirectAccess server acting as the ISATAP router, and have an ISATAP IPv6 connection. If you generate an ipconfig /all on that internal machine, you will see the ISATAP adapter and address now listed as shown in the following screenshot: And now if you try to ping the name of a client computer that is connected via DirectAccess, you will see that it now resolves to the IPv6 record. Perfect! But wait, why are my pings timing out? Let's explore that next.
Read more
  • 0
  • 2
  • 14604

article-image-team-project-setup-0
Packt
10 Feb 2016
12 min read
Save for later

Building Your Application

Packt
10 Feb 2016
12 min read
"Measuring programming progress by lines of code is like measuring aircraft building progress by weight."                                                                --Bill Gates In this article, by Tarun Arora, the author of the book Microsoft Team Foundation Server 2015 Cookbook, provides you information about: Configuring TFBuild Agent, Pool, and Queues Setting up a TFBuild Agent using an unattended installation (For more resources related to this topic, see here.) As a developer, compiling code and running unit tests gives you an assurance that your code changes haven't had an impact on the existing codebase. Integrating your code changes into the source control repository enables other users to validate their changes with yours. As a best practice, Teams integrate changes into the shared repository several times a day to reduce the risk of introducing breaking changes or worse, overwriting each other's. Continuous integration (CI) is a development practice that requires developers to integrate code into a shared repository several times a day. Each check-in is verified by an automated build, allowing Teams to detect problems early. The automated build that runs as part of the CI process is often referred to as the CI build. There isn't a clear definition of what the CI build should do, but at the very minimum, it is expected to compile code and run unit tests. Running the CI build on a non-developer remote workspace helps identify the dependencies that may otherwise go unnoticed into the release process. We can talk endlessly about the benefits of CI; the key here is that it enables you to have potentially deployable software at all times. Deployable software is the most tangible asset to customers. Moving from concept to application, in this article, you'll learn how to leverage the build tooling in TFS to set up a quality-focused CI process. But first, let's have a little introduction to the build system in TFS. The following image illustrates the three generations of build systems in TFS: TFS has gone through three generations of build systems. The very first was MSBuild using XML for configuration; the next one was XAML using Windows Workflow Foundation for configuration, and now, there's TFBuild using JSON for configuration. The XAML-based build system will continue to be supported in TFS 2015. No automated migration path is available from XAML build to TFBuild. This is generally because of the difference in the architecture between the two build systems. The new build system in TFS is called Team Foundation Build (TFBuild). It is an extensible task-based execution system with a rich web interface that allows authoring, queuing, and monitoring builds. TFBuild is fully cross platform with the underlying build agents that are capable of running natively on both Windows and non-Windows platforms. TFBuild provides out-of-the-box integration with Centralized Version Control such as TFVC and Distributed Version Controls such as Git and GitHub. TFBuild supports building .NET, Java, Android, and iOS applications. All the recipes in this article are based on TFBuild. TFBuild is a task orchestrator that allows you to run any build engine, such as Ant, CMake, Gradle, Gulp, Grunt, Maven, MSBuild, Visual Studio, Xamarin, XCode, and so on. TFBuild supports work item integration, publishing drops, and publishing test execution results into the TFS that is independent of the build engine that you choose. The build agents are xCopyable and do not require any installation. The agents are auto-updating in nature; there's no need to update every agent in your infrastructure: TFBuild offers a rich web-based interface. It does not require Visual Studio to author or modify a build definition. From simple to complex, all build definitions can easily be created in the web portal. The web interface is accessible from any device and any platform: The build definition can be authored from the web portal directly A build definition is a collection of tasks. A task is simply a build step. Build definition can be composed by dragging and dropping tasks. Each task supports Enabled, Continue on error, and Always run flags making it easier to manage build definitions as the task list grows: The build system supports invoking PowerShell, batch, command line, and shell scripts. All out-of-the-box tasks are open source. If a task does not satisfy your requirements, you can download the task from GitHub at https://github.com/Microsoft/vso-agent-tasks and customize it. If you can't find a task, you can easily create one. You'll learn more about custom tasks in this article. Changes to build definitions can be saved as drafts. Build definitions maintain a history of all changes in the History tab. A side-by-side comparison of the changes is also possible. Comments entered when changing the build definition show up in the change history: Build definitions can be saved as templates. This helps standardize the use of certain tasks across new build definitions: An existing build definition can be saved as a template Multiple triggers can be set for the same build, including CI triggers and multiple scheduled triggers: Rule-based retention policies support the setting up of multiple rules. Retention can be specified by "days" or "number" of the builds: The build output logs are displayed in web portal in real time. The build log can be accessed from the console even after the build gets completed: The build reports have been revamped to offer more visibility into the build execution, and among other things, the test results can now directly be accessed from the web interface. The .trx file does not need to be downloaded into Visual Studio to view the test results: The old build system had restrictions on one Team Project Collection per build controller and one controller per build machine. TFBuild removes this restriction and supports the reuse of queues across multiple Team Project Collections. The following image illustrates the architecture of the new build system: In the preceding diagram, we observe the following: Multiple agents can be configured on one machine Agents from across different machines can be grouped into a pool Each pool can have only one queue One queue can be used across multiple Team Project Collections To demonstrate the capabilities of TFBuild, we'll use the FabrikamTFVC and FabrikamGit Team Projects. Configuring TFBuild Agent, Pool, and Queues In this recipe, you'll learn how to configure agents and create pools and queues. You'll also learn how a queue can be used across multiple Team Project Collections. Getting ready Scenario: At Fabrikam, the FabrikamTFVC and FabrikamGit Team Projects need their own build queues. The FabrikamTFVC Teams build process can be executed on a Windows Server. The FabrikamGit Team build process needs both Windows and OS X. The Teams want to set up three build agents on a Windows Server; one build agent on an OS X machine. The Teams want to group two Windows Agents into a Windows Pool for FabrikamTFVC Team and group one Windows and one Mac Agent into another pool for the FabrikamGit Team: Permission: To configure a build agent, you should be in the Build Administrators Group. The prerequisites for setting up the build agent on a Windows-based machine are as follows: The build agent should have a supporting version of Windows. The list of supported versions is listed at https://msdn.microsoft.com/en-us/Library/vs/alm/TFS/administer/requirements#Operatingsystems. The build agent should have Visual Studio 2013 or 2015. The build agent should have PowerShell 3 or a newer version. A build agent is configured for your TFS as part of the server installation process if you leave the Configure the build service to start automatically option selected: For the purposes of this recipe, we'll configure the agents from scratch. Delete the default pool or any other pool you have by navigating to the Agent pools option in the TFS Administration Console http://tfs2015:8080/tfs/_admin/_AgentPool: How to do it Log into the Windows machine that you desire to set the agents upon. Navigate to the Agent pools in the TFS Administration Console by browsing to http://tfs2015:8080/tfs/_admin/_AgentPool. Click on New Pool, enter the pool name as Pool 1, and uncheck Auto-Provision Queue in Project Collections: Click on the Download agent icon. Copy the downloaded folder into E: and unzip it into E:Win-A1. You can use any drive; however, it is recommended to use the non-operating system drive: Run the PowerShell console as an administrator and change the current path in PowerShell to the location of the agent in this case E:Win-A1. Call the ConfigureAgent.ps1 script in the PowerShell console and click on Enter. This will launch the Build Agent Configuration utility: Enter the configuration details as illustrated in the following screenshot: It is recommended to install the build agent as a service; however, you have an option to run the agent as an interactive process. This is great when you want to debug a build or want to temporarily use a machine as a build agent. The configuration process creates a JSON settings file; it creates the working and diagnostics folders: Refresh the Agent pools page in the TFS Administration Console. The newly configured agent shows up under Pool 1: Repeat steps 2 to 5 to configure Win-A2 in Pool 1. Repeat steps 1 to 5 to configure Win-A3 in Pool 2. It is worth highlighting that each agent runs from its individual folder: Now, log into the Mac machine and launch terminal: Install the agent installer globally by running the commands illustrated here. You will be required to enter the machine password to authorize the install: This will download the agent in the user profile, shown as follows: The summary of actions performed when the agent is downloaded Run the following command to install the agent installer globally for the user profile: Running the following command will create a new directory called osx-A1 for the agent; create the agent in the directory: The agent installer has been copied from the user profile into the agent directory, shown as follows: Pass the following illustrated parameters to configure the agent: This completes the configuration of the xPlatform agent on the Mac. Refresh the Agent pools page in the TFS Administration Console to see the agent appear in Pool 2: The build agent has been configured at the Team Foundation Server level. In order to use the build agent for a Team Project Collection, a mapping between the build agent and Team Project Collection needs to be established. This is done by creating queues. To configure queues, navigate to the Collection Administration Console by browsing to http://tfs2015:8080/tfs/DefaultCollection/_admin/_BuildQueue. From the Build tab, click on New queue; this dialog allows you to reference the pool as a queue: Map Pool 1 as Queue 1 and Pool 2 as Queue 2 as shown here: The TFBuild Agent, Pools, and Queues are now ready to use. The green bar before the agent name and queue in the administration console indicates that the agent and queues are online. How it works... To test the setup, create a new build definition by navigating to the FabrikamTFVC Team Project Build hub by browsing to http://tfs2015:8080/tfs/DefaultCollection/FabrikamTFVC/_build. Click on the Add a new build definition icon. In the General tab, you'll see that the queues show up under the Queue dropdown menu. This confirms that the queues have been correctly configured and are available for selection in the build definition: Pools can be used across multiple Team Project Collections. As illustrated in the following screenshot, in Team Project Collection 2, clicking on the New queue... shows that the existing pools are already mapped in the default collection: Setting up a TFBuild Agent using an unattended installation The new build framework allows the unattended setup of build agents by injecting a set of parameter values via script. This technique can be used to spin up new agents to be attached into an existing agent pool. In this recipe, you'll learn how to configure and unconfigure a build agent via script. Getting ready Scenario: The FabrikamTFVC Team wants the ability to install, configure, and unconfigure a build agent directly via script without having to perform this operation using the Team Portal. Permission: To configure a build agent, you should be in the Build Administrators Group. Download the build agent as discussed in the earlier recipe Configuring TFBuild Agent, Pool, and Queues. Copy the folder to E:Agent. The script refers to this Agent folder. How to do it... Launch PowerShell in the elevated mode and execute the following command: .AgentVsoAgent.exe /Configure /RunningAsService /ServerUrl:"http://tfs2015:8080/tfs" /WindowsServiceLogonAccount:svc_build /WindowsServiceLogonPassword:xxxxx /Name:WinA-10 /PoolName:"Pool 1" /WorkFolder:"E:Agent_work" /StartMode:Automatic Replace the value of the username and password accordingly. Executing the script will result in the following output: The script installs an agent by the name WinA-10 as Windows Service running as svc_build. The agent is added to Pool 1: To unconfigure WinA-10, run the following command in an elevated PowerShell prompt: .AgentVsoAgent.exe /Unconfigure "vsoagent.tfs2015.WinA-10" To unconfigure, script needs to be executed from outside the scope of the Agent folder. Running the script from within the Agent folder scope will result in an error message. How it works... The new build agent natively allows configuration via script. A new capability called Personal Access Token (PAT) is due for release in the future updates of TFS 2015. PAT allows you to generate a personal OAuth token for a specific scope; it replaces the need to key in passwords into configuration files. Summary In this article, we have looked at configuring TFBuild Agent, Pool, and Queues and setting up a TFBuild Agent using an unattended installation. Resources for Article: Further resources on this subject: Overview of Process Management in Microsoft Visio 2013 [article] Introduction to the Raspberry Pi's Architecture and Setup [article] Implementing Microsoft Dynamics AX [article]
Read more
  • 0
  • 0
  • 14599

article-image-swift-programming-language
Packt
06 Oct 2015
25 min read
Save for later

The Swift Programming Language

Packt
06 Oct 2015
25 min read
This article is by Chuck Gaffney, the author of the book iOS 9 Game Development Essentials. This delves into some vital specifics of the Swift language. (For more resources related to this topic, see here.) At the core of all game development is your game's code. It is the brain of your project and outside of the art, sound, and various asset developments, it is where you will spend most of your time creating and testing your game. Up until Apple's Worldwide Developers Conference WWDC14 in June of 2014, the code of choice for iOS game and app development was Objective-C. At WWDC14, a new and faster programming language Swift, was announced and is now the recommended language for all current and future iOS games and general app creation. As of the writing of this book, you can still use Objective-C to design your games, but both programmers new and seasoned will see why writing in Swift is not only easier with expressing your game's logic but even more preformat. Keeping your game running at that critical 60 FPS is dependent on fast code and logic. Engineers at Apple developed the Swift Programming Language from the ground up with performance and readability in mind, so this language can execute certain code iterations faster than Objective-C while also keeping code ambiguity to a minimum. Swift also uses many of the methodologies and syntax found in more modern languages like Scala, JavaScript, Ruby, and Python. So let's dive into the Swift language. It is recommended that some basic knowledge of Object Oriented Programming (OOP) be known prior but we will try to keep the build up and explanation of code simple and easy to follow as we move on to the more advanced topics related to game development. Hello World! It's somewhat tradition in the education of programming languages to begin with a Hello World example. A Hello World program is simply using your code to display or log the text Hello World. It's always been the general starting point because sometimes just getting your code environment set up and having your code executing correctly is half the battle. At least, this was more the case in previous programming languages. Swift makes this easier than ever, without going into the structure of a Swift file (which we shall do later on and is also much easier than Objective-C and past languages), here's how you create a Hello World program: print("Hello, World!") That's it! That is all you need to have the text "Hello, World" appear in XCode's Debug Area output. No more semicolons Those of us who have been programming for some time might notice that the usually all important semicolon (;) is missing. This isn't a mistake, in Swift we don't have to use a semicolon to mark the end of an expression. We can if we'd like and some of us might still do it as a force of habit, but Swift has omitted that common concern. The use of the semicolon to mark the end of an expression stems from the earliest days of programming when code was written in simple word processors and needed a special character to represent when the code's expression ends and the next begins. Variables, constants, and primitive data types When programming any application, either if new to programming or trying to learn a different language, first we should get an understanding of how a language handles variables, constants, and various data types, such as Booleans, integers, floats, strings, and arrays. You can think of the data in your program as boxes or containers of information. Those containers can be of different flavors, or types. Throughout the life of your game, the data could change (variables, objects, and so on) or they can stay the same. For example, the number of lives a player has would be stored as a variable, as that is expected to change during the course of the game. That variable would then be of the primitive data type integer, which are basically whole numbers. Data that stores, say the name of a certain weapon or power up in your game would be stored in what's known as a constant, as the name of that item is never going to change. In a game where the player can have interchangeable weapons or power-ups, the best way to represent the currently equipped item would be to use a variable. A variable is a piece of data that is bound to change. That weapon or power-up will also most likely have a bit more information to it than just a name or number; the primitive types we mentioned prior. The currently equipped item would be made up of properties like its name, power, effects, index number, and the sprite or 3D model that visually represents it. Thus the currently equipped item wouldn't just be a variable of a primitive data type, but be what is known as type of object. Objects in programming can hold a number of properties and functionality that can be thought of as a black box of both function and information. The currently equipped item in our case would be sort of a placeholder that can hold an item of that type and interchange it when needed; fulfilling its purpose as a replaceable item. Swift is what's known as a type-safe language, so keeping track of the exact type of data and even it's future usage (that is, if the data is or will be NULL) as it's very important when working with Swift compared to other languages. Apple made Swift behave this way to help keep runtime errors and bugs in your applications to a minimum and so we can find them much earlier in the development process. Variables Let's look at how variables are declared in Swift: var lives = 3 //variable of representing the player's lives lives = 1 //changes that variable to a value of 1 Those of us who have been developing in JavaScript will feel right at home here. Like JavaScript, we use the keyword var to represent a variable and we named the variable, lives. The compiler implicitly knows that the type of this variable is a whole number, and the data type is a primitive one: integer. The type can be explicitly declared as such: var lives: Int = 3 //variable of type Int We can also represent lives as the floating point data types as double or float: // lives are represented here as 3.0 instead of 3 var lives: Double = 3 //of type Double var lives: Float = 3 //of type Float Using a colon after the variable's name declaration allows us to explicitly typecast the variable. Constants During your game there will be points of data that don't change throughout the life of the game or the game's current level or scene. This can be various data like gravity, a text label in the Heads-Up Display (HUD), the center point of character's 2D animation, an event declaration, or time before your game checks for new touches or swipes. Declaring constants is almost the same as declaring variables. Using a colon after the variable's name declaration allows us to explicitly typecast the variable. let gravityImplicit = -9.8 //implicit declaration let gravityExplicit: Float = -9.8 //explicit declaration As we can see, we use the keyword let to declare constants. Here's another example using a string that could represent a message displayed on the screen during the start or end of a stage. let stageMessage = "Start!" stageMessage = "You Lose!" //error Since the string stageMessage is a constant, we cannot change it once it has been declared. Something like this would be better as a variable using var instead of let. Why don't we declare everything as a variable? This is a question sometimes asked by new developers and is understandable why it's asked especially since game apps tend to have a large number of variables and more interchangeable states than an average application. When the compiler is building its internal list of your game's objects and data, more goes on behind the scenes with variables than with constants. Without getting too much into topics like the program's stack and other details, in short, having objects, events, and data declared as constants with the let keyword is more efficient than var. In a small app on the newest devices today, though not recommended, we could possibly get away with this without seeing a great deal of loss in app performance. When it comes to video games however, performance is critical. Buying back as much performance as possible can allow for a better player experience. Apple recommends that when in doubt, always use let when declaring and have the complier say when to change to var. More about constants As of Swift version 1.2, constants can have a conditionally controlled initial value. Prior to this update, we had to initialize a constant with a single starting value or be forced to make the property a variable. In XCode 6.3 and newer, we can perform the following logic: let x : SomeThing if condition { x = foo() } else { x = bar() } use(x) An example of this in a game could be let stageBoss : Boss if (stageDifficulty == gameDifficulty.hard) { stageBoss = Boss.toughBoss() } else { stageBoss = Boss.normalBoss() } loadBoss(stageBoss) With this functionality, a constant's initialization can have a layer of variance while still keeping it unchangeable, or immutable through its use. Here, the constant, stageBoss can be one of two types based on the game's difficulty: Boss.toughBoss() or Boss.normalBoss(). The boss won't change for the course of this stage, so it makes sense to keep it as a constant. More on if and else statements is covered later in the article. Arrays, matrices, sets, and dictionaries Variables and constants can represent a collection of various properties and objects. The most common collection types are arrays, matrices, sets, and dictionaries. An Array is an ordered list of distinct objects, a Matrix is, in short, an array of arrays, a Set is an unordered list of distinct objects and a Dictionary is an unordered list that utilizes a key : value association to the data. Arrays Here's an example of an Array in Swift. let stageNames : [String] = ["Downtown Tokyo","Heaven Valley", "Nether"] The object stageNames is a collection of strings representing names of a game's stages. Arrays are ordered by subscripts from 0 to array length 1. So stageNames[0] would be Downtown Tokyo, stageNames[2] would be Nether, and stageNames[4] would give an error since that's beyond the limits of the array and doesn't exist. We use [] brackets around the class type of stageNames, [String] to tell the compiler that we are dealing with an array of Strings. Brackets are also used around the individual members of this array. 2D arrays or matrices A common collection type used in physics calculations, graphics, and game design, particularly grid-based puzzle games, are two-dimensional arrays or matrices. 2D arrays are simply arrays that have arrays as their members. These arrays can be expressed in a rectangular fashion in rows and columns. For example, the 4x4 (4 rows, 4 columns) tile board in the 15 Puzzle Game can be represented as such: var tileBoard = [[1,2,3,4], [5,6,7,8], [9,10,11,12], [13,14,15,""]] In the 15 Puzzle game, your goal is to shift the tiles using the one empty spot (represented with the blank String ""), to all end up in the 1—15 order we see up above. The game would start with the numbers arranged in a random and solvable order and player would then have to swap the numbers and the blank space. To better perform various actions on AND or OR, store information about each tile in the 15 Game (and other games); it'd be better to create a tile object as opposed to using raw values seen here. For the sake of understanding what a matrix or 2D array is, simply take note on how the array is surrounded by doubly encapsulated brackets [[]]. We will later use one of our example games, SwiftSweeper, to better understand how puzzle games use 2D arrays of objects to create a full game. Here are ways to declare blank 2D arrays with strict types: var twoDTileArray : [[Tiles]] = [] //blank 2D array of type,Tiles var anotherArray = Array<Array<Tile>>() //same array, using Generics The variable twoDTileArray uses the double brackets [[Tiles]] to declare it as a blank 2D array or matrix for the made up type, tiles. The variable anotherArray is a rather oddly declared array that uses angle bracket characters <> for enclosures. It utilizes what's known as Generics. Generics is a rather advanced topic that we will touch more on later. They allow for very flexible functionality among a wide array of data types and classes. For the moment we can think of them as a catchall way of working with Objects. To fill in the data for either version of this array, we would then use for-loops to fill in the data. More on loops and iterations later in the article! Sets This is how we would make a set of various game items in Swift: var keyItems = Set([Dungeon_Prize, Holy_Armor, Boss_Key,"A"]) This set keyItems has various objects and a character A. Unlike an Array, a Set is not ordered and contains unique items. So unlike stageNames, attempting to get keyItems[1] would return an error and items[1] might not necessarily be the Holy_Armor object, as the placement of objects is internally random in a set. The advantage Sets have over Arrays is that Sets are great at checking for duplicated objects and for specific content searching in the collection overall. Sets make use of hashing to pinpoint the item in the collections; so checking for items in Set's content can be much faster than an array. In game development, a game's key items which the player may only get once and should never have duplicates of, could work great as a Set. Using the function keyItems, contains(Boss_Key) returns the Boolean value of true in this case. Sets were added in Swift 1.2 / XCode 6.3. Their class is represented by the Generic type Set<T> where T is the class type of the collection. In other words, the set Set([45, 66, 1233, 234]) would be of the type Set<Int> and our example here would be a Set<NSObject> instance due to it having a collection of various data types. We will discuss more on Generics and Class Hierarchy later in this article. Dictionaries A Dictionary can be represented this way in Swift: var playerInventory: [Int : String] = [1 : "Buster Sword", 43 : "Potion", 22: "StrengthBooster"] Dictionaries use a key : value association, so playerInventory[22] returns the value StrengthBooster based on the key, 22. Both the key and value could be initialized to almost any class type*. In addition to the inventory example given, we can have the code as following: var stageReward: [Int : GameItem] = [:] //blank initialization //use of the Dictionary at the end of a current stage stageReward = [currentStage.score : currentStage.rewardItem] *The values of a Dictionary, though rather flexible in Swift, do have limitations. The key must conform to what's known as the Hashable protocol. Basic data types like integer and string already have this functionality, so if you are to make your own classes or data structures that are to be used in Dictionaries, say mapping a player actions with player input, this protocol must be utilized first. We will discuss more about Protocols, later in this article. Dictionaries are like Sets in that they are unordered but with the additional layer of having a key and a value associated with their content instead of just the hashed key. As with Sets, Dictionaries are great for quick insertion and retrieval of specific data. In IOS Apps and in web applications, Dictionaries are what's used to parse and select items from JSON (JavaScript Object Notation) data. In the realm of game development, Dictionaries using JSON or via Apple's internal data class, NSUserDefaults, can be used to save and load game data, set up game configurations or access specific members of a game's API. For example, here's one way to save a player's high score in an IOS game using Swift: let newBestScore : Void = NSUserDefaults.standardUserDefaults().setInteger(bestScore, forKey: "bestScore") This code comes directly from a published Swift—developed game called PikiPop, which we will use from time to time to show code used in actual game applications. Again, note that Dictionaries are unordered but Swift has ways to iterate or search through an entire Dictionary. Mutable or immutable collections One rather important discussion that we left out is how to subtract, edit or add to Arrays, Sets, and Dictionaries, but before we do that, we should understand the concept of mutable and immutable data or collections. A mutable collection is simply data that can be changed, added to or subtracted from, while an immutable collection cannot be changed, added to or subtracted from. To work with mutable and immutable collections efficiently in Objective-C, we had to explicitly state the mutability of the collection beforehand. For example, an array of the type NSArray in Objective-C is always immutable. There are methods we can call on NSArray that would edit the collection but behind the scenes this would be creating brand new NSArrays, thus would be rather inefficient if doing this often in the life of our game. Objective-C solved this issue with class type, NSMutableArray. Thanks to the flexibility of Swift's type inference, we already know how to make a collection mutable or immutable! The concept of constants and variables has us covered when it comes to data mutability in Swift. Using the keyword let when creating a collection will make that collection immutable while using var will initialize it as a mutable collection. //mutable Array var unlockedLevels : [Int] = [1, 2, 5, 8] //immutable Dictionary let playersForThisRound : [PlayerNumber:PlayerUserName] = [453:"userName3344xx5", 233:"princeTrunks", 6567: "noScopeMan98", 211: "egoDino"] The Array of Int, unlockedLevels can be edited simply because it's a variable. The immutable Dictionary playersForThisRound, can't be changed since it's already been declared as a constant; no additional layers of ambiguity concerning additional class types. Editing or accessing collection data As long as a collection type is a variable, using the var keyword, we can do various edits to the data. Let's go back to our unlockedLevels array. Many games have the functionality of unlocking levels as the player progresses. Say the player reached the high score needed to unlock the previously locked level 3 (as 3 isn't a member of the array). We can add 3 to the array using the append function: unlockedLevels.append(3) Another neat attribute of Swift is that we can add data to an array using the += assignment operator: unlockedLevels += [3] Doing it this way however will simply add 3 to the end of the array. So our previous array of [1, 2, 5, 8] is now [1, 2, 5, 8, 3]. This probably isn't a desirable order, so to insert the number 3 in the third spot, unlockedLevels[2], we can use the following method: unlockedLevels.insert(3, atIndex: 2) Now our array of unlocked levels is ordered to [1, 2, 3, 5, 8]. This is assuming though that we know a member of the array prior to 3 is sorted already. There's various sorting functionalities provided by Swift that could assist in keeping an array sorted. We will leave the details of sorting to our discussions of loops and control flow later on in this article. Removing items from an array is just as simple. Let's use again our unlockedLevels array. Imagine our game has an over world for the player to travel to and from, and the player just unlocked a secret that triggered an event, which blocked off access to level 1. Level 1 would now have to be removed from the unlocked levels. We can do it like this: unlockedLevels.removeAtIndex(0) // array is now [2, 3, 5, 8] Alternately, imagine the player lost all of their lives and got a Game Over. A penalty to that could be to lock up the furthest level. Though probably a rather infuriating method and us knowing that Level 8 is the furthest level in our array, we can remove it using the .removeLast() function of Array types. unlockedLevels.removeLast() // array is now [2,3,5] That this is assuming we know the exact order of the collection. Sets or Dictionaries might be better at controlling certain aspects of your game. Here's some ways to edit a set or a dictionary as a quick guide. Set inventory.insert("Power Ring") //.insert() adds items to a set inventory.remove("Magic Potion") //.remove() removes a specific item inventory.count //counts # of items in the Set inventory.union(EnemyLoot) //combines two Sets inventory.removeAll() //removes everything from the Set inventory.isEmpty //returns true Dictionary var inventory = [Float : String]() //creates a mutable dictionary /* one way to set an equipped weapon in a game; where 1.0 could represent the first "item slot" that would be placeholder for the player's "current weapon" */ inventory.updateValue("Broadsword", forKey: 1.0) //removes an item from a Dictionary based on the key value inventory.removeValueForKey("StatusBooster") inventory.count //counts items in the Dictionary inventory.removeAll(keepCapacity: false) //deletes the Dictionary inventory.isEmpty //returns false //creates an array of the Dictionary's values let inventoryNames = [String](inventory.values) //creates an array of the Dictionary's keys let inventoryKeys = [String](inventory.keys) Iterating through collection types We can't discuss about collection types without mentioning how to iterate through them in mass. Here's some ways we'd iterate though an Array, Set or a Dictionary in Swift: //(a) outputs every item through the entire collection //works for Arrays, Sets, and Dictionaries but output will vary for item in inventory { print(item) } //(b) outputs sorted item list using Swift's sorted() function //works for Sets for item in sorted(inventory) { print("(item)") } //(c) outputs every item as well as it's current index //works for Arrays, Sets, and Dictionaries for (index, value) in enumerate(inventory) { print("Item (index + 1): (value)") } //(d) //Iterate through and through the keys of a Dictionary for itemCode in inventory.keys { print("Item code: (itemCode)") } //(e) //Iterate through and through the values of a Dictionary for itemName in inventory.values { print("Item name: (itemName)") } As stated previously, this is done with what's known as a for-loop; with these examples we show how Swift utilizes the for-in variation using the in keyword. The code will repeat until it reaches the end of the collection in all of these examples. In example (c) we also see the use of the Swift function, enumerate(). This function returns a compound value, (index,value), for each item. This compound value is known as a tuple and Swift's use of tuples makes for a wide variety of functionality for functions loops as well as code blocks. We will delve more into tuples, loops, and blocks later on. Comparing Objective-C and Swift Here's a quick review of our Swift code with a comparison of the Objective-C equivalent. Objective-C An example code in Objective-C is as follows: const int MAX_ENEMIES = 10; //constant float playerPower = 1.3; //variable //Array of NSStrings NSArray * stageNames = @[@"Downtown Tokyo", @"Heaven Valley", @" Nether"]; //Set of various NSObjects NSSet *items = [NSSet setWithObjects: Weapons, Armor, HealingItems,"A", nil]; //Dictionary with an Int:String key:value NSDictionary *inventory = [NSDictionary dictionaryWithObjectsAndKeys: [NSNumber numberWithInt:1], @"Buster Sword", [NSNumber numberWithInt:43], @"Potion", [NSNumber numberWithInt:22], @"Strength", nil]; Swift An example code in Objective-C is as follows: let MAX_ENEMIES = 10 //constant var playerPower = 1.3 //variable //Array of Strings let stageNames : [String] = ["Downtown Tokyo","Heaven Valley","Nether"] //Set of various NSObjects var items = Set([Weapons, Armor, HealingItems,"A"]) //Dictionary with an Int:String key:value var playerInventory: [Int : String] = [1 : "Buster Sword", 43 : "Potion", 22: "StrengthBooster"] In the preceding code, we some examples of variables, constants, Arrays, Sets, and Dictionaries. First we see their Objective-C syntax and then we see the equivalent declarations using Swift's syntax. We can see from this example how compact Swift is compared to Objective-C. Characters and strings For some time in this article we've been mentioning Strings. Strings are also a collection of data type but a specially dealt collection of Characters, of the class type, String. Swift is Unicode-compliant so we can have Strings like this: let gameOverText = "Game Over!" We have strings with emoji characters like this: let cardSuits = "♠ ♥ ♣ ♦" What we did now was create what's known as a string literal. A string literal is when we explicitly define a String around two quotes "". We can create empty String variables for later use in our games as such: var emptyString = "" // empty string literal var anotherEmptyString = String() // using type initializer Both are valid ways to create an empty String "". String interpolation We can also create a string from a mixture of other data types, known as string interpolation. String Interpolation is rather common in game development, debugging, and string use in general. The most notable of examples are displaying the player's score and lives. This is how one our example games, PikiPop uses string interpolation to display current player stats: //displays the player's current lives var livesLabel = "x (currentScene.player!.lives)" //displays the player's current score var scoreText = "Score: (score)" Take note of the "(variable_name)" formatting. We've actually seen this before in our past code snippets. In the various print() outputs, we used this to display the variable, collection, and so on we wanted to get info on. In Swift, the way to output the value of a data type in a String is by using this formatting. For those of us who came from Objective-C, it's the same as this: NSString *livesLabel = @"Lives: "; int lives = 3; NSString *livesText = [NSString stringWithFormat:@" %@ (%d days ago)", livesLabel, lives]; Notice how Swift makes string interpolation much cleaner and easier to read than its Objective-C predecessor. Mutating strings There are various ways to change strings. We can also add to a string just the way we did while working with collection objects. Here's a basic example: var gameText = "The player enters the stage" gameText += " and quickly lost due to not leveling up" /* gameText now says "The player enters the stage and lost due to not leveling up" */ Since Strings are essentially arrays of characters, like arrays, we can use the += assignment operator to add to the previous String. Also, akin to arrays, we can use the append() function to add a character to the end of a string: let exclamationMark: Character = "!" gameText.append(exclamationMark) /*gameText now says "The player enters the stage and lost due to not leveling up!"*/ Here's how we iterate through the Characters in a string in Swift: for character in "Start!" { print(character) } //outputs: //S //t //a //r //t //! Notice how again we use the for-in loop and even have the flexibility of using a string literal if we'd so like to be what's iterated through by the loop. String indices Another similarity between Arrays and Strings is the fact that a String's individual characters can be located via indices. Unlike Arrays however, since a character can be a varying size of data, broken in 21-bit numbers known as Unicode scalars, they can not be located in Swift with Int type index values. Instead, we can use the .startIndex and .endIndex properties of a String and move one place ahead or one place behind the index with the .successor() and .predecessor() functions respectively to retrieve the needed character or characters of a String. gameText[gameText.startIndex] // = T gameText[gameText.endIndex] // = ! gameText[gameText.startIndex.successor()] // = h gameText[gameText.endIndex.predecessor()] // = p Here are some examples that use these properties and functions using our previous gameText String: There are many ways to manipulate, mix, remove, and retrieve various aspects of a String and Characters. For more information, be sure to check out the official Swift documentation on Characters and Strings here: https://developer.apple.com/library/ios/documentation/Swift/Conceptual/Swift_Programming_Language/StringsAndCharacters.html. Summary There's much more about the Swift programming language than we can fit here. Throughout the course of this book we will throw in a few extra tidbits and nuances about Swift as it becomes relevant to our upcoming gaming programming needs. If you wish to become more versed in the Swift programming language, Apple actually provides a wonderful tool for us in what's known as a Playground. Playgrounds were introduced with the Swift programming language at WWDC14 in June of 2014 and allow us to test various code output and syntax without having to create a project, build, run, and repeat again when in many cases we simply needed to tweak a few variables and function loop iterations. There are a number of resources to check out on the official Swift developer page (https://developer.apple.com/swift/resources/). Two highly recommended Playgrounds to check out are as follows: Guided Tour Playground (https://developer.apple.com/library/ios/documentation/Swift/Conceptual/Swift_Programming_Language/GuidedTour.playground.zip) This Playground covers many of the topics we mentioned in this article and more, from Hello World all the way to Generics. The second playground to test out is the Balloons Playground (https://developer.apple.com/swift/blog/downloads/Balloons.zip). The Balloons Playground was the keynote Playgrounds demonstration from WWDC14 and shows off many of the features Playgrounds have to offer, particularly for making and testing games. Sometimes the best way to learn a programming language is to test live code; and that's exactly what Playgrounds allow us to do.
Read more
  • 0
  • 0
  • 14598

article-image-google-bard-everything-you-need-to-know-to-get-started
Sangita Mahala
05 Oct 2023
6 min read
Save for later

Google Bard: Everything You Need to Know to Get Started

Sangita Mahala
05 Oct 2023
6 min read
Dive deeper into the world of AI innovation and stay ahead of the AI curve! Subscribe to our AI_Distilled newsletter for the latest insights. Don't miss out – sign up today!IntroductionGoogle Bard, a generative AI conversational chatbot. Initially from the LaMDA family of large language models and later the PaLM LLMs. It will follow your instructions and complete your requests in a better way. Bard uses its knowledge to answer your questions in an informative manner. It can generate different creative text formats, like poems, code, scripts, emails, letters, etc. Currently, it is available in 238 countries and 46 languages.Bard is a powerful tool which can be used for many different things, including:Writing and editing contentResearching and learning new thingsTranslating languagesGenerating new creative ideasAnswering questionsWhat is Google Bard and How does it work?Bard is a large language model, commonly referred to as a chatbot or conversational AI, that has been programmed to be extensive and informative. It is able to communicate and generate human-like text in response to a wide range of prompts and questions.Bard operates by using a method known as deep learning. Artificial neural networks are used in deep learning to learn from data. Deep learning is a subset of machine learning. The structure and operation of the human brain served as the inspiration for neural networks, which are capable of learning intricate patterns from data.The following illustrates how Bard works:You enter a command or query into Bard.The input or query is processed by Bard's neural network, which then produces a response.The reply from Bard is then shown to you.How to get started with Bard?It’s pretty simple and easy to get started using Google Bard. The following steps will let you know how you will be able to start your journey in Google Bard.Step-1Go to the Google Bard website by clicking this link: https://bard.google.com/Step-2Go to the top right corner of your screen and then click on the “Sign in” button.Step-3Once you are signed in, you can start typing your prompts or queries into the text box at the bottom of the screen.Step-4Your prompt or query will trigger Bard to produce an answer, which you may then read and evaluate.For Example:You can provide the prompt to Google Bard such as “Write a 'Hello, World!' program in the Rust programming language.”Prompt:Common Bard commands and how to use themGoogle Bard does not have any specific commands, but you can use certain keywords and phrases to get Bard to perform certain tasks. For example, you can use the following keywords and phrases to get Bard to:Create many creative text forms, such as "Write a script for...", "Write a poem about...", and "Write a code for...".Bard will perfectly answer your questions in a comprehensive and informative way: "What is the capital of India?", "How do I build a website?", "What are the benefits of using Bard?" Examples of what Bard can do and how to use it for specific tasksHere are some examples of what Bard can do and how to use it for specific tasks:Generate creative text formats: Bard allows you to generate a variety of unique text formats, including code, poems, emails, and letters. You have to simply input the required format, followed by a question or query, to get the things done. For example, to generate an email to your manager requesting a raise in salary, you would type “Write an email to your manager asking for a raise."Prompt:Answer your questions in a comprehensive and informative way: No matter how difficult or complex your query might be, Bard can help you to find the solution. So you have to simply enter your query into the text box, and Bard will generate a response likewise. For example, to ask Bard what is the National Flower of India is, you would type "What is the National Flower of India?".Prompt: Translate languages: Bard will allow to convert text between different languages. To do this, simply type the text that you want to translate into the text box, followed by the provided language from your side. For example, to translate the sentence: I am going to the store to buy some groceries into Hindi, you would type "I am going to the store to buy some groceries to Hindi".Prompt:How Bard Can Be Used for Different PurposesA writer can use Bard to generate new concepts for fresh stories or articles or to summarize research results.Students can utilize Bard to create essays and reports, get assistance with their assignments, and master new topics.A business owner can use Bard to produce marketing materials, develop chatbots for customer support, or perform data analysis.Software developers can use Bard to produce code, debug applications, or find solutions for technical issues.The future of Google BardGoogle Bard is probably going to get increasingly more capable and adaptable as it keeps developing and acquiring knowledge. It might be utilized to produce new works of art and entertainment, advance scientific research, and provide solutions to some of the most critical global problems.It's also crucial to remember that Google Bard is not the only significant language model being developed. Similar technologies are being created as well by various other companies, such as Microsoft and OpenAI. This competition is likely to drive innovation and lead to even more powerful and sophisticated language models in the future.Overall, the future of Google Bard and other substantial language models seems quite bright overall. These innovations have the power to completely transform the way we study, work, and produce. There is no doubt that these technologies have the potential to improve the world, but it is necessary to use them effectively.ConclusionGoogle Bard is a powerful AI tool that has the potential to be very beneficial, including writing and editing content, researching and learning new things, translating languages, generating new creative ideas, and answering questions. Being more productive and saving time are two of the greatest advantages of utilizing Google Bard. This can free up your time so that you can concentrate on other things, including developing new ideas or enhancing your skills. Bard can assist you in finding and understanding the data quickly and easily because it has access to a wide amount of information. It has the potential to be a game-changer for many people. If you are looking for a way to be more productive, I encourage you to try using Bard.Author BioSangita Mahala is a passionate IT professional with an outstanding track record, having an impressive array of certifications, including 12x Microsoft, 11x GCP, 2x Oracle, and LinkedIn Marketing Insider Certified. She is a Google Crowdsource Influencer and IBM champion learner gold. She also possesses extensive experience as a technical content writer and accomplished book blogger. She is always Committed to staying with emerging trends and technologies in the IT sector.
Read more
  • 0
  • 0
  • 14591
Unlock access to the largest independent learning library in Tech for FREE!
Get unlimited access to 7500+ expert-authored eBooks and video courses covering every tech area you can think of.
Renews at $19.99/month. Cancel anytime
article-image-writing-reddit-reader-rxphp
Packt
09 Jan 2017
9 min read
Save for later

Writing a Reddit Reader with RxPHP

Packt
09 Jan 2017
9 min read
In this article by Martin Sikora, author of the book, PHP Reactive Programming, we will cover writing a CLI Reddit reader app using RxPHP, and we will see how Disposables are used in the default classes that come with RxPHP, and how these are going to be useful for unsubscribing from Observables in our app. (For more resources related to this topic, see here.) Examining RxPHP's internals As we know, Disposables as a means for releasing resources used by Observers, Observables, Subjects, and so on. In practice, a Disposable is returned, for example, when subscribing to an Observable. Consider the following code from the default RxObservable::subscribe() method: function subscribe(ObserverI $observer, $scheduler = null) { $this->observers[] = $observer; $this->started = true; return new CallbackDisposable(function () use ($observer) { $this->removeObserver($observer); }); } This method first adds the Observer to the array of all subscribed Observers. It then marks this Observable as started and, at the end, it returns a new instance of the CallbackDisposable class, which takes a Closure as an argument and invokes it when it's disposed. This is probably the most common use case for Disposables. This Disposable just removes the Observer from the array of subscribers and therefore, it receives no more events emitted from this Observable. A closer look at subscribing to Observables It should be obvious that Observables need to work in such way that all subscribed Observables iterate. Then, also unsubscribing via a Disposable will need to remove one particular Observer from the array of all subscribed Observables. However, if we have a look at how most of the default Observables work, we find out that they always override the Observable::subscribe() method and usually completely omit the part where it should hold an array of subscribers. Instead, they just emit all available values to the subscribed Observer and finish with the onComplete() signal immediately after that. For example, we can have a look at the actual source code of the subscribe() method of the RxReturnObservable class: function subscribe(ObserverI $obs, SchedulerI $sched = null) { $value = $this->value; $scheduler = $scheduler ?: new ImmediateScheduler(); $disp = new CompositeDisposable(); $disp->add($scheduler->schedule(function() use ($obs, $val) { $obs->onNext($val); })); $disp->add($scheduler->schedule(function() use ($obs) { $obs->onCompleted(); })); return $disp; } The ReturnObservable class takes a single value in its constructor and emits this value to every Observer as they subscribe. The following is a nice example of how the lifecycle of an Observable might look: When an Observer subscribes, it checks whether a Scheduler was also passed as an argument. Usually, it's not, so it creates an instance of ImmediateScheduler. Then, an instance of CompositeDisposable is created, which is going to keep an array of all Disposables used by this method. When calling CompositeDisposable::dispose(), it iterates all disposables it contains and calls their respective dispose() methods. Right after that we start populating our CompositeDisposable with the following: $disposable->add($scheduler->schedule(function() { ... })); This is something we'll see very often. SchedulerInterface::schedule() returns a DisposableInterface, which is responsible for unsubscribing and releasing resources. In this case, when we're using ImmediateScheduler, which has no other logic, it just evaluates the Closure immediately: function () use ($obs, $val) { $observer->onNext($val); } Since ImmediateScheduler::schedule() doesn't need to release any resources (it didn't use any), it just returns an instance of RxDisposableEmptyDisposable that does literally nothing. Then the Disposable is returned, and could be used to unsubscribe from this Observable. However, as we saw in the preceding source code, this Observable doesn't let you unsubscribe, and if we think about it, it doesn't even make sense because ReturnObservable class's value is emitted immediately on subscription. The same applies to other similar Observables, such as IteratorObservable, RangeObservable or ArrayObservable. These just contain recursive calls with Schedulers but the principle is the same. A good question is, why on Earth is this so complicated? All the preceding code does could be stripped into the following three lines (assuming we're not interested in using Schedulers): function subscribe(ObserverI $observer) { $observer->onNext($this->value); $observer->onCompleted(); } Well, for ReturnObservable this might be true, but in real applications, we very rarely use any of these primitive Observables. It's true that we usually don't even need to deal with Schedulers. However, the ability to unsubscribe from Observables or clean up any resources when unsubscribing is very important and we'll use it in a few moments. A closer look at Operator chains Before we start writing our Reddit reader, we should talk briefly about an interesting situation that might occur, so it doesn't catch us unprepared later. We're also going to introduce a new type of Observable, called ConnectableObservable. Consider this simple Operator chain with two subscribers: // rxphp_filters_observables.php use RxObservableRangeObservable; use RxObservableConnectableObservable; $connObs = new ConnectableObservable(new RangeObservable(0, 6)); $filteredObs = $connObs ->map(function($val) { return $val ** 2; }) ->filter(function($val) { return $val % 2;, }); $disposable1 = $filteredObs->subscribeCallback(function($val) { echo "S1: ${val}n"; }); $disposable2 = $filteredObs->subscribeCallback(function($val) { echo "S2: ${val}n"; }); $connObservable->connect(); The ConnectableObservable class is a special type of Observable that behaves similarly to Subject (in fact, internally, it really uses an instance of the Subject class). Any other Observable emits all available values right after you subscribe to it. However, ConnectableObservable takes another Observable as an argument and lets you subscribe Observers to it without emitting anything. When you call ConnectableObservable::connect(), it connects Observers with the source Observables, and all values go one by one to all subscribers. Internally, it contains an instance of the Subject class and when we called subscribe(), it just subscribed this Observable to its internal Subject. Then when we called the connect() method, it subscribed the internal Subject to the source Observable. In the $filteredObs variable we keep a reference to the last Observable returned from filter() call, which is an instance of AnnonymousObservable where, on next few lines, we subscribe both Observers. Now, let's see what this Operator chain prints: $ php rxphp_filters_observables.php S1: 1 S2: 1 S1: 9 S2: 9 S1: 25 S2: 25 As we can see, each value went through both Observers in the order they were emitted. Just out of curiosity, we can also have a look at what would happen if we didn't use ConnectableObservable, and used just the RangeObservable instead: $ php rxphp_filters_observables.php S1: 1 S1: 9 S1: 25 S2: 1 S2: 9 S2: 25 This time, RangeObservable emitted all values to the first Observer and then, again, all values to the second Observer. Right now, we can tell that the Observable had to generate all the values twice, which is inefficient, and with a large dataset, this might cause a performance bottleneck. Let's go back to the first example with ConnectableObservable, and modify the filter() call so it prints all the values that go through: $filteredObservable = $connObservable ->map(function($val) { return $val ** 2; }) ->filter(function($val) { echo "Filter: $valn"; return $val % 2; }); Now we run the code again and see what happens: $ php rxphp_filters_observables.php Filter: 0 Filter: 0 Filter: 1 S1: 1 Filter: 1 S2: 1 Filter: 4 Filter: 4 Filter: 9 S1: 9 Filter: 9 S2: 9 Filter: 16 Filter: 16 Filter: 25 S1: 25 Filter: 25 S2: 25 Well, this is unexpected! Each value is printed twice. This doesn't mean that the Observable had to generate all the values twice, however. It's not obvious at first sight what happened, but the problem is that we subscribed to the Observable at the end of the Operator chain. As stated previously, $filteredObservable is an instance of AnnonymousObservable that holds many nested Closures. By calling its subscribe() method, it runs a Closure that's created by its predecessor, and so on. This leads to the fact that every call to subscribe() has to invoke the entire chain. While this might not be an issue in many use cases, there are situations where we might want to do some special operation inside one of the filters. Also, note that calls to the subscribe() method might be out of our control, performed by another developer who wanted to use an Observable we created for them. It's good to know that such a situation might occur and could lead to unwanted behavior. It's sometimes hard to see what's going on inside Observables. It's very easy to get lost, especially when we have to deal with multiple Closures. Schedulers are prime examples. Feel free to experiment with the examples shown here and use debugger to examine step-by-step what code gets executed and in what order. So, let's figure out how to fix this. We don't want to subscribe at the end of the chain multiple times, so we can create an instance of Subject class, where we'll subscribe both Observers, and the Subject class itself will subscribe to the AnnonymousObservable as discussed a moment ago: // ... use RxSubjectSubject; $subject = new Subject(); $connObs = new ConnectableObservable(new RangeObservable(0, 6)); $filteredObservable = $connObs ->map(function($val) { return $val ** 2; }) ->filter(function($val) { echo "Filter: $valn"; return $val % 2; }) ->subscribe($subject); $disposable1 = $subject->subscribeCallback(function($val) { echo "S1: ${val}n"; }); $disposable2 = $subject->subscribeCallback(function($val) { echo "S2: ${val}n"; }); $connObservable->connect(); Now we can run the script again and see that it does what we wanted it to do: $ php rxphp_filters_observables.php Filter: 0 Filter: 1 S1: 1 S2: 1 Filter: 4 Filter: 9 S1: 9 S2: 9 Filter: 16 Filter: 25 S1: 25 S2: 25 This might look like an edge case, but soon we'll see that this issue, left unhandled, could lead to some very unpredictable behavior. We'll bring out both these issues (proper usage of Disposables and Operator chains) when we start writing our Reddit reader. Summary In this article, we looked in more depth at how to use Disposables and Operators, how these work internally, and what it means for us. We also looked at a couple of new classes from RxPHP, such as ConnectableObservable, and CompositeDisposable. Resources for Article: Further resources on this subject: Working with JSON in PHP jQuery [article] Working with Simple Associations using CakePHP [article] Searching Data using phpMyAdmin and MySQL [article]
Read more
  • 0
  • 0
  • 14589

Packt
24 Sep 2013
8 min read
Save for later

Quick start – using Haml

Packt
24 Sep 2013
8 min read
(For more resources related to this topic, see here.) Step 1 – integrating with Rails and creating a simple view file Let's create a simple rails application and change one of the view files to Haml from ERB: Create a new rails application named blog: rails new blog Add the Haml gem to this application's Gemfile and run it: bundle install When the gem has been added, generate some views that we can convert to Haml and learn about its basic features. Run the Rails scaffold generator to create a scffold named post. rails g scaffold post After this, you need to run the migrations to create the database tables: rake db:migrate You should get an output as shown in the following screenshot: Our application is not yet generating Haml views automatically. We will switch it to this mode in the next steps. The index.html.erb file that has been generated and is located in app/views/posts/index.html.erb looks as follows: <h1>Listing posts</h1><table> <tr> <th></th> <th></th> <th></th> </tr><% @posts.each do |post| %> <tr> <td><%= link_to 'Show', post %></td> <td><%= link_to 'Edit', edit_post_path(post) %></td> <td><%= link_to 'Destroy', post, method: :delete, data: { confirm:'Are you sure?' } %></td> </tr><% end %></table><br><%= link_to 'New Post', new_post_path %>] Let's convert it to an Haml view step-by-step. First, let's understand the basic features of Haml: Any HTML tags are written with a percent sign and then the name of the tag Whitespace (tabs) is being used to create nested tags Any part of an Haml line that is not interpreted as something else is taken to be plain text Closing tags as well as end statements are omitted for Ruby blocks Knowing the previous features we can write the first lines of our example view in Haml. Open the index.html.erb file in an editor and replace <h1>, <table>, <th>, and <tr> as follows: <h1>Listing posts</h1> can be written as %h1 Listing posts <table> can be written as %table <tr> becomes %tr <th> becomes %th After those first replacements our view file should look like: %h1 Listing posts%table %tr %th %th %th<% @posts.each do |post| %> <tr> <td><%= link_to 'Show', post %></td> <td><%= link_to 'Edit', edit_post_path(post) %></td> <td><%= link_to 'Destroy', post, method: :delete, data: { confirm:'Are you sure?' } %></td> </tr><% end %><br><%= link_to 'New Post', new_post_path %> Please notice how %tr is nested within the %table tag using a tab and also how %th is nested within %tr using a tab. Next, let's convert the Ruby parts of this view. Ruby is evaluated and its output is inserted into the view when using the equals character. In ERB we had to use <%=, whereas in Haml, this is shortened to just a =. The following examples illustrate this: <%= link_to 'Show', post %> becomes = link_to 'Show', post and all the other <%= parts are changed accordingly The equals sign can be used at the end of the tag to insert the Ruby code within that tag Empty (void) tags, such as <br>, are created by adding a forward slash at the end of the tag Please note that you have to leave a space after the equals sign. After the changes are incorporated, our view file will look like %h1 Listing posts %table %tr %th %th %th<% @posts.each do |post| %> %tr %td= link_to 'Show',post %td= link_to 'Edit',edit_post_path(post) %td= link_to 'Destroy',post,method: :delete,data: { confirm: 'Areyou sure?' }%br/= link_to 'New Post',new_post_path The only thing left to do now is to convert the Ruby block part: <% @posts.each do | post | %>. Code that needs to be run, but does not generate any output, is written using a hyphen character. Here is how this conversion works: Ruby blocks do not need to be closed, they end when the indentation decreases HTML tags and the Ruby code that is nested within the block are indented by one tab more than the block <% @posts.each do |post| %> becomes - @ posts.each do |post| Remember about a space after the hyphen character After we replace the remaining part in our view file according to the previous rules, it should look as follows: %h1 Listing posts%table %tr %th %th %th- @posts.each do |post| %tr %td= link_to 'Show', post %td= link_to 'Edit', edit_post_path(post) %td= link_to 'Destroy', post, method: :delete, data: { confirm:'Are you sure?' }%br/= link_to 'New Post', new_post_path Save the view file and change its name to index.html.haml. This is now an Haml-based template. Start our example Rails application and visit http: //localhost:3000/posts to see the view being rendered by Rails, as shown in the following screenshot: Step 2 – switching Rails application to use Haml as the templating engine In the previous step, we have enabled Haml in the test application. However, if you generate new view files using any of the Rails built-in generators, it will still use ERB. Let's switch the application to use Haml as the templating engine. Edit the blog application Gemfile and add a gem named haml-rails to it. You can add it to the :development group because the generators are only used during development and this functionality is not needed in production or test environments. Our application Gemfile now looks as shown in the following code: source 'https://rubygems.org'gem 'rails', '3.2.13'gem 'sqlite3'gem 'haml'gem 'haml-rails', :group => :developmentgroup :assets do gem 'sass-rails', '~> 3.2.3' gem 'coffee-rails', '~> 3.2.1' gem 'uglifier', '>= 1.0.3'endgem 'jquery-rails' Then run following bundle command to install the gem: bundle install Let's say the posts in our application need to have categories. Run the scaffold generator to create some views for categories. This generator will create views using Haml, as shown in the following screenshot: Please note that new views have a .html.haml extension and are using Haml. For example, the _form.html.haml view for the form looks as follows: = form_for @category do |f|Asjd12As - if @category.errors.any? #error_explanation %h2= "#{pluralize(@category.errors.count, "error")} prohibitedthis category from being saved:" %ul - @category.errors.full_messages.each do |msg| %li= msg .actions = f.submit 'Save' There are two very useful shorthand notations for creating a <div> tag with a class or a <div> tag with an ID. To create a div with an ID, use the hash symbol followed by the name of the ID. For example, #error_explanation will result in <div id="error_explanation"> To create a <div>tag with a class attribute, use a dot followed by the name of the class. For example, .actions will create <div class="actions"> Step 3 – converting existing view templates to Haml Our example blog app still has some leftover templates which are using ERB as well as an application.html.erb layout file. We would like to convert those to Haml. There is no need to do it all individually, because there is a handy gem which will automatically convert all the ERB files to Haml, shown as follows: Let's install the html2haml gem: gem install html2haml Using the cd command, change the current working directory to the app directory of our example application and run the following bash command to convert all the ERB files to Haml (to run this command you need a bash shell. On Windows, you can use the embedded bash shell which ships with GitHub for Windows, Cygwin bash, MinGW bash, or the MSYS bash shell which is bundled with Git for Windows). for file in $(find . -type f -name \*.html.erb); do html2haml -e ${file} "$(dirname ${file})/$(basename ${file}.erb).haml";done Then to remove the ERB files and run this command: find ./ -name *erb | while read line; do rm $line; done Those two Bash snippets will first convert all the ERB files recursively in the app directory of our application and then remove the remaining ERB view templates. Summary This article covered integrating with Rails and creating a simple view file, switching Rails application to use Haml as the templating engine, and converting existing view templates to Haml. Resources for Article : Further resources on this subject: Building HTML5 Pages from Scratch [Article] URL Shorteners – Designing the TinyURL Clone with Ruby [Article] Building tiny Web-applications in Ruby using Sinatra [Article]
Read more
  • 0
  • 0
  • 14585

article-image-neural-network-model-multi-layer-perceptrons-classifying-iris-flower-species
Sunith Shetty
16 Dec 2017
9 min read
Save for later

Deep Learning Algorithms: How to classify Irises using multi-layer perceptrons

Sunith Shetty
16 Dec 2017
9 min read
[box type="note" align="" class="" width=""]This article is an excerpt taken from the book Big Data Analytics with Java by Rajat Mehta. Java is the de facto language for major big data environments like Hadoop, MapReduce etc. This book will teach you how to perform analytics on big data with production-friendly Java.[/box] From our below given post, we help you learn how to classify flower species from Iris dataset using multi-layer perceptrons. Code files are available for download towards the end of the post. Flower species classification using multi-layer perceptrons This is a simple hello world-style program for performing classification using multi-layer perceptrons. For this, we will be using the famous Iris dataset, which can be downloaded from the UCI Machine Learning Repository at https://archive.ics.uci.edu/ml/datasets/Iris. This dataset has four types of datapoints, shown as follows: Attribute name Attribute description Petal Length Petal length in cm Petal Width Petal width in cm Sepal Length Sepal length in cm Sepal Width Sepal width in cm Class The type of iris flower that is Iris Setosa, Iris Versicolour, Iris Virginica This is a simple dataset with three types of Iris classes, as mentioned in the table. From the perspective of our neural network of perceptrons, we will be using the multi-perceptron algorithm bundled inside the spark ml library and will demonstrate how you can club it with the Spark-provided pipeline API for the easy manipulation of the machine learning workflow. We will also split our dataset into training and testing bundles so as to separately train our model on the training set and finally test its accuracy on the test set. Let's now jump into the code of this simple example. First, create the Spark configuration object. In our case, we also mention that the master is local as we are running it on our local machine: SparkConf sc = new SparkConf().setMaster("local[*]"); Next, build the SparkSession with this configuration and provide the name of the application; in our case, it is JavaMultilayerPerceptronClassifierExample: SparkSession spark = SparkSession  .builder()  .config(sc)  .appName("JavaMultilayerPerceptronClassifierExample")  .getOrCreate(); Next, provide the location of the iris dataset file: String path = "data/iris.csv"; Now load this dataset file into a Spark dataset object. As the file is in an csv format, we also specify the format of the file while reading it using the SparkSession object: Now load this dataset file into a Spark dataset object. As the file is in an csv format, we also specify the format of the file while reading it using the SparkSession object: Dataset<Row> dataFrame1 = spark.read().format("csv").load(path); After loading the data from the file into the dataset object, let's now extract this data from the dataset and put it into a Java class, IrisVO. This IrisVO class is a plain POJOand has the attributes to store the data point types, as shown: public class IrisVO { private Double sepalLength; private Double petalLength; private Double petalWidth; private Double sepalWidth; private String labelString; On the dataset object dataFrame1, we invoke the to JavaRDD method to convert it into an RDD object and then invoke the map function on it. The map function is linked to a lambda function, as shown. In the lambda function, we go over each row of the dataset and pull the data items from it and fill it in the IrisVO POJO object before finally returning this object from the lambda function. This way, we get a dataMap rdd object filled with IrisVO objects: JavaRDD<IrisVO> dataMap = dataFrame1.toJavaRDD().map( r -> {  IrisVO irisVO = new IrisVO();  irisVO.setLabelString(r.getString(5));  irisVO.setPetalLength(Double.parseDouble(r.getString(3)));  irisVO.setSepalLength(Double.parseDouble(r.getString(1)));  irisVO.setPetalWidth(Double.parseDouble(r.getString(4)));  irisVO.setSepalWidth(Double.parseDouble(r.getString(2)));  return irisVO; }); As we are using the latest Spark ML library for applying our machine learning algorithms from Spark, we need to convert this RDD back to a dataset. In this case, however, this dataset would have the schema for the individual data points as we had mapped them to the IrisVO object attribute types earlier: Dataset<Row> dataFrame = spark.createDataFrame(dataMap.rdd(), IrisVO. class); We will now split the dataset into two portions: one for training our multi-layer perceptron model and one for testing its accuracy later. For this, we are using the prebuilt randomSplit method available on the dataset object and will provide the parameters. We keep 70 percent for training and 30 percent for testing. The last entry is the 'seed' value supplied to the randomSplit method. Dataset<Row>[] splits = dataFrame.randomSplit(new double[]{0.7, 0.3}, 1234L); Next, we extract the splits into individual datasets for training and testing: Dataset<Row> train = splits[0]; Dataset<Row> test = splits[1]; Until now we had seen the code that was pretty much generic across most of the Spark machine learning implementations. Now we will get into the code that is specific to our multi-layer perceptron model. We will create an int array that will contain the count for the various attributes needed by our model: int[] layers = new int[] {4, 5, 4, 3}; Let's now look at the attribute types of this int array, as shown in the following table: Attribute value at array index Description 0 This is the number of neurons or perceptrons at the input layer of the network. This is the count of the number of features that are passed to the model. 1 This is a hidden layer containing five perceptrons (sigmoid neurons only, ignore the terminology). 2 This is another hidden layer containing four sigmoid neurons. 3 This is the number of neurons representing the output label classes. In our case, we have three types of Iris flowers, hence three classes. After creating the layers for the neural network and specifying the number of neurons in each layer, next build a StringIndexer class. Since our models are mathematical and look for mathematical inputs for their computations, we have to convert our string labels for classification (that is, Iris Setosa, Iris Versicolour, and Iris Virginica) into mathematical numbers. To do this, we use the StringIndexer class that is provided by Apache Spark. In the instance of this class, we also provide the place from where we can read the data for the label and the column where it will output the numerical representation for that label: StringIndexer labelIndexer = new StringIndexer(). setInputCol("labelString").setOutputCol("label"); Now we build the features array. These would be the features that we use when training our model: String[] featuresArr = {"sepalLength","sepalWidth","petalLength","pet alWidth"}; Next, we build a features vector as this needs to be fed to our model. To put the feature in vector form, we use the VectorAssembler class from the Spark ML library. We also provide a features array as input and provide the output column where the vector array will be printed: VectorAssembler va = new VectorAssembler().setInputCols(featuresArr). setOutputCol("features"); Now we build the multi-layer perceptron model that is bundled within the Spark ML library. To this model we supply the array of layers we created earlier. This layer array has the number of neurons (sigmoid neurons) that are needed in each layer of the multi-perceptron network: MultilayerPerceptronClassifier trainer = new MultilayerPerceptronClassifier()  .setLayers(layers)  .setBlockSize(128)  .setSeed(1234L)  .setMaxIter(25); The other parameters that are being passed to this multi-layer perceptron model are: Block Size Block size for putting input data in matrices for faster computation. The default value is 128. Seed Seed for weight initialization if weights are not set. Maximum iterations Maximum number of iterations to be performed on the dataset while learning. The default value is 100. Finally, we hook all the workflow pieces together using the pipeline API. To this pipeline API, we pass the different pieces of the workflow, that is, the labelindexer and vector assembler, and finally provide the model: Pipeline pipeline = new Pipeline().setStages(new PipelineStage[] {labelIndexer, va, trainer}); Once our pipeline object is ready, we fit the model on the training dataset to train our model on the underlying training data: PipelineModel model = pipeline.fit(train); Once the model is trained, it is not yet ready to be run on the test data to figure out its predictions. For this, we invoke the transform method on our model and store the result in a Dataset object: Dataset<Row> result = model.transform(test); Let's see the first few lines of this result by invoking a show method on it: result.show(); This would print the result of the first few lines of the result dataset as shown: As seen in the previous image, the last column depicts the predictions made by our model. After making the predictions, let's now check the accuracy of our model. For this, we will first select two columns in our model which represent the predicted label, as well as the actual label (recall that the actual label is the output of our StringIndexer): Dataset<Row> predictionAndLabels = result.select("prediction", "label"); Finally, we will use a standard class called MulticlassClassificationEvaluator, which is provided by Spark for checking the accuracy of the models. We will create an instance of this class. Next, we will set the metric name of the metric, that is, accuracy, for which we want to get the value from our predicted results: MulticlassClassificationEvaluator evaluator = new MulticlassClassificationEvaluator() .setMetricName("accuracy"); Next, using the instance of this evaluator, invoke the evaluate method and pass the parameter of the dataset that contains the column for the actual result and predicted result (in our case, it is the predictionAndLabels column): System.out.println("Test set accuracy = " + evaluator.evaluate(predictionAndLabels)); This would print the output as: If we get this value in a percentage, this means that our model is 95% accurate. This is the beauty of neural networks - they can give us very high accuracy when tweaked properly. With this, we come to an end for our small hello world-type program on multi-perceptrons. Unfortunately, Spark support on neural networks and deep learning is not extensive; at least not until now. To summarize, we covered a sample case study for the classification of Iris flower species based on the features that were used to train our neural network. If you are keen to know more about real-time analytics using deep learning methodologies such as neural networks and multi-layer perceptrons, you can refer to the book Big Data Analytics with Java. [box type="download" align="" class="" width=""]Download Code files[/box]      
Read more
  • 0
  • 0
  • 14580

article-image-sdlc-puts-process-at-the-center-of-software-engineering
Richard Gall
22 May 2018
7 min read
Save for later

SDLC puts process at the center of software engineering

Richard Gall
22 May 2018
7 min read
What is SDLC? SDLC stands for software development lifecycle. It refers to all of the different steps that software engineers need to take when building software. This includes planning, creating, building and then deploying software, but maintenance is also crucial too. In some instances you may need to change or replace software - that is part of the software development lifecycle as well. SDLC is about software quality and development efficiency SDLC is about more than just the steps in the software development process. It's also about managing that process in a way that improves quality while also improving efficiency. Ultimately, there are numerous ways of approaching the software development lifecycle - Waterfall and Agile are the too most well known methodologies for managing the development lifecycle. There are plenty of reasons you might choose one over another. What is most important is that you pay close attention to what the software development lifecycle looks like. It sounds obvious, but it is very difficult to build software without a plan in place. Things can get chaotic very quickly. If it does, that's bad news for you, as the developer, and bad news for users as well. When you don't follow the software development lifecycle properly, you're likely to miss user requirements, and faults will also find their way into your code. The stages of the software development lifecycle (SDLC) There are a number of ways you might see an SDLC presented but the core should always be the same. And yes, different software methodologies like Agile and Waterfall outline very different ways of working, but broadly the steps should be the same. What differs between different software methodologies is how each step fits together. Step 1: Requirement analysis This is the first step in any SDLC. This is about understanding everything that needs to be understood in as much practical detail as possible. It might mean you need to find out about specific problems that need to be solved. Or, there might be certain things that users need that you need to make sure are in the software. To do this, you need to do good quality research, from discussing user needs with a product manager to revisiting documentation on your current systems. It is often this step that is the most challenging in the software development lifecycle. This is because you need to involve a wide range of stakeholders. Some of these might not be technical, and sometimes you might simply use a different vocabulary. It's essential that you have a shared language to describe everything from the user needs to the problems you might be trying to solve. Step 2: Design the software Once you have done a requirement analysis you can begin designing the software. You do this by turning all the requirements and software specifications into a design document. This might feel like it slows down the development process, but if you don't do this, not only are you wasting the time taken to do your requirement analysis, you're also likely to build poor quality or even faulty software. While it's important not to design by committee or get slowed down by intensive nave-gazing, keeping stakeholders updated and requesting feedback and input where necessary can be incredibly important. Sometimes its worth taking that extra bit of time, as it could solve a lot of problems later in the SDLC. Step 3: Plan the project Once you have captured requirements and feel you have properly understood exactly what needs to be delivered - as well as any potential constraints - you need to plan out how you're going to build that software. To do this you'll need to have an overview of the resources at your disposal. These are the sorts of questions you'll need to consider at this stage: Who is available? Are there any risks? How can we mitigate them? What budget do we have for this project? Are there any other competing projects? In truth, you'll probably do this during the design stage. The design document you create should, of course, be developed with context in mind. It's pointless creating a stunning design document, outlining a detailed and extensive software development project if it's simply not realistic for your team to deliver it. Step 4: Start building the software Now you can finally get down to the business of actually writing code. With all the work you have done in the previous steps this should be a little easier. However, it's important to remember that imperfection is part and parcel of software engineering. There will always be flaws in your software. That doesn't necessarily mean bugs or errors, however; it could be small compromises that need to be made in order to ensure something works. The best approach here is to deliver rapidly. The sooner you can get software 'out there' the faster you can make changes and improvements if (or more likely when) they're needed. It's worth involving stakeholders at this stage - transparency in the development process is a good way to build collaboration and ensure the end result delivers on what was initially in the requirements. Step 5: Testing the software Testing is, of course, an essential step in the software development lifecycle. This is where you identify any problems. That might be errors or performance issues, but you may find you haven't quite been able to deliver what you said you would in the design document. The continuous integration server is important here, as the continuous integration server can help to detect any problems with the software. The rise of automated software testing has been incredibly valuable; it means that instead of spending time manually running tests, engineers can dedicate more time to fixing problems and optimizing code. Step 6: Deploy the software The next step is to deploy the software to production. All the elements of the software should now be in place, and you want it to simply be used. It's important to remember that there will be problems here. Testing can never capture every issue, and feedback and insight from users are going to be much more valuable than automated tests run on a server. Continuous delivery pipelines allow you to deploy software very efficiently. This makes the build-test-deploy steps of the software development lifecycle to be relatively frictionless. Okay, maybe not frictionless - there's going to be plenty of friction when you're developing software. But it does allow you to push software into production very quickly. Step 7: Maintaining software Software maintenance is a core part of the day-to-day life of a software engineer. Its a crucial step in the SDLC. There are two forms of software maintenance; both are of equal importance. Evolutive maintenance and corrective maintenance. Evolutive maintenance As the name suggests, evolutive maintenance is where you evolve software by adding in new functionality or making larger changes to the logic of the software. These changes should be a response to feedback from stakeholders or, more importantly, users. There may be times when business needs dictate this type of maintenance - this is never ideal, but it is nevertheless an important part of a software engineer's work. Corrective maintenance Corrective maintenance isn't quite as interesting or creative as evolutive maintenance - it's about fixing bugs and errors in the code. This sort of maintenance can feel like a chore, and ideally you want to minimize the amount of time you spend doing this. However, if you're following SDLC closely, you shouldn't find too many bugs in your software. The benefits of SDLC are obvious The benefits of SDLC are clear. It puts process at the center of software engineering. Without those processes it becomes incredibly difficult to build the software that stakeholders and users want. And if you don't care about users then, really, why build software at all. It's true that DevOps has done a lot to change SDLC. Arguably, it is an area that is more important and more hotly debated than ever before. It's not difficult to find someone with an opinion on the best way to build something. Equally, as software becomes more fragmented and mutable, thanks to the emergence of cloud and architectural trends like microservices and serverless, the way we design, build and deploy software has never felt more urgent. Read next DevOps Engineering and Full-Stack Development – 2 Sides of the Same Agile Coin
Read more
  • 0
  • 0
  • 14572
article-image-did-you-know-facebook-shares-the-data-you-share-with-them-for-security-reasons-with-advertisers
Natasha Mathur
28 Sep 2018
5 min read
Save for later

Did you know Facebook shares the data you share with them for ‘security’ reasons with advertisers?

Natasha Mathur
28 Sep 2018
5 min read
Facebook is constantly under the spotlight these days when it comes to controversies regarding user’s data and privacy. A new research paper published by the Princeton University researchers states that Facebook shares the contact information you handed over for security purposes, with their advertisers. This study was first brought to light by a Gizmodo writer, Kashmir Hill. “Facebook is not content to use the contact information you willingly put into your Facebook profile for advertising. It is also using contact information you handed over for security purposes and contact information you didn’t hand over at all, but that was collected from other people’s contact books, a hidden layer of details Facebook has about you that I’ve come to call “shadow contact information”, writes Hill. Recently, Facebook introduced a new feature called custom audiences. Unlike traditional audiences, the advertiser is allowed to target specific users. To do so, the advertiser uploads user’s PII (personally identifiable information) to Facebook. After the uploading is done, Facebook then matches the given PII against platform users. Facebook then develops an audience that comprises the matched users and allows the advertiser to further track the specific audience. Essentially with Facebook, the holy grail of marketing, which is targeting an audience of one, is practically possible; nevermind whether that audience wanted it or not. In today’s world, different social media platforms frequently collect various kinds of personally identifying information (PII), including phone numbers, email addresses, names and dates of birth. Majority of this PII often represent extremely accurate, unique, and verified user data. Because of this, these services have the incentive to exploit and use this personal information for other purposes. One such scenario includes providing advertisers with more accurate audience targeting. The paper titled ‘Investigating sources of PII used in Facebook’s targeted advertising’ is written by Giridhari Venkatadri, Elena Lucherini, Piotr Sapiezynski, and Alan Mislove. “In this paper, we focus on Facebook and investigate the sources of PII used for its PII-based targeted advertising feature. We develop a novel technique that uses Facebook’s advertiser interface to check whether a given piece of PII can be used to target some Facebook user and use this technique to study how Facebook’s advertising service obtains users’ PII,” reads the paper. The researchers developed a novel methodology, which involved studying how Facebook obtains the PII to provide custom audiences to advertisers. “We test whether PII that Facebook obtains through a variety of methods (e.g., directly from the user, from two-factor authentication services, etc.) is used for targeted advertising, whether any such use is clearly disclosed to users, and whether controls are provided to users to help them limit such use,” reads the paper. The paper uses size estimates to study what sources of PII are used for PII-based targeted advertising. Researchers used this methodology to investigate which range of sources of PII was actually used by Facebook for its PII-based targeted advertising platform. They also examined what information gets disclosed to users and what control users have over PII. What sources of PII are actually being used by Facebook? Researchers found out that Facebook allows its users to add contact information (email addresses and phone numbers) on their profiles. While any arbitrary email address or phone number can be added, it is not displayed to other users unless verified (through a confirmation email or confirmation SMS message, respectively). This is the most direct and explicit way of providing PII to advertisers. Researchers then further moved on to examine whether PII provided by users for security purposes such as two-factor authentication (2FA) or login alerts are being used for targeted advertising. They added and verified a phone number for 2FA to one of the authors’ accounts. The added phone number became targetable after 22 days. This proved that a phone number provided for 2FA was indeed used for PII-based advertising, despite having set the privacy controls to the choice. What control do users have over PII? Facebook allows users the liberty of choosing who can see each PII listed on their profiles, the current list of possible general settings being: Public, Friends, Only Me.   Users can also restrict the set of users who can search for them using their email address or their phone number. Users are provided with the following options: Everyone, Friends of Friends, and Friends. Facebook provides users a list of advertisers who have included them in a custom audience using their contact information. Users can opt out of receiving ads from individual advertisers listed here. But, information about what PII is used by advertisers is not disclosed. What information about how Facebook uses PII gets disclosed to the users? On adding mobile phone numbers directly to one’s Facebook profile, no information about the uses of that number is directly disclosed to them. This Information is only disclosed to users when adding a number from the Facebook website. As per the research results, there’s very little disclosure to users, often in the form of generic statements that do not refer to the uses of the particular PII being collected or that it may be used to allow advertisers to target users. “Our paper highlights the need to further study the sources of PII used for advertising, and shows that more disclosure and transparency needs to be provided to the user,” says the researchers in the paper. For more information, check out the official research paper. Ex-employee on contract sues Facebook for not protecting content moderators from mental trauma How far will Facebook go to fix what it broke: Democracy, Trust, Reality Mark Zuckerberg publishes Facebook manifesto for safeguarding against political interference
Read more
  • 0
  • 0
  • 14569

article-image-building-applications-spring-data-redis
Packt
03 Dec 2012
9 min read
Save for later

Building Applications with Spring Data Redis

Packt
03 Dec 2012
9 min read
(For more resources related on Spring, see here.) Designing a Redis data model The most important rules of designing a Redis data model are: Redis does not support ad hoc queries and it does not support relations in the same way than relational databases. Thus, designing a Redis data model is a total different ballgame than designing the data model of a relational database. The basic guidelines of a Redis data model design are given as follows: Instead of simply modeling the information stored in our data model, we have to also think how we want to search information from it. This often leads to a situation where we have to duplicate data in order to fulfill the requirements given to us. Don't be afraid to do this. We should not concentrate on normalizing our data model. Instead, we should combine the data that we need to handle as an unit into an aggregate. Since Redis does not support relations, we have to design and implement these relations by using the supported data structures. This means that we have to maintain these relations manually when they are changed. Because this might require a lot of effort and code, it could be wise to simply duplicate the information instead of using relations. It is always wise to spend a moment to verify that we are using the correct tool for the job. NoSQL Distilled, by Martin Fowler contains explanations of different NoSQL databases and their use cases, and can be found at http://martinfowler.com/books/nosql.html. Redis supports multiple data structures. However, one question remained unanswered: which data structure should we use for our data? This question is addressed in the following table: Data type Description String A string is good choice for storing information that is already converted to a textual form. For instance, if we want to store HTML, JSON, or XML, a string should be our weapon of choice. List A list is a good choice if we will access it only near the start or end. This means that we should use it for representing queues or stacks. Set We should use a set if we need to get the size of a collection or check if a certain item belongs to it. Also, if we want to represent relations, a set is a good choice (for example, "who are John's friends?"). Sorted set Sorted sets should be used in the same situations as sets when the ordering of items is important to us. Hash A hash is a perfect data structure for representing complex objects. Key components Spring Data Redis provides certain components that are the cornerstones of each application that uses it. This section provides a brief introduction to the components that we will later use to implement our example applications. Atomic counters Atomic counters are for Redis what sequences are for relational databases. Atomic counters guarantee that the value received by a client is unique. This makes these counters a perfect tool for creating unique IDs to our data that is stored in Redis. At the moment, Spring Data Redis offers two atomic counters: RedisAtomicInteger and RedisAtomicLong . These classes provide atomic counter operations for integers and longs. RedisTemplate The RedisTemplate<K,V> class is the central component of Spring Data Redis. It provides methods that we can use to communicate with a Redis instance. This class requires that two type parameters are given during its instantiation: the type of used Redis key and the type of the Redis value. Operations The RedisTemplate class provides two kinds of operations that we can use to store, fetch, and remove data from our Redis instance: Operations that require that the key and the value are given every time an operation is performed. These operations are handy when we have to execute a single operation by using a key and a value. Operations that are bound to a specific key that is given only once. We should use this approach when we have to perform multiple operations by using the same key. The methods that require that a key and value is given every time an operation is performed are described in following list: HashOperations<K,HK,HV> opsForHash(): This method returns the operations that are performed on hashes ListOperations<K,V> opsForList(): This method returns the operations performed on lists SetOperations<K,V> opsForSet(): This method returns the operations performed on sets ValueOperations<K,V> opsForValue(): This method returns the operations performed on simple values ZSetOperations<K,HK,HV> opsForZSet(): This method returns the operations performed on sorted sets The methods of the RedisTemplate class that allow us to execute multiple operations by using the same key are described in following list: BoundHashOperarations<K,HK,HV> boundHashOps(K key): This method returns hash operations that are bound to the key given as a parameter BoundListOperations<K,V> boundListOps(K key): This method returns list operations bound to the key given as a parameter BoundSetOperations<K,V> boundSetOps(K key):: This method returns set operations, which are bound to the given key BoundValueOperations<K,V> boundValueOps(K key): This method returns operations performed to simple values that are bound to the given key BoundZSetOperations<K,V> boundZSetOps(K key): This method returns operations performed on sorted sets that are bound to the key that is given as a parameter The differences between these operations become clear to us when we start building our example applications. Serializers Because the data is stored in Redis as bytes, we need a method for converting our data to bytes and vice versa. Spring Data Redis provides an interface called RedisSerializer<T>, which is used in the serialization process. This interface has one type parameter that describes the type of the serialized object. Spring Data Redis provides several implementations of this interface. These implementations are described in the following table: Serializer Description GenericToStringSerializer<T> Serializes strings to bytes and vice versa. Uses the Spring ConversionService to transform objects to strings and vice versa. JacksonJsonRedisSerializer<T> Converts objects to JSON and vice versa. JdkSerializationRedisSerializer Provides Java based serialization to objects. OxmSerializer Uses the Object/XML mapping support of Spring Framework 3. StringRedisSerializer  Converts strings to bytes and vice versa. We can customize the serialization process of the RedisTemplate class by using the described serializers. The RedisTemplate class provides flexible configuration options that can be used to set the serializers that are used to serialize value keys, values, hash keys, hash values, and string values. The default serializer of the RedisTemplate class is JdkSerializationRedisSerializer. However, the string serializer is an exception to this rule. StringRedisSerializer is the serializer that is by default used to serialize string values. Implementing a CRUD application This section describes two different ways for implementing a CRUD application that is used to manage contact information. First, we will learn how we can implement a CRUD application by using the default serializer of the RedisTemplate class. Second, we will learn how we can use value serializers and implement a CRUD application that stores our data in JSON format. Both of these applications will also share the same domain model. This domain model consists of two classes: Contact and Address. We removed the JPA specific annotations from them We use these classes in our web layer as form objects and they no longer have any other methods than getters and setters The domain model is not the only thing that is shared by these examples. They also share the interface that declares the service methods for the Contact class. The source code of the ContactService interface is given as follows: public interface ContactService {public Contact add(Contact added);public Contact deleteById(Long id) throws NotFoundException;public List&lt;Contact&gt; findAll();public Contact findById(Long id) throws NotFoundException;public Contact update(Contact updated) throws NotFoundException;} Both of these applications will communicate with the used Redis instance by using the Jedis connector. Regardless of the user's approach, we can implement a CRUD application with Spring Data Redis by following these steps: Configure the application context. Implement the CRUD functions. Let's get started and find out how we can implement the CRUD functions for contact information. Using default serializers This subsection describes how we can implement a CRUD application by using the default serializers of the RedisTemplate class. This means that StringRedisSerializer is used to serialize string values, and JdkSerializationRedisSerializer serializes other objects. Configuring the application context We can configure the application context of our application by making the following changes to the ApplicationContext class: Configuring the Redis template bean. Configuring the Redis atomic long bean. Configuring the Redis template bean We can configure the Redis template bean by adding a redisTemplate() method to the ApplicationContext class and annotating this method with the @Bean annotation. We can implement this method by following these steps: Create a new RedisTemplate object. Set the used connection factory to the created RedisTemplate object. Return the created object. The source code of the redisTemplate() method is given as follows: @Beanpublic RedisTemplate redisTemplate() {RedisTemplate&lt;String, String&gt; redis = new RedisTemplate&lt;String,String&gt;();redis.setConnectionFactory(redisConnectionFactory());return redis;} Configuring the Redis atomic long bean We start the configuration of the Redis atomic long bean by adding a method called redisAtomicLong() to the ApplicationContext class and annotating the method with the @Bean annotation. Our next task is to implement this method by following these steps: Create a new RedisAtomicLong object. Pass the name of the used Redis counter and the Redis connection factory as constructor parameters. Return the created object. The source code of the redisAtomicLong() method is given as follows: @Beanpublic RedisAtomicLong redisAtomicLong() {return new RedisAtomicLong("contact", redisConnectionFactory());} If we need to create IDs for instances of different classes, we can use the same Redis counter. Thus, we have to configure only one Redis atomic long bean.
Read more
  • 0
  • 0
  • 14558

article-image-heart-it-all
Packt
27 Jan 2016
15 min read
Save for later

The Heart of It All

Packt
27 Jan 2016
15 min read
In this article by Thomas Hamilton, the author of Building a Media Center with Raspberry Pi ,you will learn how to find the operating system that you will use on the system that you chose. Just like with hardware, there are a plethora of options for the operating systems for the Raspberry Pi. For this book, we are going to focus on transforming the Raspberry Pi into a media center. At the time of writing this book, there are two operating systems available that are well known for being geared specifically to do just this. The first one is called the Open Embedded Linux Entertainment Center (openELEC) and is a slimmed-down operating system that has been optimized to be a media center and nothing else. The second option, and the one that we will be using for this project, is called the Open Source Media Center (OSMC). The main advantage of this specific version is that there is a full operating system running in the background. This will be important for some of the add-ons to work correctly. Once you can do this, if you want to try openELEC, you will be fully prepared to be able to do this on your own. In fact, the information in this article will enable you to install practically any operating system that's designed for a Raspberry Pi onto an SD card for you to use and experiment with as you see fit. In this article, we will cover the following topics: Downloading an operating system Installing an operating system to an SD card using Windows Install an operating system to an SD card using Linux (For more resources related to this topic, see here.) The Operating System It is now time to find the correct version of OSMC so that we can download and install it. If you are primarily a Windows or an iOS user, it may feel strange to think that you can search online for operating systems and just download them to your computer. In the Linux world, the world in which the Raspberry Pi resides, this is very normal and one of the great things about open source. The Raspberry Pi is built as a learning tool. It was designed in such a way that it will allow you to modify and add to it. In this way, the community can develop it and make it better. Open source software does the same thing. If you know programming, you can contribute to and change software that someone else developed, and this is encouraged! More eyes on the code means less bugs and vulnerabilities. Most versions of Linux follow this open source principle. Versions of Linux? Yes. This is another point of confusion for Windows and Mac users. For the computers that you buy in a normal retail or computer store, you do not have many choices related to the OS that is already installed. You can either buy an Apple product with the newest version of their OS, or a Windows-based computer with Windows 7, 8, or 10 pre-installed. In this example, Windows 7, 8, and 10 are just newer and older versions of each other. Linux works off a different principle. Linux itself is not an operating system. Think of it more like a type of operating system or maybe as a brand such as Microsoft and Apple. Because it is open source and free, developers can take it and turn it into whatever they need it to be. The most popular versions of Linux are Ubuntu, Fedora, Suse, Mint, and CentOS. They each have a different look and feel and can have different functions. They are also operating systems that can be used daily for your normal computing needs. This article is based on a combination of Ubuntu and Fedora operating systems. The world of Linux and open source software can be confusing at first. Don't be scared! After you get past the shock, you will find that this openness is very exciting and helpful and can actually make your life much easier. Now, lets download OSMC. Raspberrypi.org If you haven't come across this already, it is the official website for the Raspberry Pi. From this website, you can find information about the Raspberry Pi, instructional how-tos and forums to talk with other Raspberry Pi users. This site can point you to their official retailers for the versions of the Raspberry Pi that are currently in production, and for the purpose of this article, it points us to the most popular operating systems for the Raspberry Pi (though not nearly all the ones that can work on it). From the main page, click on the link that says DOWNLOADS near the top of the page. This will bring you to the page that lists the most popular operating systems. Raspbian is the official OS of the Raspberry Pi and what OSMC is based on. Noobs is worth looking at for your next project. It isn't an OS itself, but it gives you the ability to choose from a list of operating systems and install them with a single click. If you want to see what the Raspberry Pi is capable of, start with Noobs. Under these options, you will have a list of third-party operating systems. The names may sound familiar at this point, as we have mentioned most of them already. This list is where you will find OSMC. Click on its link to go to their website. We could have gone straight to this website to download OSMC, but this allowed you to see what other options are available and which is the easiest place to find them. OSMC gives a few different ways to install the OS onto different types of computers. If you want to use their automated way of installing OSMC to an SD card for the Raspberry Pi, you are welcome to do so; just follow their instructions for the operation system that you are using on your main computer. For learning purposes, I am going to explain the method of downloading a disk image and doing it ourselves, as this is how most operating systems are installed to the Raspberry Pi. Under the heading named Get Started, where you can choose the automated installation methods, there is a line just under it that allows you to download disk images. This is what we are going to do. Click on that link. Now, we are presented with choices, namely Raspberry Pi 1 and Raspberry Pi 2. The Raspberry Pi 1 refers to any of the single-core Raspberry Pi devices while the Raspberry Pi 2 refers to the newest Pi with a quad-core processor and more RAM. Click on the link under whichever heading applies for the type of Pi that you will be using and select the newest release option that is available. Verifying the download While OSMC is downloading, let's take a minute to understand what the MD5 Checksum is. An MD5 Checksum is used to verify a file's integrity. The number that you see beside the download is the Checksum that was created when the file that you are downloading was created. After the image has finished downloading, we will check the MD5 Checksum of the file on your computer as well. These numbers should be identical. If they are not, it indicates that the image is corrupt and you will need to download it again. From a security standpoint, a checksum can also be used to ensure that data hasn't been tampered with in the time span between when it was created and when it was given to you. This could indicate malicious software or a data breech. Now that OSMC has been downloaded, we can verify its integrity. In Linux, this is easy. Open a terminal and navigate to the Downloads folder or wherever you downloaded the file. Now type in the following command: [md5sum name-of-file] The output that this gives should match the MD5 Checksum that was beside the file that you clicked on to download. If it doesn't, delete the file and try doing this again. To verify the file integrity using Windows, you will need to install a program that can do this. Search online for MD5 checksum Windows, and you will see that Microsoft has a program that can be downloaded from their website. Once you download and install it, it will work in a fashion that's similar to the Linux method, where you use the Windows command prompt. It comes with a readme file to explain how to use it. If you are unable to find a program to verify the checksum, do not worry. This step isn't required, but it helps you troubleshoot whether the Raspberry Pi will not boot after you install the OS onto the SD card. Installing OSMC - for Windows users For Windows, you need to install two more applications to successfully write OSMC to an SD card. Because the OSMC file that you downloaded is compressed using gzip, you need a program that can unzip it. The recommended program for all of your compression needs in Windows is WinRAR. It is free and can be found at www.filehippo.com along with the next program that you will need. After you unzip the OSMC file, you will need a program that can write (burn) it to your SD card. There are many options to choose from, and these options can be found under the CD/DVD option of Categories on the homepage. ImgBurn and DeepBurner appear to be the most popular image burning software at the time of writing this article. Preparing everything Ensure that you have the appropriate type of SD card for the Raspberry Pi that you own. The original Raspberry Pi Model A and B use full-size SD cards. Thus, if you purchased a miniSD by mistake, do not worry. The miniSD probably came with an adapter that turns it into a full-size SD. If it did not, they are easy to acquire. You will need to insert your SD card into your computer so that you can write the operating system on it. If your computer has an in-built SD card reader, then that is ideal. If it does not, there are card readers available that plug in through your USB port and which can accomplish this goal as well. Once you have inserted your SD card into your computer using either method, ensure that you have taken all the information off the card that you want to keep. Anything that's currently on the card will be erased in the following steps! Install WinRAR and your image burning program if you have not already done so. When it is , you should be able to right-click on the OSMC file that you downloaded and select the option to uncompress or extract the files in a gzip file. Burn It! Now that we have an OSMC file that ends with .img, we can open the image burning program. Each program works differently, but you want to set the destination (where the image will be burned) as your SD card and the source (or input file) as the OSMC image. Once these settings are correct, click on BurnISO to begin burning the image. Now that this is done, congratulations! Installing OSMC - for Linux users As you have seen several times already, Linux comes with nearly everything that you need already installed. The software used to install the operating system to the SD card is no different. Ensure that you have the appropriate type of the SD card for the Raspberry Pi that you own. The original Raspberry Pi Model A and B use full-size SD cards. Therefore, if you purchased a miniSD by mistake, do not worry. The miniSD probably came with an adapter that turns it into a full-size SD. If it did not, they are easy to acquire. Preparing the SD card You will need to insert your SD card into your computer so that you can write the operating system on it. If your computer has an in-built SD card reader, then that is ideal. If it does not, there are card readers available that plug in through your USB port that can accomplish this goal as well. Once you have inserted your SD card into your computer using either method, ensure that you have taken all information that you want to keep off the card. Anything that's currently on the card will be erased in the next step! If the SD card was already formatted with a filesystem, it probably automounted itself somewhere so that you can access it. We need to unmount it so that the system is not actually using it, but it is still inserted into the computer. To do this, type the following command into your command line: lsblk This command lists the block devices that are currently on your computer. In other words, it shows the storage devices and the partitions on them. Sda is most likely your hard drive; you can tell by the size of the device in the right columns. Sda1 and sda2 are the partitions on the sda device. Look for your device by its size. If you have a 4 GB SD card, then you will see something like this: NAME                    MAJ:MIN RM   SIZE    RO TYPE  MOUNTPOINT sda                          8:0          0     238.5G  0   disk  ├─sda1                      8:1          0     476M  0   part    /boot └─sda2                       8:2          0      186.3G  0   part    / sdb                          8:16        1      3.8G  0   disk  ├─sdb1                       8:17        1      2.5G   0  part   └─sdb2                       8:18        1      1.3G   0  part    /run/media/username/mountpoint In this case, my SD card is sdb and the second partition is mounted. To unmount this, we are going to issue the following command in the terminal again: sudo umount /dev/sdb* It will then ask you for your sudo (administrator) password and then unmount all the partitions for the sdb device. In this case, you could have replaced the sdb* with the partition number (sdb2) to be more specific if you only wanted to unmount one partition and not the entire device. In this example, we will erase everything on the device so that we unmount everything. Now, we can write the operating system to the SD card. Burn It! The process of installing an OSMC to the SD card is called burning an image. The process of burning an image is done with a program called dd, and it is done via the terminal. dd is a very useful tool that's used to copy disks and partitions to other disks or partitions or to images and vice versa. In this instance, we will take an image and copy it to a disk. In the terminal, navigate to the directory where you downloaded OSMC. The file that you downloaded is compressed using gzip. Before we can burn it to the disk, we need to unzip it. To do so, type in the following command: gunzip name-of-file.img.gz This will leave you with a new file that has the same name but with the .gz file no longer at the end. This file is also much bigger than the gzipped version. This .img (image) file is what we will burn to the SD card. In the previous step, we found out what device our SD card was listed under (it was sdb in the preceding example) and unmounted it. Now, we are going to use the following command to burn the image: sudo dd if=name-of-file.img of=/dev/sdb  (change /dev/sdb to whatever it is on your computer)  And that's it! This will take several minutes to complete and the terminal will look like it froze, but this is because it is working. When it is done, the prompt will come back and you can remove the SD card: Summary If your computer already uses Linux, these steps will be a little bit faster because you already have the needed software. For Windows users, hunting for the right software and installing it will take some time. Just have patience and know that the exciting part is just around the corner. Now that we have downloaded OSMC, verified the download, prepared the SD card, and burned OSMC on it, the hardest part is over. Resources for Article:   Further resources on this subject: Raspberry Pi LED Blueprints [article] Raspberry Pi and 1-Wire [article] Raspberry Pi Gaming Operating Systems [article]
Read more
  • 0
  • 0
  • 14553
article-image-introduction-openvpn
Packt
20 Aug 2015
13 min read
Save for later

Introduction to OpenVPN

Packt
20 Aug 2015
13 min read
In this article by Eric Crist and Jan Just Keijser, the authors of the book Mastering OpenVPN, you will cover: Types of VPNs Comparison of VPNs (For more resources related to this topic, see here.) What is a VPN? Put simply, a VPN allows an administrator to create a local network between multiple computers on varying network segments. In some instances, those machines can be on the same LAN, they can be distant from each other across the vast Internet, or they can even be connected across a multitude of connection media such as wireless uplinks, satellite, dial-up-networking, and so on. The P in VPN comes from the added protection to make that virtual network private. Network traffic that is flowing over a VPN is often referred to as inside the (VPN) tunnel, compared to all other traffic that is outside the tunnel. In the following figure, network traffic is shown as it traditionally traverses across multiple network segments and the general Internet. Here, this traffic is relatively open to inspection and analysis. Though protected protocols such as HTTPS and SSH are less vulnerable, they are still identifiable: if an attacker is snooping network traffic, he can still see what type of connection is made from which computer to which server. When a VPN is used, the traffic inside the tunnel is no longer identifiable. Traffic within a VPN can be anything you would send over a local or wide-area network: web traffic, e-mail, text, graphics, and so on. Examples of some applications include the following: Automated Teller Machines: ATMs may use a VPN to connect more securely to banking systems. Open / Free Wi-Fi: With the proliferation of free or open wireless networks, everyday users can utilize a VPN to protect the entirety of their Internet browsing. Corporate networks: Corporations and other organizations may use a VPN to connect multiple office locations or even entire data centers. GeoIP / Location-based services: Some websites serve data based on geographic location using GeoIP databases and other records. A VPN can allow you to "bounce" through another machine in a location closer to the content you really want. Internet video services such as Hulu, YouTube, and Netflix are common examples. Bypassing censorship / Political freedom: Some regimes, such as North Korea or China have extraordinarily restrictive censorship rules. The "Great Firewall of China" is one extreme example. Lock-downs of Internet access during political uprisings such as "Arab Spring" attempt to contain and control reports outside the conflict. VPNs can aid in getting outside those restrictive rules to the greater Internet. Here is an example of traffic within a VPN. While the VPN itself is routed across the Internet like in the preceding figure, devices along the network path only see VPN traffic; those devices are completely unaware of what in being transmitted inside the private tunnel. Protected protocols, such as HTTPS and SSH, will still be protected inside the tunnel from other VPN users, but will additionally unidentifiable from outside the tunnel. A VPN not only encrypts the traffic within, it hides and protects individual data streams from those outside the tunnel. It should be noted that the preceding figure shows both the strength and one of the greatest threats of VPN technologies. The VPN tunnel is dug through routers and firewalls on both sides. Thus, all network traffic that is flowing via the VPN tunnel is bypassing the regular network defenses, unless special measures are taken to police the VPN traffic. Most VPN implementations utilize some form of encryption and, additionally, authentication. Encryption of the VPN ensures that other parties that may be monitoring traffic between systems cannot decode and further analyze otherwise sensitive data. Authentication has two components, each in a different context. First, there is user or system authentication that ensures those connecting to the service are authorized. This type of authentication may be in the form of per-user certificates, or a username/password combination. Further, rules specific to a given user can be negotiated such as specific routes, firewall rules, or other scripts and utilities. Typically, these are unique to a single instance, though even that can be configurable (when OpenVPN is used, see --duplicate-cn). The second component of authentication is added protection to the communication stream. In this case, a method of signing each packet sent is established. Each system verifies the VPN packets it receives are properly signed before decrypting the payload. By authenticating packets that are already encrypted, a system can save processing time by not even decrypting packets that do not meet the authentication rules. In the end, this prevents a very real potential Denial of Service (DoS) attack as well as thwarts Man in the Middle (MITM) attacks, assuming the signing keys are kept secure! Types of VPNs There are many VPN products available on the market, both commercial and open source. Almost all of these VPN products can be separated into the four categories: PPTP-protocol based VPNs IPSec-protocol based VPNs SSL-based VPNs OpenVPN Some people argue that OpenVPN is also an SSL-based VPN, as it uses an SSL or TLS-like protocol to establish a secure connection. However, we have created a separate category for OpenVPN, as it is different from almost every other SSL-based VPN solution. We will now go into more detail about each of the four types of VPNs: PPTP One of the oldest VPN protocols is the Point-to-Point Tunneling Protocol (PPTP) developed by Microsoft and Ascend in 1999. It is officially registered as RFC2637 (see https://www.ietf.org/rfc/rfc2637.txt for the full standard). The PPTP client has been included in Windows ever since 1995 and is still included in most operating systems. Nowadays, the PPTP protocol is considered fundamentally insecure, as the strength of the security of the connection is directly related to the strength of the authentication mechanism chosen (example, the password). Thus, an insecure password leads to an insecure VPN connection. Most PPTP setups use the MS-CHAPv2 protocol for encrypting passwords, and it is this protocol which is fundamentally broken. The security of the PPTP protocol, including the Microsoft MS-CHAPv2 extensions, has been discussed in the article available at https://www.schneier.com/paper-pptpv2.html. It is also possible to use X.509 certificates for securing a PPTP connection, which does lead to a fairly secure connection. However, not all PPTP clients support EAP-TLS, which is needed to allow the use of X.509 certificates. PPTP uses two channels, a control channel for setting up the connection and another channel for data transport. The control channel is initiated over TCP port 1723. The data channel uses the General Routing Encapsulation (GRE) protocol, which is IP protocol 47. For comparison, "regular" TCP/IP traffic is done using IP protocol 6 (TCP) and 17 (UDP). PPTP clients are available on almost all operating systems, ranging from Windows to Linux and UNIX derivatives to iOS and Android devices. IPSec The IPSec standard is the official IEEE/IETF standard for IP security. It is officially registered as RFC2411 (see https://www.ietf.org/rfc/rfc2411.txt for the full standard). IPSec is also built into the IPv6 standard. IPSec operates at level 2 and 3 of the OSI model of the network stack. It introduces the concept of security policies, which makes it extremely flexible and powerful, but also notoriously hard to configure and troubleshoot. Security policies allow an administrator to encrypt traffic between two endpoints based on many parameters, such as the source and destination IP address as well as the source and destination TCP or UDP ports. IPSec can be configured to use pre-shared keys or X.509 certificates to secure the VPN connection. Additionally, it uses either X.509 certificates, one-time passwords, or username/password protocols to authenticate the VPN connection. There are two modes of operation in IPSec: tunneling mode and transport mode. Transport mode is used most often in combination with the Level 2 Tunneling Protocol (L2TP). This L2TP protocol performs the user authentication as described in the preceding section. The IPSec clients built into most operating systems usually perform IPSec+L2TP, although it is also possible to set up an IPSec-only connection. The IPSec VPN client built into Microsoft Windows uses IPSec+L2TP by default, but it is possible to disable or bypass it. However, this involves cryptic commands and security policy changes. Like PPTP, IPSec also uses two channels: a control channel for setting up the connection and one for data transport. The control channel is initiated over UDP port 500 or 4500. The data channel uses the Encapsulated Security Payload (ESP) protocol, which is IP protocol 50. For comparison, "regular" TCP/IP traffic is done using IP protocol 6 (TCP) and 17 (UDP). The integrity of IPSec packets is ensured using Hash-based Message Authentication Code (HMAC), which is the same method that OpenVPN uses. One of the main disadvantages of IPSec is that many vendors have implemented extensions to the standard, which makes it hard (if not impossible) to connect two IPSec endpoints from different vendors. IPSec software is included in almost all operating systems, as well as firewall, router, and switch firmware. SSL-based VPNs The most commonly used VPNs nowadays are SSL-based VPNs, which are based on the SSL/TLS protocol. SSL-based VPNs are often called client-less VPNs or web-based VPNs, although there are some vendors that provide separate client software, such as Cisco AnyConnect and Microsoft SSTP. Most SSL-based VPNs use the same network protocol as is used for secure website (HTTPS), while OpenVPN uses a custom format for encrypting and signing data traffic. This is the main reason why OpenVPN is listed as a separate VPN category. There is no well-defined standard for SSL-based VPNs, but most use the SSL/TLS protocol to set up and secure the connection. The connection is secured using X.509 certificates in most cases, with one-time password or username/password protocols for authenticating the connection. SSL-based VPNs are very similar to connections used to secure websites (HTTPS) and the same protocol and channel (TCP and port 443) is often used. Even though SSL-based VPNs are often called web-based or client-less, there are quite a few vendors that use a browser plugin or ActiveX control to "enhance" the VPN connection. This makes the VPN noninteroperable with unsupported browsers or operating systems. OpenVPN OpenVPN is often called an SSL-based VPN, as it uses the SSL/TLS protocol to secure the connection. However, OpenVPN also uses HMAC in combination with a digest (or hashing) algorithm for ensuring the integrity of the packets delivered. It can be configured to use pre-shared keys as well as X.509 certificates. These features are not typically offered by other SSL-based VPNs. Furthermore, OpenVPN uses a virtual network adapter (a tun or tap device) as an interface between the user-level OpenVPN software and the operating system. In general, any operating system that has support for a tun/tap device can run OpenVPN. This currently includes Linux, Free/Open/NetBSD, Solaris, AIX, Windows, Mac OS as well as iOS/Android devices. For all these platforms, client software needs to be installed, which sets OpenVPN apart from client-less or web-based VPNs. The OpenVPN protocol is not defined in an RFC standard, but the protocol is publicly available as OpenVPN is open source software. The fact that it is open source actually makes OpenVPN more secure than closed-source VPNs, as the code is continually inspected by different people. Also, there is very little chance of secret backdoors being built into OpenVPN. OpenVPN has the notion of a control channel and a data channel, both of which are encrypted and secured differently. However, all traffic passes over a single UDP or TCP connection. The control channel is encrypted and secured using SSL/TLS, the data channel is encrypted using a custom encryption protocol. The default protocol and port for OpenVPN is UDP and port 1194. Before IANA granted OpenVPN an official port assignment, older clients (2.0-beta16 and older) defaulted to port 5000. Comparison of VPNs Each of the different VPN technologies has its own characteristics, advantages, and disadvantages. Even though this article is about OpenVPN, there are use-cases where, for example, an IPSec-based VPN is more suitable, depending on the requirement of the users. Advantages and disadvantages of PPTP The main advantage of PPTP-based VPNs is that the VPN client software is built into most operating systems. Also, the startup time for configuring and initializing a PPTP VPN connection is quite short. Disadvantages of PPTP-based VPNs are the lack of security and the lack of configuration options on both the client and server side. Furthermore, the EAP-TLS extension that enables the use of X.509 certificates is fully supported only on Microsoft Windows, although a patch exists for the open source pppd package to enable EAP-TLS support. The pppd package is included in almost every Linux distribution. Also, if one must resort to using EAP-TLS, then the ease of setting up a PPTP VPN is greatly diminished. This is because EAP-TLS requires setting up a public key infrastructure, just like IPSec and OpenVPN. Another major disadvantage of PPTP is the use of the GRE protocol, which does not integrate well with NAT'ing devices. Advantages and disadvantages of IPSec Advantages of the IPSec protocol are its strong security, good support by different vendors and platforms, including xDSL and Wi-Fi routers, as well as the ability to use fine-grained security policies to control the flow of traffic. The downsides of IPSec are that it is notoriously difficult to configure and troubleshoot, different IPSec implementations from different vendors do not play nicely together, and IPSec does not integrate well with NAT'ted networks. Most notably, it is not recommended and sometimes not even possible to run an IPSec server that is on a NAT'ted network. Advantages and disadvantages of SSL-based VPNs SSL-based VPNs or web-based VPNs have the advantage that there is no or very little client software involved. This makes installation and initialization on the client side very easy. The disadvantage of a web-based VPN is that it is often not a full-blown VPN and allows access to a single server or set of servers. Also, it is harder to share local data with the remote site or server. Advantages and disadvantages of OpenVPN Advantages of OpenVPN are its ease of deployment, its configurability, and the ability to deploy OpenVPN in restricted networks, including NAT'ted networks. Also, OpenVPN includes security features that are as strong as IPSec-based solutions, including hardware token security and support for different user authentication mechanisms. Disadvantages of OpenVPN are its current lack of scalability and its dependence on the installation of client-side software. Another disadvantage is the lack of a GUI for configuration and management. Especially, the tap interface driver for Microsoft Windows has often caused deployment issues when a new version of Windows is released. Summary In this article, we started out by explaining what a VPN is. We then discussed some examples of different types of VPN protocols, including PPTP, IPSec, and OpenVPN. At the end, we compared the various VPNs. Resources for Article: Further resources on this subject: Untangle VPN Services [article] Setting up VPNaaS in OpenStack [article] Securing OpenStack Networking [article]
Read more
  • 0
  • 0
  • 14545

article-image-building-cloud-spy-camera-and-creating-gps-tracker
Packt
30 Oct 2015
4 min read
Save for later

Building a Cloud Spy Camera and Creating a GPS Tracker

Packt
30 Oct 2015
4 min read
In this article by Marco Schwartz author of the book Arduino for Secret Agents, we will build a GPS location tracker and use a spy camera for live streaming. Building a GPS location tracker It's now time to build a real GPS location tracker. For this project, we'll get the location just as before, using the GPS if available, and the GPRS location otherwise. However here, we are going to use the GPRS capabilities of the shield to send the latitude and longitude data to Dweet.io, which is a service we already used before. Then, we'll plot this data in Google Maps, allowing you to follow your device in real-time from anywhere in the world. (For more resources related to this topic, see here.) You need to define a name for the Thing that will contain the GPS location data: String dweetThing = "mysecretgpstracker"; Then, after getting the current location, we prepare the data to be sent to Dweet.io: uint16_t statuscode; int16_t length; char url[80]; String request = "www.dweet.io/dweet/for/" + dweetThing + "?latitude=" + latitude + "andlongitude=" + longitude; request.toCharArray(url, request.length()); After that, we actually send the data to Dweet.io: if (!fona.HTTP_GET_start(url, andstatuscode, (uint16_t *)andlength)) { Serial.println("Failed!"); } while (length > 0) { while (fona.available()) { char c = fona.read(); // Serial.write is too slow, we'll write directly to Serial register! #if defined(__AVR_ATmega328P__) || defined(__AVR_ATmega168__) loop_until_bit_is_set(UCSR0A, UDRE0); /* Wait until data register empty. */ UDR0 = c; #else Serial.write(c); #endif length--; } } fona.HTTP_GET_end(); Now, before testing the project, we are going to prepare our dashboard that will host the Google Maps widget. We are going to use Freeboard.io for this purpose. If you don't have an account yet, go to http://freeboard.io/. Create a new dashboard, and also a new datasource. Insert the name of your Thing inside the THING NAME field: Then, create a new Pane with a Google Maps widget. Link this widget to the latitude and longitude of your Location datasource: It's now time to test the project. Make sure to grab all the code, for example from the GitHub repository of the book. Also don't forget to modify the Thing name, as well as your GPRS settings. Then, upload the sketch to the board, and open the Serial monitor. This is what you should see: The most important line is the last one, which confirms that data has been sent to Dweet.io and has been stored there. Now, simply go back to the dashboard you just created: you can now see that the location on the map has been updated: Note that this map is also updated in real-time, as new measurements arrive from the board. You can also modify the delay between two updates of the position of the tracker, by changing the delay() function in the sketch. Congratulations, you just built your own GPS tracking device! Live streaming from the spy camera We are now going to use the camera to stream live video in a web browser. This stream will be accessible from any device connected to the same WiFi network as the Yun. To start with this project, log into your Yun using the following command (by changing the name of the board with the name of your Yun): ssh root@arduinoyun.local Then, type: mjpg_streamer -i "input_uvc.so -d /dev/video0 -r 640x480 -f 25" -o "output_http.so -p 8080 -w /www/webcam" & This will start the streaming from your Yun. You can now simply go the URL of your Yun, and add ':8080' at the end. For example, http://arduinoyun.local:8080. You should arrive on the streaming interface: You can now stream this video live to your mobile phone or any other device within the same network. It's the perfect project to spy in a room while you are sitting outside for example. Summary In this article, we built a device that allows us to track the GPS location of any object it is attached to and we built a spy camera project that can send pictures in the cloud whenever motion is detected Resources for Article: Further resources on this subject: Getting Started with Arduino [article] Arduino Development [article] Prototyping Arduino Projects using Python [article]
Read more
  • 0
  • 0
  • 14544
Modal Close icon
Modal Close icon