Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Save more on your purchases! discount-offer-chevron-icon
Savings automatically calculated. No voucher code required.
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Newsletter Hub
Free Learning
Arrow right icon
timer SALE ENDS IN
0 Days
:
00 Hours
:
00 Minutes
:
00 Seconds

How-To Tutorials

7019 Articles
article-image-understanding-php-basics
Packt
17 Feb 2016
27 min read
Save for later

Understanding PHP basics

Packt
17 Feb 2016
27 min read
In this article by Antonio Lopez Zapata, the author of the book Learning PHP 7, you need to understand not only the syntax of the language, but also its grammatical rules, that is, when and why to use each element of the language. Luckily, for you, some languages come from the same root. For example, Spanish and French are romance languages as they both evolved from spoken Latin; this means that these two languages share a lot of rules, and learning Spanish if you already know French is much easier. (For more resources related to this topic, see here.) Programming languages are quite the same. If you already know another programming language, it will be very easy for you to go through this chapter. If it is your first time though, you will need to understand from scratch all the grammatical rules, so it might take some more time. But fear not! We are here to help you in this endeavor. In this chapter, you will learn about these topics: PHP in web applications Control structures Functions PHP in web applications Even though the main purpose of this chapter is to show you the basics of PHP, doing so in a reference-manual way is not interesting enough. If we were to copy paste what the official documentation says, you might as well go there and read it by yourself. Instead, let's not forget the main purpose of this book and your main goal—to write web applications with PHP. We will show you how can you apply everything you are learning as soon as possible, before you get too bored. In order to do that, we will go through the journey of building an online bookstore. At the very beginning, you might not see the usefulness of it, but that is just because we still haven't seen all that PHP can do. Getting information from the user Let's start by building a home page. In this page, we are going to figure out whether the user is looking for a book or just browsing. How do we find this out? The easiest way right now is to inspect the URL that the user used to access our application and extract some information from there. Save this content as your index.php file: <?php $looking = isset($_GET['title']) || isset($_GET['author']); ?> <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <title>Bookstore</title> </head> <body> <p>You lookin'? <?php echo (int) $looking; ?></p> <p>The book you are looking for is</p> <ul> <li><b>Title</b>: <?php echo $_GET['title']; ?></li> <li><b>Author</b>: <?php echo $_GET['author']; ?></li> </ul> </body> </html> And now, access http://localhost:8000/?author=Harper Lee&title=To Kill a Mockingbird. You will see that the page is printing some of the information that you passed on to the URL. For each request, PHP stores in an array—called $_GET- all the parameters that are coming from the query string. Each key of the array is the name of the parameter, and its associated value is the value of the parameter. So, $_GET contains two entries: $_GET['author'] contains Harper Lee and $_GET['title'] contains To Kill a Mockingbird. On the first highlighted line, we are assigning a Boolean value to the $looking variable. If either $_GET['title'] or $_GET['author'] exists, this variable will be true; otherwise, false. Just after that, we close the PHP tag and then we start printing some HTML, but as you can see, we are actually mixing HTML with PHP code. Another interesting line here is the second highlighted line. We are printing the content of $looking, but before that, we cast the value. Casting means forcing PHP to transform a type of value to another one. Casting a Boolean to an integer means that the resultant value will be 1 if the Boolean is true or 0 if the Boolean is false. As $looking is true since $_GET contains valid keys, the page shows 1. If we try to access the same page without sending any information as in http://localhost:8000, the browser will say "Are you looking for a book? 0". Depending on the settings of your PHP configuration, you will see two notice messages complaining that you are trying to access the keys of the array that do not exist. Casting versus type juggling We already knew that when PHP needs a specific type of variable, it will try to transform it, which is called type juggling. But PHP is quite flexible, so sometimes, you have to be the one specifying the type that you need. When printing something with echo, PHP tries to transform everything it gets into strings. Since the string version of the false Boolean is an empty string, this would not be useful for our application. Casting the Boolean to an integer first assures that we will see a value, even if it is just "0". HTML forms HTML forms are one of the most popular ways to collect information from users. They consist a series of fields called inputs in the HTML world and a final submit button. In HTML, the form tag contains two attributes: the action points, where the form will be submitted and method that specifies which HTTP method the form will use—GET or POST. Let's see how it works. Save the following content as login.html and go to http://localhost:8000/login.html: <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <title>Bookstore - Login</title> </head> <body> <p>Enter your details to login:</p> <form action="authenticate.php" method="post"> <label>Username</label> <input type="text" name="username" /> <label>Password</label> <input type="password" name="password" /> <input type="submit" value="Login"/> </form> </body> </html> This form contains two fields, one for the username and one for the password. You can see that they are identified by the name attribute. If you try to submit this form, the browser will show you a Page Not Found message, as it is trying to access http://localhost:8000/authenticate.phpand the web server cannot find it. Let's create it then: <?php $submitted = !empty($_POST); ?> <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <title>Bookstore</title> </head> <body> <p>Form submitted? <?php echo (int) $submitted; ?></p> <p>Your login info is</p> <ul> <li><b>username</b>: <?php echo $_POST['username']; ?></li> <li><b>password</b>: <?php echo $_POST['password']; ?></li> </ul> </body> </html> As with $_GET, $_POST is an array that contains the parameters received by POST. In this piece of code, we are first asking whether that array is not empty—note the ! operator. Afterwards, we just display the information received, just as in index.php. Note that the keys of the $_POST array are the values for the name argument of each input field. Control structures So far, our files have been executed line by line. Due to that, we are getting some notices on some scenarios, such as when the array does not contain what we are looking for. Would it not be nice if we could choose which lines to execute? Control structures to the rescue! A control structure is like a traffic diversion sign. It directs the execution flow depending on some predefined conditions. There are different control structures, but we can categorize them in conditionals and loops. A conditional allows us to choose whether to execute a statement or not. A loop will execute a statement as many times as you need. Let's take a look at each one of them. Conditionals A conditional evaluates a Boolean expression, that is, something that returns a value. If the expression is true, it will execute everything inside its block of code. A block of code is a group of statements enclosed by {}. Let's see how it works: <?php echo "Before the conditional."; if (4 > 3) { echo "Inside the conditional."; } if (3 > 4) { echo "This will not be printed."; } echo "After the conditional."; In this piece of code, we are using two conditionals. A conditional is defined by the keyword if followed by a Boolean expression in parentheses and by a block of code. If the expression is true, it will execute the block; otherwise, it will skip it. You can increase the power of conditionals by adding the keyword else. This tells PHP to execute a block of code if the previous conditions were not satisfied. Let's see an example: if (2 > 3) { echo "Inside the conditional."; } else { echo "Inside the else."; } This will execute the code inside else as the condition of if was not satisfied. Finally, you can also add an elseif keyword followed by another condition and block of code to continue asking PHP for more conditions. You can add as many elseif as you need after if. If you add else, it has to be the last one of the chain of conditions. Also keep in mind that as soon as PHP finds a condition that resolves to true, it will stop evaluating the rest of the conditions: <?php if (4 > 5) { echo "Not printed"; } elseif (4 > 4) { echo "Not printed"; } elseif (4 == 4) { echo "Printed."; } elseif (4 > 2) { echo "Not evaluated."; } else { echo "Not evaluated."; } if (4 == 4) { echo "Printed"; } In this last example, the first condition that evaluates to true is the one that is highlighted. After that, PHP does not evaluate any more conditions until a new if starts. With this knowledge, let's try to clean up a bit of our application, executing statements only when needed. Copy this code to your index.php file: <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <title>Bookstore</title> </head> <body> <p> <?php if (isset($_COOKIE[username'])) { echo "You are " . $_COOKIE['username']; } else { echo "You are not authenticated."; } ?> </p> <?php if (isset($_GET['title']) && isset($_GET['author'])) { ?> <p>The book you are looking for is</p> <ul> <li><b>Title</b>: <?php echo $_GET['title']; ?></li> <li><b>Author</b>: <?php echo $_GET['author']; ?></li> </ul> <?php } else { ?> <p>You are not looking for a book?</p> <?php } ?> </body> </html> In this new code, we are mixing conditionals and HTML code in two different ways. The first one opens a PHP tag and adds an if-else clause that will print whether we are authenticated or not with echo. No HTML is merged within the conditionals, which makes it clear. The second option—the second highlighted block—shows an uglier solution, but this is sometimes necessary. When you have to print a lot of HTML code, echo is not that handy, and it is better to close the PHP tag; print all the HTML you need and then open the tag again. You can do that even inside the code block of an if clause, as you can see in the code. Mixing PHP and HTML If you feel like the last file we edited looks rather ugly, you are right. Mixing PHP and HTML is confusing, and you have to avoid it by all means. Let's edit our authenticate.php file too, as it is trying to access $_POST entries that might not be there. The new content of the file would be as follows: <?php $submitted = isset($_POST['username']) && isset($_POST['password']); if ($submitted) { setcookie('username', $_POST['username']); } ?> <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <title>Bookstore</title> </head> <body> <?php if ($submitted): ?> <p>Your login info is</p> <ul> <li><b>username</b>: <?php echo $_POST['username']; ?></li> <li><b>password</b>: <?php echo $_POST['password']; ?></li> </ul> <?php else: ?> <p>You did not submitted anything.</p> <?php endif; ?> </body> </html> This code also contains conditionals, which we already know. We are setting a variable to know whether we've submitted a login or not and to set the cookies if we have. However, the highlighted lines show you a new way of including conditionals with HTML. This way, tries to be more readable when working with HTML code, avoiding the use of {} and instead using : and endif. Both syntaxes are correct, and you should use the one that you consider more readable in each case. Switch-case Another control structure similar to if-else is switch-case. This structure evaluates only one expression and executes the block depending on its value. Let's see an example: <?php switch ($title) { case 'Harry Potter': echo "Nice story, a bit too long."; break; case 'Lord of the Rings': echo "A classic!"; break; default: echo "Dunno that one."; break; } The switch case takes an expression; in this case, a variable. It then defines a series of cases. When the case matches the current value of the expression, PHP executes the code inside it. As soon as PHP finds break, it will exit switch-case. In case none of the cases are suitable for the expression, if there is a default case         , PHP will execute it, but this is optional. You also need to know that breaks are mandatory if you want to exit switch-case. If you do not specify any, PHP will keep on executing statements, even if it encounters a new case. Let's see a similar example but without breaks: <?php $title = 'Twilight'; switch ($title) { case 'Harry Potter': echo "Nice story, a bit too long."; case 'Twilight': echo 'Uh...'; case 'Lord of the Rings': echo "A classic!"; default: echo "Dunno that one."; } If you test this code in your browser, you will see that it is printing "Uh...A classic!Dunno that one.". PHP found that the second case is valid so it executes its content. But as there are no breaks, it keeps on executing until the end. This might be the desired behavior sometimes, but not usually, so we need to be careful when using it! Loops Loops are control structures that allow you to execute certain statements several times—as many times as you need. You might use them on several different scenarios, but the most common one is when interacting with arrays. For example, imagine you have an array with elements but you do not know what is in it. You want to print all its elements so you loop through all of them. There are four types of loops. Each of them has their own use cases, but in general, you can transform one type of loop into another. Let's see them closely While While is the simplest of the loops. It executes a block of code until the expression to evaluate returns false. Let's see one example: <?php $i = 1; while ($i < 4) { echo $i . " "; $i++; } Here, we are defining a variable with the value 1. Then, we have a while clause in which the expression to evaluate is $i < 4. This loop will execute the content of the block of code until that expression is false. As you can see, inside the loop we are incrementing the value of $i by 1 each time, so after 4 iterations, the loop will end. Check out the output of that script, and you will see "0 1 2 3". The last value printed is 3, so by that time, $i was 3. After that, we increased its value to 4, so when the while was evaluating whether $i < 4, the result was false. Whiles and infinite loops One of the most common problems with while loops is creating an infinite loop. If you do not add any code inside while, which updates any of the variables considered in the while expression so it can be false at some point, PHP will never exit the loop! For This is the most complex of the four loops. For defines an initialization expression, an exit condition, and the end of the iteration expression. When PHP first encounters the loop, it executes what is defined as the initialization expression. Then, it evaluates the exit condition, and if it resolves to true, it enters the loop. After executing everything inside the loop, it executes the end of the iteration expression. Once this is done, it will evaluate the end condition again, going through the loop code and the end of iteration expression until it evaluates to false. As always, an example will help clarify this: <?php for ($i = 1; $i < 10; $i++) { echo $i . " "; } The initialization expression is $i = 1 and is executed only the first time. The exit condition is $i < 10, and it is evaluated at the beginning of each iteration. The end of the iteration expression is $i++, which is executed at the end of each iteration. This example prints numbers from 1 to 9. Another more common usage of the for loop is with arrays: <?php $names = ['Harry', 'Ron', 'Hermione']; for ($i = 0; $i < count($names); $i++) { echo $names[$i] . " "; } In this example, we have an array of names. As it is defined as a list, its keys will be 0, 1, and 2. The loop initializes the $i variable to 0, and it will iterate until the value of $i is not less than the amount of elements in the array 3 The first iteration $i is 0, the second will be 1, and the third one will be 2. When $i is 3, it will not enter the loop as the exit condition evaluates to false. On each iteration, we are printing the content of the $i position of the array; hence, the result of this code will be all three names in the array. We careful with exit conditions It is very common to set an exit condition. This is not exactly what we need, especially with arrays. Remember that arrays start with 0 if they are a list, so an array of 3 elements will have entries 0, 1, and 2. Defining the exit condition as $i <= count($array) will cause an error on your code, as when $i is 3, it also satisfies the exit condition and will try to access to the key 3, which does not exist. Foreach The last, but not least, type of loop is foreach. This loop is exclusive for arrays, and it allows you to iterate an array entirely, even if you do not know its keys. There are two options for the syntax, as you can see in these examples: <?php $names = ['Harry', 'Ron', 'Hermione']; foreach ($names as $name) { echo $name . " "; } foreach ($names as $key => $name) { echo $key . " -> " . $name . " "; } The foreach loop accepts an array; in this case, $names. It specifies a variable, which will contain the value of the entry of the array. You can see that we do not need to specify any end condition, as PHP will know when the array has been iterated. Optionally, you can specify a variable that will contain the key of each iteration, as in the second loop. Foreach loops are also useful with maps, where the keys are not necessarily numeric. The order in which PHP will iterate the array will be the same order in which you used to insert the content in the array. Let's use some loops in our application. We want to show the available books in our home page. We have the list of books in an array, so we will have to iterate all of them with a foreach loop, printing some information from each one. Append the following code to the body tag in index.php: <?php endif; $books = [ [ 'title' => 'To Kill A Mockingbird', 'author' => 'Harper Lee', 'available' => true, 'pages' => 336, 'isbn' => 9780061120084 ], [ 'title' => '1984', 'author' => 'George Orwell', 'available' => true, 'pages' => 267, 'isbn' => 9780547249643 ], [ 'title' => 'One Hundred Years Of Solitude', 'author' => 'Gabriel Garcia Marquez', 'available' => false, 'pages' => 457, 'isbn' => 9785267006323 ], ]; ?> <ul> <?php foreach ($books as $book): ?> <li> <i><?php echo $book['title']; ?></i> - <?php echo $book['author']; ?> <?php if (!$book['available']): ?> <b>Not available</b> <?php endif; ?> </li> <?php endforeach; ?> </ul> The highlighted code shows a foreach loop using the : notation, which is better when mixing it with HTML. It iterates all the $books arrays, and for each book, it will print some information as a HTML list. Also note that we have a conditional inside a loop, which is perfectly fine. Of course, this conditional will be executed for each entry in the array, so you should keep the block of code of your loops as simple as possible. Functions A function is a reusable block of code that, given an input, performs some actions and optionally returns a result. You already know several predefined functions, such as empty, in_array, or var_dump. These functions come with PHP so you do not have to reinvent the wheel, but you can create your own very easily. You can define functions when you identify portions of your application that have to be executed several times or just to encapsulate some functionality. Function declaration Declaring a function means to write it down so that it can be used later. A function has a name, takes arguments, and has a block of code. Optionally, it can define what kind of value is returning. The name of the function has to follow the same rules as variable names; that is, it has to start by a letter or underscore and can contain any letter, number, or underscore. It cannot be a reserved word. Let's see a simple example: function addNumbers($a, $b) { $sum = $a + $b; return $sum; } $result = addNumbers(2, 3); Here, the function's name is addNumbers, and it takes two arguments: $a and $b. The block of code defines a new variable $sum that is the sum of both the arguments and then returns its content with return. In order to use this function, you just need to call it by its name, sending all the required arguments, as shown in the highlighted line. PHP does not support overloaded functions. Overloading refers to the ability of declaring two or more functions with the same name but different arguments. As you can see, you can declare the arguments without knowing what their types are, so PHP would not be able to decide which function to use. Another important thing to note is the variable scope. We are declaring a $sum variable inside the block of code, so once the function ends, the variable will not be accessible any more. This means that the scope of variables declared inside the function is just the function itself. Furthermore, if you had a $sum variable declared outside the function, it would not be affected at all since the function cannot access that variable unless we send it as an argument. Function arguments A function gets information from outside via arguments. You can define any number of arguments—including 0. These arguments need at least a name so that they can be used inside the function, and there cannot be two arguments with the same name. When invoking the function, you need to send the arguments in the same order as we declared them. A function may contain optional arguments; that is, you are not forced to provide a value for those arguments. When declaring the function, you need to provide a default value for those arguments, so in case the user does not provide a value, the function will use the default one: function addNumbers($a, $b, $printResult = false) { $sum = $a + $b; if ($printResult) { echo 'The result is ' . $sum; } return $sum; } $sum1 = addNumbers(1, 2); $sum1 = addNumbers(3, 4, false); $sum1 = addNumbers(5, 6, true); // it will print the result This new function takes two mandatory arguments and an optional one. The default value is false, and is used as a normal value inside the function. The function will print the result of the sum if the user provides true as the third argument, which happens only the third time that the function is invoked. For the first two times, $printResult is set to false. The arguments that the function receives are just copies of the values that the user provided. This means that if you modify these arguments inside the function, it will not affect the original values. This feature is known as sending arguments by a value. Let's see an example: function modify($a) { $a = 3; } $a = 2; modify($a); var_dump($a); // prints 2 We are declaring the $a variable with the value 2, and then we are calling the modify method, sending $a. The modify method modifies the $a argument, setting its value to 3. However, this does not affect the original value of $a, which reminds to 2 as you can see in the var_dump function. If what you want is to actually change the value of the original variable used in the invocation, you need to pass the argument by reference. To do that, you add & in front of the argument when declaring the function: function modify(&$a) { $a = 3; } Now, after invoking the modify function, $a will be always 3. Arguments by value versus by reference PHP allows you to do it, and in fact, some native functions of PHP use arguments by reference—remember the array sorting functions; they did not return the sorted array; instead, they sorted the array provided. But using arguments by reference is a way of confusing developers. Usually, when someone uses a function, they expect a result, and they do not want their provided arguments to be modified. So, try to avoid it; people will be grateful! The return statement You can have as many return statements as you want inside your function, but PHP will exit the function as soon as it finds one. This means that if you have two consecutive return statements, the second one will never be executed. Still, having multiple return statements can be useful if they are inside conditionals. Add this function inside your functions.php file: function loginMessage() { if (isset($_COOKIE['username'])) { return "You are " . $_COOKIE['username']; } else { return "You are not authenticated."; } } Let's use it in your index.php file by replacing the highlighted content—note that to save some tees, I replaced most of the code that was not changed at all with //…: //... <body> <p><?php echo loginMessage(); ?></p> <?php if (isset($_GET['title']) && isset($_GET['author'])): ?> //... Additionally, you can omit the return statement if you do not want the function to return anything. In this case, the function will end once it reaches the end of the block of code. Type hinting and return types With the release of PHP7, the language allows developers to be more specific about what functions get and return. You can—always optionally—specify the type of argument that the function needs, for example, type hinting, and the type of result the function will return—return type. Let's first see an example: <?php declare(strict_types=1); function addNumbers(int $a, int $b, bool $printSum): int { $sum = $a + $b; if ($printSum) { echo 'The sum is ' . $sum; } return $sum; } addNumbers(1, 2, true); addNumbers(1, '2', true); // it fails when strict_types is 1 addNumbers(1, 'something', true); // it always fails This function states that the arguments need to be an integer, and Boolean, and that the result will be an integer. Now, you know that PHP has type juggling, so it can usually transform a value of one type to its equivalent value of another type, for example, the string 2 can be used as integer 2. To stop PHP from using type juggling with the arguments and results of functions, you can declare the strict_types directive as shown in the first highlighted line. This directive has to be declared on the top of each file, where you want to enforce this behavior. The three invocations work as follows: The first invocation sends two integers and a Boolean, which is what the function expects. So, regardless of the value of strict_types, it will always work. The second invocation sends an integer, a string, and a Boolean. The string has a valid integer value, so if PHP was allowed to use type juggling, the invocation would resolve to just normal. But in this example, it will fail because of the declaration on top of the file. The third invocation will always fail as the something string cannot be transformed into a valid integer. Let's try to use a function within our project. In our index.php file, we have a foreach loop that iterates the books and prints them. The code inside the loop is kind of hard to understand as it is mixing HTML with PHP, and there is a conditional too. Let's try to abstract the logic inside the loop into a function. First, create the new functions.php file with the following content: <?php function printableTitle(array $book): string { $result = '<i>' . $book['title'] . '</i> - ' . $book['author']; if (!$book['available']) { $result .= ' <b>Not available</b>'; } return $result; } This file will contain our functions. The first one, printableTitle, takes an array representing a book and builds a string with a nice representation of the book in HTML. The code is the same as before, just encapsulated in a function. Now, index.php will have to include the functions.php file and then use the function inside the loop. Let's see how this is done: <?php require_once 'functions.php' ?> <!DOCTYPE html> <html lang="en"> //... ?> <ul> <?php foreach ($books as $book): ?> <li><?php echo printableTitle($book); ?> </li> <?php endforeach; ?> </ul> //... Well, now our loop looks way cleaner, right? Also, if we need to print the title of the book somewhere else, we can reuse the function instead of duplicating code! Summary In this article, we went through all the basics of procedural PHP while writing simple examples in order to practice them. You now know how to use variables and arrays with control structures and functions and how to get information from HTTP requests among others. Resources for Article: Further resources on this subject: Getting started with Modernizr using PHP IDE[article] PHP 5 Social Networking: Implementing Public Messages[article] Working with JSON in PHP jQuery[article]
Read more
  • 0
  • 0
  • 16342

article-image-chrome-custom-tabs
Packt
17 Feb 2016
16 min read
Save for later

Chrome Custom Tabs

Packt
17 Feb 2016
16 min read
Well, most of us know tabs from every day Internet browsing. It doesn't really matter which browser you use; all browsers support tabs and multiple tabs' browsing. This allows us to have more than one website open at the same time and navigate between the opened instances. In Android, things are much the same, but when using WebView, you don't have tabs. This article will give highlights about WebView and the new feature of Android 6, Chrome custom tabs. (For more resources related to this topic, see here.) What is WebView? WebView is the part in the Android OS that's responsible for rendering web pages in most Android apps. If you see web content in an Android app, chances are you're looking at WebView. The major exceptions to this rule are some of the Android browsers, such as Chrome, Firefox, and so on. In Android 4.3 and lower, WebView uses code based on Apple's Webkit. In Android 4.4 and higher, WebView is based on the Chromium project, which is the open source base of Google Chrome. In Android 5.0, WebView was decoupled into a separate app that allowed timely updates through Google Play without requiring firmware updates to be issued, and the same technique was used with Google Play services. Now, let's talk again about a simple scenario: we want to display web content (URL-related) in our application. We have two options: either launch a browser or build our own in-app browser using WebView. Both options have trade-offs or disadvantages if we write them down. A browser is an external application and you can't really change its UI; while using it, you push the users to other apps and you may lose them in the wild. On the other hand, using WebView will keep the users tightly inside. However, actually dealing with all possible actions in WebView is quite an overhead. Google heard our rant and came to the rescue with Chrome custom tabs. Now we have better control over the web content in our application, and we can stitch web content into our app in a cleaner, prettier manner. Customization options Chrome custom tabs allow several modifications and tweaks: The toolbar color Enter and exit animations Custom actions for the toolbar and overflow menu Prestarted and prefetched content for faster loading When to use Chrome custom tabs Ever since WebView came out, applications have been using it in multiple ways, embedding content—local static content inside the APK and dynamic content as loading web pages that were not designed for mobile devices at the beginning. Later on we saw the rise of the mobile web era complete with hybrid applications). Chrome custom tabs are a bit more than just loading local content or mobile-compatible web content. They should be used when you load web data and want to allow simple implementation and easier code maintenance and, furthermore, make the web content part of your application—as if it's always there within your app. Among the reasons why you should use custom tabs are the following: Easy implementation: you use the support library when required or just add extras to your View intent. It's that simple. In app UI modifications, you can do the following: Set the toolbar color Add/change the action button Add custom menu items to the overflow menu Set and create custom in/out animations when entering the tab or exiting to the previous location Easier navigation and navigation logic: you can get a callback notifying you about an external navigation, if required. You know when the user navigates to web content and where they should return when done. Chrome custom tabs allow added performance optimizations that you can use: You can keep the engine running, so to speak, and actually give the custom tab a head start to start itself and do some warm up prior to using it. This is done without interfering or taking away precious application resources. You can provide a URL to load in advance in the background while waiting for other user interactions. This speeds up the user-visible page loading time and gives the user a sense of blazing fast application where all the content is just a click away. While using the custom tab, the application won't be evicted as the application level will still be in the foreground even though the tab is on top of it. So, we remain at the top level for the entire usage time (unless a phone call or some other user interaction leads to a change). Using the same Chrome container means that users are already signed in to sites they connected to in the past; specific permissions that were granted previously apply here as well; even fill data, autocomplete, and sync work here. Chrome custom tabs allow us give the users the latest browser implementation on pre-Lollipop devices where WebView is not the latest version. The implementation guide As discussed earlier, we have a couple of features integrated into Chrome custom tabs. The first customizes the UI and interaction with the custom tabs. The second allows pages to be loaded faster and keeps the application alive. Can we use Chrome custom tabs? Before we start using custom tabs, we want to make sure they're supported. Chrome custom tabs expose a service, so the best check for support is to try and bind to the service. Success means that custom tabs are supported and can be used. You can check out this gist, which shows a helper how to to check it, or check the project source code later on at https://gist.github.com/MaTriXy/5775cb0ff98216b2a99d. After checking and learning that support exists, we will start with the UI and interaction part. Custom UI and tab interaction Here, we will use the well-known ACTION_VIEW intent action, and by appending extras to the intent sent to Chrome, we will trigger changes in the UI. Remember that the ACTION_VIEW intent is compatible with all browsers, including Chrome. There are some phones without Chrome out there, or there are instances where the device's default browser isn't Chrome. In these cases, the user will navigate to the specific browser application. Intent is a convenient way to pass that extra data we want Chrome to get. Don't use any of these flags when calling to the Chrome custom tabs: FLAG_ACTIVITY_NEW_TASK FLAG_ACTIVITY_NEW_DOCUMENT Before using the API, we need to add it to our gradle file: compile 'com.android.support:customtabs:23.1.0' This will allow us to use the custom tab support library in our application: CustomTabsIntent.EXTRA_SESSION The preceding code is an extra from the custom tabs support library; it's used to match the session. It must be included in the intent when opening a custom tab. It can be null if there is no need to match any service-side sessions with the intent. We have a sample project to show the options for the UI called ChubbyTabby at https://github.com/MaTriXy/ChubbyTabby. We will go over the important parts here as well. Our main interaction comes from a special builder from the support library called CustomTabsIntent.Builder; this class will help us build the intent we need for the custom tab: CustomTabsIntent.Builder intentBuilder = new CustomTabsIntent.Builder(); //init our Builder //Setting Toolbar Color int color = getResources().getColor(R.color.primary); //we use primary color for our toolbar as well - you can define any color you want and use it. intentBuilder.setToolbarColor(color); //Enabling Title showing intentBuilder.setShowTitle(true); //this will show the title in the custom tab along the url showing at the bottom part of the tab toolbar. //This part is adding custom actions to the over flow menu String menuItemTitle = getString(R.string.menu_title_share); PendingIntent menuItemPendingIntent = createPendingShareIntent(); intentBuilder.addMenuItem(menuItemTitle, menuItemPendingIntent); String menuItemEmailTitle = getString(R.string.menu_title_email); PendingIntent menuItemPendingIntentTwo = createPendingEmailIntent(); intentBuilder.addMenuItem(menuItemEmailTitle, menuItemPendingIntentTwo); //Setting custom Close Icon. intentBuilder.setCloseButtonIcon(mCloseButtonBitmap); //Adding custom icon with custom action for the share action. intentBuilder.setActionButton(mActionButtonBitmap, getString(R.string.menu_title_share), createPendingShareIntent()); //Setting start and exit animation for the custom tab. intentBuilder.setStartAnimations(this, R.anim.slide_in_right, R.anim.slide_out_left); intentBuilder.setExitAnimations(this, android.R.anim.slide_in_left, android.R.anim.slide_out_right); CustomTabActivityHelper.openCustomTab(this, intentBuilder.build(), Uri.parse(URL), new WebviewFallback(), useCustom);  A few things to notice here are as follows: Every menu item uses a pending intent; if you don't know what a pending intent is, head to http://developer.android.com/reference/android/app/PendingIntent.html When we set custom icons, such as close buttons or an action button, for that matter, we use bitmaps and we must decode the bitmap prior to passing it to the builder Setting animations is easy and you can use animations' XML files that you created previously; just make sure that you test the result before releasing the app The following screenshot is an example of a Chrome custom UI and tab: The custom action button As developers, we have full control over the action buttons presented in our custom tab. For most use cases, we can think of a share action or maybe a more common option that your users will perform. The action button is basically a bundle with an icon of the action button and a pending intent that will be called by Chrome when your user hits the action button. The icon should be 24 dp in height and 24-48 dp in width according to specifications: //Adding custom icon with custom action for the share action intentBuilder.setActionButton(mActionButtonBitmap, getString(R.string.menu_title_share), createPendingShareIntent()); Configuring a custom menu By default, Chrome custom tabs usually have a three-icon row with Forward, Page Info, and Refresh on top at all times and Find in page and Open in Browser (Open in Chrome can appear as well) at the footer of the menu. We, developers, have the ability to add and customize up to three menu items that will appear between the icon row and foot items as shown in the following screenshot: The menu we see is actually represented by an array of bundles, each with menu text and a pending intent that Chrome will call on your behalf when the user taps the item: //This part is adding custom buttons to the over flow menu String menuItemTitle = getString(R.string.menu_title_share); PendingIntent menuItemPendingIntent = createPendingShareIntent(); intentBuilder.addMenuItem(menuItemTitle, menuItemPendingIntent); String menuItemEmailTitle = getString(R.string.menu_title_email); PendingIntent menuItemPendingIntentTwo = createPendingEmailIntent(); intentBuilder.addMenuItem(menuItemEmailTitle, menuItemPendingIntentTwo); Configuring custom enter and exit animations Nothing is complete without a few animations to tag along. This is no different, as we have two transitions to make: one for the custom tab to enter and another for its exit; we have the option to set a specific animation for each start and exit animation: //Setting start and exit animation for the custom tab. intentBuilder.setStartAnimations(this,R.anim.slide_in_right, R.anim.slide_out_left); intentBuilder.setExitAnimations(this, android.R.anim.slide_in_left, android.R.anim.slide_out_right); Chrome warm-up Normally, after we finish setting up the intent with the intent builder, we should call CustomTabsIntent.launchUrl (Activity context, Uri url), which is a nonstatic method that will trigger a new custom tab activity to load the URL and show it in the custom tab. This can take up quite some time and impact the impression of smoothness the app provides. We all know that users demand a near-instantaneous experience, so Chrome has a service that we can connect to and ask it to warm up the browser and its native components. Calling this will ask Chrome to perform the following: The DNS preresolution of the URL's main domain The DNS preresolution of the most likely subresources Preconnection to the destination, including HTTPS/TLS negotiation The process to warm up Chrome is as follows: Connect to the service. Attach a navigation callback to get notified upon finishing the page load. On the service, call warmup to start Chrome behind the scenes. Create newSession; this session is used for all requests to the API. Tell Chrome which pages the user is likely to load with mayLaunchUrl. Launch the intent with the session ID generated in step 4. Connecting to the Chrome service Connecting to the Chrome service involves dealing with Android Interface Definition Language (AIDL). If you don't know about AIDL, read http://developer.android.com/guide/components/aidl.html. The interface is created with AIDL, and it automatically creates a proxy service class for you: CustomTabsClient.bindCustomTabsService() So, we check for the Chrome package name; in our sample project, we have a special method to check whether Chrome is present in all variations. After we set the package, we bind to the service and get a CustomTabsClient object that we can use until we're disconnected from the service: pkgName - This is one of several options checking to see if we have a version of Chrome installed can be one of the following static final String STABLE_PACKAGE = "com.android.chrome"; static final String BETA_PACKAGE = "com.chrome.beta"; static final String DEV_PACKAGE = "com.chrome.dev"; static final String LOCAL_PACKAGE = "com.google.android.apps.chrome"; private CustomTabsClient mClient; // Binds to the service. CustomTabsClient.bindCustomTabsService(myContext, pkgName, new CustomTabsServiceConnection() { @Override public void onCustomTabsServiceConnected(ComponentName name, CustomTabsClient client) { // CustomTabsClient should now be valid to use mClient = client; } @Override public void onServiceDisconnected(ComponentName name) { // CustomTabsClient is no longer valid which also invalidates sessions. mClient = null; } });  After we bind to the service, we can call the proper methods we need. Warming up the browser process The method for this is as follows: boolean CustomTabsClient.warmup(long flags) //With our valid client earlier we call the warmup method. mClient.warmup(0); Flags are currently not being used, so we pass 0 for now. The warm-up procedure loads native libraries and the browser process required to support custom tab browsing later on. This is asynchronous, and the return value indicates whether the request has been accepted or not. It returns true to indicate success. Creating a new tab session The method for this is as follows: boolean CustomTabsClient.newSession(ICustomTabsCallback callback) The new tab session is used as the grouping object tying the mayLaunchUrl call, the VIEW intent that we build, and the tab generated altogether. We can get a callback associated with the created session that would be passed for any consecutive mayLaunchUrl calls. This method returns CustomTabsSession when a session is created successfully; otherwise, it returns Null. Setting the prefetching URL The method for this is as follows: boolean CustomTabsSession.mayLaunchUrl (Uri url, Bundle extras, List<Bundle> otherLikelyBundles) This method will notify the browser that a navigation to this URL will happen soon. Make sure that you call warmup() prior to calling this method—this is a must. The most likely URL has to be specified first, and you can send an optional list of other likely URLs (otherLikelyBundles). Lists have to be sorted in a descending order and the optional list may be ignored. A new call to this method will lower the priority of previous calls and can result in URLs not being prefetched. Boolean values inform us whether the operation has been completed successfully. Custom tabs connection callback The method for this is as follows: void CustomTabsCallback.onNavigationEvent (int navigationEvent, Bundle extras) We have a callback triggered upon each navigation event in the custom tab. The int navigationEvent element is one of the six that defines the state the page is in. Refer to the following code for more information: //Sent when the tab has started loading a page. public static final int NAVIGATION_STARTED = 1; //Sent when the tab has finished loading a page. public static final int NAVIGATION_FINISHED = 2; //Sent when the tab couldn't finish loading due to a failure. public static final int NAVIGATION_FAILED = 3; //Sent when loading was aborted by a user action. public static final int NAVIGATION_ABORTED = 4; //Sent when the tab becomes visible. public static final int TAB_SHOWN = 5; //Sent when the tab becomes hidden. public static final int TAB_HIDDEN = 6; private static class NavigationCallback extends CustomTabsCallback { @Override public void onNavigationEvent(int navigationEvent, Bundle extras) { Log.i(TAG, "onNavigationEvent: Code = " + navigationEvent); } } Summary In this article, we learned about a newly added feature, Chrome custom tabs, which allows us to embed web content into our application and modify the UI. Chrome custom tabs allow us to provide a fuller, faster in-app web experience for our users. We use the Chrome engine under the hood, which allows faster loading than regular WebViews or loading the entire Chrome (or another browser) application. We saw that we can preload pages in the background, making it appear as if our data is blazing fast. We can customize the look and feel of our Chrome tab so that it matches our app. Among the changes we saw were the toolbar color, transition animations, and even the addition of custom actions to the toolbar. Custom tabs also benefit from Chrome features such as saved passwords, autofill, tap to search, and sync; these are all available within a custom tab. For developers, integration is quite easy and requires only a few extra lines of code in the basic level. The support library helps with more complex integration, if required. This is a Chrome feature, which means you get it on any Android device where the latest versions of Chrome are installed. Remember that the Chrome custom tab support library changes with new features and fixes, which is the same as other support libraries, so please update your version and make sure that you use the latest API to avoid any issues. To learn more about Chrome custom tabs and Android 6, refer to the following books: Android 6 Essentials (https://www.packtpub.com/application-development/android-6-essentials) Augmented Reality for Android Application Development (https://www.packtpub.com/application-development/augmented-reality-android-application-development) Resources for Article: Further resources on this subject: Android and iOS Apps Testing at a Glance [Article] Working with Xamarin.Android [Article] Mobile Phone Forensics – A First Step into Android Forensics [Article]
Read more
  • 0
  • 0
  • 5817

article-image-hive-security
Packt
17 Feb 2016
13 min read
Save for later

Hive Security

Packt
17 Feb 2016
13 min read
In this article by Hanish Bansal, Saurabh Chauhan, and Shrey Mehrotra, the authors of the book, Apache Hive Cookbook, we will cover the following recipes: Securing Hadoop Authorizing Hive Security is a major concern in all the big data frameworks. It is little complex to implement security in distributed systems because the components of different machines need to communicate with each other. It is very important to enable security on the data. (For more resources related to this topic, see here.) Securing Hadoop In today's era of big data, most organizations are concentrating to use Hadoop as a centralized data store. Data size is growing day by day, and organizations want to derive some insights and make decisions using the important information. While everyone is focusing on collecting the data, but having all the data at a centralized place increases the risk of data security. Securing the data access of Hadoop Distributed File System (HDFS) is very important. Hadoop security means restricting the access of data to only authorized users and groups. Further, when we talk about security, there are two major things—Authentication and Authorization. The HDFS supports a permission model for files and directories that is much equivalent to standard POSIX model. Similar to UNIX permissions, each file and directory in HDFS are associated with an owner, a group, and other users. There are three types of permissions in HDFS: read, write, and execute. In contrast to the UNIX permission model, there is no concept of executable files. So in case of files, read (r) permission is required to read a file, and write (w) permission is required to write or append to a file. In case of directories, read (r) permission is required to list the contents of directory, write (w) permission is required to create or delete the files or subdirectories, and execute (x) permission is required to access the child objects (files/subdirectories) of that directory. The following screenshot shows the level of access to each individual entity, namely OWNER, GROUP, and OTHER: The Default HDFS Permission Model As illustrated in the previous screenshot, by default, the permission set for the owner of files or directories is rwx (7), which means the owner of the file or directory is having full permission to read, write, and execute. For the members of group, the permission set is r-x, which means group members can only read and execute the files or directories and they cannot write or update anything in the files or directories. For other members, a permission set is same as a group, that is, other members can only read and execute the files or directories and they cannot write or update anything in files or directories. Although this basic permission model is sufficient to handle a large number of security requirements at the block level, but using this model, you cannot define finer level security to specifically named users or groups. HDFS also has a feature to configure Access Control List (ACL), which can be used to define fine-grained permissions at file level as well as directory level for specifically named users or groups. For example, you want to give read access to users John, Mike, and Kate; then, HDFS ACLs can be used to define such kind of permissions. HDFS ACLs are designed on the base concept of POSIX ACLs of UNIX systems. How to do it… First of all, you will need to enable ACLs in Hadoop. To enable ACL permissions, configure the following property in Hadoop-configure file named hdfs-site.xml located at <HADOOP_HOME>/etc/hadoop/hdfs-site.xml: <property> <name>dfs.namenode.acls.enabled</name> <value>true</value> </property> There are two main commands that are used to configure ACLs: setfacl and getfacl. The command setfacl is used to set Finer Access Control Lists (FACL) for files or directories, and getfacl is used to retrieve Finer Access Control Lists (FACL) for files or directories. Let's see how to use these commands: hdfs dfs -setfacl [-R] [-b |-k -m |-x <acl_specification> <path>] |[--set <acl_specification> <path>] The same command can be run using hadoop fs also, as follows: hadoop fs -setfacl [-R] [-b |-k -m |-x <acl_specification> <path>] |[--set <acl_specification> <path>] This command contains the following elements: -R is used to apply operation recursively for all files and subdirectories under a directory. -b is used to remove all ACLs except the base ACLs. -k is used to remove the default ACLs. -m is used to modify ACLs. Using this option, new entries are added to the existing set of ACLs. -x is used to remove specific ACLs. acl_specification is a comma-separated list of ACLs. path is the path of a file or directory for which ACL has to be applied. --set is used to set new ACLs. It removes all existing ACLs and set the new ACLs only. Now, let's see another command that is used to retrieve the ACLs: hdfs dfs -getfacl [-R] <path> This command can also be run using hadoop fs as follows: hadoop fs -getfacl [-R] <path> This command contains the following elements: -R is used to retrieve ACLs recursively for all files and subdirectories under a directory path is a path of a file or directory of which ACL is to be retrieve The command getfacl will list all default ACLs as well as new ACLs defined for specified files or directories. How it works… If ACLs are defined for a file or directory, then while accessing that file/directory, access is validated as given in the following algorithm: If the username is the same as the owner name of the file, then owner permissions are enforced If username matches with one of named user ACL entry, then those permissions are enforced If a user's group name matches with one of the named group ACL entry, then those permissions are enforced In case multiple ACLs entries found for a user, then the union of all those permissions is enforced If no ACL entry found for a user, then other permissions are enforced Let's assume that we have a file named stock-data containing stock market data. To retrieve all ACLs of this file, run the following command after which the output is shown in the screenshot given later: $ hadoop fs -getfacl /stock-data Because we have not defined any custom ACL for this file, as shown in the previous screenshot, command will return default ACL for this file. You can check the permissions of a file or directory using the ls command also. As shown in the previous screenshot, the permission set for stock-data file is -rw-r--r--, which means read and write access for owner as well as read access for group members and others. In the following command, we give read and write access to user named mike, and the result is shown in the following screenshot: $ hadoop fs -setfacl -m user:mike:rw- /stock-data As shown in the previous screenshot, first, we defined the ACLs for the user mike using setfacl command; then, we retrieved the ACLs using the getfacl command. The output of the getfacl command will list out all default permissions as well as all ACLs. We defined ACLs for the user mike, so in the output, there is an extra row user:mike:rw-. There is an extra row in the output mask::rw-, which defines special mask ACLs entry. Mask is a special type of ACLs that filters out the access for all named users, named groups, and unnamed groups. If you have not defined mask ACL, then its value is calculated using the union of all permissions. In addition to this, the output of the ls command is also changed after defining ACLs. There is an extra plus (+) sign in permissions list that indicates that there are additional ACLs defined for this file or directory. Revoking access of user mike. To remove a specific ACL -x option is used with the setfacl command: $ hadoop fs -setfacl -x user:mike /stock-data In the previous screenshot, after revoking access of user mike, ACLs are updated, and there is no entry for the user mike now. See also You can read more about the permission model in Hadoop at https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html. Authorizing Hive Hive authorization is to verifying that a user is authorized to perform particular action. Authentication is about verifying the identity of a user, which is different from the authorization concept. Hive can be used in the following different ways: Using HCatalog API: Hive's Hcatalog API is used to access Hive by many other frameworks such as Apache Pig, MapReduce, Facebook Presto, Spark SQL, and Cloudera Impala. Using the HCatalog API, users have direct access to HDFS data and hive metadata. Hive metadata is directly accessible using metastore server API. Using Hive CLI: Using Hive CLI also, users have direct access to HDFS data and Hive metadata. Hive CLI directly interacts with a Hive metastore server. Currently Hive CLI don't support rich authorization. In next versions of hive, Hive CLI's implementation will be changed to provide better security, and also Hive CLI will interact with HiveServer2 rather than directly interacting with the metastore server. Using ODBC/JDBC and other HiveServer2 clients such as Beeline: These clients don't have direct access to HDFS data and metadata but through HiveServer2. For security purpose, this is the best way to access Hive. How to do it… The following are the various ways of authorization in Hive: Default authorization: the legacy mode: The legacy authorization mode was available in earlier versions of Hive. This authorization scheme prevents the users from doing some unwanted actions. This scheme doesn't prevent malicious users from doing activities. It manages the access control using grant and revoke statements. This mode supports Hive Command Line Interface (Hive CLI). In case of Hive CLI, users have direct access to HDFS files and directories, so they can easily break the security checks. Also in this model for granting privileges, the permissions needed for a user are not defined, which means that any user can grant the access to themselves, so it is not secure to use this model. Storage-based Authorization: As storage perspective, both HDFS data as well as Hive metadata must be accessed only to authorized users. If users use the HCatalog API or Hive CLI, then they have direct access to data. To protect the data, HDFS ACLs are being enabled. In this mode, HDFS permissions work as a single source of truth for protecting data.Generally in Hive metastore, database credentials are configured in Hive configuration file hive-site.xml. Malicious users can easily read the metastore credentials and then cause serious damage to data as well as metadata, so Hive metastore server can also be secured. In this authorization, you can also enable security at metastore level. After enabling metastore security, it will restrict the access on metadata objects by verifying that users have respective system permissions corresponding to different files and directories of metadata objects. To configure the storage-based authorization, set the following properties in the hive-site.xml file: Property Value hive.metastore.pre.event.listeners org.apache.hadoop.hive.ql.security.authorization.AuthorizationPreEventListener hive.security.metastore.authorization.manager org.apache.hadoop.hive.ql.security.authorization.StorageBasedAuthorizationProvider hive.security.metastore.authenticator.manager org.apache.hadoop.hive.ql.security.HadoopDefaultMetastoreAuthenticator hive.security.metastore.authorization.auth.reads true After setting all these configurations, Hive configuration file hive-site.xml will look as follows: <configuration> <property> <name>hive.metastore.pre.event.listeners</name> <value>org.apache.hadoop.hive.ql.security.authorization.AuthorizationPreEventListener</value> </property> <property> <name>hive.security.metastore.authorization.manager</name> <value>org.apache.hadoop.hive.ql.security.authorization.StorageBasedAuthorizationProvider</value> </property> <property> <name>hive.security.metastore.authenticator.manager</name> <value>org.apache.hadoop.hive.ql.security.HadoopDefaultMetastoreAuthenticator</value> </property> <property> <name>hive.security.metastore.authorization.auth.reads</name> <value>true</value> </property> </configuration> hive.metastore.pre.event.listeners: This property is used to define pre-event listener class, which is loaded on the metastore side. APIs of this class are executed before occurring of any event such as creating a database, table, or partition; altering a database, table, or partition; or dropping a database, table, or partition. Configuring this property turns on security at a metastore level. Set value of this property to org.apache.hadoop.hive.ql.security.authorization.AuthorizationPreEventListener. hive.security.metastore.authorization.manager: This property is used to define the authorization provider class for metastore security. The default value of this property is DefaultHiveMetastoreAuthorizationProvider, which provides default legacy authorization described in the previous bullet. To enable storage-based authorization based on Hadoop ACLs, set value of this property to org.apache.hadoop.hive.ql.security.authorization.StorageBasedAuthorizationProvider. You can also write your own custom class to manage authorization and configure this property to enable custom authorization manager. The custom authorization manager class must implement an interface org.apache.hadoop.hive.ql.security.authorization.HiveMetastoreAuthorizationProvider. hive.security.metastore.authenticator.manager: This property is used to define an authentication manager class. Set value of this property to org.apache.hadoop.hive.ql.security.HadoopDefaultMetastoreAuthenticator. You can also write your custom class to manage authentication and configure to this property. The custom authentication manager class must implement an interface org.apache.hadoop.hive.ql.security.HiveAuthenticationProvider. hive.security.metastore.authorization.auth.reads: This property is used to define whether metastore authorization should check for read access or not. The default value of this property is true. SQL Standard-based Authorization: SQL standard-based authorization is the third way of authorizing Hive. Although the previous methodology storage-based authorization also provides access control at level of partitions, tables, and databases, that methodology does not provide access control at more granular level such as columns and rows. This is because storage-based authorization depends on access control provided by HDFS using ACL that controls the access on the level of files and directories. SQL Standard-based authorization can be used to enforce fine-grained security. It is recommended to use as it is fully SQL compliant in its authorization model. There's more Many things can be done with SQL standard-based authorization. Use SQL standard-based authorization for more details. Summary In this article, we learned two different recipes Securing Hadoop and Authorizing Hive. You also learned different terminology of access permissions and their types. You went through various steps to secure Hadoop and learned different ways to perform authorization in Hive. Resources for Article: Further resources on this subject: Hive in Hadoop[article] Processing Tweets with Apache Hive[article] Using Hive non-interactively (Simple)[article]
Read more
  • 0
  • 0
  • 3495

article-image-scraping-web-python-quick-start
Packt
17 Feb 2016
9 min read
Save for later

Scraping the Web with Python - Quick Start

Packt
17 Feb 2016
9 min read
In this article we're going to acquire intelligence data from a variety of sources. We might interview people. We might steal files from a secret underground base. We might search the World Wide Web (WWW). (For more resources related to this topic, see here.) Accessing data from the Internet The WWW and Internet are based on a series of agreements called Request for Comments (RFC). The RFCs define the standards and protocols to interconnect different networks, that is, the rules for internetworking. The WWW is defined by a subset of these RFCs that specifies the protocols, behaviors of hosts and agents (servers and clients), and file formats, among other details. In a way, the Internet is a controlled chaos. Most software developers agree to follow the RFCs. Some don't. If their idea is really good, it can catch on, even though it doesn't precisely follow the standards. We often see this in the way some browsers don't work with some websites. This can cause confusion and questions. We'll often have to perform both espionage and plain old debugging to figure out what's available on a given website. Python provides a variety of modules that implement the software defined in the Internet RFCs. We'll look at some of the common protocols to gather data through the Internet and the Python library modules that implement these protocols. Background briefing – the TCP/IP protocols The essential idea behind the WWW is the Internet. The essential idea behind the Internet is the TCP/IP protocol stack. The IP part of this is the internetworking protocol. This defines how messages can be routed between networks. Layered on top of IP is the TCP protocol to connect two applications to each other. TCP connections are often made via a software abstraction called a socket. In addition to TCP, there's also UDP; it's not used as much for the kind of WWW data we're interested in. In Python, we can use the low-level socket library to work with the TCP protocol, but we won't. A socket is a file-like object that supports open, close, input, and output operations. Our software will be much simpler if we work at a higher level of abstraction. The Python libraries that we'll use will leverage the socket concept under the hood. The Internet RFCs defines a number of protocols that build on TCP/IP sockets. These are more useful definitions of interactions between host computers (servers) and user agents (clients). We'll look at two of these: Hypertext Transfer Protocol (HTTP) and File Transfer Protocol (FTP). Using http.client for HTTP GET The essence of web traffic is HTTP. This is built on TCP/IP. HTTP defines two roles: host and user agent, also called server and client, respectively. We'll stick to server and client. HTTP defines a number of kinds of request types, including GET and POST. A web browser is one kind of client software we can use. This software makes GET and POST requests, and displays the results from the web server. We can do this kind of client-side processing in Python using two library modules. The http.client module allows us to make GET and POST requests as well as PUT and DELETE. We can read the response object. Sometimes, the response is an HTML page. Sometimes, it's a graphic image. There are other things too, but we're mostly interested in text and graphics. Here's a picture of a mysterious device we've been trying to find. We need to download this image to our computer so that we can see it and send it to our informant from http://upload.wikimedia.org/wikipedia/commons/7/72/IPhone_Internals.jpg: Here's a picture of the currency we're supposed to track down and pay with: We need to download this image. Here is the link: http://upload.wikimedia.org/wikipedia/en/c/c1/1drachmi_1973.jpg Here's how we can use http.client to get these two image files: import http.client import contextlib path_list = [ "/wikipedia/commons/7/72/IPhone_Internals.jpg", "/wikipedia/en/c/c1/1drachmi_1973.jpg", ] host = "upload.wikimedia.org" with contextlib.closing(http.client.HTTPConnection( host )) as connection: for path in path_list: connection.request( "GET", path ) response= connection.getresponse() print("Status:", response.status) print("Headers:", response.getheaders()) _, _, filename = path.rpartition("/") print("Writing:", filename) with open(filename, "wb") as image: image.write( response.read() ) We're using http.client to handle the client side of the HTTP protocol. We're also using the contextlib module to politely disentangle our application from network resources when we're done using them. We've assigned a list of paths to the path_list variable. This example introduces list objects without providing any background. It's important that lists are surrounded by [] and the items are separated by ,. Yes, there's an extra , at the end. This is legal in Python. We created an http.client.HTTPConnection object using the host computer name. This connection object is a little like a file; it entangles Python with operating system resources on our local computer plus a remote server. Unlike a file, an HTTPConnection object isn't a proper context manager. As we really like context managers to release our resources, we made use of the contextlib.closing() function to handle the context management details. The connection needs to be closed; the closing() function assures that this will happen by calling the connection's close() method. For all of the paths in our path_list, we make an HTTP GET request. This is what browsers do to get the image files mentioned in an HTML page. We print a few things from each response. The status, if everything worked, will be 200. If the status is not 200, then something went wrong and we'll need to read up on the HTTP status code to see what happened. If you use a coffee shop Wi-Fi connection, perhaps you're not logged in. You might need to open a browser to set up a connection. An HTTP response includes headers that provide some additional details about the request and response. We've printed the headers because they can be helpful in debugging any problems we might have. One of the most useful headers is ('Content-Type', 'image/jpeg'). This confirms that we really did get an image. We used _, _, filename = path.rpartition("/") to locate the right-most / character in the path. Recall that the partition() method locates the left-most instance. We're using the right-most one here. We assigned the directory information and separator to the variable _. Yes, _ is a legal variable name. It's easy to ignore, which makes it a handy shorthand for we don't care. We kept the filename in the filename variable. We create a nested context for the resulting image file. We can then read the body of the response—a collection of bytes—and write these bytes to the image file. In one quick motion, the file is ours. The HTTP GET request is what underlies much of the WWW. Programs such as curl and wget are expansions of this example. They execute batches of GET requests to locate one or more pages of content. They can do quite a bit more, but this is the essence of extracting data from the WWW. Changing our client information An HTTP GET request includes several headers in addition to the URL. In the previous example, we simply relied on the Python http.client library to supply a suitable set of default headers. There are several reasons why we might want to supply different or additional headers. First, we might want to tweak the User-Agent header to change the kind of browser that we're claiming to be. We might also need to provide cookies for some kinds of interactions. For information on the user agent string, see http://en.wikipedia.org/wiki/User_agent_string#User_agent_identification. This information may be used by the web server to determine if a mobile device or desktop device is being used. We can use something like this: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.75.14 (KHTML, like Gecko) Version/7.0.3 Safari/537.75.14 This makes our Python request appear to come from the Safari browser instead of a Python application. We can use something like this to appear to be a different browser on a desktop computer: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:28.0) Gecko/20100101 Firefox/28.0 We can use something like this to appear to be an iPhone instead of a Python application: Mozilla/5.0 (iPhone; CPU iPhone OS 7_1_1 like Mac OS X) AppleWebKit/537.51.2 (KHTML, like Gecko) Version/7.0 Mobile/11D201 Safari/9537.53 We make this change by adding headers to the request we're making. The change looks like this: connection.request( "GET", path, headers= { 'User-Agent': 'Mozilla/5.0 (iPhone; CPU iPhone OS 7_1_1 like Mac OS X) AppleWebKit/537.51.2 (KHTML, like Gecko) Version/7.0 Mobile/11D201 Safari/9537.53', }) This will make the web server treat our Python application like it's on an iPhone. This might lead to a more compact page of data than might be provided to a full desktop computer that makes the same request. The header information is a structure with the { key: value, } syntax. It's important that dictionaries are surrounded by {}, the keys and values are separated by :, and each key-value pair is separated by ,. Yes, there's an extra , at the end. This is legal in Python. There are many more HTTP headers we can provide. The User-Agent header is perhaps most important to gather different kinds of intelligence data from web servers. You can refer more book related to this topic on the following links: Python for Secret Agents - Volume II: (https://www.packtpub.com/application-development/python-secret-agents-volume-ii) Expert Python Programming: (https://www.packtpub.com/application-development/expert-python-programming) Raspberry Pi for Secret Agents: (https://www.packtpub.com/hardware-and-creative/raspberry-pi-secret-agents) Resources for Article: Further resources on this subject: Python Libraries[article] Optimization in Python[article] Introduction to Object-Oriented Programming using Python, JavaScript, and C#[article]
Read more
  • 0
  • 0
  • 15858

article-image-asking-permission-getting-your-head-around-marshmallows-runtime-permissions
Packt
17 Feb 2016
8 min read
Save for later

Asking Permission: Getting your head around Marshmallow's Runtime Permissions

Packt
17 Feb 2016
8 min read
In Android, each application runs with distinct system IDs known as Linux user ID and Group ID. The system parts are also separated into distinct IDs, forming isolated zones for applications—from each other and from the system. As part of this isolated life cycle scheme, accessing services or other applications' data requires that you declare this desire in advance by requesting a permission. This is done by adding the uses-permission element to your AndroidManifest.xml file. Your manifest may have zero or more uses-permission elements, and all of them must be the direct children of the root <manifest> element. Trying to access data or features without proper permission should give out a security exception (using a SecurityException class), informing you about the missing permission in most cases. The sendBroadcast(Intent) method is exceptional as it checks permissions after the method call has returned, so you will not receive an exception if there are permission failures. A permission failure should be printed to the system log. Note that in Android versions prior to Marshmallow, missing permissions were due to missing declarations in the manifest. Hence, it is important that you keep permissions in mind when you come up with the feature list for your app. (For more resources related to this topic, see here.) Understanding Android Marshmallow permissions Android Marshmallow introduces a new application permissions model, allowing a simpler process for users when installing and/or upgrading applications. Applications running on Marshmallow should work according to a new permissions model, where the user can grant or revoke permissions after the installation—permissions are not given until there is user acceptance. Supporting the new permissions model is backward-compatible, which means your apps can still An overview With the Android Marshmallow version, a new application permissions model has been introduced. Let's review it a bit more thoroughly: Declaring permissions: All permissions an app needs are declared in the manifest, which is done to preserve backward compatibility in a manner similar to earlier Android platform versions. Permission groups: As discussed previously, permissions are divided into permission groups based on their functionalities: PROTECTION_NORMAL permissions: Some of the permissions are granted when users install the app. Upon the installation, the system checks your app's manifest and automatically grants permissions that match the PROTECTION_NORMAL group. INTERNET permission: One important permission is the INTERNET permission, which will be granted upon the installation, and the user can't revoke it. App signature permissions granted: The user is not prompted to grant any permissions at the time of installation. Permissions granted by users at runtime: You as an app developer need to request a permission in your app; a system dialog is shown to the user, and the user response is passed back to your app, notifying whether the permission is granted. Permissions can be revoked: Users can revoke permissions that were granted previously. We must learn how to handle these cases, as we'll learn later on. If an app targets an Android Marshmallow version, it must use the new permissions model. Permission groups When working with permissions, we divide them into groups. This division is done for fast user interaction when reviewing and approving permissions. Granting is done only once per permission group. If you add a new permission or request a new permission from the same permission group and the user has already approved that group, the system will grant you the added permission without bothering the user about the approval. For more information on this, visit https://developer.android.com/reference/android/content/pm/PermissionInfo.html#constants[GS1] . When the user installs an app, the app is granted only those permissions that are listed in the manifest that belongs to the PROTECTION_NORMAL group. Requesting permissions from the PROTECTION_SIGNATURE group will be granted only if the application is signed with the same certificate as the app with the declared permission. Apps cannot request signature permissions at runtime. System components automatically receive all the permissions listed in their manifests. Runtime permissions Android Marshmallow showcased a new permissions model where users were able to directly manage app permissions at application runtime. Google has altered the old permissions model, mostly to enable easier and frictionless installations and auto-updates for users as well as for app developers. This allows users to install the app without the need to preapprove each permission the application needs. The user can install the app without going through the phase of checking each permission and declining the installation due to a single permission. Users can grant or revoke permissions for installed apps, leaving the tweaking and the freedom of choice in the users' hands. Most of the applications will need to address these issues when updating the target API to 23. Taking coding permissions into account Well, after all the explanations, we've reached the coding part, and this is where we will get our coding hands dirty. The following are the methods used for coding permissions: Context.checkSelfPermission(): This checks whether your app has been granted a permission Activity.requestPermission(): This requests a permission at runtime Even if your app is not yet targeting Android Marshmallow, you should test your app and prepare to support it. Testing permissions In the Android Marshmallow permissions model, your app must ask the user for individual permissions at runtime. There is limited compatibility support for legacy apps, and you should test your app and also test a version to make sure it's supported. You can use the following test guide and conduct app testing with the new behavior: Map your app's permissions Test flows with permissions granted and revoked The adb command shell can be quite helpful to check for permissions: Listing application permissions and status by group can be done using the following adb command: adb shell pm list permissions -g You can grant or revoke permissions using the following adb syntax: adb shell pm [grant|revoke] <permission.name> You can grant permissions and install apk using the following adb command: adb install -g <path_to_apk> Coding for runtime permissions When we want to adjust our application to the new model, we need to make sure that we organize our steps and leave no permission stranded: Check what platform the app is running on: When running a piece of code that is sensitive at the API level, we start by checking the version/API level that we are running on. By now, you should be familiar with Build.VERSION.SDK_INT. Check whether the app has the required permission: Here, we get ourselves a brand new API call: Context.checkSelfPermission(String permission_name) With this, we silently check whether permissions are granted or not. This method returns immediately, so any permission-related controls/flows should be dealt with by checking this first. Prompting for permissions: We have a new API call, Activity.requestPermissions (String[] permissions, int requestCode). This call triggers the system to show the dialog requesting a permission. This method functions asynchronously. You can request more than one permission at once. The second argument is a simple request code returned in the callback so that you can recognize the calls. This is just like how we've been dealing with startActivityForResult() and onActivityResult() for years. Another new API is Activity.shouldShowRequestPermissionRationale(String permission). This method returns true when you have requested a permission and the user denied the request. It's considered a good practice after verifying that you explain to the user why you need that exact permission. The user can decide to turn down the permission request and select the Don't ask again option; then, this method will return false. The following sample code checks whether the app has permission to read the user's contacts. It requests the permission if required, and the result callback returns to onRequestPermissionsResult: if (checkSelfPermission(Manifest.permission.READ_CONTACTS) != PackageManager.PERMISSION_GRANTED) { requestPermissions(new String[]{Manifest.permission.READ_CONTACTS}, SAMPLE_MATRIXY_READ_CONTACTS); } //Now this is our callback @Override public void onRequestPermissionsResult(int requestCode, String permissions[], int[] grantResults) { switch (requestCode) { case SAMPLE_MATRIXY_READ_CONTACTS: if (grantResults[0] == PackageManager.PERMISSION_GRANTED) { // permission granted - we can continue the feature flow. } else { // permission denied! - we should disable the functionality that depends on this permission. } } } Just to make sure we all know the constants used, here's the explanation: public static final int PERMISSION_DENIED=-1: Since it's API level 1, permission has not been granted to the given package public static final int PERMISSION_GRANTED=0: Since it's API level 1. permission has been granted to the given package. If the user denies your permission request, your app should take the appropriate action, such as notifying the user why this permission is required or explaining that the feature can't work without it. Your app cannot assume user interaction has taken place because the user can choose to reject granting a permission along with the do not show again option; your permission request is automatically rejected and onRequestPermissionsResult gets the result back. Summary To learn more about Android 6 Essentials, the following books published by Packt Publishing (https://www.packtpub.com/) are recommended: Android User Interface Development: Beginner's Guide (https://www.packtpub.com/application-development/android-user-interface-development-beginners-guide) Android Native Development Kit Cookbook (https://www.packtpub.com/application-development/android-native-development-kit-cookbook) Resources for Article: Further resources on this subject: Practical How-To Recipes for Android [article] Working with Xamarin.Android [article] Detecting Shapes Employing Hough Transform [article]
Read more
  • 0
  • 0
  • 2227

article-image-setting-project-atomic-host
Packt
17 Feb 2016
6 min read
Save for later

Setting up a Project Atomic host

Packt
17 Feb 2016
6 min read
With DockerTM, containers are becoming mainstream and enterprises are ready to use them. Docker and its ecosystem are evolving at a very high pace, so it is very important to understand the basics and build group up to adopt to new concepts and tools. In this article, we will cover the following recipes: Setting up a Project Atomic host Doing atomic update/rollback with Project Atomic (For more resources related to this topic, see here.) Setting up a Project Atomic host Project Atomic facilitates application-centric IT architecture by providing an end-to-end solution to deploy containerized applications quickly and reliably, with atomic update and rollback for the application and host alike. This is achieved by running applications in containers on a Project Atomic host, which is a lightweight operating system specially designed to run containers. The hosts can be based on Fedora, CentOS, or Red Hat Enterprise Linux. Next, we will elaborate on the building blocks of the Project Atomic host. OSTree and rpm-OSTree OSTree (https://wiki.gnome.org/action/show/Projects/OSTree) is a tool to manage bootable, immutable, and versioned filesystem trees. Using this, we can build client-server architecture in which the server hosts an OSTree repository and the client subscribed to it can incrementally replicate the content. rpm-OSTree is a system to decompose RPMs on the server side into the OSTree repository to which the client can subscribe and perform updates. With each update, a new root is created, which is used for the next reboot. During updates, /etc is rebased and /var is untouched. Container runtime As of now Project Atomic only supports Docker as container runtime. systemd Project Atomic uses Kubernetes (http://kubernetes.io/) for application deployment over clusters of container hosts. Project Atomic can be installed on bare metal, cloud providers, VMs, and so on. In this recipe, let’s see how we can install it on a VM using virt-manager on Fedora. Getting ready Download the image: $ wget http://download.fedoraproject.org/pub/fedora/linux/releases/test/22_Beta/Cloud/x86_64/Images/Fedora-Cloud-Atomic-22_Beta-20150415.x86_64.raw.xz I have downloaded the beta image for Fedora 22 Cloud image For Containers. You should look for the latest cloud image For Containers at https://getfedora.org/en/cloud/download/. Uncompress this image by using the following command: $ xz -d Fedora-Cloud-Atomic-22_Beta-20150415.x86_64.raw.xz How to do it… We downloaded the cloud image that does not have any password set for the default user fedora. While booting the VM, we have to provide a cloud configuration file through which we can customize the VM. To do this, we need to create two files, meta-data and user-data, as follows: $ cat meta-data instance-id: iid-local01 local-hostname: atomichost $ cat user-data #cloud-config password: atomic ssh_pwauth: True chpasswd: { expire: False } ssh_authorized_keys: - ssh-rsa AAAAB3NzaC1yc......... In the preceding code, we need to provide the complete SSH public key. We then need to create an ISO image consisting of these files, which we will use to boot to the VM. As we are using a cloud image, our setting will be applied to the VM during the boot process. This means the hostname will be set to atomichost, the password will be set to atomic, and so on. To create the ISO, run the following command: $ genisoimage -output init.iso -volid cidata -joliet -rock user-data meta-data Start virt-manager. Select New Virtual Machine and then import the existing disk image. Enter the image path of the Project Atomic image we downloaded earlier. Select OS type as Linux and Version as Fedora 20/Fedora 21 (or later), and click on Forward. Next, assign CPU and Memory and click on Forward. Then, give a name to the VM and select Customize configuration before install. Finally, click on Finish and review the details. Next, click on Add Hardware, and after selecting Storage, attach the ISO (init.iso) file we created to the VM and select Begin Installation: Once booted, you can see that its hostname is correctly set and you will be able to log in through the password given in the cloud init file. The default user is fedora and password is atomic as set in the user-data file. How it works… In this recipe, we took a Project Atomic Fedora cloud image and booted it using virt-manager after supplying the cloud init file. There’s more… After logging in, if you do file listing at /, you will see that most of the traditional directories are linked to /var because it is preserved across upgrades. After logging in, you can run the Docker command as usual $sudo docker run -it fedora bash See also The virtual manager documentation at https://virt-manager.org/documentation/ More information on package systems, image systems, and RPM-OSTree at https://github.com/projectatomic/rpm-ostree/blob/master/doc/background.md The quick-start guide on the Project Atomic website at http://www.projectatomic.io/docs/quickstart/ The resources on cloud images at https://www.technovelty.org//linux/running-cloud-images-locally.html and http://cloudinit.readthedocs.org/en/latest/ How to set up Kubernetes with an Atomic host at http://www.projectatomic.io/blog/2014/11/testing-kubernetes-with-an-atomic-host/ and https://github.com/cgwalters/vagrant-atomic-cluster Doing atomic update/rollback with Project Atomic To get to the latest version or to roll back to the older version of Project Atomic, we use the atomic host command, which internally calls rpm-ostree. Getting ready Boot and log in to the Atomic host. How to do it… Just after the boot, run the following command: $ atomic host status You will see details about one deployment that is in use now. To upgrade, run the following command: This changes and/or adds new packages. After the upgrade, we will need to reboot the system to use the new update. Let’s reboot and see the outcome: As we can see, the system is now booted with the new update. The *, which is at the beginning of the first line, specifies the active build. To roll back, run the following command: $ sudo atomic host rollback We will have to reboot again if we want to use older bits. How it works… For updates, the Atomic host connects to the remote repository hosting the newer build, which is downloaded and used from the next reboot onwards until the user upgrades or rolls back. In the case rollback older build available on the system used after the reboot. See also The documentation Project Atomic website, which can be found at http://www.projectatomic.io/docs/os-updates/ Summary Docker and its ecosystem are evolving at a very high pace, so it is very important to understand the basics and build group up to adopt to new concepts and tools. Additional information on Docker can be gained be referring to: https://www.packtpub.com/networking-and-servers/docker-high-performance https://www.packtpub.com/virtualization-and-cloud/monitoring-docker Resources for Article: Further resources on this subject: Understanding Docker [article] Hands On with Docker Swarm [article] Introduction to Docker [article]
Read more
  • 0
  • 0
  • 2952
Unlock access to the largest independent learning library in Tech for FREE!
Get unlimited access to 7500+ expert-authored eBooks and video courses covering every tech area you can think of.
Renews at $19.99/month. Cancel anytime
article-image-using-collider-based-system
Packt
17 Feb 2016
10 min read
Save for later

Using a collider-based system

Packt
17 Feb 2016
10 min read
In this article by Jorge Palacios, the author of the book Unity 5.x Game AI Programming Cookbook, you will learn how to implement agent awareness using a mixed approach that considers the previous learnt sensory-level algorithms. (For more resources related to this topic, see here.) Seeing using a collider-based system This is probably the easiest way to simulate vision. We take a collider, be it a mesh or a Unity primitive, and use it as the tool to determine whether an object is inside the agent's vision range or not. Getting ready It's important to have a collider component attached to the same game object using the script on this recipe, as well as the other collider-based algorithms in this chapter. In this case, it's recommended that the collider be a pyramid-based one in order to simulate a vision cone. The lesser the polygons, the faster it will be on the game. How to do it… We will create a component that is able to see enemies nearby by performing the following steps: Create the Visor component, declaring its member variables. It is important to add the corresponding tags into Unity's configuration: using UnityEngine; using System.Collections; public class Visor : MonoBehaviour { public string tagWall = "Wall"; public string tagTarget = "Enemy"; public GameObject agent; } Implement the function for initializing the game object in case the component is already assigned to it: void Start() { if (agent == null) agent = gameObject; } Declare the function for checking collisions for every frame and build it in the following steps: public void OnCollisionStay(Collision coll) { // next steps here } Discard the collision if it is not a target: string tag = coll.gameObject.tag; if (!tag.Equals(tagTarget)) return; Get the game object's position and compute its direction from the Visor: GameObject target = coll.gameObject; Vector3 agentPos = agent.transform.position; Vector3 targetPos = target.transform.position; Vector3 direction = targetPos - agentPos; Compute its length and create a new ray to be shot soon: float length = direction.magnitude; direction.Normalize(); Ray ray = new Ray(agentPos, direction); Cast the created ray and retrieve all the hits: RaycastHit[] hits; hits = Physics.RaycastAll(ray, length); Check for any wall between the visor and target. If none, we can proceed to call our functions or develop our behaviors to be triggered: int i; for (i = 0; i < hits.Length; i++) { GameObject hitObj; hitObj = hits[i].collider.gameObject; tag = hitObj.tag; if (tag.Equals(tagWall)) return; } // TODO // target is visible // code your behaviour below How it works… The collider component checks every frame to know whether it is colliding with any game object in the scene. We leverage the optimizations to Unity's scene graph and engine, and focus only on how to handle valid collisions. After checking whether a target object is inside the vision range represented by the collider, we cast a ray in order to check whether it is really visible or there is a wall in between. Hearing using a collider-based system In this recipe, we will emulate the sense of hearing by developing two entities; a sound emitter and a sound receiver. It is based on the principles proposed by Millington for simulating a hearing system, and uses the power of Unity colliders to detect receivers near an emitter. Getting ready As with the other recipes based on colliders, we will need collider components attached to every object to be checked and rigid body components attached to either emitters or receivers. How to do it… We will create the SoundReceiver class for our agents and SoundEmitter for things such as alarms: Create the class for the SoundReceiver object: using UnityEngine; using System.Collections; public class SoundReceiver : MonoBehaviour { public float soundThreshold; } We define the function for our own behavior to handle the reception of sound: public virtual void Receive(float intensity, Vector3 position) { // TODO // code your own behavior here } Now, let's create the class for the SoundEmitter object: using UnityEngine; using System.Collections; using System.Collections.Generic; public class SoundEmitter : MonoBehaviour { public float soundIntensity; public float soundAttenuation; public GameObject emitterObject; private Dictionary<int, SoundReceiver> receiverDic; } Initialize the list of receivers nearby and emitterObject in case the component is attached directly: void Start() { receiverDic = new Dictionary<int, SoundReceiver>(); if (emitterObject == null) emitterObject = gameObject; } Implement the function for adding new receivers to the list when they enter the emitter bounds: public void OnCollisionEnter(Collision coll) { SoundReceiver receiver; receiver = coll.gameObject.GetComponent<SoundReceiver>(); if (receiver == null) return; int objId = coll.gameObject.GetInstanceID(); receiverDic.Add(objId, receiver); } Also, implement the function for removing receivers from the list when they are out of reach: public void OnCollisionExit(Collision coll) { SoundReceiver receiver; receiver = coll.gameObject.GetComponent<SoundReceiver>(); if (receiver == null) return; int objId = coll.gameObject.GetInstanceID(); receiverDic.Remove(objId); } Define the function for emitting sound waves to nearby agents: public void Emit() { GameObject srObj; Vector3 srPos; float intensity; float distance; Vector3 emitterPos = emitterObject.transform.position; // next step here } Compute sound attenuation for every receiver: foreach (SoundReceiver sr in receiverDic.Values) { srObj = sr.gameObject; srPos = srObj.transform.position; distance = Vector3.Distance(srPos, emitterPos); intensity = soundIntensity; intensity -= soundAttenuation * distance; if (intensity < sr.soundThreshold) continue; sr.Receive(intensity, emitterPos); } How it works… The collider triggers help register agents in the list of agents assigned to an emitter. The sound emission function then takes into account the agent's distance from the emitter in order to decrease its intensity using the concept of sound attenuation. There is more… We can develop a more flexible algorithm by defining different types of walls that affect sound intensity. It works by casting rays and adding up their values to the sound attenuation: Create a dictionary to store wall types as strings (using tags) and their corresponding attenuation: public Dictionary<string, float> wallTypes; Reduce sound intensity this way: intensity -= GetWallAttenuation(emitterPos, srPos); Define the function called in the previous step: public float GetWallAttenuation(Vector3 emitterPos, Vector3 receiverPos) { // next steps here } Compute the necessary values for ray casting: float attenuation = 0f; Vector3 direction = receiverPos - emitterPos; float distance = direction.magnitude; direction.Normalize(); Cast the ray and retrieve the hits: Ray ray = new Ray(emitterPos, direction); RaycastHit[] hits = Physics.RaycastAll(ray, distance); For every wall type found via tags, add up its value (stored in the dictionary): int i; for (i = 0; i < hits.Length; i++) { GameObject obj; string tag; obj = hits[i].collider.gameObject; tag = obj.tag; if (wallTypes.ContainsKey(tag)) attenuation += wallTypes[tag]; } return attenuation; Smelling using a collider-based system Smelling can be simulated by computing collision between an agent and odor particles, scattered throughout the game level. Getting ready In this recipe based on colliders, we will need collider components attached to every object to be checked, which can be simulated by computing a collision between an agent and odor particles. How to do it… We will develop the scripts needed to represent odor particles and agents able to smell: Create the particle's script and define its member variables for computing its lifespan: using UnityEngine; using System.Collections; public class OdorParticle : MonoBehaviour { public float timespan; private float timer; } Implement the Start function for proper validations: void Start() { if (timespan < 0f) timespan = 0f; timer = timespan; } Implement the timer and destroy the object after its life cycle: void Update() { timer -= Time.deltaTime; if (timer < 0f) Destroy(gameObject); } Create the class for representing the sniffer agent: using UnityEngine; using System.Collections; using System.Collections.Generic; public class Smeller : MonoBehaviour { private Vector3 target; private Dictionary<int, GameObject> particles; } Initialize the dictionary for storing odor particles: void Start() { particles = new Dictionary<int, GameObject>(); } Add to the dictionary the colliding objects that have the odor-particle component attached: public void OnCollisionEnter(Collision coll) { GameObject obj = coll.gameObject; OdorParticle op; op = obj.GetComponent<OdorParticle>(); if (op == null) return; int objId = obj.GetInstanceID(); particles.Add(objId, obj); UpdateTarget(); } Release the odor particles from the local dictionary when they are out of the agent's range or are destroyed: public void OnCollisionExit(Collision coll) { GameObject obj = coll.gameObject; int objId = obj.GetInstanceID(); bool isRemoved; isRemoved = particles.Remove(objId); if (!isRemoved) return; UpdateTarget(); } Create the function for computing the odor centroid according to the current elements in the dictionary: private void UpdateTarget() { Vector3 centroid = Vector3.zero; foreach (GameObject p in particles.Values) { Vector3 pos = p.transform.position; centroid += pos; } target = centroid; } Implement the function for retrieving the odor centroid, if any: public Vector3? GetTargetPosition() { if (particles.Keys.Count == 0) return null; return target; } How it works… Just like the hearing recipe based on colliders, we use the trigger colliders to register odor particles to an agent's perception (implemented using a dictionary). When a particle is included or removed, the odor centroid is computed. However, we implement a function to retrieve that centroid because when no odor particle is registered, the internal centroid position is not updated. There is more… The particle emission logic is left behind to be implemented according to our game's needs and it basically instantiates odor-particle prefabs. Also, it is recommended to attach the rigid body components to the agents. Odor particles are prone to be massively instantiated, reducing the game's performance. Seeing using a graph-based system We will start a recipe oriented to use graph-based logic in order to simulate sense. Again, we will start by developing the sense of vision. Getting ready It is important to grasp the chapter regarding path finding in order to understand the inner workings of the graph-based recipes. How to do it… We will just implement a new file: Create the class for handling vision: using UnityEngine; using System.Collections; using System.Collections.Generic; public class VisorGraph : MonoBehaviour { public int visionReach; public GameObject visorObj; public Graph visionGraph; } Validate the visor object: void Start() { if (visorObj == null) visorObj = gameObject; } Define and start building the function needed to detect visibility of a given set of nodes: public bool IsVisible(int[] visibilityNodes) { int vision = visionReach; int src = visionGraph.GetNearestVertex(visorObj); HashSet<int> visibleNodes = new HashSet<int>(); Queue<int> queue = new Queue<int>(); queue.Enqueue(src); } Implement a breath-first search algorithm: while (queue.Count != 0) { if (vision == 0) break; int v = queue.Dequeue(); List<int> neighbours = visionGraph.GetNeighbors(v); foreach (int n in neighbours) { if (visibleNodes.Contains(n)) continue; queue.Enqueue(v); visibleNodes.Add(v); } } Compare the set of visible nodes with the set of nodes reached by the vision system: foreach (int vn in visibleNodes) { if (visibleNodes.Contains(vn)) return true; } Return false if there is no match between the two sets of nodes: return false; How it works… The recipe uses the breath-first search algorithm in order to discover nodes within its vision reach, and then compares this set of nodes with the set of nodes where the agents reside. Summary In this article, we explained some algorithms for simulating senses and agent awareness. Resources for Article: Further resources on this subject: Animation and Unity3D Physics[article] Unity 3-0 Enter the Third Dimension[article] Animation features in Unity 5[article]
Read more
  • 0
  • 0
  • 31157

article-image-probabilistic-graphical-models
Packt
17 Feb 2016
6 min read
Save for later

Probabilistic Graphical Models

Packt
17 Feb 2016
6 min read
Probabilistic graphical models, or simply graphical models as we will refer to them in this article, are models that use the representation of a graph to describe the conditional independence relationships between a series of random variables. This topic has received an increasing amount of attention in recent years and probabilistic graphical models have been successfully applied to tasks ranging from medical diagnosis to image segmentation. In this article, we'll present some of the necessary background that will pave the way to understanding the most basic graphical model, the Naïve Bayes classifier. We will then look at a slightly more complicated graphical model, known as the Hidden Markov Model, or HMM for short. To get started in this field, we must first learn about graphs. (For more resources related to this topic, see here.) A Little Graph Theory Graph theory is a branch of mathematics that deals with mathematical objects known as graphs. Here, a graph does not have the everyday meaning that we are more used to talking about, in the sense of a diagram or plot with an x and y axis. In graph theory, a graph consists of two sets. The first is a set of vertices, which are also referred to as nodes. We typically use integers to label and enumerate the vertices. The second set consists of edges between these vertices. Thus, a graph is nothing more than a description of some points and the connections between them. The connections can have a direction so that an edge goes from the source or tail vertex to the target or head vertex. In this case, we have a directed graph. Alternatively, the edges can have no direction, so that the graph is undirected. A common way to describe a graph is via the adjacency matrix. If we have V vertices in the graph, an adjacency matrix is a V×V matrix whose entries are 0 if the vertex represented by the row number is not connected to the vertex represented by the column number. If there is a connection, the entry is 1. With undirected graphs, both nodes at each edge are connected to each other so the adjacency matrix is symmetric. For directed graphs, a vertex vi is connected to a vertex vj via an edge (vi,vj); that is, an edge where vi is the tail and vj is the head. Here is an example adjacency matrix for a graph with seven nodes: > adjacency_m 1 2 3 4 5 6 7 1 0 0 0 0 0 1 0 2 1 0 0 0 0 0 0 3 0 0 0 0 0 0 1 4 0 0 1 0 1 0 1 5 0 0 0 0 0 0 0 6 0 0 0 1 1 0 1 7 0 0 0 0 1 0 0 This matrix is not symmetric, so we know that we are dealing with a directed graph. The first 1 value in the first row of the matrix denotes the fact that there is an edge starting from vertex 1 and ending on vertex 6. When the number of nodes is small, it is easy to visualize a graph. We simply draw circles to represent the vertices and lines between them to represent the edges. For directed graphs, we use arrows on the lines to denote the directions of the edges. It is important to note that we can draw the same graph in an infinite number of different ways on the page. This is because the graph tells us nothing about the positioning of the nodes in space; we only care about how they are connected to each other. Here are two different but equally valid ways to draw the graph described by the adjacency matrix we just saw: Two vertices are said to be connected with each other if there is an edge between them (taking note of the order when talking about directed graphs). If we can move from vertex vi to vertex vj by starting at the first vertex and finishing at the second vertex, by moving on the graph along the edges and passing through an arbitrary number of graph vertices, then these intermediate edges form a path between these two vertices. Note that this definition requires that all the vertices and edges along the path are distinct from each other (with the possible exception of the first and last vertex). For example, in our graph, vertex 6 can be reached from vertex 2 by a path leading through vertex 1. Sometimes, there can be many such possible paths through the graph, and we are often interested in the shortest path, which moves through the fewest number of intermediary vertices. We can define the distance between two nodes in the graph as the length of the shortest path between them. A path that begins and ends at the same vertex is known as a cycle. A graph that does not have any cycles in it is known as an acyclic graph. If an acyclic graph has directed edges, it is known as a directed acyclic graph, which is often abbreviated as a DAG. There are many excellent references on graph theory available. One such reference which is available online, is Graph Theory, Reinhard Diestel, Springer. This landmark reference is now in its 4th edition and can be found at http://diestel-graph-theory.com/. It might not seem obvious at first, but it turns out that a large number of real world situations can be conveniently described using graphs. For example, the network of friendships on social media sites, such as Facebook, or followers on Twitter, can be represented as graphs. On Facebook, the friendship relation is reciprocal, and so the graph is undirected. On Twitter, the follower relation is not, and so the graph is directed. Another graph is the network of websites on the Web, where links from one web page to the next form directed edges. Transport networks, communication networks, and electricity grids can be represented as graphs. For the predictive modeler, it turns out that a special class of models known as probabilistic graphical models, or graphical models for short, are models that involve a graph structure. In a graphical model, the nodes represent random variables and the edges in between represent the dependencies between them. Before we can go into further detail, we'll need to take a short detour in order to visit Bayes' Theorem, a classic theorem in statistics that despite its simplicity has implications both profound and practical when it comes to statistical inference and prediction. Summary In this article, we learned that graphs are consist of nodes and edges. We also learned the way of describing a graph is via the adjacency matrix. For more information on graphical models, you can refer to the books published by Packt (https://www.packtpub.com/): Mastering Predictive Analytics with Python (https://www.packtpub.com/big-data-and-business-intelligence/mastering-predictive-analytics-python) R Graphs Cookbook Second Edition (https://www.packtpub.com/big-data-and-business-intelligence/r-graph-cookbook-%E2%80%93-second-edition) Resources for Article: Further resources on this subject: Data Analytics[article] Big Data Analytics[article] Learning Data Analytics with R and Hadoop[article]
Read more
  • 0
  • 0
  • 3502

article-image-setting-kubernetes-cluster
Packt
17 Feb 2016
12 min read
Save for later

Setting up a Kubernetes Cluster

Packt
17 Feb 2016
12 min read
In this article, we will cover the following recipes: Setting up a Kubernetes cluster Scaling up and down in a Kubernetes cluster Setting up WordPress with a Kubernetes cluster (For more resources related to this topic, see here.) Introduction Running Docker on a single host may be good for the development environment, but the real value comes when we span multiple hosts. However, this is not an easy task. You have to orchestrate these containers. So, in this article, we'll discuss Kubernetes, an orchestration tool. Google started Kubernetes (http://kubernetes.io/) for Docker orchestration. Kubernetes provides mechanisms for application deployment, scheduling, updating, maintenance, and scaling. Setting up a Kubernetes cluster Kubernetes is an open source container orchestration tool across multiple nodes in the cluster. Currently, it only supports Docker. It was started by Google, and now developers from other companies are contributing to it. It provides mechanisms for application deployment, scheduling, updating, maintenance, and scaling. Kubernetes' auto-placement, auto-restart, auto-replication features make sure that the desired state of the application is maintained, which is defined by the user. Users define applications through YAML or JSON files, which we'll see later in the recipe. These YAML and JSON files also contain the API Version (the apiVersion field) to identify the schema. The following is the architectural diagram of Kubernetes: https://raw.githubusercontent.com/GoogleCloudPlatform/kubernetes/master/docs/architecture.png Let's look at some of the key components and concepts of Kubernetes. Pods: A pod, which consists of one or more containers, is the deployment unit of Kubernetes. Each container in a pod shares different namespaces with other containers in the same pod. For example, each container in a pod shares the same network namespace, which means they can all communicate through localhost. Node/Minion: A node, which was previously known as a minion, is a worker node in the Kubernetes cluster and is managed through master. Pods are deployed on a node, which has the necessary services to run them: docker, to run containers kubelet, to interact with master proxy (kube-proxy), which connects the service to the corresponding pod Master: Master hosts cluster-level control services such as the following: API server: This has RESTful APIs to interact with master and nodes. This is the only component that talks to the etcd instance. Scheduler: This schedules jobs in clusters, such as creating pods on nodes. Replication controller: This ensures that the user-specified number of pod replicas is running at any given time. To manage replicas with replication controller, we have to define a configuration file with the replica count for a pod. Master also communicates with etcd, which is a distributed key-value pair. etcd is used to store the configuration information, which is used by both master and nodes. The watch functionality of etcd is used to notify the changes in the cluster. etcd can be hosted on master or on a different set of systems. Services: In Kubernetes, each pod gets its own IP address, and pods are created and destroyed every now and then based on the replication controller configuration. So, we cannot rely on a pod's IP address to cater an app. To overcome this problem, Kubernetes defines an abstraction, which defines a logical set of pods and policies to access them. This abstraction is called a service. Labels are used to define the logical set, which a service manages. Labels: Labels are key-value pairs that can be attached to objects like, using which we select a subset of objects. For example, a service can select all pods with the label mysql. Volumes: A volume is a directory that is accessible to the containers in a pod. It is similar to Docker volumes but not the same. Different types of volumes are supported in Kubernetes, some of which are EmptyDir (ephemeral), HostDir, GCEPersistentDisk, and NFS. Active development is happening to support more types of volumes. More details can be found at https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/volumes.md. Kubernetes can be installed on VMs, physical machines, and the cloud. For the complete matrix, take a look at https://github.com/GoogleCloudPlatform/kubernetes/tree/master/docs/getting-started-guides. In this recipe, we'll see how to install it on VMs, using Vagrant with VirtualBox provider. This recipe and the following recipes on Kubernetes, were tried on v0.17.0 of Kubernetes. Getting ready Install latest Vagrant >= 1.6.2 from http://www.vagrantup.com/downloads.html. Install the latest VirtualBox from https://www.virtualbox.org/wiki/Downloads. How to do it… Run the following command to set up Kubernetes on Vagrant VMs: $ export KUBERNETES_PROVIDER=vagrant $ export VAGRANT_DEFAULT_PROVIDER=virtualbox $ curl -sS https://get.k8s.io | bash How it works… The bash script downloaded from the curl command, first downloads the latest Kubernetes release and then runs the ./kubernetes/cluster/kube-up.sh bash script to set up the Kubernetes environment. As we have specified Vagrant as KUBERNETES_PROVIDER, the script first downloads the Vagrant images and then, using Salt (http://saltstack.com/), configures one master and one node (minion) VM. Initial setup takes a few minutes to run. Vagrant creates a credential file in ~/.kubernetes_vagrant_auth for authentication. There's more… Similar to ./cluster/kube-up.sh, there are other helper scripts to perform different operations from the host machine itself. Make sure you are in the kubernetes directory, which was created with the preceding installation, while running the following commands: Get the list of nodes: $ ./cluster/kubectl.sh get nodes Get the list of pods: $ ./cluster/kubectl.sh get pods Get the list of services: $ ./cluster/kubectl.sh get services Get the list of replication controllers: $ ./cluster/kubectl.sh get replicationControllers Destroy the vagrant cluster: $ ./cluster/kube-down.sh Then bring back the vagrant cluster: $ ./cluster/kube-up.sh You will see some pods, services, and replicationControllers listed, as Kubernetes creates them for internal use. See also Setting up the Vagrant environment at https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/getting-started-guides/vagrant.md The Kubernetes user guide at https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/user-guide.md Kubernetes API conventions at https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/api-conventions.md Scaling up and down in a Kubernetes cluster In the previous section, we mentioned that the replication controller ensures that the user-specified number of pod replicas is running at any given time. To manage replicas with the replication controller, we have to define a configuration file with the replica count for a pod. This configuration can be changed at runtime. Getting ready Make sure the Kubernetes setup is running as described in the preceding recipe and that you are in the kubernetes directory, which was created with the preceding installation. How to do it… Start the nginx container with a replica count of 3: $ ./cluster/kubectl.sh run-container my-nginx --image=nginx --replicas=3 --port=80 This will start three replicas of the nginx container. List the pods to get the status: $ ./cluster/kubectl.sh get pods Get the replication controller configuration: $ ./cluster/kubectl.sh get replicationControllers As you can see, we have a my-nginx controller, which has a replica count of 3. There is a replication controller for kube-dns, which we will explore in next recipe. Request the replication controller service to scale down to replica of 1 and update the replication controller: $ ./cluster/kubectl.sh resize rc my-nginx –replicas=1 $ ./cluster/kubectl.sh get rc Get the list of pods to verify; you should see only one pod for nginx: $ ./cluster/kubectl.sh get pods How it works… We request the replication controller service running on master to update the replicas for a pod, which updates the configuration and requests nodes/minions to act accordingly to honor the resizing. There's more… Get the services: $ ./cluster/kubectl.sh get services As you can see, we don't have any service defined for our nginx containers started earlier. This means that though we have a container running, we cannot access them from outside because the corresponding service is not defined. See also Setting up the Vagrant environment at https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/getting-started-guides/vagrant.md The Kubernetes user guide at https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/user-guide.md Setting up WordPress with a Kubernetes cluster In this recipe, we will use the WordPress example given in the Kubernetes GitHub (https://github.com/GoogleCloudPlatform/kubernetes/tree/master/examples/mysql-wordpress-pd). The given example requires some changes, as we'll be running it on the Vagrant environment instead of the default Google Compute engine. Also, instead of using the helper functions (for example, <kubernetes>/cluster/kubectl.sh), we'll log in to master and use the kubectl binary. Getting ready Make sure the Kubernetes cluster has been set up as described in the previous recipe. In the kubernetes directory that was downloaded during the setup, you will find an examples directory that contains many examples. Let's go to the mysql-wordpress-pd directory: $ cd kubernetes/examples/mysql-wordpress-pd $ ls *.yaml mysql-service.yaml mysql.yaml wordpress-service.yaml wordpress.yaml These .yaml files describe pods and services for mysql and wordpress respectively. In the pods files (mysql.yaml and wordpress.yaml), you will find the section on volumes and the corresponding volumeMount file. The original example assumes that you have access to Google Compute Engine and that you have the corresponding storage setup. For simplicity, we will not set up that and instead use ephemeral storage with the EmptyDir volume option. For reference, our mysql.yaml will look like the following: Make the similar change to wordpress.yaml. How to do it… With SSH, log in to the master node and look at the running pods: $ vagrant ssh master $ kubectl get pods The kube-dns-7eqp5 pod consists of three containers: etcd, kube2sky, and skydns, which are used to configure an internal DNS server for service name to IP resolution. We'll see it in action later in this recipe. The Vagrantfile used in this example is created so that the kubernetes directory that we created earlier is shared under /vagrant on VM, which means that the changes we made to the host system will be visible here as well. From the master node, create the mysql pod and check the running pods: $ kubectl create -f /vagrant/examples/mysql-wordpress-pd/mysql.yaml $ kubectl get pods As we can see, a new pod with the mysql name has been created and it is running on host 10.245.1.3, which is our node (minion). Now let's create the service for mysql and look at all the services: $ kubectl create -f /vagrant/examples/mysql-wordpress-pd/mysql-service.yaml $ kubectl get services As we can see, a service named mysql has been created. Each service has a Virtual IP. Other than the kubernetes services, we see a service named kube-dns, which is used as the service name for the kube-dns pod we saw earlier. Similar to mysql, let's create a pod for wordpress: $ kubectl create -f /vagrant/examples/mysql-wordpress-pd/wordpress.yaml With this command, there are a few things happening in the background: The wordpress image gets downloaded from the official Docker registry and the container runs. By default, whenever a pod starts, information about all the existing services is exported as environment variables. For example, if we log in to the wordpress pod and look for MYSQL-specific environment variables, we will see something like the following: When the WordPress container starts, it runs the /entrypoint.sh script, which looks for the environment variables mentioned earlier to start the service. https://github.com/docker-library/wordpress/blob/master/docker-entrypoint.sh. With the kube-dns service, PHP scripts of wordpress are able to the reserve lookup to proceed forward. After starting the pod, the last step here is to set up the wordpress service. In the default example, you will see an entry like the following in the service file (/vagrant/examples/mysql-wordpress-pd/mysql-service.yaml): createExternalLoadBalancer: true This has been written to keep in mind that this example will run on the Google Compute Engine. So it is not valid here. In place of that, we will need to make an entry like the following: publicIPs: - 10.245.1.3 We have replaced the load-balancer entry with the public IP of the node, which in our case is the IP address of the node (minion). So, the wordpress file would look like the following: To start the wordpress service, run the following command from the master node: $ kubectl create -f /vagrant/examples/mysql-wordpress-pd/wordpress-service.yaml We can see here that our service is also available through the node (minion) IP. To verify if everything works fine, we can install the links package on master by which we can browse a URL through the command line and connect to the public IP we mentioned: $ sudo yum install links -y $ links 10.245.1.3 With this, you should see the wordpress installation page. How it works… In this recipe, we first created a mysql pod and service. Later, we connected it to a wordpress pod, and to access it, we created a wordpress service. Each YAML file has a kind key that defines the type of object it is. For example, in pod files, the kind is set to pod and in service files, it is set to service. There's more… In this example setup, we have only one Node (minion). If you log in to it, you will see all the running containers: $ vagrant ssh minion-1 $ sudo docker ps In this example, we have not configured replication controllers. We can extend this example by creating them. See also Setting up the Vagrant environment at https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/getting-started-guides/vagrant.md The Kubernetes User Guide at https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/user-guide.md The documentation on kube-dns at https://github.com/GoogleCloudPlatform/kubernetes/tree/master/cluster/addons/dns Summary Docker and its ecosystem are evolving at a very high pace, so it is very important to understand the basics and build group up to adopt to new concepts and tools. For more information you can refer to: https://www.packtpub.com/virtualization-and-cloud/beginning-docker-video https://www.packtpub.com/virtualization-and-cloud/getting-started-kubernetes Resources for Article: Further resources on this subject: Integration with Continuous Delivery [Article] Network and Data Management for Containers [Article] CoreOS Networking and Flannel Internals [Article]
Read more
  • 0
  • 0
  • 4038

Packt
17 Feb 2016
72 min read
Save for later

Speaking Java – Your First Game

Packt
17 Feb 2016
72 min read
In this article, we will start writing our very own Java code at the same time as we begin understanding Java syntax. We will learn how to store, retrieve, and manipulate different types of values stored in the memory. We will also look at making decisions and branching the flow of our code based on the values of this data. In this order, we will: Learn some Java syntax and see how it is turned into a running app by the compiler Store data and use it with variables Express yourself in Java with expressions Continue with the math game by asking a question Learn about decisions in Java Continue with the math game by getting and checking the answer (For more resources related to this topic, see here.) Acquiring the preceding Java skills will enable us to build the next two phases of our math game. This game will be able to ask the player a question on multiplication, check the answer and give feedback based on the answer given, as shown in the following screenshot: Java syntax Throughout this book, we will use plain English to discuss some fairly technical things. You will never be asked to read a technical explanation of a Java or Android concept that has not been previously explained in a non-technical way. Occasionally, I might ask or imply that you accept a simplified explanation in order to offer a fuller explanation at a more appropriate time, like the Java class as a black box; however, never will you need to scurry to Google in order to get your head around a big word or a jargon-filled sentence. Having said that, the Java and Android communities are full of people who speak in technical terms and to join in and learn from these communities, you need to understand the terms they use. So the approach this book takes is to learn a concept or appreciate an idea using an entirely plain speaking language, but at the same time, it introduces the jargon as part of the learning. Then, much of the jargon will begin to reveal its usefulness, usually as a way of clarification or keeping the explanation/discussion from becoming longer than it needs to be. The very term, "Java syntax," could be considered technical or jargon. So what is it? The Java syntax is the way we put together the language elements of Java in order to produce code that works in the Java/Dalvik virtual machine. Syntax should also be as clear as possible to a human reader, not least ourselves when we revisit our programs in the future. The Java syntax is a combination of the words we use and the formation of those words into sentence like structures. These Java elements or words are many in number, but when taken in small chunks are almost certainly easier to learn than any human-spoken language. The reason for this is that the Java language and its syntax were specifically designed to be as straightforward as possible. We also have Android Studio on our side which will often let us know if we make a mistake and will even sometimes think ahead and prompt us. I am confident that if you can read, you can learn Java; because learning Java is much easy. What then separates someone who has finished an elementary Java course from an expert programmer? The same things that separate a student of language from a master poet. Mastery of the language comes through practice and further study. The compiler The compiler is what turns our human-readable Java code into another piece of code that can be run in a virtual machine. This is called compiling. The Dalvik virtual machine will run this compiled code when our players tap on our app icon. Besides compiling Java code, the compiler will also check for mistakes. Although we might still have mistakes in our released app, many are stopped discovered at the when our code is compiled. Making code clear with comments As you become more advanced in writing Java programs, the solutions you use to create your programs will become longer and more complicated. Java was designed to manage complexity by having us divide our code into separate chunks, very often across multiple files. Comments are a part of the Java program that do not have any function in the program itself. The compiler ignores them. They serve to help the programmer to document, explain, and clarify their code to make it more understandable to themselves at a later date or to other programmers who might need to use or modify the code. So, a good piece of code will be liberally sprinkled with lines like this: //this is a comment explaining what is going on The preceding comment begins with the two forward slash characters, //. The comment ends at the end of the line. It is known as a single-line comment. So anything on that line is for humans only, whereas anything on the next line (unless it's another comment) needs to be syntactically correct Java code: //I can write anything I like here but this line will cause an error We can use multiple single-line comments: //Below is an important note //I am an important note //We can have as many single line comments like this as we like Single-line comments are also useful if we want to temporarily disable a line of code. We can put // in front of the code and it will not be included in the program. Recall this code, which tells Android to load our menu UI: //setContentView(R.layout.activity_main); In the preceding situation, the menu will not be loaded and the app will have a blank screen when run, as the entire line of code is ignored by the compiler. There is another type of comment in Java—the multiline comment. This is useful for longer comments and also to add things such as copyright information at the top of a code file. Also like the single-line comment, it can be used to temporarily disable code, in this case usually multiple lines. Everything in between the leading /* signs and the ending */ signs is ignored by the compiler. Here are some examples: /* This program was written by a Java expert You can tell I am good at this because my code has so many helpful comments in it. */ There is no limit to the number of lines in a multiline comment. Which type of comment is best to use will depend upon the situation. In this book, I will always explain every line of code explicitly but you will often find liberally sprinkled comments within the code itself that add further explanation, insight or clarification. So it's always a good idea to read all of the code: /* The winning lottery numbers for next Saturday are 9,7,12,34,29,22 But you still want to learn Java? Right? */ All the best Java programmers liberally sprinkle their code with comments. Storing data and using it with variables We can think of a variable as a labeled storage box. They are also like a programmer's window to the memory of the Android device, or whatever device we are programming. Variables can store data in memory (the storage box), ready to be recalled or altered when necessary by using the appropriate label. Computer memory has a highly complex system of addressing that we, fortunately, do not need to interact with in Java. Java variables allow us to make up convenient names for all the data that we want our program to work with; the JVM will handle all the technicalities that interact with the operating system, which in turn, probably through several layers of buck passing, will interact with the hardware. So we can think of our Android device's memory as a huge warehouse. When we assign names to our variables, they are stored in the warehouse, ready when we need them. When we use our variable's name, the device knows exactly what we are referring to. We can then tell it to do things such as "get box A and add it to box C, delete box B," and so on. In a game, we will likely have a variable named as something along the lines of score. It would be this score variable using which we manage anything related to the user's score, such as adding to it, subtracting or perhaps just showing it to the player. Some of the following situations that might arise: The player gets a question right, so add 10 to their existing score The player views their stats screen, so print score on the screen The player gets the best score ever, so make hiScore the same as their current score These are fairly arbitrary examples of names for variables and as long as you don't use any of the characters keywords that Java restricts, you can actually call your variables whatever you like. However, in practice, it is best to adopt a naming convention so that your variable names will be consistent. In this book, we will use a loose convention of variable names starting with a lowercase letter. When there is more than one word in the variable's name, the second word will begin with an uppercase letter. This is called "camel casing." Here are some examples of camel casing: score hiScore playersPersonalBest Before we look at some real Java code with some variables, we need to first look at the types of variables we can create and use. Types of variables It is not hard to imagine that even a simple game will probably have quite a few variables. What if the game has a high score table that remembers the names of the top 10 players? Then we might need variables for each player. And what about the case when a game needs to know if a playable character is dead or alive, or perhaps has any lives/retries left? We might need code that tests for life and then ends the game with a nice blood spurt animation if the playable character is dead. Another common requirement in a computer program, including games, is the right or wrong calculation: true or false. To cover almost these and many other types of information you might want to keep track of, every possibility Java has types. There are many types of variables and, we can also invent our own types or use other people's types. But for now, we will look at the built-in Java types. To be fair, they cover just about every situation we are likely to run into for a while. Some examples are the best way to explain this type of stuff. We have already discussed the hypothetical but highly likely score variable. The sWell score is likely to be a number, so we have to convey this (that the score is a number) to the Java compiler by giving the score an appropriate type. The hypothetical but equally likely playerName will of course, hold the characters that make up the player's name. Jumping ahead a couple of paragraphs, the type that holds a regular number is called an int, and the type that holds name-like data is called a string. And if we try and store a player name, perhaps "Ada Lovelace" in score, which is meant for numbers, we will certainly run into trouble. The compiler says no! Actually, the error would say this: As we can see, Java was designed to make it impossible for such errors to make it to a running program. Did you also spot in the previous screenshot that I had forgotten the semicolon at the end of the line? With this compiler identifying our errors, what could possibly go wrong? Here are the main types in Java. Later, we will see how to start using them: int: This type is used to store integers. It uses 32 pieces (bits) of memory and can therefore store values with a magnitude a little in excess of 2 billion, including negative values. long: As the name hints at, this data types can be used when even larger numbers are required. A long data type uses 64 bits of memory and 2 to the power of 63 is what we can store in this type. If you want to see what that looks like, try this: 9,223,372,036,854,775,807. Perhaps surprisingly, there are uses for long variables but if a smaller variable will do, we should use it so that our program uses less memory. You might be wondering when you might use numbers of this magnitude. The obvious examples would be math or science applications that do complex calculations but another use might be for timing. When you time how long something takes, the Java Date class uses the number of milliseconds since January 1, 1970. The long data type could be useful to subtract a start time from an end time to determine an elapsed time. float: This is for floating-point numbers, that is, numbers where there is precision beyond the decimal point. As the fractional part of a number takes memory space just as the whole number portion, the range of numbers possible in a float is therefore decreased compared to non-floating-point numbers. So, unless our variable will definitely use the extra precision, float would not be our data type of choice. double: When the precision in float is not enough we have double. short: When even an int data type is overkill, the super-skinny short fits into the tiniest of storage boxes, but we can only store around 64,000 values, from -32,768 to 32,767. byte: This is an even smaller storage box than a short type. There is plenty of room for these in memory but a byte can only store values from -128 to 127. boolean: We will be using plenty of Booleans throughout the book. A Boolean variable can be either true or false—nothing else. Perhaps Boolean answer questions such as: Is the player alive? Has a new high score been reached? Are two examples for a Boolean variable enough? char: This stores a single alphanumeric character. It's not going to change anything on its own but it could be useful if we put lots of them together. I have kept this discussion of data types to a practical level that is useful in the context of this book. If you are interested in how a data type's value is stored and why the limits are what they are, visit the Oracle Java tutorials site at http://docs.oracle.com/javase/tutorial/java/nutsandbolts/datatypes.html. Note that you do not need any more information than we have already discussed to continue with this book. As we just learned, each type of data that we might want to store will require a specific amount of memory. So we must let the Java compiler know the type of the variable before we begin to use it. The preceding variables are known as the primitive types. They use predefined amounts of memory and so, using our storage analogy, fit into predefined sizes of the storage box. As the "primitive" label suggests, they are not as sophisticated as the reference types. Reference types You might have noticed that we didn't cover the string variable type that we previously used to introduce the concept of variables. Strings are a special type of variable known as a reference type. They quite simply refer to a place in memory where the storage of the variable begins, but the reference type itself does not define a specific amount of memory. The reason for this is fairly straightforward: we don't always know how much data will need to be stored until the program is actually run. We can think of strings and other reference types as continually expanding and contracting storage boxes. So won't one of these string reference types bump into another variable eventually? If you think about the devices memory as a huge warehouse full of racks of labeled storage boxes, then you can think of the Dalvik virtual machine as a super-efficient forklift truck driver that puts the different types of storage boxes in the most appropriate place. And if it becomes necessary, the virtual machine will quickly move stuff around in a fraction of a second to avoid collisions. It will even incinerate unwanted storage boxes when appropriate. This happens at the same time as constantly unloading new storage boxes of all types and placing them in the best place, for that type of variable. Dalvik tends to keep reference variables in a part of the warehouse that is different from the part for the primitive variables. So strings can be used to store any keyboard character, like a char data type but of almost any length. Anything from a player's name to an entire book can be stored in a single string. There are a couple more reference types we will explore. Arrays are a way to store lots of variables of the same type, ready for quick and efficient access. Think of an array as an aisle in our warehouse with all the variables of a certain type lined up in a precise order. Arrays are reference types, so Dalvik keeps these in the same part of the warehouse as strings. So we know that each type of data that we might want to store will require an amount of memory. Hence, we must let the Java compiler know the type of the variable before we begin to use it. Declaration That's enough of theory. Let's see how we would actually use our variables and types. Remember that each primitive type requires a specific amount of real device memory. This is one of the reasons that the compiler needs to know what type a variable will be of. So we must first declare a variable and its type before we attempt to do anything with it. To declare a variable of type int with the name score, we would type: int score; That's it! Simply state the type, in this case int, then leave a space, and type the name you want to use for this variable. Also note the semicolon on the end of the line as usual to show the compiler that we are done with this line and what follows, if anything, is not part of the declaration. For almost all the other variable types, declaration would occur in the same way. Here are some examples. The variable names are arbitrary. This is like reserving a labeled storage box in the warehouse: long millisecondsElapsed; float gravity; double accurrateGravity; boolean isAlive; char playerInitial; String playerName; Initialization Here, for each type, we initialize a value to the variable. Think about placing a value inside the storage box, as shown in the following code: score = 0; millisecondsElapsed = 1406879842000;//1st Aug 2014 08:57:22 gravity = 1.256; double accurrateGravity =1.256098; isAlive = true; playerInitial = 'C'; playerName = "Charles Babbage"; Notice that the char variable uses single quotes (') around the initialized value while the string uses double quotes ("). We can also combine the declaration and initialization steps. In the following snippet of code, we declare and initialize the same variables as we did previously, but in one step each: int score = 0; long millisecondsElapsed = 1406879842000;//1st Aug 2014 08:57:22 float gravity = 1.256; double accurrateGravity =1.256098; boolean isAlive = true; char playerInitial = 'C'; String playerName = "Charles Babbage"; Whether we declare and initialize separately or together is probably dependent upon the specific situation. The important thing is that we must do both: int a; //The line below attempts to output a to the console Log.i("info", "int a = " + a); The preceding code would cause the following result: Compiler Error: Variable a might not have been initialized There is a significant exception to this rule. Under certain circumstances variables can have default values. Changing variables with operators Of course, in almost any program, we are going to need to do something with these values. Here is a list of perhaps the most common Java operators that allow us to manipulate variables. You do not need to memorize them as we will look at every line of code when we use them for the first time: The assignment operator (=): This makesthe variable to the left of the operator the same as the value to the right. For example, hiScore = score; or score = 100;. The addition operator (+): This adds the values on either side of the operator. It is usually used in conjunction with the assignment operator, such as score = aliensShot + wavesCleared; or score = score + 100;. Notice that it is perfectly acceptable to use the same variable simultaneously on both sides of an operator. The subtraction operator (-): This subtracts the value on the right side of the operator from the value on the left. It is usually used in conjunction with the assignment operator, such as lives = lives - 1; or balance = income - outgoings;. The division operator (/): This divides the number on the left by the number on the right. Again, it is usually used in conjunction with the assignment operator, as shown in fairShare = numSweets / numChildren; or recycledValueOfBlock = originalValue / .9;. The multiplication operator (*): This multiplies variables and numbers, such as answer = 10 * 10; or biggerAnswer = 10 * 10 * 10;. The increment operator (++): This is a really neat way to add 1 to the value of a variable. The myVariable = myVariable + 1; statement is the same as myVariable++;. The decrement operator (--): You guessed it: a really neat way to subtract 1 from something. The myVariable = myVariable -1; statement is the same as myVariable--;. The formal names for these operators are slightly different from the names used here for explanation. For example, the division operator is actually one of the multiplicative operators. But the preceding names are far more useful for the purpose of learning Java and if you used the term "division operator", while conversing with someone from the Java community, they would know exactly what you mean. There are actually many more operators than these in Java. If you are curious about operators there is a complete list of them on the Java website at http://docs.oracle.com/javase/tutorial/java/nutsandbolts/operators.html. All the operators required to complete the projects in this book will be fully explained in this book. The link is provided for the curious among us. Expressing yourself in Java Let's try using some declarations, assignments and operators. When we bundle these elements together into some meaningful syntax, we call it an expression. So let's write a quick app to try some out. Here we will make a little side project so we can play with everything we have learned so far. Instead, we will simply write some Java code and examine its effects by outputting the values of variables to the Android console, called LogCat. We will see exactly how this works by building the simple project and examining the code and the console output: The following is a quick reminder of how to create a new project. Close any currently open projects by navigating to File | Close Project. Click on Start a new Android Studio project. The Create New Project configuration window will appear. Fill in the Application name field and Company Domain with packtpub.com or you could use your own company website name here instead. Now click on the Next button. On the next screen, make sure the Phone and Tablet checkbox has a tick in it. Now we have to choose the earliest version of Android we want to build our app for. Go ahead and play with a few options in the drop-down selector. You will see that the earlier the version we select, the greater is the percentage of devices our app can support. However, the trade-off here is that the earlier the version we select, the less are cutting-edge Android features available in our apps. A good balance is to select API 8: Android 2.2 (Froyo). Go ahead and do that now as shown in the next screenshot. Click on Next. Now select Blank Activity as shown in the next screenshot and click on Next again. On the next screen, simply change Activity Name to MainActivity and click on Finish. To keep our code clear and simple, you can delete the two unneeded methods (onCreateOptionsMenu and onOptionsItemSelected) and their associated @override and @import statements. However, this is not necessary for the example to work. As with all the examples and projects in this book, you can copy or review the code from the download bundle. Just create the project as described previously and paste the code from MainActivity.java file from the download bundle to the MainActivity.java file that was generated when you created the project in Android Studio. Just ensure that the package name is the same as the one you chose when the project was created. However, I strongly recommend going along with the tutorial so that we can learn how to do everything for ourselves. As this app uses the LogCat console to show its output, you should run this app on the emulator only and not on a real Android device. The app will not harm a real device, but you just won't be able to see anything happening. Create a new blank project called Expressions In Java. Now, in the onCreate method just after the line where we use the setContentView method, add this code to declare and initialize some variables: //first we declare and initialize a few variables int a = 10; String b = "Alan Turing"; boolean c = true; Now add the following code. This code simply outputs the value of our variables in a form where we can closely examine them in a minute: //Let's look at how Android 'sees' these variables //by outputting them, one at a time to the console Log.i("info", "a = " + a); Log.i("info", "b = " + b); Log.i("info", "c = " + c); Now let's change our variables using the addition operator and another new operator. See if you can work out the output values for variables a, b, and c before looking at the output and the code explanation: //Now let's make some changes a++; a = a + 10; b = b + " was smarter than the average bear Booboo"; b = b + a; c = (1 + 1 == 3);//1 + 1 is definitely 2! So false. Let's output the values once more in the same way we did in step 3, but this time, the output should be different: //Now to output them all again Log.i("info", "a = " + a); Log.i("info", "b = " + b); Log.i("info", "c = " + c); Run the program on an emulator in the usual way. You can see the output by clicking on the Android tab from our "useful tabs" area below the Project Explorer. Here is the output, with some of the unnecessary formatting stripped off: info? a = 10 info? b = Alan Turing info? c = true info? a = 21 info? b = Alan Turing was smarter than the average bear Booboo21 info? c = false Now let's discuss what happened. In step 2, we declared and initialized three variables: a: This is an int that holds the value 10 b: This is a string that holds the name of an eminent computer scientist. c: This is a Boolean that holds the value false So when we output the values in step 3, it should be no surprise that we get the following: info? a = 10 info? b = Alan Turing info? c = true In step 4, all the fun stuff happens. We add 1 to the value of our int a using the increment operator like this: a++;. Remember that a++ is the same as a = a + 1. We then add 10 to a. Note we are adding 10 to a after having already added 1. So we get this output for a 10 + 1 + 10 operation: info? a = 21 Now let's examine our string, b. We appear to be using the addition operator on our eminent scientist. What is happening is what you could probably guess. We are adding together two strings "Alan Turing" and "was smarter than the average bear Booboo." When you add two strings together it is called concatenating and the + symbol doubles as the concatenation operator. Finally, for our string, we appear to be adding int a to it. This is allowed and the value of a is concatenated to the end of b. info? b = Alan Turing was smarter than the average bear Booboo21 This does not work the other way round; you cannot add a string to an int. This makes sense as there is no logical answer. a = a + b Finally, let's look at the code that changes our Boolean, c, from true to false: c = (1+1=3);. Here, we are assigning to c the value of the expression contained within the brackets. This would be straightforward, but why the double equals (==)? We have jumped ahead of ourselves a little. The double equals sign is another operator called the comparison operator. So we are really asking, does 1+1 equal 3? Clearly the answer is false. You might ask, "why use == instead of =?" Simply to make it clear to the compiler when we mean to assign and when we mean to compare. Inadvertently using = instead of == is a very common error. The assignment operator (=) assigns the value on the right to the value on the left, while the comparison operator (==) compares the values on either side. The compiler will warn us with an error when we do this but at first glance you might swear the compiler is wrong. Now let's use everything we know and a bit more to make our Math game project. Math game – asking a question Now that we have all that knowledge under our belts, we can use it to improve our math game. First, we will create a new Android activity to be the actual game screen as opposed to the start menu screen. We will then use the UI designer to lay out a simple game screen so that we can use our Java skills with variables, types, declaration, initialization, operators, and expressions to make our math game generate a question for the player. We can then link the start menu and game screens together with a push button. If you want to save typing and just review the finished project, you can use the code downloaded from the Packt website. If you have any trouble getting any of the code to work, you can review, compare, or copy and paste the code from the already completed code provided in the download bundle. The completed code is in the following files that correspond to the filenames we will be using in this tutorial: Chapter 3/MathGameChapter3a/java/MainActivity.java Chapter 3/MathGameChapter3a/java/GameActivity.java Chapter 3/MathGameChapter3a/layout/activity_main.xml Chapter 3/MathGameChapter3a/layout/activity_game.xml As usual, I recommend following this tutorial to see how we can create all of the code for ourselves. Creating the new game activity We will first need to create a new Java file for the game activity code and a related layout file to hold the game Activity UI. Run Android Studio and select your Math Game Chapter 2 project. It might have been opened by default. Now we will create the new Android Activity that will contain the actual game screen, which will run when the player taps the Play button on our main menu screen. To create a new activity, we know need another layout file and another Java file. Fortunately Android Studio will help us do this. To get started with creating all the files we need for a new activity, right-click on the src folder in the Project Explorer and then go to New | Activity. Now click on Blank Activity and then on Next. We now need to tell Android Studio a little bit about our new activity by entering information in the above dialog box. Change the Activity Name field to GameActivity. Notice how the Layout Name field is automatically changed for us to activity_game and the Title field is automatically changed to GameActivity. Click on Finish. Android Studio has created two files for us and has also registered our new activity in a manifest file, so we don't need to concern ourselves with it. If you look at the tabs at the top of the editor window, you will see that GameActivity.java has been opened up ready for us to edit, as shown in the following screenshot: Ensure that GameActivity.java is active in the editor window by clicking on the GameActivity.java tab shown previously. Android overrides some methods for us by default, and that most of them were not necessary. Here again, we can see the code that is unnecessary. If we remove it, then it will make our working environment simpler and cleaner. To avoid this here, we will simply use the code from MainActivity.java as a template for GameActivity.java. We can then make some minor changes. Click on the MainActivity.java tab in the editor window. Highlight all of the code in the editor window using Ctrl + A on the keyboard. Now copy all of the code in the editor window using the Ctrl + C on the keyboard. Now click on the GameActivity.java tab. Highlight all of the code in the editor window using Ctrl + A on the keyboard. Now paste the copied code and overwrite the currently highlighted code using Ctrl + V on the keyboard. Notice that there is an error in our code denoted by the red underlining as shown in the following screenshot. This is because we pasted the code referring to MainActivity in our file which is called GameActivity. Simply change the text MainActivity to GameActivity and the error will disappear. Take a moment to see if you can work out what the other minor change is necessary, before I tell you. Remember that setContentView loads our UI design. Well what we need to do is change setContentView to load the new design (that we will build next) instead of the home screen design. Change setContentView(R.layout.activity_main); to setContentView(R.layout.activity_game);. Save your work and we are ready to move on. Note the Project Explorer where Android Studio puts the two new files it created for us. I have highlighted two folders in the next screenshot. In future, I will simply refer to them as our java code folder or layout files folder. You might wonder why we didn't simply copy and paste the MainActivity.java file to begin with and saved going through the process of creating a new activity? The reason is that Android Studio does things behind the scenes. Firstly, it makes the layout template for us. It also registers the new Activity for use through a file we will see later, called AndroidManifest.xml. This is necessary for the new activity to be able to work in the first place. All things considered, the way we did it is probably the quickest. The code at this stage is exactly the same as the code for the home menu screen. We state the package name and import some useful classes provided by Android: package com.packtpub.mathgamechapter3a.mathgamechapter3a; import android.app.Activity; import android.os.Bundle; We create a new activity, this time called GameActivity: public class GameActivity extends Activity { Then we override the onCreate method and use the setContentView method to set our UI design as the contents of the player's screen. Currently, however, this UI is empty: super.onCreate(savedInstanceState); setContentView(R.layout.activity_main); We can now think about the layout of our actual game screen. Laying out the game screen UI As we know, our math game will ask questions and offer the player some multiple choices to choose answers from. There are lots of extra features we could add, such as difficulty levels, high scores, and much more. But for now, let's just stick to asking a simple, predefined question and offering a choice of three predefined possible answers. Keeping the UI design to the bare minimum suggests a layout. Our target UI will look somewhat like this: The layout is hopefully self-explanatory, but let's ensure that we are really clear; when we come to building this layout in Android Studio, the section in the mock-up that displays 2 x 2 is the question and will be made up of three text views (both numbers, and the = sign is also a separate view). Finally, the three options for the answer are made up of Button layout elements. We used all of these UI elements, but this time, as we are going to be controlling them using our Java code, there are a few extra things we need to do to them. So let's go through it step by step: Open the file that will hold our game UI in the editor window. Do this by double-clicking on activity_game.xml. This is located in our UI layout folder which can be found in the project explorer. Delete the Hello World TextView, as it is not required. Find the Large Text element on the palette. It can be found under the Widgets section. Drag three elements onto the UI design area and arrange them near the top of the design as shown in the next screenshot. It does not have to be exact; just ensure that they are in a row and not overlapping, as shown in the following screenshot: Notice in the Component Tree window that each of the three TextViews has been assigned a name automatically by Android Studio. They are textView , textView2 and textView3: Android Studio refers to these element names as an id. This is an important concept that we will be making use of. So to confirm this, select any one of the textViews by clicking on its name (id), either in the component tree as shown in the preceding screenshot or directly on it in the UI designer shown previously. Now look at the Properties window and find the id property. You might need to scroll a little to do this: Notice that the value for the id property is textView. It is this id that we will use to interact with our UI from our Java code. So we want to change all the IDs of our TextViews to something useful and easy to remember. If you look back at our design, you will see that the UI element with the textView id is going to hold the number for the first part of our math question. So change the id to textPartA. Notice the lowercase t in text, the uppercase P in Part, and the uppercase A. You can use any combination of cases and you can actually name the IDs anything you like. But just as with naming conventions with Java variables, sticking to conventions here will make things less error-prone as our program gets more complicated. Now select textView2 and change id to textOperator. Select the element currently with id textView3 and change it to textPartB. This TextView will hold the later part of our question. Now add another Large Text from the palette. Place it after the row of the three TextViews that we have just been editing. This Large Text will simply hold our equals to sign and there is no plan to ever change it. So we don't need to interact with it in our Java code. We don't even need to concern ourselves with changing the ID or knowing what it is. If this situation changed, we could always come back at a later time and edit its ID. However, this new TextView currently displays Large Text and we want it to display an equals to sign. So in the Properties window, find the text property and enter the value =. We have changed the text property, and you might also like to change the text property for textPartA, textPartB, and textOperator. This is not absolutely essential because we will soon see how we can change it via our Java code; however, if we change the text property to something more appropriate, then our UI designer will look more like it will when the game runs on a real device. So change the text property of textPartA to 2, textPartB to 2, and textOperator to x. Your UI design and Component tree should now look like this: For the buttons to contain our multiple choice answers, drag three buttons in a row, below the = sign. Line them up neatly like our target design. Now, just as we did for the TextViews, find the id properties of each button, and from left to right, change the id properties to buttonChoice1, buttonChoice2, and buttonChoice3. Why not enter some arbitrary numbers for the text property of each button so that the designer more accurately reflects what our game will look like, just as we did for our other TextViews? Again, this is not absolutely essential as our Java code will control the button appearance. We are now actually ready to move on. But you probably agree that the UI elements look a little lost. It would look better if the buttons and text were bigger. All we need to do is adjust the textSize property for each TextView and for each Button. Then, we just need to find the textSize property for each element and enter a number with the sp syntax. If you want your design to look just like our target design from earlier, enter 70sp for each of the TextView textSize properties and 40sp for each of the Buttons textSize properties. When you run the game on your real device, you might want to come back and adjust the sizes up or down a bit. But we have a bit more to do before we can actually try out our game. Save the project and then we can move on. As before, we have built our UI. This time, however, we have given all the important parts of our UI a unique, useful, and easy to identify ID. As we will see we are now able to communicate with our UI through our Java code. Coding a question in Java With our current knowledge of Java, we are not yet able to complete our math game but we can make a significant start. We will look at how we can ask the player a question and offer them some multiple choice answers (one correct and two incorrect). At this stage, we know enough of Java to declare and initialize some variables that will hold the parts of our question. For example, if we want to ask the times tables question 2 x 2, we could have the following variable initializations to hold the values for each part of the question: int partA = 2; int partB = 2; The preceding code declares and initializes two variables of the int type, each to the value of 2. We use int because we will not be dealing with any decimal fractions. Remember that the variable names are arbitrary and were just chosen because they seemed appropriate. Clearly, any math game worth downloading is going to need to ask more varied and advanced questions than 2 x 2, but it is a start. Now we know that our math game will offer multiple choices as answers. So, we need a variable for the correct answer and two variables for two incorrect answers. Take a look at these combined declarations and initializations: int correctAnswer = partA * partB; int wrongAnswer1 = correctAnswer - 1; int wrongAnswer2 = correctAnswer + 1; Note that the initialization of the variables for the wrong answers depends on the value of the correct answer, and the variables for the wrong answers are initialized after initializing the correctAnswer variable. Now we need to put these values, held in our variables, into the appropriate elements on our UI. The question variables (partA and partB) need to be displayed in our UI elements, textPartA and textPartB, and the answer variables (correctAnswer, wrongAnswer1, and wrongAnswer2) need to be displayed in our UI elements with the following IDs: buttonChoice1, buttonChoice2, and buttonChoice3. We will see how we do this in the next step-by-step tutorial. We will also implement the variable declaration and initialization code that we discussed a moment ago: First, open GameActivity.java in the editor window. Remember that you can do this by double-clicking on GameActivity in our java folder or clicking on its tab above the editor window if GameActivity.java is already open. All of our code will go into the onCreate method. It will go after the setContentView(R.layout.activity_game); line but before the closing curly brace } of the onCreate method. Perhaps, it's a good idea to leave a blank line for clarity and a nice explanatory comment as shown in the following code. We can see the entire onCreate method as it stands after the latest amendments. The parts in bold are what you need to add. Feel free to add helpful comments like mine if you wish: @Override     protected void onCreate(Bundle savedInstanceState) {         super.onCreate(savedInstanceState);         //The next line loads our UI design to the screen         setContentView(R.layout.activity_game);           //Here we initialize all our variables         int partA = 9;         int partB = 9;         int correctAnswer = partA * partB;         int wrongAnswer1 = correctAnswer - 1;         int wrongAnswer2 = correctAnswer + 1;       }//onCreate ends here Now we need to add the values contained within the variables to the TextView and Button of our UI. But first, we need to get access to the UI elements we created. We do that by creating a variable of the appropriate class and linking it via the ID property of the appropriate UI element. We already know the class of our UI elements: TextView and Button. Here is the code that creates our special class variables for each of the necessary UI elements. Take a close look at the code, but don't worry if you don't understand all of it now. We will dissect the code in detail once everything is working. Enter the code immediately after the code entered in the previous step. You can leave a blank line for clarity if you wish. Just before you proceed, note that at two points while typing in this code, you will be prompted to import another class. Go ahead and do so on both occasions: /*Here we get a working object based on either the button or TextView class and base as well as link our new   objects directly to the appropriate UI elements that we created previously*/   TextView textObjectPartA =   (TextView)findViewById(R.id.textPartA);   TextView textObjectPartB =   (TextView)findViewById(R.id.textPartB);   Button buttonObjectChoice1 =   (Button)findViewById(R.id.buttonChoice1);   Button buttonObjectChoice2 =   (Button)findViewById(R.id.buttonChoice2);   Button buttonObjectChoice3 =   (Button)findViewById(R.id.buttonChoice3); In the preceding code, if you read the multiline comment, you will see that I used the term object. When we create a variable type based on a class, we call it an object. Once we have an object of a class, we can do anything that that class was designed to do. Now we have five new objects linked to the elements of our UI that we need to manipulate. What precisely are we going to do with them? We need to display the values of our variables in the text of the UI elements. We can use the objects we just created combined with a method provided by the class, and use our variables as values for that text. As usual, we will dissect this code further at the end of this tutorial. Here is the code to enter directly after the code in the previous step. Try and work out what is going on before we look at it together: //Now we use the setText method of the class on our objects //to show our variable values on the UI elements. //Just like when we output to the console in the exercise - //Expressions in Java, only now we use setText method //to put the values in our variables onto the actual UI. textObjectPartA.setText("" + partA); textObjectPartB.setText("" + partB);   //which button receives which answer, at this stage is arbitrary.   buttonObjectChoice1.setText("" + correctAnswer); buttonObjectChoice2.setText("" + wrongAnswer1); buttonObjectChoice3.setText("" + wrongAnswer2); Save your work. If you play with the assignment values for partA and partB, you can make them whatever you like and the game adjusts the answers accordingly. Obviously, we shouldn't need to reprogram our game each time we want a new question and we will solve that problem soon. All we need to do now is link the game section we have just made to the start screen menu. We will do that in the next tutorial. Now lets explore the trickier and newer parts of our code in more detail. In step 2, we declared and initialized the variables required so far: //Here we initialize all our variables int partA = 2; int partB = 2; int correctAnswer = partA * partB; int wrongAnswer1 = correctAnswer - 1; int wrongAnswer2 = correctAnswer + 1; Then in step 3, we got a reference to our UI design through our Java code. For the TextViews, it was done like this: TextView textObjectPartA = (TextView)findViewById(R.id.textPartA); For each of the buttons, a reference to our UI design was obtained like this: Button buttonObjectChoice1 =   Button)findViewById(R.id.buttonChoice1); In step 4, we did something new. We used a the setText method to show the values of our variables on our UI elements (TextView and Button) to the player. Let's break down one line completely to see how it works. Here is the code that shows the correctAnswer variable being displayed on buttonObjectChoice1. buttonObjectChoice1.setText("" + correctAnswer); By typing buttonObjectChoice1 and adding a period, as shown in the following line of code, we have access to all the preprogrammed methods of that object's class type that are provided by Android: The power of Button and the Android API There are actually lots of methods that we can perform on an object of the Button type. If you are feeling brave, try this to get a feeling of just how much functionality there is in Android. Type the following code: buttonObjectChoice1 Be sure to type the period on the end. Android Studio will pop up a list of possible methods to use on this object. Scroll through the list and get a feel of the number and variety of options: If a mere button can do all of this, think of the possibilities for our games once we have mastered all the classes contained in Android. A collection of classes designed to be used by others is collectively known as an Application Programmers Interface (API). Welcome to the Android API! In this case, we just want to set the button's text. So, we use setText and concatenate the value stored in our correctAnswer variable to the end of an empty string, like this: setText("" + correctAnswer); We do this for each of the UI elements we require to show our variables. Playing with autocomplete If you tried the previous tip, The power of Button and the Android API, and explored the methods available for objects of the Button type, you will already have some insight into autocomplete. Note that as you type, Android Studio is constantly making suggestions for what you might like to type next. If you pay attention to this, you can save a lot of time. Simply select the correct code completion statement that is suggested and press Enter. You can even see how much time you saved by selecting Help | Productivity Guide from the menu bar. Here you will see statistics for every aspect of code completion and more. Here are a few entries from mine: As you can see, if you get used to using shortcuts early on, you can save a lot of time in the long run. Linking our game from the main menu At the moment, if we run the app, we have no way for the player to actually arrive at our new game activity. We want the game activity to run when the player clicks on the Play button on the main MainActivity UI. Here is what we need to do to make that happen: Open the file activity_main.xml, either by double-clicking on it in the project explorer or by clicking its tab in the editor window. Now, just like we did when building the game UI, assign an ID to the Play button. As a reminder, click on the Play button either on the UI design or in the component tree. Find the id property in the Properties window. Assign the buttonPlay value to it. We can now make this button do stuff by referring to it in our Java code. Open the file MainActivity.java, either by double-clicking on it in the project explorer or clicking on its tab in the editor window. In our onCreate method, just after the line where we setContentView, add the following highlighted line of code: setContentView(R.layout.activity_main); Button buttonPlay = (Button)findViewById(R.id.buttonPlay); We will dissect this code in detail once we have got this working. Basically we are making a connection to the Play button by creating a reference variable to a Button object. Notice that both words are highlighted in red indicating an error. Just as before, we need to import the Button class to make this code work. Use the Alt + Enter keyboard combination. Now click on Import class from the popped-up list of options. This will automatically add the required import directive at the top of our MainActivity.java file. Now for something new. We will give the button the ability to listen to the user clicking on it. Type this immediately after the last line of code we entered: buttonPlay.setOnClickListener(this); Notice how the this keyword is highlighted in red indicating an error. Setting that aside, we need to make a modification to our code now in order to allow the use of an interface that is a special code element that allows us to add a functionality, such as listening for button clicks. Edit the line as follows. When prompted to import another class, click on OK: public class MainActivity extends Activity { to public class MainActivity extends Activity implements View.OnClickListener{ Now we have the entire line underlined in red. This indicates an error but it's where we should be at this point. We mentioned that by adding implements View.OnClickListener, we have implemented an interface. We can think of this like a class that we can use but with extra rules. The rules of the OnClickListener interface state that we must implement/use one of its methods. Notice that until now, we have optionally overridden/used methods as and when they have suited us. If we wish to use the functionality this interface provides, namely listening for button presses, then we have to add/implement the onClick method. This is how we do it. Notice the opening curly brace, {, and the closing curly brace, }. These denote the start and end of the method. Notice that the method is empty and it doesn't do anything, but an empty method is enough to comply with the rules of the OnClickListener interface, and the red line indicating an that our code has an error has gone. Make sure that you type the following code, outside the closing curly brace (}) of the onCreate method but inside the closing curly brace of our MainActivity class: @Override     public void onClick(View view) {               } Notice that we have an empty line between { and } of the onClick method. We can now add code in here to make the button actually do something. Type the following highlighted code between { and } of onClick: @Override     public void onClick(View view) {         Intent i;         i = new Intent(this, GameActivity.class);         startActivity(i);     } OK, so that code is a bit of a mouthful to comprehend all at once. See if you can guess what is happening. The clue is in the method named startActivity and the hopefully familiar term, GameActivity. Notice that we are assigning something to i. We will quickly get our app working and then diagnose the code in full. Notice that we have an error: all instances of the word Intent are red. We can solve this by importing the classes required to make Intent work. As before press Alt + Enter. Run the game in the emulator or on your device. Our app will now work. This is what the new game screen looks like after pressing Play on the menu screen: Almost every part of our code has changed a little and we have added a lot to it as well. Let's go over the contents of MainActivity.java and look at it line by line. For context, here it is in full: package com.packtpub.mathgamechapter3a.mathgamechapter3a;   import android.app.Activity; import android.content.Intent; import android.os.Bundle; import android.view.View; import android.widget.Button;     public class MainActivity extends Activity implements View.OnClickListener{       @Override     protected void onCreate(Bundle savedInstanceState) {         super.onCreate(savedInstanceState);         setContentView(R.layout.activity_main);         final Button buttonPlay =           (Button)findViewById(R.id.buttonPlay);         buttonPlay.setOnClickListener(this);     }       @Override     public void onClick(View view) {         Intent i;         i = new Intent(this, GameActivity.class);         startActivity(i);     }   } We have seen much of this code before, but let's just go over it a chunk at a time before moving on so that it is absolutely clear. The code works like this: package com.packtpub.mathgamechapter3a.mathgamechapter3a; import android.app.Activity; import android.content.Intent; import android.os.Bundle; import android.view.View; import android.widget.Button; You would probably remember that this first block of code defines what our package is called and makes available all the Android API stuff we need for Button, TextView, and Activity. From our MainActivity.java file, we have this: public class MainActivity extends Activity implements View.OnClickListener{ Our MainActivity declaration with our new bit of code implements View.OnClickListener that gives us the ability to detect button clicks. Next in our code is this: @Override     protected void onCreate(Bundle savedInstanceState) {         super.onCreate(savedInstanceState);         setContentView(R.layout.activity_main); It is at the start of our onCreate method where we first ask the hidden code of onCreate to do its stuff using super.onCreate(savedInstanceState);. Then we set our UI to the screen with setContentView(R.layout.activity_main);. Next, we get a reference to our button with an ID of buttonPlay: Button buttonPlay = (Button)findViewById(R.id.buttonPlay); buttonPlay.setOnClickListener(this); Finally, our onClick method uses the Intent class to send the player to our GameActivity class and the related UI when the user clicks on the Play button: @Override     public void onClick(View view) {         Intent i;         i = new Intent(this, GameActivity.class);         startActivity(i);     } If you run the app, you will notice that we can now click on the Play button and our math game will ask us a question. Of course, we can't answer it yet. Although we have very briefly looked at how to deal with button presses, we need to learn more of Java in order to intelligently react to them. We will also reveal how to write code to handle presses from several buttons. This will be necessary to receive input from our multiple-choice-centric game_activity UI. Decisions in Java We can now summon enough of Java prowess to ask a question but a real math game must obviously do much more than this. We need to capture the player's answer, and we are nearly there with that—we can detect button presses. From there, we need to be able to decide whether their answer is right or wrong. Then, based on this decision, we have to choose an appropriate course of action. Let's leave the math game aside for now and look at how Java might help us by learning some more fundamentals and syntax of the Java language. More operators Let's look at some more operators: we can already add (+), take away (-), multiply (*), divide (/), assign (=), increment (++), compare (==), and decrement (--) with operators. Let's introduce some more super-useful operators, and then we will go straight to actually understanding how to use them in Java. Don't worry about memorizing every operator given here. Glance at them and their explanations. There, we will put some operators to use and they will become much clearer as we see a few examples of what they allow us to do. They are presented here in a list just to make the variety and scope of operators plain from the start. The list will also be more convenient to refer back to when not intermingled with the discussion about implementation that follows it. ==: This is a comparison operator we saw this very briefly before. It tests for equality and is either true or false. An expression like (10 == 9);, for example, is false. !: The logical NOT operator. The expression, ! (2+2==5).), is true because 2+2 is NOT 5. !=: This is another comparison operator which tests if something is NOT equal. For example, the expression, (10 != 9);), is true, that is, 10 is not equal to 9. >: This is another comparison operator which tests if something is greater than something else. The expression, (10 > 9);), is true. There are a few more comparison operators as well. <: You guessed it. This tests whether the value to the left is less than the value to the right or not. The expression, (10 < 9);, is false. >=: This operator tests whether one value is greater than or equal to the other, and if either is true, the result is true. For example, the expression, (10 >= 9);, is true. The expression, (10 >= 10);, is also true. <=: Like the preceding operator, this operator tests for two conditions but this time, less than and equal to. The expression, (10 <= 9);, is false. The expression, (10 <= 10);, is true. &&: This operator is known as logical AND. It tests two or more separate parts of an expression and all parts must be true in order for the result to be true. Logical AND is usually used in conjunction with the other operators to build more complex tests. The expression, ((10 > 9) && (10 < 11));, is true because both parts are true. The expression, ((10 > 9) && (10 < 9));, is false because only one part of the expression is true and the other is false. ||: This operator is called logical OR. It is just like logical AND except that only one of two or more parts of an expression need to be true for the expression to be true. Let's look at the last example we used but replace the && sign with ||. The expression, ((10 > 9) || (10 < 9));, is now true because one part of the expression is true. All of these operators are virtually useless without a way of properly using them to make real decisions that affect real variables and code. Let's look at how to make decisions in Java. Decision 1 – If they come over the bridge, shoot them As we saw, operators serve hardly any purpose on their own but it was probably useful to see just a part of the wide and varied range available to us. Now, when we look at putting the most common operator, ==, to use, we can start to see the powerful yet fine control that operators offer us. Let's make the previous examples less abstract using the Java if keyword, a few conditional operators, and a small story: in use with some code and a made up military situation that will hopefully make the following examples less abstract. The captain is dying and, knowing that his remaining subordinates are not very experienced, he decides to write a Java program to convey his last orders after he has died. The troops must hold one side of a bridge while awaiting reinforcements. The first command the captain wants to make sure his troops understand is this: If they come over the bridge, shoot them. So how do we simulate this situation in Java? We need a Boolean variable isComingOverBridge. The next bit of code assumes that the isComingOverBridge variable has been declared and initialized. We can then use it like this: if(isComingOverBridge){   //Shoot them } If the isComingOverBridge Boolean is true, the code inside the opening and closing curly braces will run. If not, the program continues after the if block without running it. Decision 2 – Else, do this The captain also wants to tell his troops what to do (stay put) if the enemy is not coming over the bridge. Now we introduce another Java keyword, else. When we want to explicitly do something and the if block does not evaluate to true, we can use else. For example, to tell the troops to stay put if the enemy is not coming over the bridge, we use else: if(isComingOverBridge){   //Shoot them }else{   //Hold position } The captain then realized that the problem wasn't as simple as he first thought. What if the enemy comes over the bridge and has more troops? His squad will be overrun. So, he came up with this code (we'll use some variables as well this time): boolean isComingOverTheBridge; int enemyTroops; int friendlyTroops; //Code that initializes the above variables one way or another   //Now the if if(isComingOverTheBridge && friendlyTroops > enemyTroops){   //shoot them }else if(isComingOverTheBridge && friendlyTroops < enemyTroops) {   //blow the bridge }else{   //Hold position } Finally, the captain's last concern was that if the enemy came over the bridge waving the white flag of surrender and were promptly slaughtered, then his men would end up as war criminals. The Java code needed was obvious. Using the wavingWhiteFlag Boolean variable he wrote this test: if (wavingWhiteFlag){   //Take prisoners } But where to put this code was less clear. In the end, the captain opted for the following nested solution and changing the test for wavingWhiteFlag to logical NOT, like this: if (!wavingWhiteFlag){//not surrendering so check everything else   if(isComingOverTheBridge && friendlyTroops > enemyTroops){     //shoot them   }else if(isComingOverTheBridge && friendlyTroops <   enemyTroops) {     //blow the bridge   } }else{//this is the else for our first if   //Take prisoners { //Holding position This demonstrates that we can nest if and else statements inside of one another to create even deeper decisions. We could go on making more and more complicated decisions but what we have seen is more than sufficient as an introduction. Take the time to reread this if anything is unclear. It is also important to point out that very often, there are two or more ways to arrive at the solution. The right way will usually be the way that solves the problem in the clearest and simplest manner. Switching to make decisions We have seen the vast and virtually limitless possibilities of combining the Java operators with if and else statements. But sometimes a decision in Java can be better made in other ways. When we have to make a decision based on a clear list of possibilities that doesn't involve complex combinations, then switch is usually the way to go. We start a switch decision like this: switch(argument){   } In the previous example, an argument could be an expression or a variable. Then within the curly braces, we can make decisions based on the argument with case and break elements: case x:   //code to for x   break;   case y:   //code for y   break; You can see that in the previous example, each case states a possible result and each break denotes the end of that case as well as the point at which no further case statements should be evaluated. The first break encountered takes us out of the switch block to proceed with the next line of code. We can also use default without a value to run some code if none of the case statements evaluate to true, like this: default://Look no value   //Do something here if no other case statements are true break; Supposing we are writing an old-fashioned text adventure game—the kind of game where the player types commands such as "Go East", "Go West", "Take Sword", and so on. In this case, switch could handle that situation like this example code and we could use default to handle the case of the player typing a command that is not specifically handled: //get input from user in a String variable called command switch(command){     case "Go East":":   //code to go east   break;     case "Go West":   //code to go west   break;   case "Take sword":   //code to take the sword   break;     //more possible cases     default:   //Sorry I don't understand your command   break;   } We will use switch so that our onClick method can handle the different multiple-choice buttons of our math game. Java has even more operators than we have covered here. We have looked at all the operators we are going to need in this book and probably the most used in general. If you want the complete lowdown on operators, take a look at the official Java documentation at http://docs.oracle.com/javase/tutorial/java/nutsandbolts/operators.html. Math game – getting and checking the answer Here we will detect the right or wrong answer and provide a pop-up message to the player. Our Java is getting quite good now, so let's dive in and add these features. I will explain things as we go and then, as usual, dissect the code thoroughly at the end. The already completed code is in the download bundle, in the following files that correspond to the filenames we will create/autogenerate in Android Studio in a moment: Chapter 3/MathGameChapter3b/java/MainActivity.java Chapter 3/MathGameChapter3b/java/GameActivity.java Chapter 3/MathGameChapter3b/layout/activity_main.xml Chapter 3/MathGameChapter3b/layout/activity_game.xml As usual, I recommend following this tutorial step by step to see how we can create all of the code for ourselves. Open the GameActivity.java file visible in the editor window. Now we need to add the click detection functionality to our GameActivity, just as we did for our MainActivity. However, we will go a little further than the last time. So let's do it step by step as if it is totally new. Once again, we will give the buttons the ability to listen to the user clicking on them. Type this immediately after the last line of code we entered in the onCreate method but before the closing }. This time of course, we need to add some code to listen to three buttons: buttonObjectChoice1.setOnClickListener(this); buttonObjectChoice2.setOnClickListener(this); buttonObjectChoice3.setOnClickListener(this); Notice how the this keyword is highlighted in red indicating an error. Again, we need to make a modification to our code in order to allow the use of an interface, the special code element that allows us to add functionalities such as listening to button clicks. Edit the line as follows. When prompted to import another class, click on OK. Consider this line of code: public class GameActivity extends Activity { Change it to the following line: public class GameActivity extends Activity implements View.OnClickListener{   Now we have the entire preceding line underlined in red. This indicates an error but it is where we should be at this point. We mentioned that by adding implements View.OnClickListener, we have implemented an interface. We can think of this like a class that we can use, but with extra rules. One of the rules of the OnClickListener interface is that we must implement one of its methods, as you might remember. If we wish to use the useful functionality this interface provides (listening to button presses). Now we will add the onClick method. Type the following code. Notice the opening curly brace, {, and the closing curly brace, }. These denote the start and end of the method. Notice that the method is empty; it doesn't do anything but an empty method is enough to comply with the rules of the OnClickListener interface and the red line that indicated an error has gone. Make sure that you type the following code outside the closing curly brace (}) of the onCreate method but inside the closing curly brace of our MainActivity class: @Override     public void onClick(View view) {     } Notice that we have an empty line between the { and } braces of our onClick method. We can now put some code in here to make the buttons actually do something. Type the following in between { and } of onClick. This is where things get different from our code in MainActivity. We need to differentiate between the three possible buttons that could be pressed. We will do this with the switch statement that we discussed earlier. Look at the case criteria; they should look familiar. Here is the code that uses the switch statements: switch (view.getId()) {               case R.id.buttonChoice1:             //button 1 stuff goes here                 break;               case R.id.buttonChoice2:             //button 2 stuff goes here                 break;               case R.id.buttonChoice3:            //button 3 stuff goes here                 break;           }   Each case element handles a different button. For each button case, we need to get the value stored in the button that was just pressed and see if it matches our correctAnswer variable. If it does, we must tell the player they got it right, and if not, we must tell them they got it wrong. However, there is still one problem we have to solve. The onClick method is separate from the onCreate method and the Button objects. In fact, all the variables are declared in the onCreate method. If you try typing the code from step 9 now, you will get lots of errors. We need to make all the variables that we need in onClick available in onClick. To do this, we will move their declarations from above the onCreate method to just below the opening { of GameActivity. This means that these variables become variables of the GameActivity class and can be seen anywhere within GameActivity. Declare the following variables like this: int correctAnswer; Button buttonObjectChoice1; Button buttonObjectChoice2; Button buttonObjectChoice3; Now change the initialization of these variables within onCreate as follows. The actual parts of code that need to be changed are highlighted. The rest is shown for context: //Here we initialize all our variables int partA = 9; int partB = 9; correctAnswer = partA * partB; int wrongAnswer1 = correctAnswer - 1; int wrongAnswer2 = correctAnswer + 1; and TextView textObjectPartA =   (TextView)findViewById(R.id.textPartA);   TextView textObjectPartB =   (TextView)findViewById(R.id.textPartB);   buttonObjectChoice1 = (Button)findViewById(R.id.buttonChoice1);         buttonObjectChoice2 = (Button)findViewById(R.id.buttonChoice2);         buttonObjectChoice3 = (Button)findViewById(R.id.buttonChoice3);   Here is the top of our onClick method as well as the first case statement for our onClick method: @Override     public void onClick(View view) {         //declare a new int to be used in all the cases         int answerGiven=0;         switch (view.getId()) {               case R.id.buttonChoice1:             //initialize a new int with the value contained in buttonObjectChoice1             //Remember we put it there ourselves previously                 answerGiven = Integer.parseInt("" +                     buttonObjectChoice1.getText());                   //is it the right answer?                 if(answerGiven==correctAnswer) {//yay it's the right answer                     Toast.makeText(getApplicationContext(),                       "Well done!",                       Toast.LENGTH_LONG).show();                 }else{//uh oh!                     Toast.makeText(getApplicationContext(),"Sorry that's     wrong", Toast.LENGTH_LONG).show();                 }                 break;   Here are the rest of the case statements that do the same steps as the code in the previous step except handling the last two buttons. Enter the following code after the code entered in the previous step:   case R.id.buttonChoice2:                 //same as previous case but using the next button                 answerGiven = Integer.parseInt("" +                   buttonObjectChoice2.getText());                 if(answerGiven==correctAnswer) {                     Toast.makeText(getApplicationContext(), "Well done!", Toast.LENGTH_LONG).show();                 }else{                     Toast.makeText(getApplicationContext(),"Sorry that's wrong", Toast.LENGTH_LONG).show();                 }                 break;               case R.id.buttonChoice3:                 //same as previous case but using the next button                 answerGiven = Integer.parseInt("" +                     buttonObjectChoice3.getText());                 if(answerGiven==correctAnswer) {                     Toast.makeText(getApplicationContext(), "Well done!", Toast.LENGTH_LONG).show();                 }else{                     Toast.makeText(getApplicationContext(),"Sorry that's wrong", Toast.LENGTH_LONG).show();                 }                 break;           } Run the program, and then we will look at the code carefully, especially that odd-looking Toast thing. Here is what happens when we click on the leftmost button: This is how we did it: In steps 1 through 6, we set up handling for our multi-choice buttons, including adding the ability to listen to clicks using the onClick method and a switch block to handle decisions depending on the button pressed. In steps 7 and 8, we had to alter our code to make our variables available in the onClick method. We did this by making them member variables of our GameActivity class. When we make a variable a member of a class, we call it a field. In steps 9 and 10, we implemented the code that actually does the work in our switch statement in onClick. Let's take a line-by-line look at the code that runs when button1 is pressed. case R.id.buttonChoice1: First, the case statement is true when the button with an id of buttonChoice1 is pressed. Then the next line of code to execute is this: answerGiven = Integer.parseInt(""+ buttonObjectChoice1.getText()); The preceding line gets the value on the button using two methods. First, getText gets the number as a string and then Integer.parseInt converts it to an int. The value is stored in our answerGiven variable. The following code executes next: if(answerGiven==correctAnswer) {//yay it's the right answer   Toast.makeText(getApplicationContext(), "Well done!",     Toast.LENGTH_LONG).show(); }else{//uh oh!     Toast.makeText(getApplicationContext(),"Sorry that's wrong",       Toast.LENGTH_LONG).show();                 } The if statement tests to see if the answerGiven variable is the same as correctAnswer using the == operator. If so, the makeText method of the Toast object is used to display a congratulatory message. If the values of the two variables are not the same, the message displayed is a bit more negative one. The Toast line of code is possibly the most evil thing we have seen thus far. It looks exceptionally complicated and it does need a greater knowledge of Java than we have at the moment to understand. All we need to know for now is that we can use the code as it is and just change the message, and it is a great tool to announce something to the player. If you really want an explanation now, you can think of it like this: when we made button objects, we got to use all the button methods. But with Toast, we used the class directly to access its setText method without creating an object first. We can do this process when the class and its methods are designed to allow it. Finally, we break out of the whole switch statement as follows: break; Self-test questions Q1) What does this code do? // setContentView(R.layout.activity_main); Q2) Which of these lines causes an error? String a = "Hello"; String b = " Vinton Cerf"; int c = 55; a = a + b c = c + c + 10; a = a + c; c = c + a; Q3) We talked a lot about operators and how different operators can be used together to build complicated expressions. Expressions, at a glance, can sometimes make the code look complicated. However, when looked at closely, they are not as tough as they seem. Usually, it is just a case of splitting the expressions into smaller pieces to work out what is going on. Here is an expression that is more convoluted than anything else you will ever see in this book. As a challenge, can you work out: what will x be? int x = 10; int y = 9; boolean isTrueOrFalse = false; isTrueOrFalse = (((x <=y)||(x == 10))&&((!isTrueOrFalse) || (isTrueOrFalse))); Summary We went from knowing nothing about Java syntax to learning about comments, variables, operators, and decision making. As with any language, mastery of Java can be achieved by simply practicing, learning, and increasing our vocabulary. At this point, the temptation might be to hold back until mastery of the current Java syntax has been achieved, but the best way is to move on to new syntax at the same time as revisiting what we have already begun to learn. We will finally finish our math game by adding random questions of multiple difficulties as well as using more appropriate and random wrong answers for the multiple choice buttons. To enable us to do this, we will first learn some more new on Java and syntax. Resources for Article: Further resources on this subject: Asking Permission: Getting your head around Marshmallow's Runtime Permissions [article] Android and iOS Apps Testing at a Glance [article] Practical How-To Recipes for Android [article]
Read more
  • 0
  • 0
  • 8977
article-image-training-and-visualizing-a-neural-network-with-r
Oli Huggins
16 Feb 2016
8 min read
Save for later

Training and Visualizing a neural network with R

Oli Huggins
16 Feb 2016
8 min read
The development of a neural network is inspired by human brain activities. As such, this type of network is a computational model that mimics the pattern of the human mind. In contrast to this, support vector machines first, map input data into a high dimensional feature space defined by the kernel function, and find the optimum hyperplane that separates the training data by the maximum margin. In short, we can think of support vector machines as a linear algorithm in a high dimensional space. In this article, we will cover: Training a neural network with neuralnet Visualizing a neural network trained by neuralnet (For more resources related to this topic, see here.) Training a neural network with neuralnet The neural network is constructed with an interconnected group of nodes, which involves the input, connected weights, processing element, and output. Neural networks can be applied to many areas, such as classification, clustering, and prediction. To train a neural network in R, you can use neuralnet, which is built to train multilayer perceptron in the context of regression analysis, and contains many flexible functions to train forward neural networks. In this recipe, we will introduce how to use neuralnet to train a neural network. Getting ready In this recipe, we will use an iris dataset as our example dataset. We will first split the irisdataset into a training and testing datasets, respectively. How to do it... Perform the following steps to train a neural network with neuralnet: First load the iris dataset and split the data into training and testing datasets: > data(iris) > ind <- sample(2, nrow(iris), replace = TRUE, prob=c(0.7, 0.3)) > trainset = iris[ind == 1,]> testset = iris[ind == 2,] Then, install and load the neuralnet package: > install.packages("neuralnet")> library(neuralnet) Add the columns versicolor, setosa, and virginica based on the name matched value in the Species column: > trainset$setosa = trainset$Species == "setosa" > trainset$virginica = trainset$Species == "virginica" > trainset$versicolor = trainset$Species == "versicolor" Next, train the neural network with the neuralnet function with three hidden neurons in each layer. Notice that the results may vary with each training, so you might not get the same result: > network = neuralnet(versicolor + virginica + setosa~ Sepal.Length + Sepal.Width + Petal.Length + Petal.Width, trainset, hidden=3) > network Call: neuralnet(formula = versicolor + virginica + setosa ~ Sepal.Length + Sepal.Width + Petal.Length + Petal.Width, data = trainset, hidden = 3) 1 repetition was calculated. Error Reached Threshold Steps 1 0.8156100175 0.009994274769 11063 Now, you can view the summary information by accessing the result.matrix attribute of the built neural network model: > network$result.matrix 1 error 0.815610017474 reached.threshold 0.009994274769 steps 11063.000000000000 Intercept.to.1layhid1 1.686593311644 Sepal.Length.to.1layhid1 0.947415215237 Sepal.Width.to.1layhid1 -7.220058260187 Petal.Length.to.1layhid1 1.790333443486 Petal.Width.to.1layhid1 9.943109233330 Intercept.to.1layhid2 1.411026063895 Sepal.Length.to.1layhid2 0.240309549505 Sepal.Width.to.1layhid2 0.480654059973 Petal.Length.to.1layhid2 2.221435192437 Petal.Width.to.1layhid2 0.154879347818 Intercept.to.1layhid3 24.399329878242 Sepal.Length.to.1layhid3 3.313958088512 Sepal.Width.to.1layhid3 5.845670010464 Petal.Length.to.1layhid3 -6.337082722485 Petal.Width.to.1layhid3 -17.990352566695 Intercept.to.versicolor -1.959842102421 1layhid.1.to.versicolor 1.010292389835 1layhid.2.to.versicolor 0.936519720978 1layhid.3.to.versicolor 1.023305801833 Intercept.to.virginica -0.908909982893 1layhid.1.to.virginica -0.009904635231 1layhid.2.to.virginica 1.931747950462 1layhid.3.to.virginica -1.021438938226 Intercept.to.setosa 1.500533827729 1layhid.1.to.setosa -1.001683936613 1layhid.2.to.setosa -0.498758815934 1layhid.3.to.setosa -0.001881935696 Lastly, you can view the generalized weight by accessing it in the network: > head(network$generalized.weights[[1]]) How it works... The neural network is a network made up of artificial neurons (or nodes). There are three types of neurons within the network: input neurons, hidden neurons, and output neurons. In the network, neurons are connected; the connection strength between neurons is called weights. If the weight is greater than zero, it is in an excitation status. Otherwise, it is in an inhibition status. Input neurons receive the input information; the higher the input value, the greater the activation. Then, the activation value is passed through the network in regard to weights and transfer functions in the graph. The hidden neurons (or output neurons) then sum up the activation values and modify the summed values with the transfer function. The activation value then flows through hidden neurons and stops when it reaches the output nodes. As a result, one can use the output value from the output neurons to classify the data. Artificial Neural Network The advantages of a neural network are: firstly, it can detect a nonlinear relationship between the dependent and independent variable. Secondly, one can efficiently train large datasets using the parallel architecture. Thirdly, it is a nonparametric model so that one can eliminate errors in the estimation of parameters. The main disadvantages of neural network are that it often converges to the local minimum rather than the global minimum. Also, it might over-fit when the training process goes on for too long. In this recipe, we demonstrate how to train a neural network. First, we split the iris dataset into training and testing datasets, and then install the neuralnet package and load the library into an R session. Next, we add the columns versicolor, setosa, and virginica based on the name matched value in the Species column, respectively. We then use the neuralnet function to train the network model. Besides specifying the label (the column where the name equals to versicolor, virginica, and setosa) and training attributes in the function, we also configure the number of hidden neurons (vertices) as three in each layer. Then, we examine the basic information about the training process and the trained network saved in the network. From the output message, it shows the training process needed 11,063 steps until all the absolute partial derivatives of the error function were lower than 0.01 (specified in the threshold). The error refers to the likelihood of calculating Akaike Information Criterion (AIC). To see detailed information on this, you can access the result.matrix of the built neural network to see the estimated weight. The output reveals that the estimated weight ranges from -18 to 24.40; the intercepts of the first hidden layer are 1.69, 1.41 and 24.40, and the two weights leading to the first hidden neuron are estimated as 0.95 (Sepal.Length), -7.22 (Sepal.Width), 1.79 (Petal.Length), and 9.94 (Petal.Width). We can lastly determine that the trained neural network information includes generalized weights, which express the effect of each covariate. In this recipe, the model generates 12 generalized weights, which are the combination of four covariates (Sepal.Length, Sepal.Width, Petal.Length, Petal.Width) to three responses (setosa, virginica, versicolor). See also For a more detailed introduction on neuralnet, one can refer to the following paper: Günther, F., and Fritsch, S. (2010). neuralnet: Training of neural networks. The R journal, 2(1), 30-38. Visualizing a neural network trained by neuralnet The package, neuralnet, provides the plot function to visualize a built neural network and the gwplot function to visualize generalized weights. In following recipe, we will cover how to use these two functions. Getting ready You need to have completed the previous recipe by training a neural network and have all basic information saved in network. How to do it... Perform the following steps to visualize the neural network and the generalized weights: You can visualize the trained neural network with the plot function: > plot(network) Figure 10: The plot of trained neural network Furthermore, You can use gwplot to visualize the generalized weights: > par(mfrow=c(2,2)) > gwplot(network,selected.covariate="Petal.Width") > gwplot(network,selected.covariate="Sepal.Width") > gwplot(network,selected.covariate="Petal.Length") > gwplot(network,selected.covariate="Petal.Width") Figure 11: The plot of generalized weights How it works... In this recipe, we demonstrate how to visualize the trained neural network and the generalized weights of each trained attribute. Also, the plot includes the estimated weight, intercepts and basic information about the training process. At the bottom of the figure, one can find the overall error and number of steps required to converge. If all the generalized weights are close to zero on the plot, it means the covariate has little effect. However, if the overall variance is greater than one, it means the covariate has a nonlinear effect. See also For more information about gwplot, one can use the help function to access the following document: > ?gwplot Summary To learn more about machine learning with R, the following books published by Packt Publishing (https://www.packtpub.com/) are recommended: Machine Learning with R (Second Edition) - Read Online Mastering Machine Learning with R - Read Online Resources for Article: Further resources on this subject: Introduction to Machine Learning with R [article] Hive Security [article] Spark - Architecture and First Program [article]
Read more
  • 0
  • 0
  • 14821

article-image-python-data-science-and-running
Packt
16 Feb 2016
10 min read
Save for later

Python Data Science Up and Running

Packt
16 Feb 2016
10 min read
In this article we will learn how to build NumPy, SciPy, matplotlib, and IPython and using Ipython as shell and then to create a simple application using it. (For more resources related to this topic, see here.) Building NumPY, SciPy, matplotlib, and IPython from source As a last resort or if we want to have the latest code, we can build from source.  In practice, it shouldn't be that hard, although depending on your operating system, you might run into problems. As operating systems and related software are rapidly evolving, in such cases, the best you can do is search online or ask for help. In this chapter, we give pointers on good places to look for help. The source code can be retrieved with git or as an archive from GitHub. The steps  to install NumPy from source are straightforward and given here. We can retrieve the source code for NumPy with git as follows: $ git clone git://github.com/numpy/numpy.git numpy There are similar commands for SciPy, matplotlib, and IPython (refer to the table that follows after this piece of information). The IPython source code can be downloaded from https://github.com/ipython/ipython/releases as a source archive or ZIP file. You can then unpack it with your favorite tool or with the following command: $ tar -xzf ipython.tar.gz Please refer to the following table for the git commands and source archive/zip links: Library Git command Tarball/zip URL NumPy git clone git://github.com/numpy/numpy.git numpy https://github.com/numpy/numpy/releases SciPy git clone http://github.com/scipy/scipy.git scipy https://github.com/scipy/scipy/releases matplotlib git clone git://github.com/matplotlib/matplotlib.git https://github.com/matplotlib/matplotlib/releases IPython git clone --recursive https://github.com/ipython/ipython.git https://github.com/ipython/ipython/releases Install on /usr/local with the following command from the source code directory: $ python setup.py build $ sudo python setup.py install --prefix=/usr/local To build, we need a C compiler such as GCC and the Python header files in the python-dev or python-devel package. Installing with setuptools If you have setuptools or pip, you can install NumPy, SciPy, matplotlib, and IPython with the following commands. For each library, we give two commands, one for setuptools and one for pip. You only need to choose one command per pair: $ easy_install numpy $ pip install numpy $ easy_install scipy $ pip install scipy $ easy_install matplotlib $ pip install matplotlib $ easy_install ipython $ pip install ipython It may be necessary to prepend sudo to these commands if your current user doesn't have sufficient rights on your system. NumPy arrays After going through the installation of NumPy, it's time to have a look at NumPy arrays. NumPy arrays are more efficient than Python lists when it comes to numerical operations. NumPy arrays are, in fact, specialized objects with extensive optimizations. NumPy code requires less explicit loops than equivalent Python code. This is based  on vectorization. If we go back to highschool mathematics, then we should remember the concepts of scalars and vectors. The number 2, for instance, is a scalar. When we add 2 to 2, we are performing scalar addition. We can form a vector out of a group of scalars. In Python programming terms, we will then have a one-dimensional array. This concept can, of course, be extended to higher dimensions. Performing an operation on two arrays, such as addition, can be reduced to a group of scalar operations. In straight Python, we will do that with loops going through each element in the first array and adding it to the corresponding element in the second array. However, this is more verbose than the way it is done in mathematics. In mathematics, we treat the addition of two vectors as a single operation. That's the way NumPy arrays do it too, and there are certain optimizations using low-level C routines, which make these basic operations more efficient. Simple application Imagine that we want to add two vectors called a and b. The word vector is used here in the mathematical sense, which means a one-dimensional array. The vector a holds the squares of integers 0 to n; for instance, if n is equal to 3, a contains 0, 1, or 4. The vector b holds the cubes of integers 0 to n, so if n is equal to 3, then the vector b is equal to 0, 1, or 8. How would you do that using plain Python? After we come up with a solution, we will compare it with the NumPy equivalent. The following function solves the vector addition problem using pure Python without NumPy: def pythonsum(n): a = range(n) b = range(n) c = [] for i in range(len(a)): a[i] = i ** 2 b[i] = i ** 3 c.append(a[i] + b[i]) return c The following is a function that solves the vector addition problem with NumPy: def numpysum(n): a = numpy.arange(n) ** 2 b = numpy.arange(n) ** 3 c = a + b return c Notice that numpysum() does not need a for loop. Also, we used the arange() function from NumPy, which creates a NumPy array for us with integers from  0 to n. The arange() function was imported; that is why it is prefixed with numpy. Now comes the fun part. Remember that it was mentioned in the Preface that NumPy is faster when it comes to array operations. How much faster is Numpy, though? The following program will show us by measuring the elapsed time in microseconds for the numpysum() and pythonsum() functions. It also prints the last two elements of the vector sum. Let's check that we get the same answers using Python and NumPy: #!/usr/bin/env/python import sys from datetime import datetime import numpy as np """ This program demonstrates vector addition the Python way. Run from the command line as follows python vectorsum.py n where n is an integer that specifies the size of the vectors. The first vector to be added contains the squares of 0 up to n. The second vector contains the cubes of 0 up to n. The program prints the last 2 elements of the sum and the elapsed time. """ def numpysum(n): a = np.arange(n) ** 2 b = np.arange(n) ** 3 c = a + b return c def pythonsum(n): a = range(n) b = range(n) c = [] for i in range(len(a)): a[i] = i ** 2 b[i] = i ** 3 c.append(a[i] + b[i]) return c size = int(sys.argv[1]) start = datetime.now() c = pythonsum(size) delta = datetime.now() - start print "The last 2 elements of the sum", c[-2:] print "PythonSum elapsed time in microseconds", delta.microseconds start = datetime.now() c = numpysum(size) delta = datetime.now() - start print "The last 2 elements of the sum", c[-2:] print "NumPySum elapsed time in microseconds", delta.microseconds The output of the program for 1000, 2000, and 3000 vector elements is as follows: $ python vectorsum.py 1000 The last 2 elements of the sum [995007996, 998001000] PythonSum elapsed time in microseconds 707 The last 2 elements of the sum [995007996 998001000] NumPySum elapsed time in microseconds 171 $ python vectorsum.py 2000 The last 2 elements of the sum [7980015996, 7992002000] PythonSum elapsed time in microseconds 1420 The last 2 elements of the sum [7980015996 7992002000] NumPySum elapsed time in microseconds 168 $ python vectorsum.py 4000 The last 2 elements of the sum [63920031996, 63968004000] PythonSum elapsed time in microseconds 2829 The last 2 elements of the sum [63920031996 63968004000] NumPySum elapsed time in microseconds 274 Clearly, NumPy is much faster than the equivalent normal Python code. One thing is certain; we get the same results whether we are using NumPy or not. However, the result that is printed differs in representation. Notice that the result from the numpysum() function does not have any commas. How come? Obviously, we are not dealing with a Python list but with a NumPy array. Using IPython as a shell Scientists, data analysts, and engineers are used to experimenting. IPython was created by scientists with experimentation in mind. The interactive environment that IPython provides is viewed by many as a direct answer to MATLAB, Mathematica, and Maple. The following is a list of features of the IPython shell: Tab completion, which helps you find a command History mechanism Inline editing Ability to call external Python scripts with %run Access to system commands The pylab switch Access to the Python debugger and profiler The following list describes how to use the IPython shell: The pylab switch: The pylab switch automatically imports all the Scipy, NumPy, and matplotlib packages. Without this switch, we would have to import these packages ourselves. All we need to do is enter the following instruction on the command line: $ ipython -pylab Type "copyright", "credits" or "license" for more information. IPython 2.0.0-dev -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object', use 'object??' for extra details. Welcome to pylab, a matplotlib-based Python environment [backend: MacOSX]. For more information, type 'help(pylab)'. In [1]: quit() The quit() function or Ctrl + D quits the IPython shell. Saving a session: We might want to be able to go back to our experiments. In IPython, it is easy to save a session for later use, with the following command: In [1]: %logstart Activating auto-logging. Current session state plus future input saved. Filename : ipython_log.py Mode : rotate Output logging : False Raw input log : False Timestamping : False State : active Logging can be switched off as follows: In [9]: %logoff Switching logging OFF Executing system shell command: Execute a system shell command in  the default IPython profile by prefixing the command with the ! symbol.  For instance, the following input will get the current date: In [1]: !date In fact, any line prefixed with ! is sent to the system shell. Also, we can store the command output as shown here: In [2]: thedate = !date In [3]: thedate Displaying history: We can show the history of commands with the %hist command, for example: In [1]: a = 2 + 2 In [2]: a Out[2]: 4 In [3]: %hist a = 2 + 2 a %hist This is a common feature in Command Line Interface (CLI) environments. We can also search through the history with the -g switch as follows: In [5]: %hist -g a = 2    1: a = 2 + 2 Downloading the example code You can download the example code files for all the Packt books you have purchased from your account at https://www.packtpub.com. If you purchased this book elsewhere, you can visit https://www.packtpub.com/books/content/support and register to have the files e-mailed directly to you. We saw a number of so-called magic functions in action. These functions start with  the % character. If the magic function is used on a line by itself, the % prefix is optional. Summary In this article, we installed NumPy, SciPy, matplotlib, and IPython. We got a vector addition program working and convinced ourselves that NumPy offers superior performance. In addition, we explored the available documentation and online resources. You can also refer the following books on the similar topics: Mastering Python Data Analysis: (https://www.packtpub.com/big-data-and-business-intelligence/mastering-python-data-analysis) Python Data Analysis Cookbook: (https://www.packtpub.com/big-data-and-business-intelligence/python-data-analysis-cookbook) Getting Started with Python Data Analysis: (https://www.packtpub.com/big-data-and-business-intelligence/getting-started-python-data-analysis) Resources for Article: Further resources on this subject: Python Design Patterns in Depth – The Observer Pattern [article] Python Design Patterns in Depth: The Factory Pattern [article] Customizing IPython [article]
Read more
  • 0
  • 0
  • 1765

article-image-introduction-scikit-learn
Packt
16 Feb 2016
5 min read
Save for later

Introduction to Scikit-Learn

Packt
16 Feb 2016
5 min read
Introduction Since its release in 2007, scikit-learn has become one of the most popular open source machine learning libraries for Python. scikit-learn provides algorithms for machine learning tasks including classification, regression, dimensionality reduction, and clustering. It also provides modules for extracting features, processing data, and evaluating models. (For more resources related to this topic, see here.) Conceived as an extension to the SciPy library, scikit-learn is built on the popular Python libraries NumPy and matplotlib. NumPy extends Python to support efficient operations on large arrays and multidimensional matrices. matplotlib provides visualization tools, and SciPy provides modules for scientific computing. scikit-learn is popular for academic research because it has a well-documented, easy-to-use, and versatile API. Developers can use scikit-learn to experiment with different algorithms by changing only a few lines of the code. scikit-learn wraps some popular implementations of machine learning algorithms, such as LIBSVM and LIBLINEAR. Other Python libraries, including NLTK, include wrappers for scikit-learn. scikit-learn also includes a variety of datasets, allowing developers to focus on algorithms rather than obtaining and cleaning data. Licensed under the permissive BSD license, scikit-learn can be used in commercial applications without restrictions. Many of scikit-learn's algorithms are fast and scalable to all but massive datasets. Finally, scikit-learn is noted for its reliability; much of the library is covered by automated tests. Installing scikit-learn This book is written for version 0.15.1 of scikit-learn; use this version to ensure that the examples run correctly. If you have previously installed scikit-learn, you can retrieve the version number with the following code: >>> import sklearn >>> sklearn.__version__ '0.15.1' If you have not previously installed scikit-learn, you can install it from a package manager or build it from the source. We will review the installation processes for Linux, OS X, and Windows in the following sections, but refer to http://scikit-learn.org/stable/install.html for the latest instructions. The following instructions only assume that you have installed Python 2.6, Python 2.7, or Python 3.2 or newer. Go to http://www.python.org/download/ for instructions on how to install Python. Installing scikit-learn on Windows scikit-learn requires Setuptools, a third-party package that supports packaging and installing software for Python. Setuptools can be installed on Windows by running the bootstrap script at https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py. Windows binaries for the 32- and 64-bit versions of scikit-learn are also available. If you cannot determine which version you need, install the 32-bit version. Both versions depend on NumPy 1.3 or newer. The 32-bit version of NumPy can be downloaded from http://sourceforge.net/projects/numpy/files/NumPy/. The 64-bit version can be downloaded from http://www.lfd.uci.edu/~gohlke/pythonlibs/#scikit-learn. A Windows installer for the 32-bit version of scikit-learn can be downloaded from http://sourceforge.net/projects/scikit-learn/files/. An installer for the 64-bit version of scikit-learn can be downloaded from http://www.lfd.uci.edu/~gohlke/pythonlibs/#scikit-learn. scikit-learn can also be built from the source code on Windows. Building requires a C/C++ compiler such as MinGW (http://www.mingw.org/), NumPy, SciPy, and Setuptools. To build, clone the Git repository from https://github.com/scikit-learn/scikit-learn and execute the following command: python setup.py install Installing scikit-learn on Linux There are several options to install scikit-learn on Linux, depending on your distribution. The preferred option to install scikit-learn on Linux is to use pip. You may also install it using a package manager, or build scikit-learn from its source. To install scikit-learn using pip, execute the following command: sudo pip install scikit-learn To build scikit-learn, clone the Git repository from https://github.com/scikit-learn/scikit-learn. Then install the following dependencies: sudo apt-get install python-dev python-numpy python-numpy-dev python-setuptools python-numpy-dev python-scipy libatlas-dev g++ Navigate to the repository's directory and execute the following command: python setup.py install Installing scikit-learn on OS X scikit-learn can be installed on OS X using Macports: sudo port install py26-sklearn If Python 2.7 is installed, run the following command: sudo port install py27-sklearn scikit-learn can also be installed using pip with the following command: pip install scikit-learn Verifying the installation To verify that scikit-learn has been installed correctly, open a Python console and execute the following: >>> import sklearn >>> sklearn.__version__ '0.15.1' To run scikit-learn's unit tests, first install the nose library. Then execute the following: nosetest sklearn –exe Congratulations! You've successfully installed scikit-learn. Summary In this article, we had a brief introduction of Scikit. We also covered the installation of Scikit on various operating system Windows, Linux, OS X. You can also refer the following books on the similar topics: scikit-learn Cookbook (https://www.packtpub.com/big-data-and-business-intelligence/scikit-learn-cookbook) Learning scikit-learn: Machine Learning in Python (https://www.packtpub.com/big-data-and-business-intelligence/learning-scikit-learn-machine-learning-python) Resources for Article: Further resources on this subject: Our First Machine Learning Method – Linear Classification[article] Specialized Machine Learning Topics[article] Machine Learning[article]
Read more
  • 0
  • 0
  • 4154
article-image-getting-started-rstudio
Packt
16 Feb 2016
5 min read
Save for later

Getting Started with RStudio

Packt
16 Feb 2016
5 min read
The number of users adopting the R programming language has been increasing faster and faster in the last few years. The functions of the R console are limited when it comes to managing a lot of files, or when we want to work with version control systems. This is the reason, in combination with the increasing adoption rate, why a need for a better development environment arose. To serve this need, a team of R fans began to develop an integrated development environment (IDE) to make it easier to work on bigger projects and to collaborate with others. This IDE has the name, RStudio. In this article, we will see how to work with RStudio and projects (For more resources related to this topic, see here.) Working with RStudio and projects In the times before RStudio, it was very hard to manage bigger projects with R in the R console, as you had to create all the folder structures on your own. When you work with projects or open a project, RStudio will instantly take several actions. For example, it will start a new and clean R session, it will source the .Rprofile file in the project's main directory, and it will set the current working directory to the project directory. So, you have a complete working environment individually for every project. RStudio will even adjust its own settings, such as active tabs, splitter positions, and so on, to where they were when the project was closed. But just because you can create projects with RStudio easily, it does not mean that you should create a project for every single time that you write R code. For example, if you just want to do a small analysis, we would recommend that you create a project where you save all your smaller scripts. Creating a project with RStudio RStudio offers you an easy way to create projects. Just navigate to File | New Project and you will see a popup window as with the options shown in the following screenshot: These options let you decide from where you want to create your project. So, if you want to start it from scratch and create a new directory, associate your new project to an existing one, or if you want to create a project from a version control repository, you can avail of the respective options. For now, we will focus on creating a new directory. The following screenshot shows you the next options available: Locating your project A very important question you have to ask yourself when creating a new project is where you want to save it? There are several options and details you have to pay attention to especially when it comes to collaboration and different people working on the same project. You can save your project locally, on a cloud storage or with the help of a revision control system such as Git. Creating your first project To begin your first project, choose the New Directory option we described before and create an empty project. Then, choose a name for the directory and the location that you want to save it in. You should create a projects folder on your Dropbox. The first project will be a small data analysis based on a dataset that was extracted from the 1974 issue of the Motor Trend US magazine. It comprises fuel consumption and ten aspects of automobile design and performance, such as the weight or number of cylinders for 32 automobiles, and is included in the base R package. So, we do not have to install a separate package to work with this dataset, as it is automatically loaded when you start R: As you can see, we left the Use packrat with this project option unchecked. Packrat is a dependency management tool that makes your R code more isolated, portable, and reproducible by giving your project its own privately managed package library. This is especially important when you want to create projects in an organizational context where the code has to run on various computer systems, and has to be usable for a lot of different users. This first project will just run locally and will not focus on a specific combination of package versions. Organizing your folders RStudio creates an empty directory for you that includes just the file, Motor-Car-Trend-Analysis.Rproj. This file will store all the information on your project that RStudio will need for loading. But to stay organized, we have to create some folders in the directory. Create the following folders: data: This includes all the data that we need for our analysis code: This includes all the code files for cleaning up data, generating plots, and so on plots: This includes all graphical outputs reports: This comprises all the reports that we create from our dataset Saving the data The Motor Trend Car Road Tests dataset is part of the dataset package, which is one of the preinstalled packages in R. But, we will save the data in a CSV file in our data folder, after extracting the data from the mtcars variable, to make sure our analysis is reproducible. Put the following line of code in a new R script and save it as data.R in the code folder: #write data into csv file write.csv(mtcars, file = "data/cars.csv", row.names=FALSE) Analyzing the data The analysis script will first have to load the data from the CSV file with the following line: cars_data <- read.csv(file = "data/cars.csv", header = TRUE, sep = ",") Summary To learn more about RStudio, the following books published by Packt Publishing (https://www.packtpub.com/) are recommended: Mastering Machine Learning with R (https://www.packtpub.com/big-data-and-business-intelligence/mastering-machine-learning-r) R Data Analysis Cookbook (https://www.packtpub.com/big-data-and-business-intelligence/r-data-analysis-cookbook) Mastering Data Analysis with R (https://www.packtpub.com/big-data-and-business-intelligence/mastering-data-analysis-r) Resources for Article: Further resources on this subject: RefresheR [article] Deep learning in R [article] Aspects of Data Manipulation in R [article]
Read more
  • 0
  • 0
  • 41798

article-image-developing-your-first-cordova-application
Packt
16 Feb 2016
24 min read
Save for later

Developing Your First Cordova Application

Packt
16 Feb 2016
24 min read
In this article, you will develop, build, and deploy your first Apache Cordova application from scratch. The application you will develop is a Sound Recorder utility that you can use to record your voice or any sound and play it back. In this chapter, you will learn about the following topics: Generating your initial Apache Cordova project artifacts by utilizing the Apache Cordova Command-line Interface (CLI) Developing and building your mobile application from the initial Cordova generated code Deploying your developed mobile application to a real Android mobile device to see your application in action (For more resources related to this topic, see here.) An introduction to Cordova CLI In order to create, develop, build, and test a Cordova application, you first need to use the Cordova CLI. Using this, you can create new Apache Cordova project(s), build them on mobile platforms such as iOS, Android, Windows Phone, and so on, and run them on real devices or within emulators. Note that in this chapter, we will focus on deploying our Sound Recorder application in Android devices only. In the next chapter, we will learn how to deploy our Sound Recorder application in iOS and Windows Phone devices. Installing Apache Cordova Before installing Apache Cordova CLI, you need to make sure that you install the following software: Target platform SDK: For Android, you can download its SDK from http://developer.android.com/sdk/index.html (for other platforms, you need to download and install their corresponding SDKs) Node.js: This is accessible at http://nodejs.org and can be downloaded and installed from http://nodejs.org/download/ After installing Node.js, you should be able to run Node.js or node package manager (npm) from the command line. In order to install Apache Cordova using npm, run the following command (you can omit sudo if you are working in a Windows environment): > sudo npm install -g cordova It's worth mentioning that npm is the official package manager for Node.js and it is written completely in JavaScript. npm is a tool that allows users to install Node.js modules, which are available in the npm registry. The sudo command allows a privileged Unix user to execute a command as the super user, or as any other user, according to the sudoers file. The sudo command, by default, requires you to authenticate with a password. Once you are authenticated, you can use the command without a password, by default, for 5 minutes. After successfully installing Apache Cordova (Version 3.4.0), you should be able to execute Apache Cordova commands from the command line, for example, the following command will show you the current installed version of Apache Cordova: > cordova -version In order to execute the Cordova commands without any problem, you also need to have Apache Ant installed and configured in your operating system. You can download Apache Ant from http://ant.apache.org. The complete instructions on how to install Ant are mentioned at https://ant.apache.org/manual/install.html. Generating our Sound Recorder's initial code After installing Apache Cordova, we can start creating our Sound Recorder project by executing the following command: > cordova create soundRecorder com.jsmobile.soundrecorder SoundRecorder After successfully executing this command, you will find a message similar to the following one (note that the location path will be different on your machine): Creating a new cordova project with name "SoundRecorder" and id "com.jsmobile.soundrecorder" at location "/Users/xyz/projects/soundRecorder" If we analyze the cordova create command, we will find that its first parameter represents the path of your project. In this command, a soundRecorder directory will be generated for your project under the directory from which the cordova create command is executed. The second and third parameters are optional. The second parameter, com.jsmobile.soundrecorder, provides your project's namespace (it should be noted that in Android projects, this namespace will be translated to a Java package with this name), and the last parameter, SoundRecorder, provides the application's display text. You can edit both these values in the config.xml configuration file later, which will be illustrated soon. The following screenshot shows our SoundRecorder project's generated artifacts:   The Sound Recorder's initial structure As shown in the preceding screenshot, the generated Apache Cordova project contains the following main directories: www: This directory includes your application's HTML, JavaScript, and CSS code. You will also find the application's starting page (index.html), along with various subdirectories, which are as follows: css: This directory includes the default Apache Cordova application's CSS file (index.css) js: This directory includes the default Apache Cordova application's JavaScript file (index.js) img: This directory includes the default Apache Cordova application's logo file (logo.png) config.xml: This file contains the application configuration. The following code snippet shows the initial code of the config.xml file: <?xml version='1.0' encoding='utf-8'?> <widget id="com.jsmobile.soundrecorder" version="0.0.1" > <name>SoundRecorder</name> <description> A sample Apache Cordova application that responds to the deviceready event. </description> <author email="dev@cordova.apache.org" href="http://cordova.io"> Apache Cordova Team </author> <content src="index.html" /> <access origin="*" /> </widget> As shown in the preceding config.xml file, config.xml contains the following elements that are available on all the supported Apache Cordova platforms: The <widget> element's id attribute represents the application's namespace identifier as specified in our cordova create command, and the <widget> element's version attribute represents its full version number in the form of major.minor.patch. The <name> element specifies the application's name. The <description> and <author> elements specify the application's description and author, respectively. The <content> element (which is optional) specifies the application's starting page that is placed directly under the www directory. The default value is index.html. The <access> element(s) defines the set of external domains that the application is allowed to access. The default value is *, which means that the application is allowed to access any external server(s). Specifying the <access> element's origin to * is fine during application development, but it is considered a bad practice in production due to security concerns. Note that before moving your application to production, you should review its whitelist and declare its access to specific network domains and subdomains. There is another element that is not included in the default config.xml, and this is the <preference> element. The <preference> element(s) can be used to set the different preferences of the Cordova application and can work on all or a subset of the Apache Cordova-supported platforms. Take the example of the following code: <preference name="Fullscreen" value="true" /> If the Fullscreen preference is set to true, it means that the application will be in fullscreen mode on all Cordova-supported platforms (by default, this option is set to false). It is important to note that not all preferences work on all Cordova-supported platforms. Consider the following example: <preference name="HideKeyboardFormAccessoryBar" value="true"/> If the HideKeyboardFormAccessoryBar preference is set to true, then the additional helper toolbar, which appears above the device keyboard, will be hidden. This preference works only on iOS and BlackBerry platforms. platforms: This directory includes the application's supported platforms. After adding a new platform using Apache Cordova CLI, you will find a newly created directory that contains the platform-specific generated code under the platforms directory. The platforms directory is initially empty because we have not added any platforms yet. We will add support to the Android platform in the next step. plugins: This directory includes your application's used plugins. If you aren't already aware, a plugin is the mechanism to access the device's native functions in Apache Cordova. After adding a plugin (such as the Media plugin) to the project, you will find a newly created directory under the plugins directory, which contains the plugin code. Note that we will add three plugins in our Sound Recorder application example. merges: This directory can be used to override the common resources under the www directory. The files placed under the merges/[platform] directory will override the matching files (or add new files) under the www directory for the specified platform (the [platform] value can be iOS, Android, or any other valid supported platform). hooks: This directory contains scripts that can be used to customize Apache Cordova commands. A hook is a piece of code that executes before and/or after the Apache Cordova command runs. An insight into the www files If we look in the www directory, we will find that it contains the following three files: index.html: This file is placed under the application's www directory, and it contains the HTML content of the application page index.js: This file is placed under the www/js directory, and it contains a simple JavaScript logic that we will illustrate soon index.css: This file is placed under the www/css directory, and it contains the style classes of the HTML elements The following code snippet includes the most important part of the index.html page: <div class="app"> <h1>Apache Cordova</h1> <div id="deviceready" class="blink"> <p class="event listening">Connecting to Device</p> <p class="event received">Device is Ready</p> </div> </div> <script type="text/javascript" src="cordova.js"></script> <script type="text/javascript" src="js/index.js"></script> <script type="text/javascript"> app.initialize(); </script> The index.html page has a single div "app", which contains a child div "deviceready". The "deviceready" div has two paragraph elements, the "event listening" and "event received" paragraphs. The "event received" paragraph is initially hidden as indicated by index.css: .event.received { background-color:#4B946A; display:none; } In the index.html page, there are two main JavaScript-included files, as follows: cordova.js: This file contains Apache Cordova JavaScript APIs index.js: This file contains the application's simple logic Finally, the index.html page calls the initialize() method of the app object. Let's see the details of the app object in index.js: var app = { initialize: function() { this.bindEvents(); }, bindEvents: function() { document.addEventListener('deviceready', this.onDeviceReady, false); }, onDeviceReady: function() { app.receivedEvent('deviceready'); }, receivedEvent: function(id) { var parentElement = document.getElementById(id); var listeningElement = parentElement.querySelector('.listening'); var receivedElement = parentElement.querySelector('.received'); listeningElement.setAttribute('style', 'display:none;'); receivedElement.setAttribute('style', 'display:block;'); console.log('Received Event: ' + id); } }; The initialize() method calls the bindEvents() method, which adds an event listener for the 'deviceready' event. When the device is ready, the onDeviceReady() method is called, and this in turn calls the receivedEvent() method of the app object. In the receivedEvent() method, the "event listening" paragraph is hidden and the "event received" paragraph is shown to the user. This is to display the Device is Ready message to the user once Apache Cordova is fully loaded. It is important to note that you must not call any Apache Cordova API before the 'deviceready' event fires. This is because the 'deviceready' event fires only once Apache Cordova is fully loaded. Now you have an Apache Cordova project that has common cross-platform code, so we need to generate a platform-specific code in order to deploy our code on a real device. To generate Android platform code, you need to add the Android platform as follows: > cd soundRecorder > cordova platform add android In order to add any platform, you need to execute the cordova platform command from the application directory. Note that in order to execute the cordova platform command without problems, you need to perform the following instructions: Have Apache Ant installed and configured in your operating system as described in the Installing Apache Cordova section Make sure that the path to your Android SDK platform tools and the tools directory are added to your operating system's PATH environment variable After executing the cordova platform add command, you will find a new subdirectory Android added under the soundRecorder/platforms directory, which is added by Android. In order to build the project, use the following command: > cordova build Finally, you can run and test the generated Android project in the emulator by executing the following command: > cordova emulate android You might see the ERROR: No emulator images (avds) found message flash if no Android AVDs are available in your operating system. So, make sure you create one! The following screenshot shows our Sound Recorder application's initial screen:   It is recommended that you make your code changes in the root www directory, and not in the platforms/android/assets/www directory (especially if you are targeting multiple platforms) as the platforms directory will be overridden every time you execute the cordova build command, unless you are willing to use Apache Cordova CLI to initialize the project for a single platform only. Developing Sound Recorder application After generating the initial application code, it's time to understand what to do next. Sound Recorder functionality The following screenshot shows our Sound Recorder page:   When the user clicks on the Record Sound button, they will be able to record their voices; they can stop recording their voices by clicking on the Stop Recording button. You can see this in the following screenshot:   As shown in the following screenshot, when the user clicks on the Playback button, the recorded voice will be played back:   Sound Recorder preparation In order to implement this functionality using Apache Cordova, we need to add the following plugins using the indicated commands, which should be executed from the application directory: media: This plugin is used to record and play back sound files: > cordova plugin add https://git-wip-us.apache.org/repos/asf/cordova-plugin-media.git device: This plugin is required to access the device information: > cordova plugin add https://git-wip-us.apache.org/repos/asf/cordova-plugin-device.git file: This plugin is used to access the device's filesystem: > cordova plugin add https://git-wip-us.apache.org/repos/asf/cordova-plugin-file.git In order to apply these plugins to our Apache Cordova project, we need to run the cordova build command again from the project directory, as follows: > cordova build Sound Recorder details Now we are done with the preparation of our Sound Recorder application. Before moving to the code details, let's see the hierarchy of our Sound Recorder application, as shown in the following screenshot: The application's www directory contains the following directories: css: This directory contains the custom application CSS file(s) img: This directory contains the custom application image file(s) js: This directory contains the custom application JavaScript code jqueryMobile: This directory (which is a newly added one) contains jQuery mobile framework files Finally, the index.html file contains the application's single page whose functionality was illustrated earlier in this section. It is important to note that Apache Cordova does not require you to use a JavaScript mobile User Interface (UI) framework. However, it is recommended that you use a JavaScript mobile UI framework in addition to Apache Cordova. This is in order to facilitate building the application UI and speed up the application development process. Let's see the details of the index.html page of our Sound Recorder application. The following code snippet shows the included files in the page: <link rel="stylesheet" type="text/css" href="css/app.css" /> <link rel="stylesheet" href="jqueryMobile/jquery.mobile-1.4.0.min.css"> <script src="jqueryMobile/jquery-1.10.2.min.js"></script> <script src="jqueryMobile/jquery.mobile-1.4.0.min.js"></script> ... <script type="text/javascript" src="cordova.js"></script> <script type="text/javascript" src="js/app.js"></script> In the preceding code, the following files are included: app.css: This is the custom style file of our Sound Recorder application The files required by the jQuery mobile framework, which are: jquery.mobile-1.4.0.min.css jquery-1.10.2.min.js jquery.mobile-1.4.0.min.js cordova.js: This is the Apache Cordova JavaScript API's file app.js: This is the custom JavaScript file of our Sound Recorder application It is important to know that you can download the jQuery mobile framework files from http://jquerymobile.com/download/. The following code snippet shows the HTML content of our application's single page, whose id is "main": <div data-role="page" id="main"> <div data-role="header"> <h1>Sound Recorder</h1> </div> <div data-role="content"> <div data-role="fieldcontain"> <h1>Welcome to the Sound Recorder Application</h1> <p>Click 'Record Sound' button in order to start recording. You will be able to see the playback button once the sound recording finishes.<br/><br/></p> <input type="hidden" id="location"/> <div class="center-wrapper"> <input type="button" id="recordSound" data- icon="audio" value="Record Sound" class="center-button" data- inline="true"/> <input type="button" id="playSound" data- icon="refresh" value="Playback" class="center-button" data- inline="true"/><br/> </div> <div data-role="popup" id="recordSoundDialog" data- dismissible="false" style="width:250px"> <div data-role="header"> <h1>Recording</h1> </div> <div data-role="content"> <div class="center-wrapper"> <div id="soundDuration"></div> <input type="button" id="stopRecordingSound" value="Stop Recording" class="center-button" data- inline="true"/> </div> </div> </div> </div> </div> <div data-role="footer" data-position="fixed"> <h1>Powered by Apache Cordova</h1> </div> </div> Looking at the preceding code, our Sound Recording page ("main") is defined by setting a div's data-role attribute to "page". It has a header defined by setting a div's data-role to "header". It has content defined by setting a div's data-role to "content", which contains the recording and playback buttons. The content also contains a "recordSoundDialog" pop up, which is defined by setting a div's data-role to "popup". The "recordSoundDialog" pop up has a header and content. The pop-up content displays the recorded audio duration in the "soundDuration" div, and it has a "stopRecordingSound" button that stops recording the sound. Finally, the page has a footer defined by setting a div's data-role to "footer", which contains a statement about the application. Now, it's time to learn how we can define event handlers on the the page HTML elements and use the Apache Cordova API inside our defined event handlers to implement the application's functionality. The following code snippet shows the page initialization code: (function() { $(document).on("pageinit", "#main", function(e) { e.preventDefault(); function onDeviceReady() { $("#recordSound").on("tap", function(e) { // Action is defined here ... }); $("#recordSoundDialog").on("popupafterclose", function(event, ui) { // Action is defined here ... }); $("#stopRecordingSound").on("tap", function(e) { // Action is defined here ... }); $("#playSound").on("tap", function(e) { // Action is defined here ... }); } $(document).on('deviceready', onDeviceReady); initPage(); }); // Code is omitted here for simplicity function initPage() { $("#playSound").closest('.ui-btn').hide(); } })(); In jQuery mobile, the "pageinit" event is called once during page initialization. In this event, the event handlers are defined and the page is initialized. Note that all of the event handlers are defined after the 'deviceready' event fires. The event handlers are defined for the following: Tapping the "recordSound" button Closing the "recordSoundDailog" dialog Tapping the "stopRecordingSound" button Tapping the "playSound" button In initPage(), the "playSound" button is hidden as no voice has been recorded yet. As you noticed, in order to hide an element in jQuery mobile, you just need to call its hide() method. We can now see the details of each event handler; the next code snippet shows the "recordSound" tap event handler: var recInterval; $("#recordSound").on("tap", function(e) { e.preventDefault(); var recordingCallback = {}; recordingCallback.recordSuccess = handleRecordSuccess; recordingCallback.recordError = handleRecordError; startRecordingSound(recordingCallback); var recTime = 0; $("#soundDuration").html("Duration: " + recTime + " seconds"); $("#recordSoundDialog").popup("open"); recInterval = setInterval(function() { recTime = recTime + 1; $("#soundDuration").html("Duration: " + recTime + " seconds"); }, 1000); }); The following actions are performed in the "recordSound" tap event handler: A call to the startRecordingSound(recordingCallback) function is performed. The startRecordingSound(recordingCallback) function is a helper function that starts the sound recording process using the Apache Cordova Media API. Its recordingCallback parameter represents a JSON object, which has the recordSuccess and recordError callback attributes. The recordSuccess callback will be called if the recording operation is a success, and the recordError callback will be called if the recording operation is a failure. Then, the "recordSoundDialog" dialog is opened and its "soundDuration" div is updated every second with the duration of the recorded sound. The following code snippet shows the startRecordingSound(recordingCallback), stopRecordingSound(), and requestApplicationDirectory(callback) functions: var BASE_DIRECTORY = "CS_Recorder"; var recordingMedia; function startRecordingSound(recordingCallback) { var recordVoice = function(dirPath) { var basePath = ""; if (dirPath) { basePath = dirPath + "/"; } var mediaFilePath = basePath + (new Date()).getTime() + ".wav"; var recordingSuccess = function() { recordingCallback.recordSuccess(mediaFilePath); }; recordingMedia = new Media(mediaFilePath, recordingSuccess, recordingCallback.recordError); // Record audio recordingMedia.startRecord(); }; if (device.platform === "Android") { var callback = {}; callback.requestSuccess = recordVoice; callback.requestError = recordingCallback.recordError; requestApplicationDirectory(callback); } else { recordVoice(); } } function stopRecordingSound() { recordingMedia.stopRecord(); recordingMedia.release(); } function requestApplicationDirectory(callback) { var directoryReady = function (dirEntry) { callback.requestSuccess(dirEntry.toURL()); }; var fileSystemReady = function(fileSystem) { fileSystem.root.getDirectory(BASE_DIRECTORY, {create: true}, directoryReady); }; window.requestFileSystem(LocalFileSystem.PERSISTENT, 0, fileSystemReady, callback.requestError); } The next section illustrates the preceding code snippet. Recording and playing the audio files back In order to record the audio files using Apache Cordova, we need to create a Media object, as follows: recordingMedia = new Media(src, mediaSuccess, mediaError); The Media object constructor has the following parameters: src: This refers to the URI of the media file mediaSuccess: This refers to the callback that will be invoked if the media operation (play/record or stop function) succeeds mediaError: This refers to the callback that will be invoked if the media operation (again a play/record or stop function) fails In order to start recording an audio file, a call to the startRecord() method of the Media object must be performed. When the recording is over, a call to stopRecord() of the Media object method must be performed. In startRecordingSound(recordingCallback), the function gets the current device platform by using device.platform, as follows: If the current platform is Android, then a call to requestApplicationDirectory(callback) is performed in order to create an application directory (if it is not already created) called "CS_Recorder" under the device's SD card root directory using the Apache Cordova File API. If the directory creation operation succeeds, recordVoice() will be called by passing the application directory path as a parameter. The recordVoice() function starts recording the sound and saves the resulting audio file under the application directory. Note that if there is no SD card in your Android device, then the application directory will be created under the app's private data directory (/data/data/[app_directory]), and the audio file will be saved under it. In the else block which refers to the other supported platforms (Windows Phone 8 and iOS), recordVoice() is called without creating an application-specific directory. At the time of writing this article, in iOS and Windows Phone 8, every application has a private directory, and applications cannot store their files in any place other than this directory, using the Apache Cordova APIs. In the case of iOS, the audio files will be stored under the tmp directory of the application's sandbox directory (the application's private directory). In the case of Windows Phone 8, the audio files will be stored under the application's local directory. Note that using the native Windows Phone 8 API (Window.Storage), you can read and write files in an SD card with some restrictions. However, until the moment you cannot do this using Apache Cordova; hopefully this capability will soon be supported by Cordova (http://msdn.microsoft.com/en-us/library/windows/apps/xaml/dn611857.aspx). In recordVoice(), it starts creating a media file using the Media object's startRecord() function. After calling the media file's stopRecord() function and after the success of the recording operation, recordingCallback.recordSuccess will be called by recordingSuccess. The recordingCallback.recordSuccess function calls handleRecordSuccess, passing the audio file's full path mediaFilePath as a parameter. The following code snippet shows the handleRecordSuccess function: function handleRecordSuccess(currentFilePath) { $("#location").val(currentFilePath); $("#playSound").closest('.ui-btn').show(); } The handleRecordSuccess function stores the recorded audio filepath in the "location" hidden field, which is used later by the playback button, and shows the "playSound" button. In requestApplicationDirectory(callback), which is called in case of Android, it does the following: Calls window.requestFileSystem in order to request the device filesystem before performing any file operation(s) Calls fileSystem.root.getDirectory when the filesystem is ready in order to create our custom application directory When our custom application directory is created successfully, the path of the created directory, or the existing directory, is passed to recordVoice() that was illustrated earlier In the other application actions, the following code snippet shows the "stopRecordingSound" tapping and "recordSoundDialog" closing event handlers: $("#recordSoundDialog").on("popupafterclose", function(event, ui) { clearInterval(recInterval); stopRecordingSound(); }); $("#stopRecordingSound").on("tap", function(e) { $("#recordSoundDialog").popup("close"); }); function stopRecordingSound(recordingCallback) { recordingMedia.stopRecord(); recordingMedia.release(); } In the "stopRecordingSound" tapping event handler, it closes the open "recordSoundDialog" pop up. Generally, if "recordSoundDialog" is closed by the "stopRecordingSound" button's tapping action or by pressing special device keys, such as the back button in Android devices, then the recording timer stops as a result of calling clearInterval(recInterval), and then it calls the stopRecordingSound() function to stop recording the sound. The stopRecordingSound() function calls the Media object's stopRecord() method, and then releases it by calling the Media object's release() method. The following code snippet shows the "playSound" tap event handler: var audioMedia; var recordingMedia; $("#playSound").on("tap", function(e) { e.preventDefault(); var playCallback = {}; playCallback.playSuccess = handlePlaySuccess; playCallback.playError = handlePlayError; playSound($("#location").val(), playCallback); }); function playSound(filePath, playCallback) { if (filePath) { cleanUpResources(); audioMedia = new Media(filePath, playCallback.playSuccess, playCallback.playError); // Play audio audioMedia.play(); } } function cleanUpResources() { if (audioMedia) { audioMedia.stop(); audioMedia.release(); audioMedia = null; } if (recordingMedia) { recordingMedia.stop(); recordingMedia.release(); recordingMedia = null; } } In the "playSound" tap event handler, it calls the playSound(filePath, playCallback) function by passing the audio file location, which is stored in the "location" hidden field and playCallback. The playSound(filePath, playCallback) function uses the Media object's play() method to play back the saved audio file after releasing used Media objects. Note that this is a requirement to avoid running out of system audio resources. Building and running Sound Recorder application Now, after developing our application code, we can start building our application using the following cordova build command: > cordova build In order to run the application in your Android mobile or tablet, just make sure you enable USB debugging in your Android device. Then, plug your Android device into your development machine and execute the following command from the application directory: > cordova run android Congratulations! After running this command, you will see the Sound Recorder application deployed in your Android device; you can now start testing it on your real device. Summary In this article, you developed your first Apache Cordova application. You now know how to use the Apache Cordova Device API at a basic level. You also know how to use the Media and File APIs along with jQuery mobile to develop the Sound Recorder application. You now understand how to use Apache Cordova CLI in order to manage your Cordova mobile application. In addition, you know how to create a Cordova project, add a new platform (in our case, Android), build your own Cordova mobile application, and deploy your Cordova mobile application to the emulator, and most importantly, to a real device! To learn more, refer to these books: Creating Mobile Apps with jQuery Mobile (https://www.packtpub.com/web-development/creating-mobile-apps-jquery-mobile) Building Mobile Applications Using Kendo UI Mobile and ASP. NET Web API (https://www.packtpub.com/application-development/building-mobile-applications-using-kendo-ui-mobile-and-aspnet-web-api) jQuery Mobile First Look (https://www.packtpub.com/web-development/jquery-mobile-first-look) jQuery Mobile Web Development Essentials (https://www.packtpub.com/web-development/jquery-mobile-web-development-essentials-second-edition) Resources for Article: Further resources on this subject: Understanding mutability and immutability in Python, C#, and JavaScript[article] Object Detection Using Image Features in JavaScript[article] Learning Node.js for Mobile Application Development[article]
Read more
  • 0
  • 0
  • 7059
Modal Close icon
Modal Close icon