Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Save more on your purchases! discount-offer-chevron-icon
Savings automatically calculated. No voucher code required.
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Events
Videos
Audiobooks
Packt Hub
Free Learning
Arrow right icon
timer SALE ENDS IN
0 Days
:
00 Hours
:
00 Minutes
:
00 Seconds

How-To Tutorials

7018 Articles
article-image-implementing-apache-spark-mllib-naive-bayes-to-classify-digital-breath-test-data-for-drunk-driving
Savia Lobo
27 Feb 2018
13 min read
Save for later

Implementing Apache Spark MLlib Naive Bayes to classify digital breath test data for drunk driving

Savia Lobo
27 Feb 2018
13 min read
[box type="note" align="" class="" width=""]This article is an excerpt taken from a book Mastering Apache Spark 2.x - Second Edition written by Romeo Kienzler. In this book, you will understand how memory management and binary processing, cache-aware computation, and code generation are used to speed up things  dramatically.[/box] This article provides a working example of the Apache Spark MLlib Naive Bayes algorithm on the Road Safety - Digital Breath Test Data 2013. It will describe the theory behind the algorithm and will provide a step-by-step example in Scala to show how the algorithm may be used. Theory on Classification In order to use the Naive Bayes algorithm to classify a dataset, data must be linearly divisible; that is, the classes within the data must be linearly divisible by class boundaries. The following figure visually explains this with three datasets and two class boundaries shown via the dotted lines: Naive Bayes assumes that the features (or dimensions) within a dataset are independent of one another; that is, they have no effect on each other. The following example considers the classification of e-mails as spam. If you have 100 e-mails, then perform the following: 60% of emails are spam 80% of spam emails contain the word buy 20% of spam emails don't contain the word buy 40% of emails are not spam 10% of non spam emails contain the word buy 90% of non spam emails don't contain the word buy Let's convert this example into conditional probabilities so that a Naive Bayes classifier can pick it up: P(Spam) = the probability that an email is spam = 0.6 P(Not Spam) = the probability that an email is not spam = 0.4 P(Buy|Spam) = the probability that an email that is spam has the word buy = 0.8 P(Buy|Not Spam) = the probability that an email that is not spam has the word buy = 0.1 What is the probability that an e-mail that contains the word buy is spam? Well, this would be written as P (Spam|Buy). Naive Bayes says that it is described by the equation in the following figure: So, using the previous percentage figures, we get the following: P(Spam|Buy) = ( 0.8 * 0.6 ) / (( 0.8 * 0.6 ) + ( 0.1 * 0.4 ) ) = ( .48 ) / ( .48 + .04 ) = .48 / .52 = .923 This means that it is 92 percent more likely that an e-mail that contains the word buy is spam. That was a look at the theory; now it's time to try a real-world example using the Apache Spark MLlib Naive Bayes algorithm. Naive Bayes in practice The first step is to choose some data that will be used for classification. We have chosen some data from the UK Government data website at http://data.gov.uk/dataset/road- accidents-safety-data. The dataset is called Road Safety - Digital Breath Test Data 2013, which downloads a zipped text file called DigitalBreathTestData2013.txt. This file contains around half a million rows. The data looks as follows: Reason,Month,Year,WeekType,TimeBand,BreathAlcohol,AgeBand,Gender Suspicion of Alcohol,Jan,2013,Weekday,12am-4am,75,30-39,Male Moving Traffic Violation,Jan,2013,Weekday,12am-4am,0,20-24,Male Road Traffic Collision,Jan,2013,Weekend,12pm-4pm,0,20-24,Female In order to classify the data, we have modified both the column layout and the number of columns. We have simply used Excel, given the data volume. However, if our data size had been in the big data range, we would have had to run some Scala code on top of Apache Spark for ETL (Extract Transform Load). As the following commands show, the data now resides in HDFS in the directory named /data/spark/nbayes. The file name is called DigitalBreathTestData2013- MALE2.csv. The line count from the Linux wc command shows that there are 467,000 rows. Finally, the following data sample shows that we have selected the columns, Gender, Reason, WeekType, TimeBand, BreathAlcohol, and AgeBand to classify. We will try to classify on the Gender column using the other columns as features: [hadoop@hc2nn ~]$ hdfs dfs -cat /data/spark/nbayes/DigitalBreathTestData2013-MALE2.csv | wc -l 467054 [hadoop@hc2nn ~]$ hdfs dfs -cat /data/spark/nbayes/DigitalBreathTestData2013-MALE2.csv | head -5 Male,Suspicion of Alcohol,Weekday,12am-4am,75,30-39 Male,Moving Traffic Violation,Weekday,12am-4am,0,20-24 Male,Suspicion of Alcohol,Weekend,4am-8am,12,40-49 Male,Suspicion of Alcohol,Weekday,12am-4am,0,50-59 Female,Road Traffic Collision,Weekend,12pm-4pm,0,20-24 The Apache Spark MLlib classification function uses a data structure called LabeledPoint, which is a general purpose data representation defined at http://spark.apache.org/docs/1.0.0/api/scala/index.html#org.apache.spark.mllib.regression.LabeledPoint and https://spark.apache.org/docs/latest/mllib-data-types.html#labeled-point. This structure only accepts double values, which means that the text values in the previous data need to be classified numerically. Luckily, all of the columns in the data will convert to numeric categories, and we have provided a program in the software package with this book under the chapter2naive bayes directory to do just that. It is called convert.scala. It takes the contents of the DigitalBreathTestData2013- MALE2.csv file and converts each record into a double vector. The directory structure and files for an sbt Scala-based development environment have already been described earlier. We are developing our Scala code on the Linux server using the Linux account, Hadoop. Next, the Linux pwd and ls commands show our top-level nbayes development directory with the bayes.sbt configuration file, whose contents have already been examined: [hadoop@hc2nn nbayes]$ pwd /home/hadoop/spark/nbayes [hadoop@hc2nn nbayes]$ ls bayes.sbt target   project   src The Scala code to run the Naive Bayes example is in the src/main/scala subdirectory under the nbayes directory: [hadoop@hc2nn scala]$ pwd /home/hadoop/spark/nbayes/src/main/scala [hadoop@hc2nn scala]$ ls bayes1.scala convert.scala We will examine the bayes1.scala file later, but first, the text-based data on HDFS must be converted into numeric double values. This is where the convert.scala file is used. The code is as follows: import org.apache.spark.SparkContext import org.apache.spark.SparkContext._ import org.apache.spark.SparkConf These lines import classes for the Spark context, the connection to the Apache Spark cluster, and the Spark configuration. The object that is being created is called convert1. It is an application as it extends the App class: object convert1 extends App { The next line creates a function called enumerateCsvRecord. It has a parameter called colData, which is an array of Strings and returns String: def enumerateCsvRecord( colData:Array[String]): String = { The function then enumerates the text values in each column, so, for instance, Male becomes 0. These numeric values are stored in values such as colVal1: val colVal1 = colData(0) match { case "Male"                                                          => 0 case "Female"                                                     => 1 case "Unknown"                                                   => 2 case _                                                                     => 99 } val colVal2 = colData(1) match { case "Moving Traffic Violation"           => 0 case "Other"     => 1 case "Road Traffic Collision"                => 2 case "Suspicion of Alcohol"                                                                 => 3 case _                                                                     => 99 } val colVal3 = colData(2) match { case "Weekday"                                                     => 0 case "Weekend"                                                     => 0 case _                                                                        => 99 } val colVal4 = colData(3) match { case "12am-4am"                                                => 0 case "4am-8am"                                                   => 1 case "8am-12pm"                                                => 2 case "12pm-4pm"                                                => 3 case "4pm-8pm"                                                   => 4 case "8pm-12pm"                                                => 5 case _                                                                     => 99 } val colVal5 = colData(4) val colVal6 = colData(5) match { case "16-19"                                                          => 0 case "20-24"                                                          => 1 case "25-29"                                                          => 2 case "30-39"                                                          => 3 case "40-49"                                                          => 4 case "50-59"                                                          => 5 case "60-69"                                                          => 6 case "70-98"                                                          => 7 case "Other"                                                          => 8 case _                                                                        => 99 } Note: A comma-separated string called lineString is created from the numeric column        values and is then returned. The function closes with the final brace character. Note that the data line created next starts with a label value at column one and is followed by a vector, which represents the data. The vector is space-separated while the label is separated from the vector by a comma. Using these two separator types allows us to process both--the label and vector--in two simple steps: val lineString = colVal1+","+colVal2+" "+colVal3+" "+colVal4+" "+colVal5+" "+colVal6 return lineString } The main script defines the HDFS server name and path. It defines the input file and the output path in terms of these values. It uses the Spark URL and application name to create a new configuration. It then creates a new context or connection to Spark using these details: val hdfsServer = "hdfs://localhost:8020" val hdfsPath     = "/data/spark/nbayes/" val inDataFile = hdfsServer + hdfsPath + "DigitalBreathTestData2013- MALE2.csv" val outDataFile = hdfsServer + hdfsPath + "result" val sparkMaster = "spark://localhost:7077" val appName = "Convert 1" Val sparkConf = new SparkConf() sparkConf.setMaster(sparkMaster) sparkConf.setAppName(appName) val sparkCxt = new SparkContext(sparkConf) The CSV-based raw data file is loaded from HDFS using the Spark context textFile method. Then, a data row count is printed: val csvData = sparkCxt.textFile(inDataFile) println("Records in : "+ csvData.count() ) The CSV raw data is passed line by line to the enumerateCsvRecord function. The returned string-based numeric data is stored in the enumRddData variable: val enumRddData = csvData.map { csvLine => val colData = csvLine.split(',') enumerateCsvRecord(colData) } Finally, the number of records in the enumRddData variable is printed, and the enumerated data is saved to HDFS: println("Records out : "+ enumRddData.count() ) enumRddData.saveAsTextFile(outDataFile) } // end object In order to run this script as an application against Spark, it must be compiled. This is carried out with the sbt package command, which also compiles the code. The following command is run from the nbayes directory: [hadoop@hc2nn nbayes]$ sbt package Loading /usr/share/sbt/bin/sbt-launch-lib.bash .... [info] Done packaging. [success] Total time: 37 s, completed Feb 19, 2015 1:23:55 PM This causes the compiled classes that are created to be packaged into a JAR library, as shown here: [hadoop@hc2nn nbayes]$ pwd /home/hadoop/spark/nbayes [hadoop@hc2nn nbayes]$ ls -l target/scala-2.10 total 24 drwxrwxr-x 2 hadoop hadoop 4096 Feb 19 13:23 classes -rw-rw-r-- 1 hadoop hadoop 17609 Feb 19 13:23 naive-bayes_2.10-1.0.jar The convert1 application can now be run against Spark using the application name, Spark URL, and full path to the JAR file that was created. Some extra parameters specify memory and the maximum cores that are supposed to be used: spark-submit --class convert1 --master spark://localhost:7077 --executor-memory 700M --total-executor-cores 100 /home/hadoop/spark/nbayes/target/scala-2.10/naive-bayes_2.10-1.0.jar This creates a data directory on HDFS called /data/spark/nbayes/ followed by the result, which contains part files with the processed data: [hadoop@hc2nn nbayes]$ hdfs dfs -ls /data/spark/nbayes Found 2 items -rw-r--r--   3 hadoop supergroup 24645166 2015-01-29 21:27 /data/spark/nbayes/DigitalBreathTestData2013-MALE2.csv drwxr-xr-x   - hadoop supergroup       0 2015-02-19 13:36 /data/spark/nbayes/result [hadoop@hc2nn nbayes]$ hdfs dfs -ls /data/spark/nbayes/result Found 3 items -rw-r--r--   3 hadoop supergroup       0 2015-02-19 13:36 /data/spark/nbayes/result/_SUCCESS -rw-r--r--   3 hadoop supergroup 2828727 2015-02-19 13:36 /data/spark/nbayes/result/part-00000 -rw-r--r--   3 hadoop supergroup 2865499 2015-02-19 13:36 /data/spark/nbayes/result/part-00001 In the following HDFS cat command, we concatenated the part file data into a file called DigitalBreathTestData2013-MALE2a.csv. We then examined the top five lines of the file using the head command to show that it is numeric. Finally, we loaded it in HDFS with the put command: [hadoop@hc2nn nbayes]$ hdfs dfs -cat /data/spark/nbayes/result/part* > ./DigitalBreathTestData2013-MALE2a.csv 0,3 0 0 75 3 0,0 0 0 0 1 0,3 0 1 12 4 0,3 0 0 0 5 1,2 0 3 0 1 [hadoop@hc2nn nbayes]$ head -5 DigitalBreathTestData2013-MALE2a.csv [hadoop@hc2nn nbayes]$ hdfs dfs -put ./DigitalBreathTestData2013-MALE2a.csv /data/spark/nbayes The following HDFS ls command now shows the numeric data file stored on HDFS in the nbayes directory: [hadoop@hc2nn nbayes]$ hdfs dfs -ls /data/spark/nbayes Found 3 items -rw-r--r--   3 hadoop supergroup 24645166 2015-01-29 21:27 /data/spark/nbayes/DigitalBreathTestData2013-MALE2.csv -rw-r--r--   3 hadoop supergroup 5694226 2015-02-19 13:39 /data/spark/nbayes/DigitalBreathTestData2013-MALE2a.csv drwxr-xr-x - hadoop supergroup                           0 2015-02-19 13:36 /data/spark/nbayes/result Now that the data has been converted into a numeric form, it can be processed with the MLlib Naive Bayes algorithm; this is what the Scala file, bayes1.scala, does. This file imports the same configuration and context classes as before. It also imports MLlib classes for Naive Bayes, vectors, and the LabeledPoint structure. The application class that is created this time is called bayes1: import org.apache.spark.SparkContext import org.apache.spark.SparkContext._ import org.apache.spark.SparkConf import org.apache.spark.mllib.classification.NaiveBayes import org.apache.spark.mllib.linalg.Vectors import org.apache.spark.mllib.regression.LabeledPoint object bayes1 extends App { The HDFS data file is again defined, and a Spark context is created as before: val hdfsServer = "hdfs://localhost:8020" val hdfsPath      = "/data/spark/nbayes/" val dataFile = hdfsServer+hdfsPath+"DigitalBreathTestData2013-MALE2a.csv" val sparkMaster = "spark://loclhost:7077" val appName = "Naive Bayes 1" val conf = new SparkConf() conf.setMaster(sparkMaster) conf.setAppName(appName) val sparkCxt = new SparkContext(conf) The raw CSV data is loaded and split by the separator characters. The first column becomes the label (Male/Female) that the data will be classified on. The final columns separated by spaces become the classification features: val csvData = sparkCxt.textFile(dataFile) val ArrayData = csvData.map { csvLine => val colData = csvLine.split(',') LabeledPoint(colData(0).toDouble, Vectors.dense(colData(1) .split('') .map(_.toDouble) ) ) } The data is then randomly divided into training (70%) and testing (30%) datasets: val divData = ArrayData.randomSplit(Array(0.7, 0.3), seed = 13L) val trainDataSet = divData(0) val testDataSet = divData(1) The Naive Bayes MLlib function can now be trained using the previous training set. The trained Naive Bayes model, held in the nbTrained variable, can then be used to predict the Male/Female result labels against the testing data: val nbTrained = NaiveBayes.train(trainDataSet) val nbPredict = nbTrained.predict(testDataSet.map(_.features)) Given that all of the data already contained labels, the original and predicted labels for the test data can be compared. An accuracy figure can then be computed to determine how accurate the predictions were, by comparing the original labels with the prediction values: val predictionAndLabel = nbPredict.zip(testDataSet.map(_.label)) val accuracy = 100.0 * predictionAndLabel.filter(x => x._1 == x._2).count() / testDataSet.count() println( "Accuracy : " + accuracy ); } So, this explains the Scala Naive Bayes code example. It's now time to run the compiled bayes1 application using spark-submit and determine the classification accuracy. The parameters are the same. It's just the class name that has changed: spark-submit --class bayes1 --master spark://hc2nn.semtech-solutions.co.nz:7077 --executor-memory 700M --total-executor-cores 100 /home/hadoop/spark/nbayes/target/scala-2.10/naive-bayes_2.10-1.0.jar The resulting accuracy given by the Spark cluster is just 43 percent, which seems to imply that this data is not suitable for Naive Bayes: Accuracy: 43.30 We have seen how with the help of Apache Spark MLib, one can perform a successful classification on Naive Bayes algorithm. If you found this post useful, do check  out this book Mastering Apache Spark 2.x - Second Edition to learn about the latest enhancements to Apache Spark 2.x, such as interactive querying of live data and unifying DataFrames and Datasets.  
Read more
  • 0
  • 0
  • 8977

Packt
17 Feb 2016
72 min read
Save for later

Speaking Java – Your First Game

Packt
17 Feb 2016
72 min read
In this article, we will start writing our very own Java code at the same time as we begin understanding Java syntax. We will learn how to store, retrieve, and manipulate different types of values stored in the memory. We will also look at making decisions and branching the flow of our code based on the values of this data. In this order, we will: Learn some Java syntax and see how it is turned into a running app by the compiler Store data and use it with variables Express yourself in Java with expressions Continue with the math game by asking a question Learn about decisions in Java Continue with the math game by getting and checking the answer (For more resources related to this topic, see here.) Acquiring the preceding Java skills will enable us to build the next two phases of our math game. This game will be able to ask the player a question on multiplication, check the answer and give feedback based on the answer given, as shown in the following screenshot: Java syntax Throughout this book, we will use plain English to discuss some fairly technical things. You will never be asked to read a technical explanation of a Java or Android concept that has not been previously explained in a non-technical way. Occasionally, I might ask or imply that you accept a simplified explanation in order to offer a fuller explanation at a more appropriate time, like the Java class as a black box; however, never will you need to scurry to Google in order to get your head around a big word or a jargon-filled sentence. Having said that, the Java and Android communities are full of people who speak in technical terms and to join in and learn from these communities, you need to understand the terms they use. So the approach this book takes is to learn a concept or appreciate an idea using an entirely plain speaking language, but at the same time, it introduces the jargon as part of the learning. Then, much of the jargon will begin to reveal its usefulness, usually as a way of clarification or keeping the explanation/discussion from becoming longer than it needs to be. The very term, "Java syntax," could be considered technical or jargon. So what is it? The Java syntax is the way we put together the language elements of Java in order to produce code that works in the Java/Dalvik virtual machine. Syntax should also be as clear as possible to a human reader, not least ourselves when we revisit our programs in the future. The Java syntax is a combination of the words we use and the formation of those words into sentence like structures. These Java elements or words are many in number, but when taken in small chunks are almost certainly easier to learn than any human-spoken language. The reason for this is that the Java language and its syntax were specifically designed to be as straightforward as possible. We also have Android Studio on our side which will often let us know if we make a mistake and will even sometimes think ahead and prompt us. I am confident that if you can read, you can learn Java; because learning Java is much easy. What then separates someone who has finished an elementary Java course from an expert programmer? The same things that separate a student of language from a master poet. Mastery of the language comes through practice and further study. The compiler The compiler is what turns our human-readable Java code into another piece of code that can be run in a virtual machine. This is called compiling. The Dalvik virtual machine will run this compiled code when our players tap on our app icon. Besides compiling Java code, the compiler will also check for mistakes. Although we might still have mistakes in our released app, many are stopped discovered at the when our code is compiled. Making code clear with comments As you become more advanced in writing Java programs, the solutions you use to create your programs will become longer and more complicated. Java was designed to manage complexity by having us divide our code into separate chunks, very often across multiple files. Comments are a part of the Java program that do not have any function in the program itself. The compiler ignores them. They serve to help the programmer to document, explain, and clarify their code to make it more understandable to themselves at a later date or to other programmers who might need to use or modify the code. So, a good piece of code will be liberally sprinkled with lines like this: //this is a comment explaining what is going on The preceding comment begins with the two forward slash characters, //. The comment ends at the end of the line. It is known as a single-line comment. So anything on that line is for humans only, whereas anything on the next line (unless it's another comment) needs to be syntactically correct Java code: //I can write anything I like here but this line will cause an error We can use multiple single-line comments: //Below is an important note //I am an important note //We can have as many single line comments like this as we like Single-line comments are also useful if we want to temporarily disable a line of code. We can put // in front of the code and it will not be included in the program. Recall this code, which tells Android to load our menu UI: //setContentView(R.layout.activity_main); In the preceding situation, the menu will not be loaded and the app will have a blank screen when run, as the entire line of code is ignored by the compiler. There is another type of comment in Java—the multiline comment. This is useful for longer comments and also to add things such as copyright information at the top of a code file. Also like the single-line comment, it can be used to temporarily disable code, in this case usually multiple lines. Everything in between the leading /* signs and the ending */ signs is ignored by the compiler. Here are some examples: /* This program was written by a Java expert You can tell I am good at this because my code has so many helpful comments in it. */ There is no limit to the number of lines in a multiline comment. Which type of comment is best to use will depend upon the situation. In this book, I will always explain every line of code explicitly but you will often find liberally sprinkled comments within the code itself that add further explanation, insight or clarification. So it's always a good idea to read all of the code: /* The winning lottery numbers for next Saturday are 9,7,12,34,29,22 But you still want to learn Java? Right? */ All the best Java programmers liberally sprinkle their code with comments. Storing data and using it with variables We can think of a variable as a labeled storage box. They are also like a programmer's window to the memory of the Android device, or whatever device we are programming. Variables can store data in memory (the storage box), ready to be recalled or altered when necessary by using the appropriate label. Computer memory has a highly complex system of addressing that we, fortunately, do not need to interact with in Java. Java variables allow us to make up convenient names for all the data that we want our program to work with; the JVM will handle all the technicalities that interact with the operating system, which in turn, probably through several layers of buck passing, will interact with the hardware. So we can think of our Android device's memory as a huge warehouse. When we assign names to our variables, they are stored in the warehouse, ready when we need them. When we use our variable's name, the device knows exactly what we are referring to. We can then tell it to do things such as "get box A and add it to box C, delete box B," and so on. In a game, we will likely have a variable named as something along the lines of score. It would be this score variable using which we manage anything related to the user's score, such as adding to it, subtracting or perhaps just showing it to the player. Some of the following situations that might arise: The player gets a question right, so add 10 to their existing score The player views their stats screen, so print score on the screen The player gets the best score ever, so make hiScore the same as their current score These are fairly arbitrary examples of names for variables and as long as you don't use any of the characters keywords that Java restricts, you can actually call your variables whatever you like. However, in practice, it is best to adopt a naming convention so that your variable names will be consistent. In this book, we will use a loose convention of variable names starting with a lowercase letter. When there is more than one word in the variable's name, the second word will begin with an uppercase letter. This is called "camel casing." Here are some examples of camel casing: score hiScore playersPersonalBest Before we look at some real Java code with some variables, we need to first look at the types of variables we can create and use. Types of variables It is not hard to imagine that even a simple game will probably have quite a few variables. What if the game has a high score table that remembers the names of the top 10 players? Then we might need variables for each player. And what about the case when a game needs to know if a playable character is dead or alive, or perhaps has any lives/retries left? We might need code that tests for life and then ends the game with a nice blood spurt animation if the playable character is dead. Another common requirement in a computer program, including games, is the right or wrong calculation: true or false. To cover almost these and many other types of information you might want to keep track of, every possibility Java has types. There are many types of variables and, we can also invent our own types or use other people's types. But for now, we will look at the built-in Java types. To be fair, they cover just about every situation we are likely to run into for a while. Some examples are the best way to explain this type of stuff. We have already discussed the hypothetical but highly likely score variable. The sWell score is likely to be a number, so we have to convey this (that the score is a number) to the Java compiler by giving the score an appropriate type. The hypothetical but equally likely playerName will of course, hold the characters that make up the player's name. Jumping ahead a couple of paragraphs, the type that holds a regular number is called an int, and the type that holds name-like data is called a string. And if we try and store a player name, perhaps "Ada Lovelace" in score, which is meant for numbers, we will certainly run into trouble. The compiler says no! Actually, the error would say this: As we can see, Java was designed to make it impossible for such errors to make it to a running program. Did you also spot in the previous screenshot that I had forgotten the semicolon at the end of the line? With this compiler identifying our errors, what could possibly go wrong? Here are the main types in Java. Later, we will see how to start using them: int: This type is used to store integers. It uses 32 pieces (bits) of memory and can therefore store values with a magnitude a little in excess of 2 billion, including negative values. long: As the name hints at, this data types can be used when even larger numbers are required. A long data type uses 64 bits of memory and 2 to the power of 63 is what we can store in this type. If you want to see what that looks like, try this: 9,223,372,036,854,775,807. Perhaps surprisingly, there are uses for long variables but if a smaller variable will do, we should use it so that our program uses less memory. You might be wondering when you might use numbers of this magnitude. The obvious examples would be math or science applications that do complex calculations but another use might be for timing. When you time how long something takes, the Java Date class uses the number of milliseconds since January 1, 1970. The long data type could be useful to subtract a start time from an end time to determine an elapsed time. float: This is for floating-point numbers, that is, numbers where there is precision beyond the decimal point. As the fractional part of a number takes memory space just as the whole number portion, the range of numbers possible in a float is therefore decreased compared to non-floating-point numbers. So, unless our variable will definitely use the extra precision, float would not be our data type of choice. double: When the precision in float is not enough we have double. short: When even an int data type is overkill, the super-skinny short fits into the tiniest of storage boxes, but we can only store around 64,000 values, from -32,768 to 32,767. byte: This is an even smaller storage box than a short type. There is plenty of room for these in memory but a byte can only store values from -128 to 127. boolean: We will be using plenty of Booleans throughout the book. A Boolean variable can be either true or false—nothing else. Perhaps Boolean answer questions such as: Is the player alive? Has a new high score been reached? Are two examples for a Boolean variable enough? char: This stores a single alphanumeric character. It's not going to change anything on its own but it could be useful if we put lots of them together. I have kept this discussion of data types to a practical level that is useful in the context of this book. If you are interested in how a data type's value is stored and why the limits are what they are, visit the Oracle Java tutorials site at http://docs.oracle.com/javase/tutorial/java/nutsandbolts/datatypes.html. Note that you do not need any more information than we have already discussed to continue with this book. As we just learned, each type of data that we might want to store will require a specific amount of memory. So we must let the Java compiler know the type of the variable before we begin to use it. The preceding variables are known as the primitive types. They use predefined amounts of memory and so, using our storage analogy, fit into predefined sizes of the storage box. As the "primitive" label suggests, they are not as sophisticated as the reference types. Reference types You might have noticed that we didn't cover the string variable type that we previously used to introduce the concept of variables. Strings are a special type of variable known as a reference type. They quite simply refer to a place in memory where the storage of the variable begins, but the reference type itself does not define a specific amount of memory. The reason for this is fairly straightforward: we don't always know how much data will need to be stored until the program is actually run. We can think of strings and other reference types as continually expanding and contracting storage boxes. So won't one of these string reference types bump into another variable eventually? If you think about the devices memory as a huge warehouse full of racks of labeled storage boxes, then you can think of the Dalvik virtual machine as a super-efficient forklift truck driver that puts the different types of storage boxes in the most appropriate place. And if it becomes necessary, the virtual machine will quickly move stuff around in a fraction of a second to avoid collisions. It will even incinerate unwanted storage boxes when appropriate. This happens at the same time as constantly unloading new storage boxes of all types and placing them in the best place, for that type of variable. Dalvik tends to keep reference variables in a part of the warehouse that is different from the part for the primitive variables. So strings can be used to store any keyboard character, like a char data type but of almost any length. Anything from a player's name to an entire book can be stored in a single string. There are a couple more reference types we will explore. Arrays are a way to store lots of variables of the same type, ready for quick and efficient access. Think of an array as an aisle in our warehouse with all the variables of a certain type lined up in a precise order. Arrays are reference types, so Dalvik keeps these in the same part of the warehouse as strings. So we know that each type of data that we might want to store will require an amount of memory. Hence, we must let the Java compiler know the type of the variable before we begin to use it. Declaration That's enough of theory. Let's see how we would actually use our variables and types. Remember that each primitive type requires a specific amount of real device memory. This is one of the reasons that the compiler needs to know what type a variable will be of. So we must first declare a variable and its type before we attempt to do anything with it. To declare a variable of type int with the name score, we would type: int score; That's it! Simply state the type, in this case int, then leave a space, and type the name you want to use for this variable. Also note the semicolon on the end of the line as usual to show the compiler that we are done with this line and what follows, if anything, is not part of the declaration. For almost all the other variable types, declaration would occur in the same way. Here are some examples. The variable names are arbitrary. This is like reserving a labeled storage box in the warehouse: long millisecondsElapsed; float gravity; double accurrateGravity; boolean isAlive; char playerInitial; String playerName; Initialization Here, for each type, we initialize a value to the variable. Think about placing a value inside the storage box, as shown in the following code: score = 0; millisecondsElapsed = 1406879842000;//1st Aug 2014 08:57:22 gravity = 1.256; double accurrateGravity =1.256098; isAlive = true; playerInitial = 'C'; playerName = "Charles Babbage"; Notice that the char variable uses single quotes (') around the initialized value while the string uses double quotes ("). We can also combine the declaration and initialization steps. In the following snippet of code, we declare and initialize the same variables as we did previously, but in one step each: int score = 0; long millisecondsElapsed = 1406879842000;//1st Aug 2014 08:57:22 float gravity = 1.256; double accurrateGravity =1.256098; boolean isAlive = true; char playerInitial = 'C'; String playerName = "Charles Babbage"; Whether we declare and initialize separately or together is probably dependent upon the specific situation. The important thing is that we must do both: int a; //The line below attempts to output a to the console Log.i("info", "int a = " + a); The preceding code would cause the following result: Compiler Error: Variable a might not have been initialized There is a significant exception to this rule. Under certain circumstances variables can have default values. Changing variables with operators Of course, in almost any program, we are going to need to do something with these values. Here is a list of perhaps the most common Java operators that allow us to manipulate variables. You do not need to memorize them as we will look at every line of code when we use them for the first time: The assignment operator (=): This makesthe variable to the left of the operator the same as the value to the right. For example, hiScore = score; or score = 100;. The addition operator (+): This adds the values on either side of the operator. It is usually used in conjunction with the assignment operator, such as score = aliensShot + wavesCleared; or score = score + 100;. Notice that it is perfectly acceptable to use the same variable simultaneously on both sides of an operator. The subtraction operator (-): This subtracts the value on the right side of the operator from the value on the left. It is usually used in conjunction with the assignment operator, such as lives = lives - 1; or balance = income - outgoings;. The division operator (/): This divides the number on the left by the number on the right. Again, it is usually used in conjunction with the assignment operator, as shown in fairShare = numSweets / numChildren; or recycledValueOfBlock = originalValue / .9;. The multiplication operator (*): This multiplies variables and numbers, such as answer = 10 * 10; or biggerAnswer = 10 * 10 * 10;. The increment operator (++): This is a really neat way to add 1 to the value of a variable. The myVariable = myVariable + 1; statement is the same as myVariable++;. The decrement operator (--): You guessed it: a really neat way to subtract 1 from something. The myVariable = myVariable -1; statement is the same as myVariable--;. The formal names for these operators are slightly different from the names used here for explanation. For example, the division operator is actually one of the multiplicative operators. But the preceding names are far more useful for the purpose of learning Java and if you used the term "division operator", while conversing with someone from the Java community, they would know exactly what you mean. There are actually many more operators than these in Java. If you are curious about operators there is a complete list of them on the Java website at http://docs.oracle.com/javase/tutorial/java/nutsandbolts/operators.html. All the operators required to complete the projects in this book will be fully explained in this book. The link is provided for the curious among us. Expressing yourself in Java Let's try using some declarations, assignments and operators. When we bundle these elements together into some meaningful syntax, we call it an expression. So let's write a quick app to try some out. Here we will make a little side project so we can play with everything we have learned so far. Instead, we will simply write some Java code and examine its effects by outputting the values of variables to the Android console, called LogCat. We will see exactly how this works by building the simple project and examining the code and the console output: The following is a quick reminder of how to create a new project. Close any currently open projects by navigating to File | Close Project. Click on Start a new Android Studio project. The Create New Project configuration window will appear. Fill in the Application name field and Company Domain with packtpub.com or you could use your own company website name here instead. Now click on the Next button. On the next screen, make sure the Phone and Tablet checkbox has a tick in it. Now we have to choose the earliest version of Android we want to build our app for. Go ahead and play with a few options in the drop-down selector. You will see that the earlier the version we select, the greater is the percentage of devices our app can support. However, the trade-off here is that the earlier the version we select, the less are cutting-edge Android features available in our apps. A good balance is to select API 8: Android 2.2 (Froyo). Go ahead and do that now as shown in the next screenshot. Click on Next. Now select Blank Activity as shown in the next screenshot and click on Next again. On the next screen, simply change Activity Name to MainActivity and click on Finish. To keep our code clear and simple, you can delete the two unneeded methods (onCreateOptionsMenu and onOptionsItemSelected) and their associated @override and @import statements. However, this is not necessary for the example to work. As with all the examples and projects in this book, you can copy or review the code from the download bundle. Just create the project as described previously and paste the code from MainActivity.java file from the download bundle to the MainActivity.java file that was generated when you created the project in Android Studio. Just ensure that the package name is the same as the one you chose when the project was created. However, I strongly recommend going along with the tutorial so that we can learn how to do everything for ourselves. As this app uses the LogCat console to show its output, you should run this app on the emulator only and not on a real Android device. The app will not harm a real device, but you just won't be able to see anything happening. Create a new blank project called Expressions In Java. Now, in the onCreate method just after the line where we use the setContentView method, add this code to declare and initialize some variables: //first we declare and initialize a few variables int a = 10; String b = "Alan Turing"; boolean c = true; Now add the following code. This code simply outputs the value of our variables in a form where we can closely examine them in a minute: //Let's look at how Android 'sees' these variables //by outputting them, one at a time to the console Log.i("info", "a = " + a); Log.i("info", "b = " + b); Log.i("info", "c = " + c); Now let's change our variables using the addition operator and another new operator. See if you can work out the output values for variables a, b, and c before looking at the output and the code explanation: //Now let's make some changes a++; a = a + 10; b = b + " was smarter than the average bear Booboo"; b = b + a; c = (1 + 1 == 3);//1 + 1 is definitely 2! So false. Let's output the values once more in the same way we did in step 3, but this time, the output should be different: //Now to output them all again Log.i("info", "a = " + a); Log.i("info", "b = " + b); Log.i("info", "c = " + c); Run the program on an emulator in the usual way. You can see the output by clicking on the Android tab from our "useful tabs" area below the Project Explorer. Here is the output, with some of the unnecessary formatting stripped off: info? a = 10 info? b = Alan Turing info? c = true info? a = 21 info? b = Alan Turing was smarter than the average bear Booboo21 info? c = false Now let's discuss what happened. In step 2, we declared and initialized three variables: a: This is an int that holds the value 10 b: This is a string that holds the name of an eminent computer scientist. c: This is a Boolean that holds the value false So when we output the values in step 3, it should be no surprise that we get the following: info? a = 10 info? b = Alan Turing info? c = true In step 4, all the fun stuff happens. We add 1 to the value of our int a using the increment operator like this: a++;. Remember that a++ is the same as a = a + 1. We then add 10 to a. Note we are adding 10 to a after having already added 1. So we get this output for a 10 + 1 + 10 operation: info? a = 21 Now let's examine our string, b. We appear to be using the addition operator on our eminent scientist. What is happening is what you could probably guess. We are adding together two strings "Alan Turing" and "was smarter than the average bear Booboo." When you add two strings together it is called concatenating and the + symbol doubles as the concatenation operator. Finally, for our string, we appear to be adding int a to it. This is allowed and the value of a is concatenated to the end of b. info? b = Alan Turing was smarter than the average bear Booboo21 This does not work the other way round; you cannot add a string to an int. This makes sense as there is no logical answer. a = a + b Finally, let's look at the code that changes our Boolean, c, from true to false: c = (1+1=3);. Here, we are assigning to c the value of the expression contained within the brackets. This would be straightforward, but why the double equals (==)? We have jumped ahead of ourselves a little. The double equals sign is another operator called the comparison operator. So we are really asking, does 1+1 equal 3? Clearly the answer is false. You might ask, "why use == instead of =?" Simply to make it clear to the compiler when we mean to assign and when we mean to compare. Inadvertently using = instead of == is a very common error. The assignment operator (=) assigns the value on the right to the value on the left, while the comparison operator (==) compares the values on either side. The compiler will warn us with an error when we do this but at first glance you might swear the compiler is wrong. Now let's use everything we know and a bit more to make our Math game project. Math game – asking a question Now that we have all that knowledge under our belts, we can use it to improve our math game. First, we will create a new Android activity to be the actual game screen as opposed to the start menu screen. We will then use the UI designer to lay out a simple game screen so that we can use our Java skills with variables, types, declaration, initialization, operators, and expressions to make our math game generate a question for the player. We can then link the start menu and game screens together with a push button. If you want to save typing and just review the finished project, you can use the code downloaded from the Packt website. If you have any trouble getting any of the code to work, you can review, compare, or copy and paste the code from the already completed code provided in the download bundle. The completed code is in the following files that correspond to the filenames we will be using in this tutorial: Chapter 3/MathGameChapter3a/java/MainActivity.java Chapter 3/MathGameChapter3a/java/GameActivity.java Chapter 3/MathGameChapter3a/layout/activity_main.xml Chapter 3/MathGameChapter3a/layout/activity_game.xml As usual, I recommend following this tutorial to see how we can create all of the code for ourselves. Creating the new game activity We will first need to create a new Java file for the game activity code and a related layout file to hold the game Activity UI. Run Android Studio and select your Math Game Chapter 2 project. It might have been opened by default. Now we will create the new Android Activity that will contain the actual game screen, which will run when the player taps the Play button on our main menu screen. To create a new activity, we know need another layout file and another Java file. Fortunately Android Studio will help us do this. To get started with creating all the files we need for a new activity, right-click on the src folder in the Project Explorer and then go to New | Activity. Now click on Blank Activity and then on Next. We now need to tell Android Studio a little bit about our new activity by entering information in the above dialog box. Change the Activity Name field to GameActivity. Notice how the Layout Name field is automatically changed for us to activity_game and the Title field is automatically changed to GameActivity. Click on Finish. Android Studio has created two files for us and has also registered our new activity in a manifest file, so we don't need to concern ourselves with it. If you look at the tabs at the top of the editor window, you will see that GameActivity.java has been opened up ready for us to edit, as shown in the following screenshot: Ensure that GameActivity.java is active in the editor window by clicking on the GameActivity.java tab shown previously. Android overrides some methods for us by default, and that most of them were not necessary. Here again, we can see the code that is unnecessary. If we remove it, then it will make our working environment simpler and cleaner. To avoid this here, we will simply use the code from MainActivity.java as a template for GameActivity.java. We can then make some minor changes. Click on the MainActivity.java tab in the editor window. Highlight all of the code in the editor window using Ctrl + A on the keyboard. Now copy all of the code in the editor window using the Ctrl + C on the keyboard. Now click on the GameActivity.java tab. Highlight all of the code in the editor window using Ctrl + A on the keyboard. Now paste the copied code and overwrite the currently highlighted code using Ctrl + V on the keyboard. Notice that there is an error in our code denoted by the red underlining as shown in the following screenshot. This is because we pasted the code referring to MainActivity in our file which is called GameActivity. Simply change the text MainActivity to GameActivity and the error will disappear. Take a moment to see if you can work out what the other minor change is necessary, before I tell you. Remember that setContentView loads our UI design. Well what we need to do is change setContentView to load the new design (that we will build next) instead of the home screen design. Change setContentView(R.layout.activity_main); to setContentView(R.layout.activity_game);. Save your work and we are ready to move on. Note the Project Explorer where Android Studio puts the two new files it created for us. I have highlighted two folders in the next screenshot. In future, I will simply refer to them as our java code folder or layout files folder. You might wonder why we didn't simply copy and paste the MainActivity.java file to begin with and saved going through the process of creating a new activity? The reason is that Android Studio does things behind the scenes. Firstly, it makes the layout template for us. It also registers the new Activity for use through a file we will see later, called AndroidManifest.xml. This is necessary for the new activity to be able to work in the first place. All things considered, the way we did it is probably the quickest. The code at this stage is exactly the same as the code for the home menu screen. We state the package name and import some useful classes provided by Android: package com.packtpub.mathgamechapter3a.mathgamechapter3a; import android.app.Activity; import android.os.Bundle; We create a new activity, this time called GameActivity: public class GameActivity extends Activity { Then we override the onCreate method and use the setContentView method to set our UI design as the contents of the player's screen. Currently, however, this UI is empty: super.onCreate(savedInstanceState); setContentView(R.layout.activity_main); We can now think about the layout of our actual game screen. Laying out the game screen UI As we know, our math game will ask questions and offer the player some multiple choices to choose answers from. There are lots of extra features we could add, such as difficulty levels, high scores, and much more. But for now, let's just stick to asking a simple, predefined question and offering a choice of three predefined possible answers. Keeping the UI design to the bare minimum suggests a layout. Our target UI will look somewhat like this: The layout is hopefully self-explanatory, but let's ensure that we are really clear; when we come to building this layout in Android Studio, the section in the mock-up that displays 2 x 2 is the question and will be made up of three text views (both numbers, and the = sign is also a separate view). Finally, the three options for the answer are made up of Button layout elements. We used all of these UI elements, but this time, as we are going to be controlling them using our Java code, there are a few extra things we need to do to them. So let's go through it step by step: Open the file that will hold our game UI in the editor window. Do this by double-clicking on activity_game.xml. This is located in our UI layout folder which can be found in the project explorer. Delete the Hello World TextView, as it is not required. Find the Large Text element on the palette. It can be found under the Widgets section. Drag three elements onto the UI design area and arrange them near the top of the design as shown in the next screenshot. It does not have to be exact; just ensure that they are in a row and not overlapping, as shown in the following screenshot: Notice in the Component Tree window that each of the three TextViews has been assigned a name automatically by Android Studio. They are textView , textView2 and textView3: Android Studio refers to these element names as an id. This is an important concept that we will be making use of. So to confirm this, select any one of the textViews by clicking on its name (id), either in the component tree as shown in the preceding screenshot or directly on it in the UI designer shown previously. Now look at the Properties window and find the id property. You might need to scroll a little to do this: Notice that the value for the id property is textView. It is this id that we will use to interact with our UI from our Java code. So we want to change all the IDs of our TextViews to something useful and easy to remember. If you look back at our design, you will see that the UI element with the textView id is going to hold the number for the first part of our math question. So change the id to textPartA. Notice the lowercase t in text, the uppercase P in Part, and the uppercase A. You can use any combination of cases and you can actually name the IDs anything you like. But just as with naming conventions with Java variables, sticking to conventions here will make things less error-prone as our program gets more complicated. Now select textView2 and change id to textOperator. Select the element currently with id textView3 and change it to textPartB. This TextView will hold the later part of our question. Now add another Large Text from the palette. Place it after the row of the three TextViews that we have just been editing. This Large Text will simply hold our equals to sign and there is no plan to ever change it. So we don't need to interact with it in our Java code. We don't even need to concern ourselves with changing the ID or knowing what it is. If this situation changed, we could always come back at a later time and edit its ID. However, this new TextView currently displays Large Text and we want it to display an equals to sign. So in the Properties window, find the text property and enter the value =. We have changed the text property, and you might also like to change the text property for textPartA, textPartB, and textOperator. This is not absolutely essential because we will soon see how we can change it via our Java code; however, if we change the text property to something more appropriate, then our UI designer will look more like it will when the game runs on a real device. So change the text property of textPartA to 2, textPartB to 2, and textOperator to x. Your UI design and Component tree should now look like this: For the buttons to contain our multiple choice answers, drag three buttons in a row, below the = sign. Line them up neatly like our target design. Now, just as we did for the TextViews, find the id properties of each button, and from left to right, change the id properties to buttonChoice1, buttonChoice2, and buttonChoice3. Why not enter some arbitrary numbers for the text property of each button so that the designer more accurately reflects what our game will look like, just as we did for our other TextViews? Again, this is not absolutely essential as our Java code will control the button appearance. We are now actually ready to move on. But you probably agree that the UI elements look a little lost. It would look better if the buttons and text were bigger. All we need to do is adjust the textSize property for each TextView and for each Button. Then, we just need to find the textSize property for each element and enter a number with the sp syntax. If you want your design to look just like our target design from earlier, enter 70sp for each of the TextView textSize properties and 40sp for each of the Buttons textSize properties. When you run the game on your real device, you might want to come back and adjust the sizes up or down a bit. But we have a bit more to do before we can actually try out our game. Save the project and then we can move on. As before, we have built our UI. This time, however, we have given all the important parts of our UI a unique, useful, and easy to identify ID. As we will see we are now able to communicate with our UI through our Java code. Coding a question in Java With our current knowledge of Java, we are not yet able to complete our math game but we can make a significant start. We will look at how we can ask the player a question and offer them some multiple choice answers (one correct and two incorrect). At this stage, we know enough of Java to declare and initialize some variables that will hold the parts of our question. For example, if we want to ask the times tables question 2 x 2, we could have the following variable initializations to hold the values for each part of the question: int partA = 2; int partB = 2; The preceding code declares and initializes two variables of the int type, each to the value of 2. We use int because we will not be dealing with any decimal fractions. Remember that the variable names are arbitrary and were just chosen because they seemed appropriate. Clearly, any math game worth downloading is going to need to ask more varied and advanced questions than 2 x 2, but it is a start. Now we know that our math game will offer multiple choices as answers. So, we need a variable for the correct answer and two variables for two incorrect answers. Take a look at these combined declarations and initializations: int correctAnswer = partA * partB; int wrongAnswer1 = correctAnswer - 1; int wrongAnswer2 = correctAnswer + 1; Note that the initialization of the variables for the wrong answers depends on the value of the correct answer, and the variables for the wrong answers are initialized after initializing the correctAnswer variable. Now we need to put these values, held in our variables, into the appropriate elements on our UI. The question variables (partA and partB) need to be displayed in our UI elements, textPartA and textPartB, and the answer variables (correctAnswer, wrongAnswer1, and wrongAnswer2) need to be displayed in our UI elements with the following IDs: buttonChoice1, buttonChoice2, and buttonChoice3. We will see how we do this in the next step-by-step tutorial. We will also implement the variable declaration and initialization code that we discussed a moment ago: First, open GameActivity.java in the editor window. Remember that you can do this by double-clicking on GameActivity in our java folder or clicking on its tab above the editor window if GameActivity.java is already open. All of our code will go into the onCreate method. It will go after the setContentView(R.layout.activity_game); line but before the closing curly brace } of the onCreate method. Perhaps, it's a good idea to leave a blank line for clarity and a nice explanatory comment as shown in the following code. We can see the entire onCreate method as it stands after the latest amendments. The parts in bold are what you need to add. Feel free to add helpful comments like mine if you wish: @Override     protected void onCreate(Bundle savedInstanceState) {         super.onCreate(savedInstanceState);         //The next line loads our UI design to the screen         setContentView(R.layout.activity_game);           //Here we initialize all our variables         int partA = 9;         int partB = 9;         int correctAnswer = partA * partB;         int wrongAnswer1 = correctAnswer - 1;         int wrongAnswer2 = correctAnswer + 1;       }//onCreate ends here Now we need to add the values contained within the variables to the TextView and Button of our UI. But first, we need to get access to the UI elements we created. We do that by creating a variable of the appropriate class and linking it via the ID property of the appropriate UI element. We already know the class of our UI elements: TextView and Button. Here is the code that creates our special class variables for each of the necessary UI elements. Take a close look at the code, but don't worry if you don't understand all of it now. We will dissect the code in detail once everything is working. Enter the code immediately after the code entered in the previous step. You can leave a blank line for clarity if you wish. Just before you proceed, note that at two points while typing in this code, you will be prompted to import another class. Go ahead and do so on both occasions: /*Here we get a working object based on either the button or TextView class and base as well as link our new   objects directly to the appropriate UI elements that we created previously*/   TextView textObjectPartA =   (TextView)findViewById(R.id.textPartA);   TextView textObjectPartB =   (TextView)findViewById(R.id.textPartB);   Button buttonObjectChoice1 =   (Button)findViewById(R.id.buttonChoice1);   Button buttonObjectChoice2 =   (Button)findViewById(R.id.buttonChoice2);   Button buttonObjectChoice3 =   (Button)findViewById(R.id.buttonChoice3); In the preceding code, if you read the multiline comment, you will see that I used the term object. When we create a variable type based on a class, we call it an object. Once we have an object of a class, we can do anything that that class was designed to do. Now we have five new objects linked to the elements of our UI that we need to manipulate. What precisely are we going to do with them? We need to display the values of our variables in the text of the UI elements. We can use the objects we just created combined with a method provided by the class, and use our variables as values for that text. As usual, we will dissect this code further at the end of this tutorial. Here is the code to enter directly after the code in the previous step. Try and work out what is going on before we look at it together: //Now we use the setText method of the class on our objects //to show our variable values on the UI elements. //Just like when we output to the console in the exercise - //Expressions in Java, only now we use setText method //to put the values in our variables onto the actual UI. textObjectPartA.setText("" + partA); textObjectPartB.setText("" + partB);   //which button receives which answer, at this stage is arbitrary.   buttonObjectChoice1.setText("" + correctAnswer); buttonObjectChoice2.setText("" + wrongAnswer1); buttonObjectChoice3.setText("" + wrongAnswer2); Save your work. If you play with the assignment values for partA and partB, you can make them whatever you like and the game adjusts the answers accordingly. Obviously, we shouldn't need to reprogram our game each time we want a new question and we will solve that problem soon. All we need to do now is link the game section we have just made to the start screen menu. We will do that in the next tutorial. Now lets explore the trickier and newer parts of our code in more detail. In step 2, we declared and initialized the variables required so far: //Here we initialize all our variables int partA = 2; int partB = 2; int correctAnswer = partA * partB; int wrongAnswer1 = correctAnswer - 1; int wrongAnswer2 = correctAnswer + 1; Then in step 3, we got a reference to our UI design through our Java code. For the TextViews, it was done like this: TextView textObjectPartA = (TextView)findViewById(R.id.textPartA); For each of the buttons, a reference to our UI design was obtained like this: Button buttonObjectChoice1 =   Button)findViewById(R.id.buttonChoice1); In step 4, we did something new. We used a the setText method to show the values of our variables on our UI elements (TextView and Button) to the player. Let's break down one line completely to see how it works. Here is the code that shows the correctAnswer variable being displayed on buttonObjectChoice1. buttonObjectChoice1.setText("" + correctAnswer); By typing buttonObjectChoice1 and adding a period, as shown in the following line of code, we have access to all the preprogrammed methods of that object's class type that are provided by Android: The power of Button and the Android API There are actually lots of methods that we can perform on an object of the Button type. If you are feeling brave, try this to get a feeling of just how much functionality there is in Android. Type the following code: buttonObjectChoice1 Be sure to type the period on the end. Android Studio will pop up a list of possible methods to use on this object. Scroll through the list and get a feel of the number and variety of options: If a mere button can do all of this, think of the possibilities for our games once we have mastered all the classes contained in Android. A collection of classes designed to be used by others is collectively known as an Application Programmers Interface (API). Welcome to the Android API! In this case, we just want to set the button's text. So, we use setText and concatenate the value stored in our correctAnswer variable to the end of an empty string, like this: setText("" + correctAnswer); We do this for each of the UI elements we require to show our variables. Playing with autocomplete If you tried the previous tip, The power of Button and the Android API, and explored the methods available for objects of the Button type, you will already have some insight into autocomplete. Note that as you type, Android Studio is constantly making suggestions for what you might like to type next. If you pay attention to this, you can save a lot of time. Simply select the correct code completion statement that is suggested and press Enter. You can even see how much time you saved by selecting Help | Productivity Guide from the menu bar. Here you will see statistics for every aspect of code completion and more. Here are a few entries from mine: As you can see, if you get used to using shortcuts early on, you can save a lot of time in the long run. Linking our game from the main menu At the moment, if we run the app, we have no way for the player to actually arrive at our new game activity. We want the game activity to run when the player clicks on the Play button on the main MainActivity UI. Here is what we need to do to make that happen: Open the file activity_main.xml, either by double-clicking on it in the project explorer or by clicking its tab in the editor window. Now, just like we did when building the game UI, assign an ID to the Play button. As a reminder, click on the Play button either on the UI design or in the component tree. Find the id property in the Properties window. Assign the buttonPlay value to it. We can now make this button do stuff by referring to it in our Java code. Open the file MainActivity.java, either by double-clicking on it in the project explorer or clicking on its tab in the editor window. In our onCreate method, just after the line where we setContentView, add the following highlighted line of code: setContentView(R.layout.activity_main); Button buttonPlay = (Button)findViewById(R.id.buttonPlay); We will dissect this code in detail once we have got this working. Basically we are making a connection to the Play button by creating a reference variable to a Button object. Notice that both words are highlighted in red indicating an error. Just as before, we need to import the Button class to make this code work. Use the Alt + Enter keyboard combination. Now click on Import class from the popped-up list of options. This will automatically add the required import directive at the top of our MainActivity.java file. Now for something new. We will give the button the ability to listen to the user clicking on it. Type this immediately after the last line of code we entered: buttonPlay.setOnClickListener(this); Notice how the this keyword is highlighted in red indicating an error. Setting that aside, we need to make a modification to our code now in order to allow the use of an interface that is a special code element that allows us to add a functionality, such as listening for button clicks. Edit the line as follows. When prompted to import another class, click on OK: public class MainActivity extends Activity { to public class MainActivity extends Activity implements View.OnClickListener{ Now we have the entire line underlined in red. This indicates an error but it's where we should be at this point. We mentioned that by adding implements View.OnClickListener, we have implemented an interface. We can think of this like a class that we can use but with extra rules. The rules of the OnClickListener interface state that we must implement/use one of its methods. Notice that until now, we have optionally overridden/used methods as and when they have suited us. If we wish to use the functionality this interface provides, namely listening for button presses, then we have to add/implement the onClick method. This is how we do it. Notice the opening curly brace, {, and the closing curly brace, }. These denote the start and end of the method. Notice that the method is empty and it doesn't do anything, but an empty method is enough to comply with the rules of the OnClickListener interface, and the red line indicating an that our code has an error has gone. Make sure that you type the following code, outside the closing curly brace (}) of the onCreate method but inside the closing curly brace of our MainActivity class: @Override     public void onClick(View view) {               } Notice that we have an empty line between { and } of the onClick method. We can now add code in here to make the button actually do something. Type the following highlighted code between { and } of onClick: @Override     public void onClick(View view) {         Intent i;         i = new Intent(this, GameActivity.class);         startActivity(i);     } OK, so that code is a bit of a mouthful to comprehend all at once. See if you can guess what is happening. The clue is in the method named startActivity and the hopefully familiar term, GameActivity. Notice that we are assigning something to i. We will quickly get our app working and then diagnose the code in full. Notice that we have an error: all instances of the word Intent are red. We can solve this by importing the classes required to make Intent work. As before press Alt + Enter. Run the game in the emulator or on your device. Our app will now work. This is what the new game screen looks like after pressing Play on the menu screen: Almost every part of our code has changed a little and we have added a lot to it as well. Let's go over the contents of MainActivity.java and look at it line by line. For context, here it is in full: package com.packtpub.mathgamechapter3a.mathgamechapter3a;   import android.app.Activity; import android.content.Intent; import android.os.Bundle; import android.view.View; import android.widget.Button;     public class MainActivity extends Activity implements View.OnClickListener{       @Override     protected void onCreate(Bundle savedInstanceState) {         super.onCreate(savedInstanceState);         setContentView(R.layout.activity_main);         final Button buttonPlay =           (Button)findViewById(R.id.buttonPlay);         buttonPlay.setOnClickListener(this);     }       @Override     public void onClick(View view) {         Intent i;         i = new Intent(this, GameActivity.class);         startActivity(i);     }   } We have seen much of this code before, but let's just go over it a chunk at a time before moving on so that it is absolutely clear. The code works like this: package com.packtpub.mathgamechapter3a.mathgamechapter3a; import android.app.Activity; import android.content.Intent; import android.os.Bundle; import android.view.View; import android.widget.Button; You would probably remember that this first block of code defines what our package is called and makes available all the Android API stuff we need for Button, TextView, and Activity. From our MainActivity.java file, we have this: public class MainActivity extends Activity implements View.OnClickListener{ Our MainActivity declaration with our new bit of code implements View.OnClickListener that gives us the ability to detect button clicks. Next in our code is this: @Override     protected void onCreate(Bundle savedInstanceState) {         super.onCreate(savedInstanceState);         setContentView(R.layout.activity_main); It is at the start of our onCreate method where we first ask the hidden code of onCreate to do its stuff using super.onCreate(savedInstanceState);. Then we set our UI to the screen with setContentView(R.layout.activity_main);. Next, we get a reference to our button with an ID of buttonPlay: Button buttonPlay = (Button)findViewById(R.id.buttonPlay); buttonPlay.setOnClickListener(this); Finally, our onClick method uses the Intent class to send the player to our GameActivity class and the related UI when the user clicks on the Play button: @Override     public void onClick(View view) {         Intent i;         i = new Intent(this, GameActivity.class);         startActivity(i);     } If you run the app, you will notice that we can now click on the Play button and our math game will ask us a question. Of course, we can't answer it yet. Although we have very briefly looked at how to deal with button presses, we need to learn more of Java in order to intelligently react to them. We will also reveal how to write code to handle presses from several buttons. This will be necessary to receive input from our multiple-choice-centric game_activity UI. Decisions in Java We can now summon enough of Java prowess to ask a question but a real math game must obviously do much more than this. We need to capture the player's answer, and we are nearly there with that—we can detect button presses. From there, we need to be able to decide whether their answer is right or wrong. Then, based on this decision, we have to choose an appropriate course of action. Let's leave the math game aside for now and look at how Java might help us by learning some more fundamentals and syntax of the Java language. More operators Let's look at some more operators: we can already add (+), take away (-), multiply (*), divide (/), assign (=), increment (++), compare (==), and decrement (--) with operators. Let's introduce some more super-useful operators, and then we will go straight to actually understanding how to use them in Java. Don't worry about memorizing every operator given here. Glance at them and their explanations. There, we will put some operators to use and they will become much clearer as we see a few examples of what they allow us to do. They are presented here in a list just to make the variety and scope of operators plain from the start. The list will also be more convenient to refer back to when not intermingled with the discussion about implementation that follows it. ==: This is a comparison operator we saw this very briefly before. It tests for equality and is either true or false. An expression like (10 == 9);, for example, is false. !: The logical NOT operator. The expression, ! (2+2==5).), is true because 2+2 is NOT 5. !=: This is another comparison operator which tests if something is NOT equal. For example, the expression, (10 != 9);), is true, that is, 10 is not equal to 9. >: This is another comparison operator which tests if something is greater than something else. The expression, (10 > 9);), is true. There are a few more comparison operators as well. <: You guessed it. This tests whether the value to the left is less than the value to the right or not. The expression, (10 < 9);, is false. >=: This operator tests whether one value is greater than or equal to the other, and if either is true, the result is true. For example, the expression, (10 >= 9);, is true. The expression, (10 >= 10);, is also true. <=: Like the preceding operator, this operator tests for two conditions but this time, less than and equal to. The expression, (10 <= 9);, is false. The expression, (10 <= 10);, is true. &&: This operator is known as logical AND. It tests two or more separate parts of an expression and all parts must be true in order for the result to be true. Logical AND is usually used in conjunction with the other operators to build more complex tests. The expression, ((10 > 9) && (10 < 11));, is true because both parts are true. The expression, ((10 > 9) && (10 < 9));, is false because only one part of the expression is true and the other is false. ||: This operator is called logical OR. It is just like logical AND except that only one of two or more parts of an expression need to be true for the expression to be true. Let's look at the last example we used but replace the && sign with ||. The expression, ((10 > 9) || (10 < 9));, is now true because one part of the expression is true. All of these operators are virtually useless without a way of properly using them to make real decisions that affect real variables and code. Let's look at how to make decisions in Java. Decision 1 – If they come over the bridge, shoot them As we saw, operators serve hardly any purpose on their own but it was probably useful to see just a part of the wide and varied range available to us. Now, when we look at putting the most common operator, ==, to use, we can start to see the powerful yet fine control that operators offer us. Let's make the previous examples less abstract using the Java if keyword, a few conditional operators, and a small story: in use with some code and a made up military situation that will hopefully make the following examples less abstract. The captain is dying and, knowing that his remaining subordinates are not very experienced, he decides to write a Java program to convey his last orders after he has died. The troops must hold one side of a bridge while awaiting reinforcements. The first command the captain wants to make sure his troops understand is this: If they come over the bridge, shoot them. So how do we simulate this situation in Java? We need a Boolean variable isComingOverBridge. The next bit of code assumes that the isComingOverBridge variable has been declared and initialized. We can then use it like this: if(isComingOverBridge){   //Shoot them } If the isComingOverBridge Boolean is true, the code inside the opening and closing curly braces will run. If not, the program continues after the if block without running it. Decision 2 – Else, do this The captain also wants to tell his troops what to do (stay put) if the enemy is not coming over the bridge. Now we introduce another Java keyword, else. When we want to explicitly do something and the if block does not evaluate to true, we can use else. For example, to tell the troops to stay put if the enemy is not coming over the bridge, we use else: if(isComingOverBridge){   //Shoot them }else{   //Hold position } The captain then realized that the problem wasn't as simple as he first thought. What if the enemy comes over the bridge and has more troops? His squad will be overrun. So, he came up with this code (we'll use some variables as well this time): boolean isComingOverTheBridge; int enemyTroops; int friendlyTroops; //Code that initializes the above variables one way or another   //Now the if if(isComingOverTheBridge && friendlyTroops > enemyTroops){   //shoot them }else if(isComingOverTheBridge && friendlyTroops < enemyTroops) {   //blow the bridge }else{   //Hold position } Finally, the captain's last concern was that if the enemy came over the bridge waving the white flag of surrender and were promptly slaughtered, then his men would end up as war criminals. The Java code needed was obvious. Using the wavingWhiteFlag Boolean variable he wrote this test: if (wavingWhiteFlag){   //Take prisoners } But where to put this code was less clear. In the end, the captain opted for the following nested solution and changing the test for wavingWhiteFlag to logical NOT, like this: if (!wavingWhiteFlag){//not surrendering so check everything else   if(isComingOverTheBridge && friendlyTroops > enemyTroops){     //shoot them   }else if(isComingOverTheBridge && friendlyTroops <   enemyTroops) {     //blow the bridge   } }else{//this is the else for our first if   //Take prisoners { //Holding position This demonstrates that we can nest if and else statements inside of one another to create even deeper decisions. We could go on making more and more complicated decisions but what we have seen is more than sufficient as an introduction. Take the time to reread this if anything is unclear. It is also important to point out that very often, there are two or more ways to arrive at the solution. The right way will usually be the way that solves the problem in the clearest and simplest manner. Switching to make decisions We have seen the vast and virtually limitless possibilities of combining the Java operators with if and else statements. But sometimes a decision in Java can be better made in other ways. When we have to make a decision based on a clear list of possibilities that doesn't involve complex combinations, then switch is usually the way to go. We start a switch decision like this: switch(argument){   } In the previous example, an argument could be an expression or a variable. Then within the curly braces, we can make decisions based on the argument with case and break elements: case x:   //code to for x   break;   case y:   //code for y   break; You can see that in the previous example, each case states a possible result and each break denotes the end of that case as well as the point at which no further case statements should be evaluated. The first break encountered takes us out of the switch block to proceed with the next line of code. We can also use default without a value to run some code if none of the case statements evaluate to true, like this: default://Look no value   //Do something here if no other case statements are true break; Supposing we are writing an old-fashioned text adventure game—the kind of game where the player types commands such as "Go East", "Go West", "Take Sword", and so on. In this case, switch could handle that situation like this example code and we could use default to handle the case of the player typing a command that is not specifically handled: //get input from user in a String variable called command switch(command){     case "Go East":":   //code to go east   break;     case "Go West":   //code to go west   break;   case "Take sword":   //code to take the sword   break;     //more possible cases     default:   //Sorry I don't understand your command   break;   } We will use switch so that our onClick method can handle the different multiple-choice buttons of our math game. Java has even more operators than we have covered here. We have looked at all the operators we are going to need in this book and probably the most used in general. If you want the complete lowdown on operators, take a look at the official Java documentation at http://docs.oracle.com/javase/tutorial/java/nutsandbolts/operators.html. Math game – getting and checking the answer Here we will detect the right or wrong answer and provide a pop-up message to the player. Our Java is getting quite good now, so let's dive in and add these features. I will explain things as we go and then, as usual, dissect the code thoroughly at the end. The already completed code is in the download bundle, in the following files that correspond to the filenames we will create/autogenerate in Android Studio in a moment: Chapter 3/MathGameChapter3b/java/MainActivity.java Chapter 3/MathGameChapter3b/java/GameActivity.java Chapter 3/MathGameChapter3b/layout/activity_main.xml Chapter 3/MathGameChapter3b/layout/activity_game.xml As usual, I recommend following this tutorial step by step to see how we can create all of the code for ourselves. Open the GameActivity.java file visible in the editor window. Now we need to add the click detection functionality to our GameActivity, just as we did for our MainActivity. However, we will go a little further than the last time. So let's do it step by step as if it is totally new. Once again, we will give the buttons the ability to listen to the user clicking on them. Type this immediately after the last line of code we entered in the onCreate method but before the closing }. This time of course, we need to add some code to listen to three buttons: buttonObjectChoice1.setOnClickListener(this); buttonObjectChoice2.setOnClickListener(this); buttonObjectChoice3.setOnClickListener(this); Notice how the this keyword is highlighted in red indicating an error. Again, we need to make a modification to our code in order to allow the use of an interface, the special code element that allows us to add functionalities such as listening to button clicks. Edit the line as follows. When prompted to import another class, click on OK. Consider this line of code: public class GameActivity extends Activity { Change it to the following line: public class GameActivity extends Activity implements View.OnClickListener{   Now we have the entire preceding line underlined in red. This indicates an error but it is where we should be at this point. We mentioned that by adding implements View.OnClickListener, we have implemented an interface. We can think of this like a class that we can use, but with extra rules. One of the rules of the OnClickListener interface is that we must implement one of its methods, as you might remember. If we wish to use the useful functionality this interface provides (listening to button presses). Now we will add the onClick method. Type the following code. Notice the opening curly brace, {, and the closing curly brace, }. These denote the start and end of the method. Notice that the method is empty; it doesn't do anything but an empty method is enough to comply with the rules of the OnClickListener interface and the red line that indicated an error has gone. Make sure that you type the following code outside the closing curly brace (}) of the onCreate method but inside the closing curly brace of our MainActivity class: @Override     public void onClick(View view) {     } Notice that we have an empty line between the { and } braces of our onClick method. We can now put some code in here to make the buttons actually do something. Type the following in between { and } of onClick. This is where things get different from our code in MainActivity. We need to differentiate between the three possible buttons that could be pressed. We will do this with the switch statement that we discussed earlier. Look at the case criteria; they should look familiar. Here is the code that uses the switch statements: switch (view.getId()) {               case R.id.buttonChoice1:             //button 1 stuff goes here                 break;               case R.id.buttonChoice2:             //button 2 stuff goes here                 break;               case R.id.buttonChoice3:            //button 3 stuff goes here                 break;           }   Each case element handles a different button. For each button case, we need to get the value stored in the button that was just pressed and see if it matches our correctAnswer variable. If it does, we must tell the player they got it right, and if not, we must tell them they got it wrong. However, there is still one problem we have to solve. The onClick method is separate from the onCreate method and the Button objects. In fact, all the variables are declared in the onCreate method. If you try typing the code from step 9 now, you will get lots of errors. We need to make all the variables that we need in onClick available in onClick. To do this, we will move their declarations from above the onCreate method to just below the opening { of GameActivity. This means that these variables become variables of the GameActivity class and can be seen anywhere within GameActivity. Declare the following variables like this: int correctAnswer; Button buttonObjectChoice1; Button buttonObjectChoice2; Button buttonObjectChoice3; Now change the initialization of these variables within onCreate as follows. The actual parts of code that need to be changed are highlighted. The rest is shown for context: //Here we initialize all our variables int partA = 9; int partB = 9; correctAnswer = partA * partB; int wrongAnswer1 = correctAnswer - 1; int wrongAnswer2 = correctAnswer + 1; and TextView textObjectPartA =   (TextView)findViewById(R.id.textPartA);   TextView textObjectPartB =   (TextView)findViewById(R.id.textPartB);   buttonObjectChoice1 = (Button)findViewById(R.id.buttonChoice1);         buttonObjectChoice2 = (Button)findViewById(R.id.buttonChoice2);         buttonObjectChoice3 = (Button)findViewById(R.id.buttonChoice3);   Here is the top of our onClick method as well as the first case statement for our onClick method: @Override     public void onClick(View view) {         //declare a new int to be used in all the cases         int answerGiven=0;         switch (view.getId()) {               case R.id.buttonChoice1:             //initialize a new int with the value contained in buttonObjectChoice1             //Remember we put it there ourselves previously                 answerGiven = Integer.parseInt("" +                     buttonObjectChoice1.getText());                   //is it the right answer?                 if(answerGiven==correctAnswer) {//yay it's the right answer                     Toast.makeText(getApplicationContext(),                       "Well done!",                       Toast.LENGTH_LONG).show();                 }else{//uh oh!                     Toast.makeText(getApplicationContext(),"Sorry that's     wrong", Toast.LENGTH_LONG).show();                 }                 break;   Here are the rest of the case statements that do the same steps as the code in the previous step except handling the last two buttons. Enter the following code after the code entered in the previous step:   case R.id.buttonChoice2:                 //same as previous case but using the next button                 answerGiven = Integer.parseInt("" +                   buttonObjectChoice2.getText());                 if(answerGiven==correctAnswer) {                     Toast.makeText(getApplicationContext(), "Well done!", Toast.LENGTH_LONG).show();                 }else{                     Toast.makeText(getApplicationContext(),"Sorry that's wrong", Toast.LENGTH_LONG).show();                 }                 break;               case R.id.buttonChoice3:                 //same as previous case but using the next button                 answerGiven = Integer.parseInt("" +                     buttonObjectChoice3.getText());                 if(answerGiven==correctAnswer) {                     Toast.makeText(getApplicationContext(), "Well done!", Toast.LENGTH_LONG).show();                 }else{                     Toast.makeText(getApplicationContext(),"Sorry that's wrong", Toast.LENGTH_LONG).show();                 }                 break;           } Run the program, and then we will look at the code carefully, especially that odd-looking Toast thing. Here is what happens when we click on the leftmost button: This is how we did it: In steps 1 through 6, we set up handling for our multi-choice buttons, including adding the ability to listen to clicks using the onClick method and a switch block to handle decisions depending on the button pressed. In steps 7 and 8, we had to alter our code to make our variables available in the onClick method. We did this by making them member variables of our GameActivity class. When we make a variable a member of a class, we call it a field. In steps 9 and 10, we implemented the code that actually does the work in our switch statement in onClick. Let's take a line-by-line look at the code that runs when button1 is pressed. case R.id.buttonChoice1: First, the case statement is true when the button with an id of buttonChoice1 is pressed. Then the next line of code to execute is this: answerGiven = Integer.parseInt(""+ buttonObjectChoice1.getText()); The preceding line gets the value on the button using two methods. First, getText gets the number as a string and then Integer.parseInt converts it to an int. The value is stored in our answerGiven variable. The following code executes next: if(answerGiven==correctAnswer) {//yay it's the right answer   Toast.makeText(getApplicationContext(), "Well done!",     Toast.LENGTH_LONG).show(); }else{//uh oh!     Toast.makeText(getApplicationContext(),"Sorry that's wrong",       Toast.LENGTH_LONG).show();                 } The if statement tests to see if the answerGiven variable is the same as correctAnswer using the == operator. If so, the makeText method of the Toast object is used to display a congratulatory message. If the values of the two variables are not the same, the message displayed is a bit more negative one. The Toast line of code is possibly the most evil thing we have seen thus far. It looks exceptionally complicated and it does need a greater knowledge of Java than we have at the moment to understand. All we need to know for now is that we can use the code as it is and just change the message, and it is a great tool to announce something to the player. If you really want an explanation now, you can think of it like this: when we made button objects, we got to use all the button methods. But with Toast, we used the class directly to access its setText method without creating an object first. We can do this process when the class and its methods are designed to allow it. Finally, we break out of the whole switch statement as follows: break; Self-test questions Q1) What does this code do? // setContentView(R.layout.activity_main); Q2) Which of these lines causes an error? String a = "Hello"; String b = " Vinton Cerf"; int c = 55; a = a + b c = c + c + 10; a = a + c; c = c + a; Q3) We talked a lot about operators and how different operators can be used together to build complicated expressions. Expressions, at a glance, can sometimes make the code look complicated. However, when looked at closely, they are not as tough as they seem. Usually, it is just a case of splitting the expressions into smaller pieces to work out what is going on. Here is an expression that is more convoluted than anything else you will ever see in this book. As a challenge, can you work out: what will x be? int x = 10; int y = 9; boolean isTrueOrFalse = false; isTrueOrFalse = (((x <=y)||(x == 10))&&((!isTrueOrFalse) || (isTrueOrFalse))); Summary We went from knowing nothing about Java syntax to learning about comments, variables, operators, and decision making. As with any language, mastery of Java can be achieved by simply practicing, learning, and increasing our vocabulary. At this point, the temptation might be to hold back until mastery of the current Java syntax has been achieved, but the best way is to move on to new syntax at the same time as revisiting what we have already begun to learn. We will finally finish our math game by adding random questions of multiple difficulties as well as using more appropriate and random wrong answers for the multiple choice buttons. To enable us to do this, we will first learn some more new on Java and syntax. Resources for Article: Further resources on this subject: Asking Permission: Getting your head around Marshmallow's Runtime Permissions [article] Android and iOS Apps Testing at a Glance [article] Practical How-To Recipes for Android [article]
Read more
  • 0
  • 0
  • 8977

article-image-creating-underwater-scene-blender-part-3
Packt
12 Apr 2010
9 min read
Save for later

Creating an Underwater Scene in Blender- Part 3

Packt
12 Apr 2010
9 min read
LIGHTING UP THE SCENE From the point we started our scene, we haven’t tried any test renders yet to see how our underwater environment looks and I know you’re all so eager to try and press that RENDER button (so am I), but let’s make sure that we don’t waste a second of our time by rendering something that’s not worth the button press at all. Let’s prepare our background or world (as Blender world) so our recently created objects could have something to lie on, not just blank void of utter nothingness. In our 3D Viewport, select your camera (which is in layer 5), and proceed to the Editing (F9) panel and under the Camera tab, click the Mist button. What we did was to enable our camera to view our mist distance, for use later in the next procedure. Next step, proceed to the Shading (F5) panel then to the World Buttons and under Mist/Stars/Physics tab, click the Mist button to enable it. As of this time, the Mist toggling we did doesn’t do anything great yet for our scene. Under the same tab, adjust the Start and Dist values accordingly, and then you’ll notice the changes take effect in your 3D Viewport via your camera. What mist does is it fades whatever objects are within its range into our world background. Enabling Mist Display   Mist Settings   After you’ve set your Mist settings, proceed to the tab right next to the current one we are in now, the Amb Occ tab. Click the Ambient Occlusion button and leave the other default settings as they were except for the diffuse energy settings which we’ll change to Sky Color and the type of Ambient Occlusion to use which we’ll change in the dropdown menu from Raytrace to Approximate. We told Blender to render out our scene with ambient occlusion which is basically a global illumination method (or something similar to it), creating a global environmental lighting and producing soft shadows due to object occlusion. I often do this to my scenes where I start out my lighting process by turning on AO (Ambient Occlusion) as my first pass and take it from there. Ambient Occlusion Settings   Now, the moment you have been waiting for, RENDER TIME! Probably, before you have even read this sentence here, you already pressed the F12 button or the RENDER button. If you didn’t, I guess you already know what to do which leads us to this image: First Render!   Pretty dark isn’t it? By default, if we had the Ambient Occlusion’s diffusion settings to Plain, which just uses a plain white occlusion color, this render would be brighter, but we would not achieve that dark blue distant color that we see right now. To fix this problem, or better said to tweak this better, we’ll add in lights, which is the primary reason of this part of this article. So why don’t we throw in some? The first light we’ll create is a spot lamp from directly above. We would have opted to have this step of the lighting in the last parts of our pass but we’re trying as much as possible to avoid resetting some settings later on. Add a spot lamp to our scene by moving your mouse pointer over to the 3D Window and pressing spacebar, choose Add > Lamp > Spot. Set the settings of lamp (as seen in the screenshot) such that it only affects our nearest terrain. Adding the Spot Lamp   Top Spot Lamp Settings   Again, let’s do a test render and see how everything looks like so far. Render with Top Spot Lamp   And finally, some light and recognition, but still a little bit dark, so however you want it, throw in some lights at the sides to kill the dark shadows. Experiment with the way you light the scene. Just for reference purposes, you can take a look at the screenshot below. Light Setup   Leading us to: With More Lights   Just for fun and more aesthetic appeal, we’ll create another light which will cast rays (popularly known as volumetric lighting) and produce caustics which we’ll see in the sand. Duplicate the top lamp and move it higher than the original spot lamp we duplicated it from and adjust the settings accordingly. Ray Spot Lamp   The key settings for this lamp are the halo and halo steps. Enabling Halo makes our spot lamp draw volumes of light and halo step enables us to block this volume to create nice looking shadows out of the volume. A halo step of 0 will not produce the shadows and non-zero values will enable us to achieve such, often, you can stay at levels of 5 which is a good compromise between the quality of shadows and speed but for this part, we’ll just choose 1 to achieve the smoothest shadows. Without anything to block out the volumetric light, it would look as though a beam from an alien spacecraft just swept through our scene. But we want smaller light streaks, so what we need to do is to create a mesh to block out the light and let the beam pass through the holes of the mesh (as seen in the screenshot). Commonly, this mesh is called the “gobo” or the pattern with which light passes through. Render with Light Beams   Now, on to the caustics. To those of you who haven’t heard about caustics before, they are the patterns you see on the floor of a pool or a shallow body of water. It is caused by the way light had been bent by the waves and distortions happening on the surface of the water. You can skip and just move on to the other one if you don’t wish to add caustics to your scene, but I highly suggest you not to since this is one of the most fun parts in creating the scene. It had been a tedious and long journey for me to discover about this technique in creating caustics and thought I might share it to those who haven’t found out a way yet. All credit goes to the Blender Artists forum and the creators of this great application that we’ll use shortly to create the caustic effect. If you’ll do a quick Google search and type in “caustics generator”, among the first few results you will see is the CausticsGenerator application (pretty repetitive, eh?). Or you could go directly to this site http://www.lysator.liu.se/~kand/caustics/ and find out about this wonderful application. Head over to their downloads section and grab a copy of the free version which is licensed under the GNU GPL or you could buy the commercial version. Under their readme file or about section, here’s a description of caustics: “Caustics can be described as the light pattern you see at the bottom of a pool on a sunny day. This tool will let you render such caustics patterns. The rendered images can be animated and used for realtime graphics and are tileable in both space and time.” After downloading the application, extract the zip file to your preferred directory, and inside that open the file CausticsGenerator.exe. You’ll be presented with lots of settings and sliders to play around it. I won’t hold you back to go ahead, play around with it, and have fun. The procedure I did to achieve the caustics effect for our scene is to just adjust the depth, resolution (width & height), supersampling, and the output file and render out an animation out of the settings I had. Caustics Generator Settings   That’s about it for our caustics and after successfully rendering out the animation, you can check the directory where you have the renders outputted and you can see a bunch of BMP images. These are what we’ll be using as image sequence textures for one of our lights back in Blender. Caustic Image Sequences Generated   Back to Blender, select the upper spot lamp casting the beams of light and go to Shading (F5) panel, then to the Lamp Buttons and scroll over until you see Texture and Input tab. Click Add New and name your texture “caustics” and use the View Coordinates. Under the Map To tab, select Shadows to have the caustics affect the shadows from the light as well (though in real life, this rarely happens). Adding a Texture to the Spot Lamp   Proceed to the textures button and change the texture type to Images. Browse through the folder where you saved the caustics images and just select the first image in the series and click load or middle mouse click the first file. Under the Image tab, click Sequence (to tell Blender that we’re not just using the single file we loaded but the whole sequence which comes with it), enable Auto Refresh, set the total number of frames, and click Cyclic to make the animation loop. Loading the Caustics Texture   If we render the scene now, here’s how it will look like with the caustics: Render with Caustics
Read more
  • 0
  • 0
  • 8973

article-image-selinux-highly-secured-web-hosting-python-based-web-applications
Packt
21 Oct 2009
10 min read
Save for later

SELinux - Highly Secured Web Hosting for Python-based Web Applications

Packt
21 Oct 2009
10 min read
When contemplating the security of a web application, there are several attack vectors that you must consider. An outsider may attack the operating system by planting a remote exploit, exercising insecure operating system settings, or brandishing some other method of privilege escalation. Or, the outsider may attack other sites contained in the same server without escalating privileges. (Note that this particular discussion does not touch upon the conditions under which an attack steals data from a single site. Instead, I'm focusing on the ability to attack different applications on the same server.) With hosts providing space for large numbers of PHP-based sites, security can be difficult as the httpd daemon traditionally runs under the same Unix user for all sites. In order to prevent these kinds of attacks from occurring, you need to concentrate on two areas: Preventing the site from reading or modifying the data of another site, and Preventing the site from escalating privileges to tamper with the operating system and bypass user-based restrictions. There are two toolboxes you use to accomplish this. In the first case, you need to find a way to run all of your sites under different Linux users. This allows the traditional Linux filesystem security model to provide protection against a hacked site attacking other sites on the same server. In the second case, you need to find a way to prevent a privilege escalation to begin with and barring that, prevent damage to the operating system should an escalation occur. Let's first take a look at a method to run different sites under different users. The Python web framework provides several versatile methods by which applications can run. There are three common methods: first, using Python's built-in http server; second, running the script as a CGI application; and third, using mod_python under Apache (similar to what mod_perl and mod_php do). These methods have various disadvantages: respectively, a lack of scalability, performance issues due to CGI application loading, and the aforementioned “all sites under one user” problem. To provide a scalable, secure, high-performance framework, you can turn to a relatively new delivery method: mod_wsgi. This Apache module, created by Graham Dumpleton, provides several methods by which you can run Python applications. In this case, we'll be focusing on the “daemon” mode of mod_wsgi. Much like mod_python, the daemon mode of mod_wsgi embeds a Python interpreter (and the requisite script) into a httpd instance. Much like with mod_python, you can configure sites based on mod_wsgi to appear at various locations in the virtual directory tree and under different virtual servers. You can also configure the number and behavior of child daemons on a per-site basis. However, there is one important difference: with mod_wsgi, you can configure each httpd instance to run as a different Linux user. During operation, the main httpd instance dispatches requests to the already-running mod_wsgi children, producing performance results that rival mod_python. But most importantly, since each httpd instance is running under a different Linux user, you can apply Linux security mechanisms to different sites running on one server. Once you have your sites running on a per-user basis, you should next turn your attention to preventing privilege escalation and protecting the operating system. By default, the Targeted mode of SELinux provided by RedHat Enterprise Linux 5 (and its free cousins such as CentOS) provides strong protection against intrusions from httpd-based applications. Because of this, you will need to configure SELinux to allow access to resources such as databases and files that reside outside of the normal httpd directories. To illustrate these concepts, I'll guide you as you install a Trac instance under mod_wsgi. The platform is CentOS 5. As a side note, it's highly recommended that you perform the installation and SELinux debugging in a XEN instance so that your environment only contains the software that is needed. The sidebar explains how to easily install the environment that was originally used to perform this exercise, and I will assume that is your primary environment. There are a few steps that require the use of a C compiler – namely, the installation of Trac – and I'll guide you through migrating these packages to your XEN-based test environment. Installing Trac In this example, you'll use a standard installation of Trac. Following the instructions provided in the URL in the Resource section, begin by installing Trac 0.10.4 with ClearSilver 0.10.5 and SilverCity 0.9.7. (Note that with many Python web applications such as Trac and Django, “installing” the application means that you're actually installing the libraries necessary for Python to run the application. You'll need to run a script to create the actual site.) Next, create a PostgreSQL user and database on a different machine. If you are using XEN for your development machine, you can use a PostgreSQL database running in your main DOM0 instance; all we are concerned with is that the PostgreSQL instance is accessed on a different machine over the network. (Note that MySQL will also work in this example, but SQLite will not. In this case, we need a database engine that is accessed over the network, not as a disk file.) After that's done, you'll need to create an actual Trac site. Create a directory under /opt, such as /opt/trac. Next, run the trac_admin command and enter the information prompted. trac-admin /opt/trac initenv Installing mod_wsgi You can find mod_wsgi at the source listed in the Resources. After you make sure the httpd_devel package is installed, installing mod_wsgi is as simple as extracting the tarball and issuing the normal ./configure and 'make install' commands. Running Trac under mod_wsgi If you look under /opt/trac, you'll notice two directories: one labeled apache, and one with the label of the project that you assigned when you installed this instance of Trac. You'll start by creating an application script in the apache directory. The application script is listed in Listing 1. Listing 1: /opt/trac/apache/trac.wsgi #!/usr/bin/python import sys sys.stdout = sys.stderr import os os.environ['TRAC_ENV'] = '/opt/trac/test_proj' import trac.web.main application = trac.web.main.dispatch_request (Note the 'sys.stdout = sys.stderr' line. This is necessary due to the way WSGI handles communications between the Python script and the httpd instance. If there is any code in the script that prints to STDOUT (such as debug messages), then the httpd instance can crash.) After creating the application script, you'll modify httpd.conf to load the wsgi module and set up the Trac application. After the LoadModule lines, insert a line for mod_wsgi: LoadModule wsgi_module modules/mod_wsgi.so Next, go to the bottom of httpd.conf and insert the text in Listing 2. This text configures the wsgi module for one particular site; it can be used under the default httpd configuration as well as under VirtualHost directives. Listing 2: Excerpt from httpd.conf: WSGIDaemonProcess trac user=trac_user group=trac_user threads=25 WSGIScriptAlias /trac /opt/trac/apache/trac.wsgi WSGIProcessGroup trac WSGISocketPrefix run/wsgi <Directory /opt/trac/apache> WSGIApplicationGroup %{GLOBAL} Order deny,allow Allow from all </Directory> Note the WSGIScriptAlias identifier. The /trac keyword (first parameter) specifies where in the directory tree the application will exist. With this configuration, If you go to your server's root address, you'll see the default CenOS splash page. If you add /trac after the address, you'll hit your Trac instance. Save the httpd.conf file. Finally, add a Linux user called trac_user. It is important that this user should not have login privileges. When the root httpd instance runs and encounters the WSGIDaemonProcess directive noted above, it will fork itself as the user specified in the directive; the fork will then load Python and the indicated script.     Securing Your Site In this section, I'll focus on the two areas noted in the introduction: User based security and SELinux. I will touch briefly on the theory of SELinux and explain the nuts and bolts of this particular implementation in more depth. I highly recommend that you read the RedHat Enterprise Linux Deployment Guide for the particulars about how RedHat implements SELinux. As with all activities involving some risk, if you plan to implement these methods, you should retain the services of a qualified security consultant to advise you about your particular situation. Setting up the user-based security is not difficult. Because the HTTPD instance containing Python and the Trac instance will run under the Trac user, you can safely set everything under /opt/trac/test_project for read and execute (for directories) for user and none for group/all. By doing this, you will isolate this site from other sites and users on the system. Now, let's configure SELinux. First, you should verify that your system is running the proper Policy and Mode. On your development system, you'll be using the Targeted policy in its Permissive mode. If you choose to move your Python applications to a production machine, you would run under the Targeted policy, in the Enforcing mode. The Targeted policy is limited to protecting the most popular network services without making the system so complex as to prevent user-level work from being done. It is the only mode that ships with RedHat 5, and by extension, CentOS 5. In Permissive mode, SELinux policy violations are trapped and sent to the audit log, but the behavior is allowed. In enforcing mode, the violation is trapped and the behavior is not allowed. To verify the Mode, run the Security Level Configuration tool from the Administration menu. The SELinux tab, shown in Figure 1, allows you to adjust the Mode. After you have verified that SELinux is running in Permissive mode, you need to do two things. First, you need to change the Type of the files under /opt/trac. Second, you need to allow Trac to connect to the Postgres database that you configured when you installed Trac. First, you need to tweak the SELinux file types attached to the files in your Trac instance. These file types dictate what processes are allowed to access them. For example, /etc/shadow has a very restrictive 'shadow' type that only allows a few applications to read and write it. By default, SELinux expects web-based applications – indeed, anything using Apache – to reside under /var/www. Files created under this directory have the SELinux Type httpd_sys_content_t. When you created the Trac instance under /opt/trac, the files were created as type usr_t. Figure 2 shows the difference between these labels To properly label the files under /opt, issue the following commands as root: cd /optchcon -R -t httpd_user_content_t trac/ After the file types are configured, there is one final step to do: allow Trac to connect to PostgreSQL. In its default state, SELinux disallows outbound network connections for the httpd type. To allow database connections, issue the following command: setsebool -P httpd_can_network_connect_db=1 In this case, we are using the -P option to make this setting persistent. If you omit this option, then the setting will be reset to its default state upon the next reboot. After the setsebool command has been run, start HTTPD by issuing the following command: /sbin/service httpd start If you visit the url http://127.0.0.1/trac, you should see the Trac screen such as that in Figure 3.    
Read more
  • 0
  • 0
  • 8965

article-image-palm-2-a-game-changer-in-tackling-real-world-challenges
Sangita Mahala
07 Nov 2023
9 min read
Save for later

PaLM 2: A Game-Changer in Tackling Real-World Challenges

Sangita Mahala
07 Nov 2023
9 min read
Dive deeper into the world of AI innovation and stay ahead of the AI curve! Subscribe to our AI_Distilled newsletter for the latest insights. Don't miss out – sign up today!IntroductionA new large language model, Google AI's PaLM2, developed from a massive textual and code database. It's a successor of the PaLM program, and is even more powerful in terms of producing text, translating language, writing various types of creative content, and answering your questions by means of information. The research and development of PaLM 2 continues, but it has the potential to shake up many industries and research areas in terms of its ability to address a broad range of complex real-world problems.PaLM 2 is a new large language model from Google AI, trained on a massive dataset of text and code. It is even more powerful than its predecessor, PaLM, and can be used to solve a wide range of complex real-world problems.Powerful Tools for NLP, Code Generation, and Creative Writing by PaLM2In order to learn the complex relationships between words and phrases, LLMs, such as PaLM 2, are trained in massive databases of text and code. For this reason, they make excellent candidates for a wide range of tasks, such as:Natural language processing (NLP): There are also NLP tasks to be performed such as machine translation, text summary, and answering questions. In order to perform these tasks with high accuracy and consistency, PaLM 2 can be used.Code generation: A number of programming languages, including Python, Java, and C++ can be used for generating code by PaLML 2. It can also be useful for tasks like the automation of software development and the creation of new algorithms.Creative writing: Different creative text formats, such as poems, code, scripts, musical notes, emails, letters, etc. may be created by PaLM 2. It could be useful to the tasks of writing advertising copy, producing scripts for films and television shows as well as composing music.Real-World ExamplesTo illustrate how PaLM 2 can be put to use in solving the complicated problems of the actual world, these are some specific examples:Example 1: Drug DiscoveryIn the area of drug discovery, there are many promising applications to be had by PaLM 2. For the generation of new drug candidates, for the prediction of their properties, and for the simulation of their interaction with biological targets, PaLM 2 can be used. This may make it more quickly and efficiently possible for scientists to identify new drugs.In order to produce new drug candidates, PaLM 2 is able to screen several millions of possible compounds with the aim of binding to a specific target protein. This is a highly complex task, but PaLM 2 can speed it up very fast.Input code:import google.cloud.aiplatform as aip def drug_discovery(target_protein): """Uses PaLM 2 to generate new drug candidates for a given target protein. Args:    target_protein: The target protein to generate drug candidates for. Returns:    A list of potential drug candidates. """ # Create a PaLM 2 client. client = aip.PredictionClient() # Set the input prompt. prompt = f"Generate new drug candidates for the target protein {target_protein}." # Make a prediction. prediction = client.predict(model_name="paLM_2", inputs={"text": prompt}) # Extract the drug candidates from the prediction. drug_candidates = prediction.outputs["drug_candidates"] return drug_candidates # Example usage: target_protein = "ACE2" drug_candidates = drug_discovery(target_protein) print(drug_candidates) Output:A list of potential therapeutic candidates for that protein is provided by the function drug_discovery(). The specific output depends on the protein being targeted, and this example is as follows:This indicates that three possible drug candidates for target protein ACE2 have been identified by PaLM 2. In order to determine the effectiveness and safety of these substances, researchers may therefore carry out additional studies.Example 2: Climate ChangeIn order to cope with climate change, PaLM 2 may also be used. In order to model a climate system, anticipate the impacts of climate change and develop mitigation strategies it is possible to use PaLM 2.Using a variety of greenhouse gas emissions scenarios, PaLM 2 can simulate the Earth's climate. This information can be used for the prediction of climate change's effects on temperature, precipitation, and other factors.Input code:import google.cloud.aiplatform as aip def climate_change_prediction(emission_scenario): """Uses PaLM 2 to predict the effects of climate change under a given emission scenario. Args:    emission_scenario: The emission scenario to predict the effects of climate change under. Returns:    A dictionary containing the predicted effects of climate change. """ # Create a PaLM 2 client. client = aip.PredictionClient() # Set the input prompt. prompt = f"Predict the effects of climate change under the emission scenario {emission_scenario}." # Make a prediction. prediction = client.predict(model_name="paLM_2", inputs={"text": prompt}) # Extract the predicted effects of climate change from the prediction. predicted_effects = prediction.outputs["predicted_effects"] return predicted_effects # Example usage: emission_scenario = "RCP8.5" predicted_effects = climate_change_prediction(emission_scenario) print(predicted_effects)  Output:The example given is RCP 8.5, which has been shown to be a large emission scenario. The model estimates that the global temperature will rise by 4.3 degrees C, with precipitation decreasing by 10 % in this scenario.Example 3: Material ScienceIn the area of material science, PaLM 2 may be used to create new materials with desired properties. In order to obtain the required properties such as durability, lightness, and conductivity, it is possible to use PaLM 2 for an assessment of millions of material possibilities.The development of new materials for batteries may be achieved with the use of PaLM 2. It is essential that the batteries be light, long lasting and have high energy density. Millions of potential material for such properties may be identified using PaLM 2.Input code:import google.cloud.aiplatform as aip def material_design(desired_properties): """Uses PaLM 2 to design a new material with the desired properties. Args:    desired_properties: A list of the desired properties of the new material. Returns:    A dictionary containing the properties of the designed material. """ # Create a PaLM 2 client. client = aip.PredictionClient() # Set the input prompt. prompt = f"Design a new material with the following desired properties: {desired_properties}" # Make a prediction. prediction = client.predict(model_name="paLM_2", inputs={"text": prompt}) # Extract the properties of the designed material from the prediction. designed_material_properties = prediction.outputs["designed_material_properties"] return designed_material_properties # Example usage: desired_properties = ["lightweight", "durable", "conductive"] designed_material_properties = material_design(desired_properties) print(designed_material_properties)Output:This means that the model designed a material with the following properties:Density: 1.0 grams per cubic centimeter (g/cm^3)Strength: 1000.0 megapascals (MPa)Conductivity: 100.0 watts per meter per kelvin (W/mK)This is only a prediction based on the language model, and further investigation and development would be needed to make this material real.Example 4: Predicting the Spread of Infectious DiseasesIn order to predict the spread of COVID-19 in a given region, PaLM 2 may be used. Factors that may be taken into account by PaLM2 include the number of infections, transmission, and vaccination rates. The PALM 2 method can also be used to predict the effects of preventive health measures, e.g. mask mandates and lockdowns.Input code:import google.cloud.aiplatform as aip def infectious_disease_prediction(population_density, transmission_rate): """Uses PaLM 2 to predict the spread of an infectious disease in a population with a given population density and transmission rate. Args:    population_density: The population density of the population to predict the spread of the infectious disease in.    transmission_rate: The transmission rate of the infectious disease. Returns:    A dictionary containing the predicted spread of the infectious disease. """ # Create a PaLM 2 client. client = aip.PredictionClient() # Set the input prompt. prompt = f"Predict the spread of COVID-19 in a population with a population density of {population_density} and a transmission rate of {transmission_rate}." # Make a prediction. prediction = client.predict(model_name="paLM_2", inputs={"text": prompt}) # Extract the predicted spread of the infectious disease from the prediction. predicted_spread = prediction.outputs["predicted_spread"] return predicted_spread # Example usage: population_density = 1000 transmission_rate = 0.5 predicted_spread = infectious_disease_prediction(population_density, transmission_rate) print(predicted_spread)Output:An estimated peak incidence for infectious disease is 50%, meaning that half of the population will be affected at a particular time during an outbreak. The total number of anticipated cases is 500,000.It must be remembered that this is a prediction, and the rate at which infectious diseases are spreading can change depending on many factors like the effectiveness of disease prevention measures or how people behave.The development of new medicines, more effective energy systems and materials with desired properties is expected to take advantage of PALM 2 in the future. In order to predict the spread of infectious agents and develop mitigation strategies for Climate Change, PaLM 2 is also likely to be used.ConclusionIn conclusion, several sectors have transformed with the emergence of PaLM 2, Google AI's advanced language model. By addressing the complex problems of today's world, it is offering the potential for a revolution in industry. The applicability of the PALM 2 system to drug discovery, prediction of climate change, materials science, and infectious disease spread forecast is an example of its flexibility and strength.Responsibility and proper use of PaLM 2 are at the heart of this evolving landscape. It is necessary to combine the Model's capacity with human expertise in order to make full use of this potential, while ensuring that its application meets ethics standards and best practices. This technology may have the potential for shaping a brighter future, helping to solve complicated world problems across different fields as we continue our search for possible PaLM 2 solutions.Author BioSangita Mahala is a passionate IT professional with an outstanding track record, having an impressive array of certifications, including 12x Microsoft, 11x GCP, 2x Oracle, and LinkedIn Marketing Insider Certified. She is a Google Crowdsource Influencer and IBM champion learner gold. She also possesses extensive experience as a technical content writer and accomplished book blogger. She is always Committed to staying with emerging trends and technologies in the IT sector.
Read more
  • 0
  • 0
  • 8963

article-image-hello-tdd
Packt
14 Sep 2016
6 min read
Save for later

Hello TDD!

Packt
14 Sep 2016
6 min read
In this article by Gaurav Sood, the author of the book Scala Test-Driven Development, tells  basics of Test-Driven Development. We will explore: What is Test-Driven Development? What is the need for Test-Driven Development? Brief introduction to Scala and SBT (For more resources related to this topic, see here.) What is Test-Driven Development? Test-Driven Development or TDD(as it is referred to commonly)is the practice of writing your tests before writing any application code. This consists of the following iterative steps: This process is also referred to asRed-Green-Refactor-Repeat. TDD became more prevalent with the use of agile software development process, though it can be used as easily with any of the Agile's predecessors like Waterfall. Though TDD is not specifically mentioned in agile manifesto (http://agilemanifesto.org), it has become a standard methodology used with agile. Saying this, you can still use agile without using TDD. Why TDD? The need for TDD arises from the fact that there can be constant changes to the application code. This becomes more of a problem when we are using agile development process, as it is inherently an iterative development process. Here are some of the advantages, which underpin the need for TDD: Code quality: Tests on itself make the programmer more confident of their code. Programmers can be sure of syntactic and semantic correctness of their code. Evolving architecture: A purely test-driven application code gives way to an evolving architecture. This means that we do not have to pre define our architectural boundaries and the design patterns. As the application grows so does the architecture. This results in an application that is flexible towards future changes. Avoids over engineering: Tests that are written before the application code define and document the boundaries. These tests also document the requirements and application code. Agile purists normally regard comments inside the code as a smell. According to them your tests should document your code. Since all the boundaries are predefined in the tests, it is hard to write application code, which breaches these boundaries. This however assumes that TDD is following religiously. Paradigm shift: When I had started with TDD, I noticed that the first question I asked myself after looking at the problem was; "How can I solve it?" This however is counterproductive. TDD forces the programmer to think about the testability of the solution before its implementation. To understand how to test a problem would mean a better understanding of the problem and its edge cases. This in turn can result into refinement of the requirements or discovery or some new requirements. Now it had become impossible for me not to think about testability of the problem before the solution. Now the first question I ask myself is; "How can I test it?". Maintainable code: I have always found it easier to work on an application that has historically been test-driven rather than on one that is not. Why? Only because when I make change to the existing code, the existing tests make sure that I do not break any existing functionality. This results in highly maintainable code, where many programmers can collaborate simultaneously. Brief introduction to Scala and SBT Let us look at Scala and SBT briefly. It is assumed that the reader is familiar with Scala and therefore will not go into the depth of it. What is Scala Scala is a general-purpose programming language. Scala is an acronym for Scalable Language. This reflects the vision of its creators of making Scala a language that grows with the programmer's experience of it. The fact that Scala and Java objects can be freely mixed, makes transition from Java to Scala quite easy. Scala is also a full-blown functional language. Unlike Haskell, which is a pure functional language, Scala allows interoperability with Java and support for objectoriented programming. Scala also allows use of both pure and impure functions. Impure functions have side affect like mutation, I/O and exceptions. Purist approach to Scala programming encourages use of pure functions only. Scala is a type-safe JVM language that incorporates both object oriented and functional programming into an extremely concise, logical, and extraordinarily powerful language. Why Scala? Here are some advantages of using Scala: Functional solution to problem is always better: This is my personal view and open for contention. Elimination of mutation from application code allows application to be run in parallelacross hosts and cores without any deadlocks. Better concurrency model: Scala has an actor model that is better than Java's model of locks on thread. Concise code:Scala code is more concise than itsmore verbose cousin Java. Type safety/ static typing: Scala does type checking at compile time. Pattern matching: Case statements in Scala are superpowerful. Inheritance:Mixin traits are great and they definitely reduce code repetition. There are other features of Scala like closure and monads, which will need more understanding of functional language concepts to learn. Scala Build Tool Scala Build Tool (SBT) is a build tool that allows compiling, running, testing, packaging, and deployment of your code. SBT is mostly used with Scala projects, but it can as easily be used for projects in other languages. Here, we will be using SBT as a build tool for managing our project and running our tests. SBT is written in Scala and can use many of the features of Scala language. Build definitions for SBT are also written in Scala. These definitions are both flexible and powerful. SBT also allows use of plugins and dependency management. If you have used a build tool like Maven or Gradlein any of your previous incarnations, you will find SBT a breeze. Why SBT? Better dependency management Ivy based dependency management Only-update-on-request model Can launch REPL in project context Continuous command execution Scala language support for creating tasks Resources for learning Scala Here are few of the resources for learning Scala: http://www.scala-lang.org/ https://www.coursera.org/course/progfun https://www.manning.com/books/functional-programming-in-scala http://www.tutorialspoint.com/scala/index.htm Resources for SBT Here are few of the resources for learning SBT: http://www.scala-sbt.org/ https://twitter.github.io/scala_school/sbt.html Summary In this article we learned what is TDD and why to use it. We also learned about Scala and SBT.  Resources for Article: Further resources on this subject: Overview of TDD [article] Understanding TDD [article] Android Application Testing: TDD and the Temperature Converter [article]
Read more
  • 0
  • 0
  • 8952
Unlock access to the largest independent learning library in Tech for FREE!
Get unlimited access to 7500+ expert-authored eBooks and video courses covering every tech area you can think of.
Renews at €18.99/month. Cancel anytime
article-image-going-mobile-first
Packt
03 Feb 2016
16 min read
Save for later

Going Mobile First

Packt
03 Feb 2016
16 min read
In this article by Silvio Moreto Pererira, author of the book, Bootstrap By Example, we will focus on mobile design and how to change the page layout for different viewports, change the content, and more. In this article, you will learn the following: Mobile first development Debugging for any device The Bootstrap grid system for different resolutions (For more resources related to this topic, see here.) Make it greater Maybe you have asked yourself (or even searched for) the reason of the mobile first paradigm movement. It is simple and makes complete sense to speed up your development pace. The main argument for the mobile first paradigm is that it is easier to make it greater than to shrink it. In other words, if you first make a desktop version of the web page (known as responsive design or mobile last) and then go to adjust the website for a mobile, it has a probability of 99% of breaking the layout at some point, and you will have to fix a lot of things in both the mobile and desktop. On the other hand, if you first create the mobile version, naturally the website will use (or show) less content than the desktop version. So, it will be easier to just add the content, place the things in the right places, and create the full responsiveness stack. The following image tries to illustrate this concept. Going mobile last, you will get a degraded, warped, and crappy layout, and going mobile first, you will get a progressively enhanced, future-friendly awesome web page. See what happens to the poor elephant in this metaphor: Bootstrap and themobile first design At the beginning of Bootstrap, there was no concept of mobile first, so it was made to work for designing responsive web pages. However, with the version 3 of the framework, the concept of mobile first was very solid in the community. For doing this, the whole code of the scaffolding system was rewritten to become mobile first from the start. They decided to reformulate how to set up the grid instead of just adding mobile styles. This made a great impact on compatibility between versions older than 3, but was crucial for making the framework even more popular. To ensure the proper rendering of the page, set the correct viewport at the <head> tag: <meta name="viewport" content="width=device-width,   initial-scale=1"> How to debug different viewports in the browser Here, you will learn how to debug different viewports using the Google Chrome web browser. If you already know this, you can skip this section, although it might be important to refresh the steps for this. In the Google Chrome browser, open the Developer tools option. There are many ways to open this menu: Right-click anywhere on the page and click on the last option called Inspect. Go in the settings (the sandwich button on the right-hand side of the address bar), click on More tools, and finally on Developer tools. The shortcut to open it is Ctrl (cmd for OS X users) + Shift + I. F12 in Windows also works (Internet Explorer legacy…). With Developer tools, click on the mobile phone to the left of a magnifier, as showed in the following image: It will change the display of the viewport to a certain device, and you can also set a specific network usage to limit the data bandwidth. Chrome will show a message telling you that for proper visualization, you may need to reload the page to get the correct rendering: For the next image, we have activated the Device mode for an iPhone 5 device. When we set this viewport, the problems start to appear because we did not make the web page with the mobile first methodology. Bootstrap scaffolding for different devices Now that we know more about mobile first development and its important role in Bootstrap starting from version 3, we will cover Bootstrap usage for different devices and viewports. For doing this, we must apply the column class for the specific viewport, for example, for medium displays, we use the .col-md-*class. The following table presents the different classes and resolutions that will be applied for each viewport class:   Extra small devices (Phones < 544 px / 34 em) Small devices (Tablets ≥ 544 px / 34 em and < 768 px / 48 em) Medium devices (Desktops ≥ 768 px /48 em < 900px / 62 em) Large devices (Desktops ≥ 900 px / 62 em < 1200px 75 em) Extra large devices (Desktops ≥ 1200 px / 75 em) Grid behavior Horizontal lines at all times Collapse at start and fit column grid Container fixed width Auto 544px or 34rem 750px or 45rem 970px or 60rem 1170px or 72.25rem Class prefix .col-xs-* .col-sm-* .col-md-* .col-lg-* .col-xl-* Number of columns 12 columns Column fixed width Auto ~ 44px or 2.75 rem ~ 62px or 3.86 rem ~ 81px or 5.06 rem ~ 97px or 6.06 rem Mobile and extra small devices To exemplify the usage of Bootstrap scaffolding in mobile devices, we will have a predefined web page and want to adapt it to mobile devices. We will be using the Chrome mobile debug tool with the device, iPhone 5. You may have noticed that for small devices, Bootstrap just stacks each column without referring for different rows. In the layout, some of the Bootstrap rows may seem fine in this visualization, although the one in the following image is a bit strange as the portion of code and image are not in the same line, as it supposed to be: To fix this, we need to add the class column's prefix for extra small devices, which is .col-xs-*, where * is the size of the column from 1 to 12. Add the .col-xs-5 class and .col-xs-7 for the columns of this respective row. Refresh the page and you will see now how the columns are put side by side: <div class="row">   <!-- row 3 -->   <div class="col-md-3 col-xs-5">     <pre>&lt;p&gt;I love programming!&lt;/p&gt;       &lt;p&gt;This paragraph is on my landing page&lt;/p&gt;       &lt;br/&gt;       &lt;br/&gt;       &lt;p&gt;Bootstrap by example&lt;/p&gt;     </pre>   </div>   <div class="col-md-9 col-xs-7">     <img src="imgs/center.png" class="img-responsive">   </div> </div> Although the image of the web browser is too small on the right, it would be better if it was a vertical stretched image, such as a mobile phone. (What a coincidence!) To make this, we need to hide the browser image in extra small devices and display an image of the mobile phone. Add the new mobile image below the existing one as follows. You will see both images stacked up vertically in the right column: <img src="imgs/center.png" class="img-responsive"> <img src="imgs/mobile.png" class="img-responsive"> Then, we need to use the new concept of availability classes present in Bootstrap. We need to hide the browser image and display the mobile image just for this kind of viewport, which is extra small. For this, add the .hidden-xs class to the browser image and add the .visible-xs class to the mobile image: <div class="row">   <!-- row 3 -->   <div class="col-md-3 col-xs-5">     <pre>&lt;p&gt;I love programming!&lt;/p&gt;       &lt;p&gt;This paragraph is on my landing page&lt;/p&gt;       &lt;br/&gt;       &lt;br/&gt;       &lt;p&gt;Bootstrap by example&lt;/p&gt;     </pre>   </div>   <div class="col-md-9 col-xs-7">     <img src="imgs/center.png" class="img-responsive hidden-xs">     <img src="imgs/mobile.png" class="img-responsive visible-xs">   </div> </div> Now this row seems nice! With this, the browser image was hidden in extra small devices and the mobile image was shown for this viewport in question. The following image shows the final display of this row: Moving on, the next Bootstrap .row contains a testimonial surrounded by two images. It would be nicer if the testimonial appeared first and both images were displayed after it, splitting the same row, as shown in the following image. For this, we will repeat almost the same techniques presented in the last example: The first change is to hide the Bootstrap image using the .hidden-xs class. After this, create another image tag with the Bootstrap image in the same column of the PACKT image. The final code of the row should be as follows: <div class="row">   <div class="col-md-3 hidden-xs">     <img src="imgs/bs.png" class="img-responsive">   </div>   <div class="col-md-6 col-xs-offset-1 col-xs-11">     <blockquote>       <p>Lorem ipsum dolor sit amet, consectetur         adipiscing elit. Integer posuere erat a ante.</p>       <footer>Testimonial from someone at         <cite title="Source Title">Source Title</cite></footer>     </blockquote>   </div>   <div class="col-md-3 col-xs-7">     <img src="imgs/packt.png" class="img-responsive">   </div>   <div class="col-xs-5 visible-xs">     <img src="imgs/bs.png" class="img-responsive">   </div> </div> We did plenty of things now; all the changes are highlighted. The first is the .hidden-xs class in the first column of the Bootstrap image, which hid the column for this viewport. Afterward, in the testimonial, we changed the grid for the mobile, adding a column offset with size 1 and making the testimonial fill the rest of the row with the .col-xs-11 class. Lastly, like we said, we want to split both images from PACKT and Bootstrap in the same row. For this, make the first image column fill seven columns with the .col-xs-7 class. The other image column is a little more complicated. As it is visible just for mobile devices, we add the .col-xs-5 class, which will make the element span five columns in extra small devices. Moreover, we hide the column for other viewports with the .visible-xs class. As you can see, this row has more than 12 columns (one offset, 11 testimonials, seven PACKT images, and five Bootstrap images). This process is called column wrapping and happens when you put more than 12 columns in the same row, so the groups of extra columns will wrap to the next lines. Availability classes Just like .hidden-*, there are the .visible-*-*classes for each variation of the display and column from 1 to 12. There is also a way to change the display of the CSS property using the .visible-*-* class, where the last * means block, inline, or inline-block. Use this to set the proper visualization for different visualizations. The following image shows you the final result of the changes. Note that we made the testimonial appear first, with one column of offset, and both images appear below it: Tablets and small devices Completing the mobile visualization devices, we move on to tablets and small devices, which are devices from 544 px (34 em) to 768 px (48 em). Most of this kind of devices are tablets or old desktops monitors. To work with this example, we are using the iPad mini in the portrait position. For this resolution, Bootstrap handles the rows just as in extra small devices by stacking up each one of the columns and making them fill the total width of the page. So, if we do not want this to happen, we have to set the column fill for each element with the .col-sm-* class manually. If you see now how our example is presented, there are two main problems. The first one is that the heading is in separate lines, whereas they could be in the same line. For this, we just need to apply the grid classes for small devices with the .col-sm-6 class for each column, splitting them into equal sizes: <div class="row">   <div class="col-md-offset-4 col-md-4 col-sm-6">     <h3>       Some text with <small>secondary text</small>     </h3>   </div>   <div class="col-md-4 col-sm-6">     <h3>       Add to your favorites       <small>         <kbd class="nowrap"><kbd>ctrl</kbd> + <kbd>d</kbd></kbd>       </small>     </h3>   </div> </div> The result should be as follows: The second problem in this viewport is again the testimonial row! Due to the classes that we have added for the mobile viewport, the testimonial now has an offset column and different column span. We must add the classes for small devices and make this row with the Bootstrap image on the left, the testimonial in the middle, and the PACKT image on the right: <div class="row">   <div class="col-md-3 hidden-xs col-sm-3">     <img src="imgs/bs.png" class="img-responsive">   </div>   <div class="col-md-6 col-xs-offset-1 col-xs-11     col-sm-6 col-sm-offset-0">     <blockquote>       <p>Lorem ipsum dolor sit amet, consectetur         adipiscing elit. Integer posuere erat a ante.</p>       <footer>Testimonial from someone at         <cite title="Source Title">Source Title</cite></footer>     </blockquote>   </div>   <div class="col-md-3 col-xs-7 col-sm-3">     <img src="imgs/packt.png" class="img-responsive">   </div>   <div class="col-xs-5 hidden-sm hidden-md hidden-lg">     <img src="imgs/bs.png" class="img-responsive">   </div> </div> As you can see, we had to reset the column offset in the testimonial column. It happened because it kept the offset that we had added for extra small devices. Moreover, we are just ensuring that the image columns had to fill just three columns with the .col-sm-3 classes in both the images. The result of the row is as follows: Everything else seems fine! These viewports were easier to set up. See how Bootstrap helps us a lot? Let's move on to the final viewport, which is a desktop or large devices. Desktop and large devices Last but not least, we enter the grid layout for a desktop and large devices. We skipped medium devices because we coded first for this viewport. Deactivate the Device mode in Chrome and put your page in a viewport with a width larger or equal to 1200 pixels. The grid prefix that we will be using is .col-lg-*, and if you take a look at the page, you will see that everything is well placed and we don't need to make changes! (Although we would like to make some tweaks to make our layout fancier and learn some stuff about the Bootstrap grid.) We want to talk about a thing called column ordering. It is possible to change the order of the columns in the same row by applying the.col-lg-push-* and .col-lg-pull-* classes. (Note that we are using the large devices prefix, but any other grid class prefix can be used.) The .col-lg-push-* class means that the column will be pushed to the right by the * columns, where * is the number of columns pushed. On the other hand, .col-lg-pull-* will pull the column in the left direction by *, where * is the number of columns as well. Let's test this trick in the second row by changing the order of both the columns: <div class="row">   <div class="col-md-offset-4 col-md-4 col-sm-6 col-lg-push-4">     <h3>       Some text with <small>secondary text</small>     </h3>   </div>   <div class="col-md-4 col-sm-6 col-lg-pull-4">     <h3>       Add to your favorites       <small>         <kbd class="nowrap"><kbd>ctrl</kbd> + <kbd>d</kbd></kbd>       </small>     </h3>   </div> </div> We just added the .col-lg-push-4 class to the first column and .col-lg-pull-4 to the other one to get this result. By doing this, we have changed the order of both the columns in the second row, as shown in the following image: Summary In this article, you learned a little about the mobile first development and how Bootstrap can help us in this task. We started from an existing Bootstrap template, which was not ready for mobile visualization, and fixed that. While fixing, we used a lot of Bootstrap scaffolding properties and Bootstrap helpers, making it much easier to fix anything. We did all of this without a single line of CSS or JavaScript; we used only Bootstrap with its inner powers! Resources for Article:   Further resources on this subject: Bootstrap in a Box [article] The Bootstrap grid system [article] Deep Customization of Bootstrap [article]
Read more
  • 0
  • 0
  • 8945

article-image-advanced-user-management
Packt
30 Dec 2015
20 min read
Save for later

Advanced User Management

Packt
30 Dec 2015
20 min read
In this article written by Bhaskarjyoti Roy, author of the book Mastering CentOS 7 Linux Server, will introduce some advanced user and group management scenarios along with some examples on how to handle advanced level options such as password aging, managing sudoers, and so on, on a day to day basis. Here, we are assuming that we have already successfully installed CentOS 7 along with a root and user credentials as we do in the traditional format. Also, the command examples, in this chapter, assume you are logged in or switched to the root user. (For more resources related to this topic, see here.)  The following topics will be covered: User and group management from GUI and command line Quotas Password aging Sudoers Managing users and groups from GUI and command line We can add a user to the system using useradd from the command line with a simple command as follows: useradd testuser This creates a user entry in the /etc/passwd file and automatically creates the home directory for the user in /home. The /etc/passwd entry looks like this: testuser:x:1001:1001::/home/testuser:/bin/bash But, as we all know, the user is in a locked state and cannot login to the system unless we add a password for the user using the command: passwd testuser This will, in turn, modify the /etc/shadow file, at the same time unlock the user, and the user will be able to login to the system. By default, the preceding set of commands will create both a user and a group for the testuser on the system. What if we want a certain set of users to be a part of a common group? We will use the -g option along with the useradd command to define the group for the user, but we have to make sure that the group already exists. So, to create users such as testuser1, testuser2, and testuser3 and make them part of a common group called testgroup, we will first create the group and then we create the users using the -g or -G switch. So we will do this: # To create the group : groupadd testgroup # To create the user with the above group and provide password and unlock user at the same time : useradd testuser1 -G testgroup passwd testuser1 useradd testuser2 -g 1002 passwd testuser2 Here, we have used both -g and -G. The difference between them is: with -G, we create the user with its default group and assign the user to the common testgroup as well, but with -g, we created the user as part of the testgroup only. In both cases, we can use either the gid or the group name obtained from the /etc/group file. There are a couple of more options that we can use for an advanced level user creation, for example, for system users with uid less than 500, we have to use the -r option, which will create a user on the system but the uid will be less than 500. We also can use -u to define a specific uid, which must be unique and greater than 499. Common options that we can use with the useradd command are: -c: This option is used for comments, generally to define the user's real name such as -c "John Doe". -d: This option is used to define home-dir; by default, the home directory is created in /home such as -d /var/<user name>. -g: This option is used for the group name or the group number for the user's default group. The group must already have been created earlier. -G: This option is used for additional group names or group numbers, separated by commas, of which the user is a member. Again, these groups must also have been created earlier. -r: This option is used to create a system account with a UID less than 500 and without a home directory. -u: This option is the user ID for the user. It must be unique and greater than 499. There are few quick options that we use with the passwd command as well. These are: -l: This option is to lock the password for the user's account -u: This option is to unlock the password for the user's account -e: This option is to expire the password for the user -x: This option is to define the maximum days for the password lifetime -n: This option is to define the minimum days for the password lifetime Quotas In order to control the disk space used in the Linux filesystem, we must use quota, which enables us to control the disk space and thus helps us resolve low disk space issues to a great extent. For this, we have to enable user and group quota on the Linux system. In CentOS 7, the user and group quota are not enabled by default so we have to enable them first. To check whether quota is enabled, or not, we issue the following command: mount | grep ' / ' The image shows that the root filesystem is enabled without quota as mentioned by the noquota in the output. Now, we have to enable quota on the root (/) filesystem, and to do that, we have to first edit the file /etc/default/grub and add the following to the GRUB_CMDLINE_LINUX: rootflags=usrquota,grpquota The GRUB_CMDLINE_LINUX line should read as follows: GRUB_CMDLINE_LINUX="rd.lvm.lv=centos/swap vconsole.font=latarcyrheb-sun16 rd.lvm.lv=centos/root crashkernel=auto  vconsole.keymap=us rhgb quiet rootflags=usrquota,grpquota" The /etc/default/grub should like the following screenshot: Since we have to reflect the changes we just made, we should backup the grub configuration using the following command: cp /boot/grub2/grub.cfg /boot/grub2/grub.cfg.original Now, we have to rebuild the grub with the changes we just made using the command: grub2-mkconfig -o /boot/grub2/grub.cfg Next, reboot the system. Once it's up, login and verify that the quota is enabled using the command we used before: mount | grep ' / ' It should now show us that the quota is enabled and will show us an output as follows: /dev/mapper/centos-root on / type xfs (rw,relatime,attr2,inode64,usrquota,grpquota) Now, since quota is enabled, we will further install quota using the following to operate quota for different users and groups, and so on: yum -y install quota Once quota is installed, we check the current quota for users using the following command: repquota -as The preceding command will report user quotas in a human readable format. From the preceding screenshot, there are two ways we can limit quota for users and groups, one is setting soft and hard limits for the size of disk space used or another is limiting the user or group by limiting the number of files they can create. In both cases, soft and hard limits are used. A soft limit is something that warns the user when the soft limit is reached and the hard limit is the limit that they cannot bypass. We will use the following command to modify a user quota: edquota -u username Now, we will use the following command to modify the group quota: edquota -g groupname If you have other partitions mounted separately, you have to modify the /etc/fstab to enable quota on the filesystem by adding usrquota and grpquota after the defaults for that specific partition as in the following screenshot, where we have enabled the quota for the /var partition: Once you are finished enabling quota, remount the filesystem and run the following commands: To remount /var : mount -o remount /var To enable quota : quotacheck -avugm quotaon -avug Quota is something all system admins use to handle disk space consumed on a server by users or groups and limit over usage of the space. It thus helps them manage the disk space usage on the system. In this regard, it should be noted that you plan before your installation and create partitions accordingly as well so that the disk space is used properly. Multiple separate partitions such as /var and /home etc are always suggested, as generally, these are the partitions, which consume most space on a Linux system. So, if we keep them on a separate partition, it will not eat up the root ('/') filesystem space and will be more failsafe than using an entire filesystem mounted as only root. Password aging It is a good policy to have password aging so that the users are forced to change their password at a certain interval. This, in turn, helps to keep the security of the system as well. We can use chage to configure the password to expire the first time the user logs in to the system. Note: This process will not work if the user logs in to the system using SSH. This method of using chage will ensure that the user is forced to change the password right away. If we use only chage <username>, it will display the current password aging value for the specified user and will allow them to be changed interactively. The following steps need to be performed to accomplish password aging: Lock the user. If the user doesn't exist, we will use the useradd command to create the user. However, we will not assign any password to the user so that it remains locked. But, if the user already exists on the system, we will use the usermod command to lock the user: Usermod -L <username> Force immediate password change using the following command: chage -d 0 <username> Unlock the account, This can be achieved in two ways. One is to assign an initial password and the other way is to assign a null password. We will take the first approach as the second one, though possible, is not a good practice in terms of security. Therefore, here is what we do to assign an initial password: Use the python command to start the command-line python interpreter: import crypt; print crypt.crypt("Q!W@E#R$","Bing0000/") Here, we have used the Q!W@E#R$ password with a salt combination of the alphanumeric character: Bing0000 followed by a (/) character. The output is the encrypted password, similar to 'BiagqBsi6gl1o'. Press Ctrl + D to exit the Python interpreter. At the shell, enter the following command with the encrypted output of the Python interpreter: usermod -p "<encrypted-password>" <username> So, here, in our case, if the username is testuser, we will use the following command: usermod -p "BiagqBsi6gl1o" testuser Now, upon initial login using the "Q!W@E#R$" password, the user will be prompted for a new password. Setting the password policy This is a set of rules defined in some files, which have to be followed when a system user is setting up. It's an important factor in security because one of the many security breach histories was started with hacking user passwords. This is the reason why most organizations set a password policy for their users. All usernames and passwords must comply with this. A password policy usually is defined by the following: Password aging Password length Password complexity Limit login failures Limit prior password reuse Configuring password aging and password length Password aging and password length are defined in /etc/login.defs. Aging basically means the maximum number of days a password might be used, minimum number of days allowed between password changes, and number of warnings before the password expires. Length refers to the number of characters required for creating the password. To configure password aging and length, we should edit the /etc/login.defs file and set different PASS values according to the policy set by the organization. Note: The password aging controls defined here does not affect existing users; it only affects the newly created users. So, we must set these policies when setting up the system or the server at the beginning. The values we modify are: PASS_MAX_DAYS: The maximum number of days a password can be used PASS_MIN_DAYS: The minimum number of days allowed between password changes PASS_MIN_LEN: The minimum acceptable password length PASS_WARN_AGE: The number of days warning to be given before a password expires Let's take a look at a sample configuration of the login.defs file: Configuring password complexity and limiting reused password usage By editing the /etc/pam.d/system-auth file, we can configure the password complexity and the number of reused passwords to be denied. A password complexity refers to the complexity of the characters used in the password, and the reused password deny refers to denying the desired number of passwords the user used in the past. By setting the complexity, we force the usage of the desired number of capital characters, lowercase characters, numbers, and symbols in a password. The password will be denied by the system until and unless the complexity set by the rules are met. We do this using the following terms: Force capital characters in passwords: ucredit=-X, where X is the number of capital characters required in the password Force lower case characters in passwords: lcredit=-X, where X is the number of lower case characters required in the password Force numbers in passwords: dcredit=-X, where X is the number numbers required in the password Force the use of symbols in passwords: ocredit=-X, where X is the number of symbols required in the password For example: password requisite pam_cracklib.so try_first_pass retry=3 type= ucredit=-2 lcredit=-2 dcredit=-2 ocredit=-2 Deny reused passwords: remember=X, where X is the number of past passwords to be denied For example: password sufficient pam_unix.so sha512 shadow nullok try_first_pass use_authtok remember=5 Let's now take a look at a sample configuration of /etc/pam.d/system-auth: Configuring login failures We set the number of login failures allowed by a user in the /etc/pam.d/password-auth, /etc/pam.d/system-auth, and /etc/pam.d/login files. When a user's failed login attempts are higher than the number defined here, the account is locked and only a system administrator can unlock the account. To configure this, make the following additions to the files. The following deny=X parameter configures this, where X is the number of failed login attempts allowed: Add these two lines to the /etc/pam.d/password-auth and /etc/pam.d/system-auth files and only the first line to the /etc/pam.d/login file: auth        required    pam_tally2.so file=/var/log/tallylog deny=3 no_magic_root unlock_time=300 account     required    pam_tally2.so The following screenshot is a sample /etc/pam.d/system-auth file: The following is a sample /etc/pam.d/login file: To see failures, use the following command: pam_tally2 –user=<User Name> To reset the failure attempts and to enable the user to login again, use the following command: pam_tally2 –user=<User Name> --reset Sudoers Separation of user privilege is one of the main features in Linux operating systems. Normal users operate in limited privileged sessions to limit the scope of their influence on the entire system. One special user exists on Linux that we know about already is root, which has super-user privileges. This account doesn't have any restrictions that are present to normal users. Users can execute commands with super-user or root privileges in a number of different ways. There are mainly three different ways to obtain root privileges on a system: Login to the system as root Login to the system as any user and then use the su - command. This will ask you for the root password and once authenticated, will give you the root shell session. We can disconnect this root shell using Ctrl + D or using the command exit. Once exited, we will come back to our normal user shell. Run commands with root privileges using sudo without spawning a root shell or logging in as root. This sudo command works as follows: sudo <command to execute> Unlike su, sudo will request the password of the user calling the command, not the root password. Sudo doesn't work by default and requires to be set up before it functions correctly. In the following section, we will see how to configure sudo and modify the /etc/sudoers file so that it works the way we want it to. visudo Sudo is modified or implemented using the /etc/sudoers file, and visudo is the command that enables us to edit the file. Note: This file should not be edited using a normal text editor to avoid potential race conditions in updating the file with other processes. Instead, the visudo command should be used. The visudo command opens a text editor normally, but then validates the syntax of the file upon saving. This prevents configuration errors from blocking sudo operations. By default, visudo opens the /etc/sudoers file in Vi Editor, but we can configure it to use the nano text editor instead. For that, we have to make sure nano is already installed or we can install nano using: yum install nano -y Now, we can change it to use nano by editing the ~/.bashrc file: export EDITOR=/usr/bin/nano Then, source the file using: . ~/.bashrc Now, we can use visudo with nano to edit the /etc/sudoers file. So, let's open the /etc/sudoers file using visudo and learn a few things. We can use different kinds of aliases for different set of commands, software, services, users, groups, and so on. For example: Cmnd_Alias NETWORKING = /sbin/route, /sbin/ifconfig, /bin/ping, /sbin/dhclient, /usr/bin/net, /sbin/iptables, /usr/bin/rfcomm, /usr/bin/wvdial, /sbin/iwconfig, /sbin/mii-tool Cmnd_Alias SOFTWARE = /bin/rpm, /usr/bin/up2date, /usr/bin/yum Cmnd_Alias SERVICES = /sbin/service, /sbin/chkconfig and many more ... We can use these aliases to assign a set of command execution rights to a user or a group. For example, if we want to assign the NETWORKING set of commands to the group netadmin we will define: %netadmin ALL = NETWORKING Otherwise, if we want to allow the wheel group users to run all the commands, we use the following command: %wheel  ALL=(ALL)  ALL If we want a specific user, john, to get access to all commands we use the following command: john  ALL=(ALL)  ALL We can create different groups of users, with overlapping membership: User_Alias      GROUPONE = abby, brent, carl User_Alias      GROUPTWO = brent, doris, eric, User_Alias      GROUPTHREE = doris, felicia, grant Group names must start with a capital letter. We can then allow members of GROUPTWO to update the yum database and all the commands assigned to the preceding software by creating a rule like this: GROUPTWO    ALL = SOFTWARE If we do not specify a user/group to run, sudo defaults to the root user. We can allow members of GROUPTHREE to shutdown and reboot the machine by creating a command alias and using that in a rule for GROUPTHREE: Cmnd_Alias      POWER = /sbin/shutdown, /sbin/halt, /sbin/reboot, /sbin/restart GROUPTHREE  ALL = POWER We create a command alias called POWER that contains commands to power off and reboot the machine. We then allow the members of GROUPTHREE to execute these commands. We can also create Run as aliases, which can replace the portion of the rule that specifies to the user to execute the command as: Runas_Alias     WEB = www-data, apache GROUPONE    ALL = (WEB) ALL This will allow anyone who is a member of GROUPONE to execute commands as the www-data user or the apache user. Just keep in mind that later, rules will override previous rules when there is a conflict between the two. There are a number of ways that you can achieve more control over how sudo handles a command. Here are some examples: The updatedb command associated with the mlocate package is relatively harmless. If we want to allow users to execute it with root privileges without having to type a password, we can make a rule like this: GROUPONE    ALL = NOPASSWD: /usr/bin/updatedb NOPASSWD is a tag that means no password will be requested. It has a companion command called PASSWD, which is the default behavior. A tag is relevant for the rest of the rule unless overruled by its twin tag later down the line. For instance, we can have a line like this: GROUPTWO    ALL = NOPASSWD: /usr/bin/updatedb, PASSWD: /bin/kill In this case, a user can run the updatedb command without a password as the root user, but entering the root password will be required for running the kill command. Another helpful tag is NOEXEC, which can be used to prevent some dangerous behavior in certain programs. For example, some programs, such as less, can spawn other commands by typing this from within their interface: !command_to_run This basically executes any command the user gives it with the same permissions that less is running under, which can be quite dangerous. To restrict this, we could use a line like this: username    ALL = NOEXEC: /usr/bin/less We should now have clear understanding of what sudo is and how do we modify and provide access rights using visudo. There are many more things left here. You can check the default /etc/sudoers file, which has a good number of examples, using the visudo command, or you can read the sudoers manual as well. One point to remember is that root privileges are not given to regular users often. It is important for us to understand what these command do when you execute with root privileges. Do not take the responsibility lightly. Learn the best way to use these tools for your use case, and lock down any functionality that is not needed. Reference Now, let's take a look at the major reference used throughout the chapter: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/System_Administrators_Guide/index.html Summary In all we learned about some advanced user management and how to manage users through the command line, along with password aging, quota, exposure to /etc/sudoers, and how to modify them using visudo. User and password management is a regular task that a system administrator performs on servers, and it has a very important role in the overall security of the system. Resources for Article: Further resources on this subject: SELinux - Highly Secured Web Hosting for Python-based Web Applications [article] A Peek Under the Hood – Facts, Types, and Providers [article] Puppet Language and Style [article]
Read more
  • 0
  • 0
  • 8945

article-image-introduction-xenconvert
Packt
04 Sep 2013
3 min read
Save for later

Introduction to XenConvert

Packt
04 Sep 2013
3 min read
(For more resources related to this topic, see here.) System requirements Since XenConvert can only convert Windows-based hosts and installs on the same host, the requirements are pretty much the same, as follows: Operating system: Windows XP, Windows Vista, Windows 7, Windows Server 2003 (SP1 or later), Windows Server 2008 (R2) .Net Framework 4.0 Disk Space: 40 MB free disk space XenServer version 6.0 or 6.1 Converting a physical machine to a virtual machine Let's take a quick look at how to convert a physical machine to a virtual machine. First we need to install XenConvert on the source physical machine. We can download XenConvert from this link: http://www.citrix.com/downloads/xenserver/tools/conversion.html. Once the standard Windows installation process is complete, launch the XenConvert tool; but before that we need to prepare the host machine for the conversion. To know more about XenConvert, refer to the XenConvert guide at http://support.citrix.com/article/CTX135017. Preparing the host machine For best results, prepare the host machine as follows: Enable Windows Automount on Windows Server operating systems. Disable Windows Autoplay. Remove any virtualization software before performing a conversion. Ensure that adequate free space exists at the destination, which is approximately 101 percent of used space of all source volumes. Remove any network interface teams; they are not applicable to a virtual machine. We need to run the XenConvert tool on the host machine to start the physical-to-virtual conversion. We can convert the physical machine directly to our XenServer if this host machine is accessible. The other options are to convert to VHD, OVF, or vDisk, which can be imported later on to XenServer using some methods. These options are more useful if we don't have enough disk space or connectivity with XenServer. I chose XenServer and clicked on Next . We can select multiple partitions to be included in the conversion, or select none from the drop-down menu in Source Volume and those disks won't be included in the conversion. We can also increase or decrease the size of the new virtual partition to be allocated for this virtual machine. Click on Next . We'll be asked to provide the details of the XenServer host. The hostname needs either an IP address or a FQDN of the XenServer; a username and password are standard login requirements. In the Workspace field, enter the path to the folder to store the intermediate OVF package that XenConvert will use during the conversion process. XenConvert will store the OVF package in the path we give. Click on Next to select the storage repositories found with XenServer and continue to the last step, in which we'll be provided with the summary of the conversion. Soon after the conversion is completed, we'll be able to have this new machine in our XenCenter. We'll need to have XenServer Tools installed on this new virtual machine. Summary In this article we covered an advanced topic that explained the process of converting a physical Windows server to a virtual machine using XenConvert. Resources for Article : Further resources on this subject: Citrix XenApp Performance Essentials [Article] Defining alerts [Article] Publishing applications [Article]
Read more
  • 0
  • 0
  • 8933

article-image-microsoft-office-outlook-programming-using-vsto-30-and-c-part-1
Packt
14 Oct 2009
8 min read
Save for later

Microsoft Office Outlook Programming Using VSTO 3.0 and C#: Part 1

Packt
14 Oct 2009
8 min read
Microsoft Office Outlook object model overview Programming in Microsoft Office Outlook differs from programming other Microsoft applications such as InfoPath and Excel. Most of the Microsoft Office applications target documents rather than data items, but Microsoft Office Outlook targets each data item stored in the database or the primary data storage used by an Outlook application. Microsoft Office Outlook stores and manages data items such as emails, appointments, notes, tasks, and contacts in tables in a structured database. The Microsoft Office Outlook object model is based on COM (Component Object Model) and is used to interact with Outlook form regions, menus, and other application customization possibilities. It is similar to other Microsoft Office application object models when thought of with respect to the object model implementation interfaces for collections, objects, properties, methods, and events. COM is a language-neutral way of implementing objects that can be used in different environments. A large numbers of objects are available for developing and customizing Outlook 2007. If the developer needs to develop application add-ins for Outlook, he or she can program using the objects provided by the Outlook object model. In the Outlook object model, the class will represent each object in the UI to the user. For example, the Microsoft.Office.Interop.Outlook.Application class represents the entire application, and the Microsoft.Office.Interop.Outlook.MailItem class represents an email message. To get used to the object models in Outlook, you should be familiar with some of the top-level objects. The Application object represents an Outlook application, and it is the highest level class in the Outlook object model. The Explorer object corresponds to the window that displays the contents of a folder, and contains Outlook data items such as email messages, tasks, appointments, and so on. The MAPIFolder object represents the folder that contains emails, contacts, tasks, and other Outlook data items. By default, there are sixteen MAPIFolder objects available. The Inspector object corresponds to a window that displays a single item such as a particular email message, or a specific contact item. Let's see how to create application-level add-ins for Microsoft Office Outlook 2007 using VSTO 3.0. We will create a Hello World application-level example for Microsoft Office Outlook 2007 using Visual Studio 2008. Open Visual Studio 2008 to create a new Outlook 2007 Add-in template project. Select New Project. Under Office select 2007, and then select the Microsoft Outlook Add-in template, and name the project as per your requirements. The solution will be created with all of the supporting files required for the development of Outlook solution, as shown in the following screenshot: Write the following code to display a message box: // Windows forms namespace to display Message boxusing System.Windows.Forms; Write the code to display the Say Hello World! message while loading: private void ThisAddIn_Startup(object sender, System.EventArgs e){// Message box display MessageBox.Show("Say Hello World!");} The result will be similar to what is shown in the following screenshot: Outlook 2007 object models are categorized and mapped correspondingly. The following Outlook object models hold all of Outlook's objects for programming in their corresponding object model categories: Items object model, Navigation bars and Outlook bars object model, Rules object model, and Views object model. In the Application object model hierarchy, the Application object is the parent of all other Outlook objects. Customization using VSTO Most enterprise business applications are loaded with a wide variety of features for users. Even though these applications have a wide range of features, many business requirements can be fulfilled only through customization because not all applications are designed to fit each and every enterprise's special needs. Customization for most of the application is a tough job to execute. Outlook 2007 is loaded with a wide variety of features that will satisfy a broad range of user categories. Some organizations need more functionality and features to be added, so that the application will satisfy their custom business requirements. VSTO 3.0 helps Office developers to customize and enhance the Outlook 2007 application as per the user's business requirements. Microsoft Office Outlook 2007 supports other Microsoft Office tools such as InfoPath and Excel, in order to provide seamless collaboration. VSTO provides an easy way to create an application-level add-in for Outlook 2007 using Visual Studio 2008. Creating add-ins for Microsoft Office Outlook 2007 has been more complex to work out, but VSTO 3.0 offers Office developers with project templates in Visual Studio 2008 to allow them to create add-ins for Outlook 2007. Adding to the project templates, VSTO offers great support for development and deployment, which improves the development work. This provides .NET framework support for Outlook 2007 add-in programming, which includes class library support, controlled exception handling, memory management, extensibility, ClickOnce deployment, and so on. Menus in Outlook A group of commands or lists of options from which you can choose your desired operation is known as a menu. Most of the latest applications are menu driven. Microsoft Office Outlook, which is a menu-driven application, provides the user with an easy fl owing UI for user interaction. Most of the menu-driven applications will provide you with basic customization such as choosing the menu for the default view of the application, and so on. In Microsoft Office, all menus and toolbars are CommandBars. A CommandBar is a static collection shared by all Windows. There are standard toolbars, menu bars, context menus, and so on. A small add-in enumerates and displays all Microsoft Outlook CommandBars. Likewise, Microsoft Office Outlook 2007 provides the option for a user to customize the menus. VSTO 3.0 provides Office developers with the ability to build custom menus and customize the existing menus using the .NET framework and support a programming language. You can even rebuild the classic menu style for Outlook 2007 by using the VSTO 3.0 application level add-ins development. Let's create a custom menu in the menu bar of the Outlook and add a new item to the menu. This way, you will get to know about custom menu development for Microsoft Office Outlook 2007. Open Visual Studio 2008 and create a new solution, as described in the previous example. Let's write a program to create a menu item and call it Say Hello World. // Defining new Menubarprivate Office.CommandBar PacktOldMenuBar;// Defining old Menubarprivate Office.CommandBarPopup PacktNewMenuBar;// Defining instance of button for menu itemprivate Office.CommandBarButton PacktButton1;// Tag string for our Menu itemprivate string strMenuString = "Outlook AddIn #1";private void ThisAddIn_Startup(object sender,System.EventArgs e){// Define the Old Menu BarPacktOldMenuBar = this.Application.ActiveExplorer().CommandBars.ActiveMenuBar;// Define the new Menu Bar into the existing menu barPacktNewMenuBar = (Office.CommandBarPopup)PacktOldMenuBar.Controls.Add(Office.MsoControlType.msoControlPopup,missing, missing, missing, false);//If PacktNewMenuBar not found then the code will add itif (PacktNewMenuBar != null){// Set caption for the MenuPacktNewMenuBar.Caption = "Packt Menu Item 1";// Tag string value passingPacktNewMenuBar.Tag = strMenuString;// Assigning button typePacktButton1 = (Office.CommandBarButton)PacktNewMenuBar.Controls.Add(Office.MsoControlType.msoControlButton, missing, missing, 1, true);// Setting up the button stylePacktButton1.Style = Office.MsoButtonStyle.msoButtonIconAndCaptionBelow;// Set button captionPacktButton1.Caption = "Say Hello World";// Set the menu visiblePacktNewMenuBar.Visible = true;}} The following screenshot displays the resulting menu developed by you using the preceding code: You can also build more custom menus for your Outlook 2007 with functionality as per your requirements. VSTO will speed up development and provide support for a hassle free environment for the developer to work on Outlook 2007 add-ins and other customization. Toolbars in Outlook Generally, toolbars provide easy access to the functionality of the application by using buttons and menus. Most application's user interfaces have a toolbar that has buttons, menus, and input or output control elements for user interaction with the application. Even applications allow users to do visual customization of toolbars as per the users' needs. Microsoft Office Outlook 2007 provides a very good visual representation of toolbars for user interaction with the application. Outlook provides support for toolbar customization to improve custom visual interaction for users. VSTO 3.0 offers wide options to build custom toolbars and to customize existing toolbars as per the user's needs. Remember that menus and CommandBars are not VSTO features, but are in the Microsoft.Office namespace. VSTO is making it easier to program for the Office object model. Let's see a demonstration of creating a custom toolbar with a button. Open Visual Studio 2008, and create a new solution, as described above. Next, add the reference needed to show the message box needed in our demonstration: // Namespace reference for message boxusing System.Windows.Forms; Let's write a program to create a toolbar and call it Hello World!. Office.CommandBar PacktCustomToolBar;// Declare the buttonOffice.CommandBarButton PacktButtonA;private void ThisAddIn_Startup(object sender, System.EventArgs e){// Verify the PacktCustomToolBar exist and add to the applicationif (PacktCustomToolBar == null){// Adding the commandbar to Active explorerOffice.CommandBars PacktBars = this.Application.ActiveExplorer().CommandBars;// Adding PacktCustomToolBar to the commandbarsPacktCustomToolBar = PacktBars.Add("NewPacktToolBar",Office.MsoBarPosition.msoBarTop, false, true);}// Adding button to the custom tool barOffice.CommandBarButton MyButton1 = (Office.CommandBarButton)PacktCustomToolBar.Controls.Add(1,missing, missing, missing, missing);// Set the button styleMyButton1.Style = Office.MsoButtonStyle.msoButtonCaption;// Set the caption and tag stringMyButton1.Caption = "PACKT BUTTON";MyButton1.Tag = "MY BUTTON";if (this.PacktButtonA == null){// Adding the event handler for the button in the toolbarthis.PacktButtonA = MyButton1;PacktButtonA.Click += new Office._CommandBarButtonEvents_ClickEventHandler(ButtonClick);}}// Button event in the custom toolbarprivate void ButtonClick(Office.CommandBarButton ButtonContrl,ref bool CancelOption){// Message box displayed on button clickMessageBox.Show(ButtonContrl.Caption + " Says Hello World!");} The following image shows the results of adding a custom toolbar with button control: The CommandBars object helps you to build variants of toolbars to your Outlook 2007 application's user interface. Microsoft Office Outlook 2007 is the only tool in the Microsoft Office 2007 family to support both standard toolbars and the new Ribbon in their UI.
Read more
  • 0
  • 0
  • 8931
article-image-cocos2d-x-installation
Packt
05 Sep 2013
10 min read
Save for later

Cocos2d-x: Installation

Packt
05 Sep 2013
10 min read
(For more resources related to this topic, see here.) Download and installation All the examples in this article were developed on a Mac using Xcode. Although you can use Cocos2d-x to develop your games for other platforms, using different systems, the examples will focus on iOS and Mac. Xcode is free and can be downloaded from the Mac App store (https://developer.apple.com/xcode/index.php), but in order to test your code on an iOS device and publish your games, you will need a developer account with Apple, which will cost you USD 99 a year. You can find more information on their website: https://developer.apple.com/ So, assuming you have an internet connection, and that Xcode is ready to rock, let's begin! Time for action – downloading and installing Cocos2d-x We start by downloading the framework: Go to http://download.cocos2d-x.org/ and download the latest stable version of Cocos2d-x. For this article I'll be using version Cocos2d-2.0-x-2.0.4, which means the 2.0.4 C++ port of version 2.0 of Cocos2d. Uncompress the files somewhere on your machine. Open Terminal and type cd (that is cd and a space). Drag the uncompressed folder you just downloaded to the Terminal window. You should see the path to the folder added to the command line. Hit returnto go to that folder in Terminal. Now type: sudo ./install-templates-xcode.sh -u Hit return again and you're done. What just happened? You have successfully installed the Cocos2d-x templates in your machine. With these in place, you can select the type of Cocos2d-x application you wish to build inside Xcode, and the templates will take care of copying all the necessary files into your application. Next, open Xcode and select Create a new Xcode Project.You should see something like this: So let's build our first application. Hello-x World-x Let's create that old chestnut in computer programming: the hello world example. Time for action – creating an application Open Xcode and select File | New | Project... and follow these steps: In the dialogue box select cocos2d-x under the iOS menu and choose the cocos2dx template. Hit Next . Give the application a name, but not HelloWorld. I'll show you why in a second. You will be then asked to select a place to save the project and you are done. Once your application is ready, click Run to build it. After that, this is what you should see in the simulator: When you run a cocos2d-x application in Xcode it is quite common for the program to post some warnings regarding your code, or most likely the frameworks. These will mostly reference deprecated methods, or statements that do not precisely follow more recent, and stricter rules of the current SDK. But that's okay. These warnings, though certainly annoying, can be ignored. What just happened? You created your first Cocos2d-x application using the cocos2dx template, sometimes referred to as the basic template. The other template options include one with Box2D, one with Chipmunk (both related to physics simulation), one with JavaScript, and one with Lua. The last two options allow you to code some or all of your game using those script languages instead of the native C++; and they work just as you would expect a scripting language to work, meaning the commands written in either Javascript or Lua are actually replaced and interpreted as C++ commands by the compiler. Now if you look at the files created by the basic template you will see a HelloWorldScene class file. That's the reason I didn't want you to call your application HelloWorld, because I didn't want you to have the impression that the file name was based on your project name. It isn't. You will always get a HelloWorldScene file unless you change the template itself. Now let's go over the sample application and its files: The folder structure First you have the Resources folder, where you find the images used by the application. The ios folder has the necessary underlying connections between your app and iOS. For other platforms, you will have their necessary linkage files in separate folders targeting their respective platform (like an android folder the Android platform, for instance.) In the libs folder you have all the cocos2dx files, plus CocosDenshion files (for sound support) and a bunch of other extensions. Using a different template for your projects will result in a different folder structure here, based on what needs to be added to your project. So you will see a Box2D folder, for example, if you choose the Box2D template. In the Classes folder you have your application. In here, everything is written in C++ and this is the home for the part of your code that will hopefully not need to change, however many platforms you target with your application. Now let us go over the main classes of the basic application. The iOS linkage classes AppController and RootViewController are responsible for setting up OpenGL in iOS as well as telling the underlying operating system that your application is about to say Hello... To the World. These classes are written with a mix of Objective-C and C++, as all the nice brackets and the .mm extensions show. You will change very little if anything in these classes; and again that will reflect in changes to the way iOS handles your application. So other targets would require the same instructions or none at all depending on the target. In AppController for instance, I could add support for multitouch. And in RootViewController, I could limit the screen orientations supported by my application. The AppDelegate class This class marks the first time your C++ app will talk to the underlying OS. It attempts to map the main events that mobile devices wants to dispatch and listen to. From here on, all your application will be written in C++ (unless you need something else). In AppDelegate you should setup CCDirector (the cocos2d-x all powerful singleton manager object) to run your application just the way you want. You can: Get rid of the application status information Change the frame rate of your application Tell CCDirector where your high definition images are, and where your standard definition images are, as well as which to use You can change the overall scale of your application to suit different screens The AppDelegate class is also the best place to start any preloading process And, most importantly, it is here you tell the CCDirector object what CCScene to begin your application with Here too you will handle what happens to your application if the OS decides to kill it, push it aside, or hang it upside down to dry. All you need to do is place your logic inside the correct event handler: applicationDidEnterBackground or applicationWillEnterForeground. The HelloWorldScene class When you run the application you get a screen with the words Hello World and a bunch of numbers in one corner. These are the display stats you decided you wanted around in the AppDelegate class. The actual screen is created by the oddly named HelloWorldScene class. It is a Layer class that creates its own scene (don't worry if you don't know what a Layer class is, or a Scene class, you will soon enough). When it initializes, HelloWorldScene puts a button on screen that you can press to exit the application. The button is actually a Menu item, part of a Menu group consisting of one button, two image states for that button, and one callback event, triggered when the said button is pressed. The Menu group automatically handles touch events targeting its members, so you don't get to see any of that code floating about. There is also the necessary Label object to show the Hello World message and the background image. Who begets whom If you never worked with either Cocos2d or Cocos2d-x before, the way the initial scene() method is instantiated may lead to dizziness. To recap, in AppDelegate you have: CCScene *pScene = HelloWorld::scene(); pDirector->runWithScene(pScene); CCDirector needs a CCScene object to run, which you can think of as being your application, basically. CCScene needs something to show, which in this case is a CCLayer class. CCScene is then said to contain a CCLayer class. Here a CCScene object is created through a static method scene inside a CCLayer derived class. So the layer creates the scene, and the scene immediately adds the layer to itself. Huh? Relax. This incestuous-like instantiation will most likely only happen once, and you have nothing to do with it when it happens. So you can easily ignore all these funny goings-on and look the other way. I promise instantiations will be much easier after this first one. Further information Follow these steps to access one of the best sources for reference material on Cocos2d-x: its Test project. Time for action – running the test samples You open the test project just like you would do for any other Xcode project: Go inside the folder you downloaded for the framework, and navigate to samples/TestCpp/proj.ios/TestCpp.xcodeproj. Open that project in Xcode. When you run the project, you will see inside the simulator a long list of tests, all nicely organized by topic. Pick any one to review. Better yet, navigate to samples/TestCpp/Classes and if you have a program like TextWrangler or some equivalent, you can open that entire directory inside a Disk Browser window and have all that information ready for referencing right at your desktop. What just happened? With the test samples you can visualize most features in Cocos2d-x and see what they do, as well as some of the ways you can initialize and customize them. I will refer to the code found in the tests quite often. As usual with programming, there is always a different way to accomplish a given task, so sometimes after showing you one way, I'll refer to another one that you can find (and by then easily understand) inside the Test classes. The other tools Now comes the part where you may need to spend a bit more money to get some extremely helpful tools. In this articles examples I use four of them: A tool to help build sprite sheets: I'll use Texture Packer (http://www.codeandweb.com/texturepacker). There are other alternatives, like Zwoptex (http://zwopple.com/zwoptex/). And they usually offer some features for free. A tool to help build particle effects: I'll use Particle Designer (http://www.71squared.com/en/particledesigner). Depending on your operating system you may find free tools online for this. Cocos2d-x comes bundled with some common particle effects that you can customize. But to do it blindly is a process I do not recommend. A tool to help build bitmap fonts: I'll use Glyph Designer (http://www.71squared.com/en/glyphdesigner). But there are others: bmGlyph (which is not as expensive), FontBuilder (which is free). It is not extremely hard to build a Bitmap font by hand, not nearly as hard as building a particle effect from scratch, but doing it once is enough to convince you to get one of these tools fast. A tool to produce sound effects: No contest. cfxr for Mac or the original sfxr for Windows. Both are free (http://www.drpetter.se/project_sfxr.html and http://thirdcog.eu/apps/cfxr respectively). Summary You just learned how to install Cocos2d-x templates and create a basic application. You also learned enough of the structure of a basic Cocos2d-x application to get started to build your first game. Resources for Article: Further resources on this subject: Getting Started With Cocos2d [Article] Cocos2d: Working with Sprites [Article] Cocos2d for iPhone: Surfing Through Scenes [Article]
Read more
  • 0
  • 0
  • 8926

article-image-android-and-ios-apps-testing-glance
Packt
02 Feb 2016
21 min read
Save for later

Android and iOS Apps Testing at a Glance

Packt
02 Feb 2016
21 min read
In this article by Vijay Velu, the author of Mobile Application Penetration Testing, we will discuss the current state of mobile application security and the approach to testing for vulnerabilities in mobile devices. We will see the major players in the smartphone OS market and how attackers target users through apps. We will deep-dive into the architecture of Android and iOS to understand the platforms and its current security state, focusing specifically on the various vulnerabilities that affect apps. We will have a look at the Open Web Application Security Project (OWASP) standard to classify these vulnerabilities. The readers will also get an opportunity to practice the security testing of these vulnerabilities via the means of readily available vulnerable mobile applications. The article will have a look at the step-by-step setup of the environment that's required to carry out security testing of mobile applications for Android and iOS. We will also explore the threats that may arise due to potential vulnerabilities and learn how to classify them according to their risks. (For more resources related to this topic, see here.) Smartphones' market share Understanding smartphones' market share will give us a clear picture about what cyber criminals are after and also what could be potentially targeted. The mobile application developers can propose and publish their applications on the stores and be rewarded by a revenue share of the selling price. The following screenshot that was taken from www.idc.com provides us with the overall smartphone OS market in 2015: Since mobile applications are platform-specific, majority of the software vendors are forced to develop applications for all the available operating systems. Android operating system Android is an open source, Linux-based operating system for mobile devices (smartphones and tablet computers). It was developed by the Open Handset Alliance, which was led by Google and other companies. Android OS is Linux-based. It can be programmed in C/C++, but most of the application development is done in Java (Java accesses C libraries via JNI, which is short for Java Native Interface). iPhone operating system (iOS) It was developed by Apple Inc. It was originally released in 2007 for iPhone, iPod Touch, and Apple TV. Apple's mobile version of the OS X operating system that's used in Apple computers is iOS. Berkeley Software Distribution (BSD) is UNIX-based and can be programmed in Objective C. Public Android and iOS vulnerabilities Before we proceed with different types of vulnerabilities on Android and iOS, this section introduces you to Android and iOS as an operating system and covers various fundamental concepts that need to be understood to gain experience in mobile application security. The following table comprises year-wise operating system releases: Year Android iOS 2007/2008 1.0 iPhone OS 1 iPhone OS 2 2009 1.1 iPhone OS 3 1.5 (Cupcake) 2.0 (Eclair) 2.0.1(Eclair) 2010 2.1 (Eclair) iOS 4 2.2 (Froyo) 2.3-2.3.2(Gingerbread) 2011 2.3.4-2.3.7 (Gingerbread) iOS 5 3.0 (HoneyComb) 3.1 (HoneyComb) 3.2 (HoneyComb) 4.0-4.0.2 (Ice Cream Sandwich) 4.0.3-4.0.4 (Ice Cream Sandwich) 2012 4.1 (Jelly Bean) iOS 6 4.2 (Jelly Bean) 2013 4.3 (Jelly bean) iOS 7 4.4 (KitKat) 2014 5.0 (Lollipop) iOS 8 5.1 (Lollipop) 2015   iOS 9 (beta) An interesting research conducted by Hewlett Packard (HP), a software giant that tested more than 2,000 mobile applications from more than 600 companies, has reported the following statistics (for more information, visit http://www8.hp.com/h20195/V2/GetPDF.aspx/4AA5-1057ENW.pdf): 97% of the applications that were tested access at least one private information source of these applications 86% of the applications failed to use simple binary-hardening protections against modern-day attacks 75% of the applications do not use proper encryption techniques when storing data on a mobile device 71% of the vulnerabilities resided on the web server 18% of the applications sent usernames and password over HTTP (of the remaining 85%, 18% implemented SSL/HTTPS incorrectly) So, the key vulnerabilities to mobile applications arise due to a lack of security awareness, "usability versus security trade-off" by developers, excessive application permissions, and a lack of privacy concerns. Coupling this with a lack of sufficient application documentation leads to vulnerabilities that developers are not aware of. Usability versus security trade-off For every developer, it would not be possible to provide users with an application with high security and usability. Making any application secure and usable takes a lot of effort and analytical thinking. Mobile application vulnerabilities are broadly categorized as follows: Insecure transmission of data: Either an application does not enforce any kind of encryption for data in transit on a transport layer, or the implemented encryption is insecure. Insecure data storage: Apps may store data either in a cleartext or obfuscated format, or hard-coded keys in the mobile device. An example e-mail exchange server configuration on Android device that uses an e-mail client stores the username and password in cleartext format, which is easy to reverse by any attacker if the device is rooted. Lack of binary protection: Apps do not enforce any anti-reversing, debugging techniques. Client-side vulnerabilities: Apps do not sanitize data provided from the client side, leading to multiple client-side injection attacks such as cross-site scripting, JavaScript injection, and so on. Hard-coded passwords/keys: Apps may be designed in such a way that hard-coded passwords or private keys are stored on the device storage. Leakage of private information: Apps may unintentionally leak private information. This could be due to the use of a particular framework and obscurity assumptions of developers. Android vulnerabilities In July 2015, a security company called Zimperium announced that it discovered a high-risk vulnerability named Stagefright inside the Android operating system. They deemed it as a unicorn in the world of Android risk, and it was practically demonstrated in one of the hacking conferences in the US on August 5, 2015. More information can be found at https://blog.zimperium.com/stagefright-vulnerability-details-stagefright-detector-tool-released/; a public exploit is available at https://www.exploit-db/exploits/38124/. This has made Google release security patches for all Android operating systems, which is believed to be 95% of the Android devices, which is an estimated 950 million users. The vulnerability is exploited through a particular library, which can let attackers take control of an Android device by sending a specifically crafted multimedia services like Multimedia Messaging Service (MMS). If we take a look at the superuser application downloads from the Play Store, there are around 1 million to 5 million downloads. It can be assumed that a major portion of Android smartphones are rooted. The following graphs show the Android vulnerabilities from 2009 until September 2015. There are currently 54 reported vulnerabilities for the Android Google operating system (for more information, visit http://www.cvedetails.com/product/19997/Google-Android.html?vendor_id=1224). More features that are introduced to the operating system in the form of applications act as additional entry points that allow cyber attackers or security researchers to circumvent and bypass the controls that were put in place. iOS vulnerabilities On June 18, 2015, password stealing vulnerability, also known as Cross Application Reference Attack (XARA), was outlined for iOS and OS X. It cracked the keychain services on jailbroken and non-jailbroken devices. The vulnerability is similar to cross-site request forgery attack in web applications. In spite of Apple's isolation protection and its App Store's security vetting, it was possible to circumvent the security controls mechanism. It clearly provided the need to protect the cross-app mechanism between the operating system and the app developer. Apple rolled a security update week after the XARA research. More information can be found at http://www.theregister.co.uk/2015/06/17/apple_hosed_boffins_drop_0day_mac_ios_research_blitzkrieg/ The following graphs show the vulnerabilities in iOS from 2007 until September 2015. There are around 605 reported vulnerabilities for Apple iPhone OS (for more information, visit http://www.cvedetails.com/product/15556/Apple-Iphone-Os.html?vendor_id=49). As you can see, the vulnerabilities kept on increasing year after year. A majority of the vulnerabilities reported are denial-of-service attacks. This vulnerability makes the application unresponsive. Primarily, the vulnerabilities arise due to insecure libraries or overwriting with plenty of buffer in the stacks. Rooting/jailbreaking Rooting/jailbreaking refers to the process of removing the limitations imposed by the operating system on devices through the use of exploit tools. Rooting/jailbreaking enables users to gain complete control over the operating system of a device. OWASP's top ten mobile risks In 2013, OWASP polled the industry for new vulnerability statistics in the field of mobile applications. The following risks were finalized in 2014 as the top ten dangerous risks as per the result of the poll data and mobile application threat landscape: M1: Weak server-side controls: Internet usage via mobiles has surpassed fixed Internet access. This is largely due to the emergence of hybrid and HTML5 mobile applications. Application servers that form the backbone of these applications must be secured on their own. The OWASP top 10 web application project defines the most prevalent vulnerabilities in this realm. Vulnerabilities such as injections, insecure direct object reference, insecure communication, and so on may lead to the complete compromise of an application server. Adversaries who have gained control over the compromised servers can push malicious content to all the application users and compromise user devices as well. M2: Insecure data storage: Mobile applications are being used for all kinds of tasks such as playing games, fitness monitors, online banking, stock trading, and so on, and most of the data used by these applications are either stored in the device itself inside SQLite files, XML data stores, log files, and so on, or they are pushed on to Cloud storage. The types of sensitive data stored by these applications may range from location information to bank account details. The application programing interfaces (API) that handle the storage of this data must securely implement encryption/hashing techniques so that an adversary with direct access to these data stores via theft or malware will not be able to decipher the sensitive information that's stored in them. M3: Insufficient transport layer protection: "Insecure Data Storage", as the name says, is about the protection of data in storage. But as all the hybrid and HTML 5 apps work on client-server architecture, emphasis on data in motion is a must, as the data will have to traverse through various channels and will be susceptible to eavesdropping and tampering by adversaries. Controls such as SSL/TLS, which enforce confidentiality and integrity of data, must be verified for correct implementations on the communication channel from the mobile application and its server. M4: Unintended data leakage: Certain functionalities of mobile applications may place users' sensitive data in locations where it can be accessed by other applications or even by malware. These functionalities may be there in order to enhance the usability or user experience but may pose adverse effects in the long run. Actions such as OS data caching, key press logging, copy/paste buffer caching, and implementations of web beacons or analytics cookies for advertisement delivery can be misused by adversaries to gain information about users. M5: Poor authorization and authentication: As mobile devices are the most "personal" devices, developers utilize this to store important data such as credentials locally in the device itself and come up with specific mechanisms to authenticate and authorize users locally for the services that users request via the application. If these mechanisms are poorly developed, adversaries may circumvent these controls and unauthorized actions can be performed. As the code is available to adversaries, they can perform binary attacks and recompile the code to directly access authorized content. M6: Broken cryptography: This is related to the weak controls that are used to protect data. Using weak cryptographic algorithms such as RC2, MD5, and so on, which can be cracked by adversaries, will lead to encryption failure. Improper encryption key management when a key is stored in locations accessible to other applications or the use of a predictable key generation technique will also break the implemented cryptography techniques. M7: Client-side injection: Injection vulnerabilities are the most common web vulnerabilities according to OWASP web top 10 dangerous risks. These are due to malformed inputs, which cause unintended action such as an alteration of database queries, command execution, and so on. In case of mobile applications, malformed inputs can be a serious threat at the local application level and server side as well (refer to M1: Weak server-side controls). Injections at a local application level, which mainly target data stores, may result in conditions such as access to paid content that's locked for trial users or file inclusions that may lead to an abuse of functionalities such as SMSes. M8: Security decisions via untrusted inputs: An implementation of certain functionalities such as the use of hidden variables to check authorization status can be bypassed by tampering them during the transit via web service calls or inter-process communication calls. This may lead to privilege escalations and unintended behavior of mobile applications. M9: Improper session handling: The application server sends back a session token on successful authentication with the mobile application. These session tokens are used by the mobile application to request for services. If these session tokens remain active for a longer duration and adversaries obtain them via malware or theft, the user account can be hijacked. M10: Lack of binary protection: A mobile application's source code is available to all. An attacker can reverse engineer the application and insert malicious code components and recompile them. If these tampered applications are installed by a user, they will be susceptible to data theft and may be the victims of unintended actions. Most applications do not ship with mechanisms such as checksum controls, which help in deducing whether the application is tampered or not. In 2015, there was another poll under the OWASP Mobile security group named the "umbrella project". This leads us to have M10 to M2, the trends look at binary protection to take over weak server-side controls. However, we will have wait until the final list for 2015. More details can be found at https://www.owasp.org/images/9/96/OWASP_Mobile_Top_Ten_2015_-_Final_Synthesis.pdf. Vulnerable applications to practice The open source community has been proactively designing plenty of mobile applications that can be utilized for practical tests. These are specifically designed to understand the OWASP top ten risks. Some of these applications are as follows: iMAS: iMAS is a collaborative research project initiated by the MITRE corporation (http://www.mitre.org/). This is for application developers and security researchers who would like to learn more about attack and defense techniques in iOS. More information about iMAS can be found at https://github.com/project-imas/about. GoatDroid: A simple functional mobile banking application for training with location tracking developed by Jack and Ken for Android application security is a great starting point for beginners. More information about GoatDroid can be found at https://github.com/jackMannino/OWASP-GoatDroid-Project. iGoat: The OWASP's iGOAT project is similar to the WebGoat web application framework. It's designed to improve the iOS assessment techniques for developers. More information on iGoat can be found at https://code.google.com/p/owasp-igoat/. Damn Vulnerable iOS Application (DVIA): DVIA is an iOS application that provides a platform for developers, testers, and security researchers to test their penetration testing skills. This application covers all the OWASP's top 10 mobile risks and also contains several challenges that one can solve and come up with custom solutions. More information on the Damn Vulnerable iOS Application can be found at http://damnvulnerableiosapp.com/. MobiSec: MobiSec is a live environment for the penetration testing of mobile environments. This framework provides devices, applications, and supporting infrastructure. It provides a great exercise for testers to view vulnerabilities from different points of view. More information on MobiSec can be found at http://sourceforge.net/p/mobisec/wiki/Home/. Android application sandboxing Android utilizes the well-established Linux protection ring model to isolate applications from each other. In Linux OS, assigning unique ID segregates every user. This ensures that there is no cross account data access. Similarly in Android OS, every app is assigned with its own unique ID and is run as a separate process. As a result, an application sandbox is formed at the kernel level, and the application will only be able to access the resources for which it is permitted to access. This subsequently ensures that the app does not breach its work boundaries and initiate any malicious activity. For example, the following screenshot provides an illustration of the sandbox mechanism: From the preceding Android Sandbox illustration, we can see how the unique Linux user ID created per application is validated every time a resource mapped to the app is accessed, thus ensuring a form of access control. Android Studio and SDK On May 16, 2013 at the Google I/O conference, an Integrated Development Environment (IDE) was released by Katherine Chou under Apache license 2.0; it was called Android Studio and it's used to develop apps on the Android platform. It entered the beta stage in 2014, and the first stable release was on December 2014 from Version 1.0 and it has been announced the official IDE on September 15, 2015. Information on Android Studio and SDK is available at http://developer.android.com/tools/studio/index.html#build-system. Android Studio and SDK heavily depends on the Java SE Development Kit. Java SE Development Kit can be downloaded at http://www.oracle.com/technetwork/java/javase/downloads/jdk7-downloads-1880260.html. Some developers prefer different IDEs such as eclipse. For them, Google only offers SDK downloads (http://dl.google.com/android/installer_r24.4.1-windows.exe). There are minimum system requirements that need to be fulfilled in order to install and use the Android Studio effectively. The following procedure is used to install the Android Studio on Windows 7 Professional 64-bit Operating System with 4 GB RAM, 500 Gig Hard Disk Space, and Java Development Kit 7 installed: Install the IDE available for Linux, Windows, and Mac OS X. Android Studio can be downloaded by visiting http://developer.android.com/sdk/index.html. Once the Android Studio is downloaded, run the installer file. By default, an installation window will be shown, as shown in the following screenshot. Click on Next: This setup will automatically check whether the system meets the requirements. Choose all the components that are required and click on Next. It is recommended to read and accept the license and click on Next. It is always recommended to create a new folder to install the tools that will help us track all the evidence in a single place. In this case, we have created a folder called Hackbox in C:, as shown in the following screenshot: Now, we can allocate the space required for the Android-accelerated environment, which will provide better performance. So, it is recommended to allocate a minimum of 2GB for this space. All the necessary files will be extracted to C:Hackbox. Once the installation is complete, you will be able to launch Android Studio, as shown in the following screenshot: Android SDK Android SDK provides developers with the ability to completely build, test, and debug apps that run on the Android platform. It has all the relevant software libraries, APIs, system images of the emulators, documentations, and other tools that help create an Android app. We have installed Android Studio with Android SDK. It is crucial to understand how to utilize the in-built SDK tools as much as possible. This section provides an overview of some of the critical tools that we will be using when attacking an Android app during the penetration testing activity. Emulator, simulators, and real devices Sometimes, we tend to believe that all virtual emulations work in exactly the same way in real devices, which is not really the case. Especially for Android, we have multiple OEMs manufacturing multiple devices, with different chipsets running different versions of Android. It would be challenge for developers to ensure that all the functionalities for the app reflect in the same way in all devices. It is very crucial to understand the difference between an emulator, simulator, and real devices. Simulators An objective of a simulator is to simulate the state of an object, which is exactly the same state as that of an object. It is preferable that the testing happens when a mobile interacts with some of the natural behavior of the available resources. These are reimplementations of the original software applications that are written, and they are difficult to debug and are mostly writing in high-level languages. Emulators Emulators predominantly aim at replicating the closest possible behavior of mobile devices. These are typically used to test a mobile's behavior internally, such as hardware, software, and firmware updates. These are typically written in machine-level languages and are easy to debug. This is again the reimplementation of the real software. Pros Fast, simple, and little or no price associated Emulators/simulators are quickly available to test the majority of the functionality of the app that is being developed It is very easy to find the defects using emulators and fix issues Cons The risk of false positives is increased; some of the functions or protection may actually not work on a real device. Differences in software and hardware will arise. Some of the emulators might be able to mimic the hardware. However, it may or may not work when it is actually installed on that particular hardware in reality. There's a lack of network interoperability. Since emulators are not really connected to a Wi-Fi or cellular network, it may not be possible to test network-based risks/functions. Real devices Real devices are physical devices that a user will be interacting with. There are pros and cons of real devices too. Pros Lesser false positives: Results are accurate Interoperability: All the test cases are on a live environment User experience: Real user experience when it comes to the CPU utilization, memory, and so on for a provided device Performance: Performance issues can be found quickly with real handsets Cons Costs: There are plenty of OEMs, and buying all the devices is not viable. A slowdown in development: It may not be possible to connect an IDE and than emulators. This will significantly slow down the development process. Other issues: The devices that are locally connected to the workstation will have to ensure that USB ports are open, thus opening an additional entry point. Threats A threat is something that can harm an asset that we are trying to protect. In mobile device security, a threat is a possible danger that might exploit a vulnerability to compromise and cause potential harm to a device. A threat can be defined by the motives; it can be any of the following ones: Intentional: An individual or a group with an aim to break an application and steal information Accidental: The malfunctioning of a device or an application may lead to a potential disclosure of sensitive information Others: Capabilities, circumstantial, and so on Threat agents A threat agent is used to indicate an individual or a group that can manifest a threat. Threat agents will be able to perform the following actions: Access Misuse Disclose Modify Deny access Vulnerability The security weakness within a system that might allow attackers to exploit it and break the security of the device is called a vulnerability. For example, if a mobile device is stolen and it does not have the PIN or pass code enabled, the phone is vulnerable to data theft. Risk The intersection between asset (A), threat (T), and vulnerability (V) is a risk. However, a risk can be included along with the probability (P) of the threat occurrences to provide more value to the business. Risk = A x T x V x P These terms will help us understand the real risk to a given asset. Business will be benefited only if these risks are accurately assessed. Understanding threat, vulnerability, and risk is the first step in threat modeling. For a given application, no vulnerabilities or a vulnerability with no threats is considered to be a low risk. Summary In this article, we saw that mobile devices are susceptible to attacks through various threats, which exist due to the lack of sufficient security measures that can be implemented at various stages of the development of a mobile application. It is necessary to understand how these threats are manifested and learn how to test and mitigate them effectively. Proper knowledge of the underlying architecture and the tools available for the testing of mobile applications will help developers and security testers alike in order to protect end users from attackers who may be attempting to leverage these vulnerabilities.
Read more
  • 0
  • 0
  • 8926

article-image-oracle-e-business-suite-adjusting-items-inventory-and-classifying-items
Packt
19 Aug 2011
4 min read
Save for later

Oracle E-Business Suite: Adjusting Items in Inventory and Classifying Items

Packt
19 Aug 2011
4 min read
Oracle E-Business Suite 12 Financials Cookbook Adjusting Items in Inventory Item quantities can be adjusted in Inventory. In this recipe, we will use miscellaneous transactions to adjust Items. Let's try to enter transactions on some of the controls we have set up. We will try and enter a Miscellaneous Transaction for five paper widgets into Inventory. How to do it... Navigate to Transactions | Miscellaneous Transactions. In the Type field, select Miscellaneous receipt from the list of values. Click on the Transaction Lines button to enter the receipt. Search for the PRD20001 Item in the Item field. Select the Subinventory list of values and the field should be automatically populated with ACME-FG. In the Locator field, enter A1.1.2. The system should display an Error message to indicate that an invalid locator has been entered. Click on OK and enter A1.1.1–the system should accept this value. Enter a value of 5 in the Quantity field. In the account field, enter 01-000-1410-0000-000. This is the account that will be charged for the Inventory transaction. Select the Lot / Serial button. Enter the Lot number–LN10001. The expiration date is generated based on the setting in the Item definition. Enter the quantity of 5. Click on the Serial button. Enter SN10001 in the Start Serial Number field and press Tab on the keyboard. The SN10005 should be automatically populated in the End Serial Number field. Click on the Done button. Click on the Done button again in the next screen. Save the record. There's more... Let's search for the Items in Inventory. Searching for Items We will use the material workbench to search for the Items: Navigate to On-hand | Availability | On-hand Quantity. Enter PRD20001 in the Item / Revision field. Click on the Find button. Expand the Organizations tree to show LN10001. Review the Item details. Close the form. Classifying Items Items are grouped into logical classifications through categories. Categories can be further grouped into category sets. A default category set can be assigned to a functional area. When an Item is created, it is automatically added to the default category set. The groupings are mainly used for reporting purposes. Let's look at how to classify Items using categories. How to do it... Let's list the steps required to create category codes: Navigate to Setup | Items | Categories | Category Codes. Click on the New button to enter the Category. In the Structure Name, select Item Categories. In the Category field, enter BOOKS.MISC. In the Description field, enter Other Books. Save the record. Let's now create the Category Set, add the Category Codes to a new set called ACME Books, and assign it to the PRD20001 Item: Navigate to Setup | Items | Categories | Category Sets. Click on the New button to enter the category set. In the Name field, enter ACME Books. Enter a description, for example, ACME Paper Books. Select Item Categories for Flex Structure. Select Controlled At as the Org level. Enter BOOKS.MISC as the Default Category. Select the checkbox Allow Multiple Item Category Assignments. Add the following Category Codes to the list: BOOKS.MISC BOOKS.NEW BOOKS.USED Save the record. Let's now assign the categories to the PRD20001 Item: Navigate to Items | Organization Items. From the Menu, select Tools and then Categories. Select ACME Books in the Category Set. Enter BOOKS.NEW in the Category field. Save the record. How it works... The structure of the Item category is defined in the Item Flexfield structure and the values are held in the individual Value Sets. The combination of the individual values forms the category code. For example, the structure we used previously is made of two segments, defined in the Flexfield structure. The segments are Family and Class. BOOKS is a value in Family and MISC, NEW, and USED are individual values in Class.
Read more
  • 0
  • 0
  • 8920
article-image-code-style-django
Packt
17 Jun 2015
16 min read
Save for later

Code Style in Django

Packt
17 Jun 2015
16 min read
In this article written by Sanjeev Jaiswal and Ratan Kumar, authors of the book Learning Django Web Development, this article will cover all the basic topics which you would require to follow, such as coding practices for better Django web development, which IDE to use, version control, and so on. We will learn the following topics in this article: Django coding style Using IDE for Django web development Django project structure This article is based on the important fact that code is read much more often than it is written. Thus, before you actually start building your projects, we suggest that you familiarize yourself with all the standard practices adopted by the Django community for web development. Django coding style Most of Django's important practices are based on Python. Though chances are you already know them, we will still take a break and write all the documented practices so that you know these concepts even before you begin. To mainstream standard practices, Python enhancement proposals are made, and one such widely adopted standard practice for development is PEP8, the style guide for Python code–the best way to style the Python code authored by Guido van Rossum. The documentation says, "PEP8 deals with semantics and conventions associated with Python docstrings." For further reading, please visit http://legacy.python.org/dev/peps/pep-0008/. Understanding indentation in Python When you are writing Python code, indentation plays a very important role. It acts as a block like in other languages, such as C or Perl. But it's always a matter of discussion amongst programmers whether we should use tabs or spaces, and, if space, how many–two or four or eight. Using four spaces for indentation is better than eight, and if there are a few more nested blocks, using eight spaces for each indentation may take up more characters than can be shown in single line. But, again, this is the programmer's choice. The following is what incorrect indentation practices lead to: >>> def a(): ...   print "foo" ...     print "bar" IndentationError: unexpected indent So, which one we should use: tabs or spaces? Choose any one of them, but never mix up tabs and spaces in the same project or else it will be a nightmare for maintenance. The most popular way of indention in Python is with spaces; tabs come in second. If any code you have encountered has a mixture of tabs and spaces, you should convert it to using spaces exclusively. Doing indentation right – do we need four spaces per indentation level? There has been a lot of confusion about it, as of course, Python's syntax is all about indentation. Let's be honest: in most cases, it is. So, what is highly recommended is to use four spaces per indentation level, and if you have been following the two-space method, stop using it. There is nothing wrong with it, but when you deal with multiple third party libraries, you might end up having a spaghetti of different versions, which will ultimately become hard to debug. Now for indentation. When your code is in a continuation line, you should wrap it vertically aligned, or you can go in for a hanging indent. When you are using a hanging indent, the first line should not contain any argument and further indentation should be used to clearly distinguish it as a continuation line. A hanging indent (also known as a negative indent) is a style of indentation in which all lines are indented except for the first line of the paragraph. The preceding paragraph is the example of hanging indent. The following example illustrates how you should use a proper indentation method while writing the code: bar = some_function_name(var_first, var_second,                                            var_third, var_fourth) # Here indentation of arguments makes them grouped, and stand clear from others. def some_function_name(        var_first, var_second, var_third,        var_fourth):    print(var_first) # This example shows the hanging intent. We do not encourage the following coding style, and it will not work in Python anyway: # When vertical alignment is not used, Arguments on the first line are forbidden foo = some_function_name(var_first, var_second,    var_third, var_fourth) # Further indentation is required as indentation is not distinguishable between arguments and source code. def some_function_name(    var_first, var_second, var_third,    var_fourth):    print(var_first) Although extra indentation is not required, if you want to use extra indentation to ensure that the code will work, you can use the following coding style: # Extra indentation is not necessary. if (this    and that):    do_something() Ideally, you should limit each line to a maximum of 79 characters. It allows for a + or – character used for viewing difference using version control. It is even better to limit lines to 79 characters for uniformity across editors. You can use the rest of the space for other purposes. The importance of blank lines The importance of two blank lines and single blank lines are as follows: Two blank lines: A double blank lines can be used to separate top-level functions and the class definition, which enhances code readability. Single blank lines: A single blank line can be used in the use cases–for example, each function inside a class can be separated by a single line, and related functions can be grouped together with a single line. You can also separate the logical section of source code with a single line. Importing a package Importing a package is a direct implication of code reusability. Therefore, always place imports at the top of your source file, just after any module comments and document strings, and before the module's global and constants as variables. Each import should usually be on separate lines. The best way to import packages is as follows: import os import sys It is not advisable to import more than one package in the same line, for example: import sys, os You may import packages in the following fashion, although it is optional: from django.http import Http404, HttpResponse If your import gets longer, you can use the following method to declare them: from django.http import ( Http404, HttpResponse, HttpResponsePermanentRedirect ) Grouping imported packages Package imports can be grouped in the following ways: Standard library imports: Such as sys, os, subprocess, and so on. import reimport simplejson Related third party imports: These are usually downloaded from the Python cheese shop, that is, PyPy (using pip install). Here is an example: from decimal import * Local application / library-specific imports: This included the local modules of your projects, such as models, views, and so on. from models import ModelFoofrom models import ModelBar Naming conventions in Python/Django Every programming language and framework has its own naming convention. The naming convention in Python/Django is more or less the same, but it is worth mentioning it here. You will need to follow this while creating a variable name or global variable name and when naming a class, package, modules, and so on. This is the common naming convention that we should follow: Name the variables properly: Never use single characters, for example, 'x' or 'X' as variable names. It might be okay for your normal Python scripts, but when you are building a web application, you must name the variable properly as it determines the readability of the whole project. Naming of packages and modules: Lowercase and short names are recommended for modules. Underscores can be used if their use would improve readability. Python packages should also have short, all-lowercase names, although the use of underscores is discouraged. Since module names are mapped to file names (models.py, urls.py, and so on), it is important that module names be chosen to be fairly short as some file systems are case insensitive and truncate long names. Naming a class: Class names should follow the CamelCase naming convention, and classes for internal use can have a leading underscore in their name. Global variable names: First of all, you should avoid using global variables, but if you need to use them, prevention of global variables from getting exported can be done via __all__, or by defining them with a prefixed underscore (the old, conventional way). Function names and method argument: Names of functions should be in lowercase and separated by an underscore and self as the first argument to instantiate methods. For classes or methods, use CLS or the objects for initialization. Method names and instance variables: Use the function naming rules—lowercase with words separated by underscores as necessary to improve readability. Use one leading underscore only for non-public methods and instance variables. Using IDE for faster development There are many options on the market when it comes to source code editors. Some people prefer full-fledged IDEs, whereas others like simple text editors. The choice is totally yours; pick up whatever feels more comfortable. If you already use a certain program to work with Python source files, I suggest that you stick to it as it will work just fine with Django. Otherwise, I can make a couple of recommendations, such as these: SublimeText: This editor is lightweight and very powerful. It is available for all major platforms, supports syntax highlighting and code completion, and works well with Python. The editor is open source and you can find it at http://www.sublimetext.com/ PyCharm: This, I would say, is most intelligent code editor of all and has advanced features, such as code refactoring and code analysis, which makes development cleaner. Features for Django include template debugging (which is a winner) and also quick documentation, so this look-up is a must for beginners. The community edition is free and you can sample a 30-day trial version before buying the professional edition. Setting up your project with the Sublime text editor Most of the examples that we will show you in this book will be written using Sublime text editor. In this section, we will show how to install and set up the Django project. Download and installation: You can download Sublime from the download tab of the site www.sublimetext.com. Click on the downloaded file option to install. Setting up for Django: Sublime has a very extensive plug-in ecosystem, which means that once you have downloaded the editor, you can install plug-ins for adding more features to it. After successful installation, it will look like this: Most important of all is Package Control, which is the manager for installing additional plugins directly from within Sublime. This will be your only manual installation of the package. It will take care of the rest of the package installation ahead. Some of the recommendations for Python development using Sublime are as follows: Sublime Linter: This gives instant feedback about the Python code as you write it. It also has PEP8 support; this plugin will highlight in real time the things we discussed about better coding in the previous section so that you can fix them.   Sublime CodeIntel: This is maintained by the developer of SublimeLint. Sublime CodeIntel have some of advanced functionalities, such as directly go-to definition, intelligent code completion, and import suggestions.   You can also explore other plugins for Sublime to increase your productivity. Setting up the pycharm IDE You can use any of your favorite IDEs for Django project development. We will use pycharm IDE for this book. This IDE is recommended as it will help you at the time of debugging, using breakpoints that will save you a lot of time figuring out what actually went wrong. Here is how to install and set up pycharm IDE for Django: Download and installation: You can check the features and download the pycharm IDE from the following link: http://www.jetbrains.com/pycharm/ Setting up for Django: Setting up pycharm for Django is very easy. You just have to import the project folder and give the manage.py path, as shown in the following figure: The Django project structure The Django project structure has been changed in the 1.6 release version. Django (django-admin.py) also has a startapp command to create an application, so it is high time to tell you the difference between an application and a project in Django. A project is a complete website or application, whereas an application is a small, self-contained Django application. An application is based on the principle that it should do one thing and do it right. To ease out the pain of building a Django project right from scratch, Django gives you an advantage by auto-generating the basic project structure files from which any project can be taken forward for its development and feature addition. Thus, to conclude, we can say that a project is a collection of applications, and an application can be written as a separate entity and can be easily exported to other applications for reusability. To create your first Django project, open a terminal (or Command Prompt for Windows users), type the following command, and hit Enter: $ django-admin.py startproject django_mytweets This command will make a folder named django_mytweets in the current directory and create the initial directory structure inside it. Let's see what kind of files are created. The new structure is as follows: django_mytweets/// django_mytweets/ manage.py This is the content of django_mytweets/: django_mytweets/ __init__.py settings.py urls.py wsgi.py Here is a quick explanation of what these files are: django_mytweets (the outer folder): This folder is the project folder. Contrary to the earlier project structure in which the whole project was kept in a single folder, the new Django project structure somehow hints that every project is an application inside Django. This means that you can import other third party applications on the same level as the Django project. This folder also contains the manage.py file, which include all the project management settings. manage.py: This is utility script is used to manage our project. You can think of it as your project's version of django-admin.py. Actually, both django-admin.py and manage.py share the same backend code. Further clarification about the settings will be provided when are going to tweak the changes. Let's have a look at the manage.py file: #!/usr/bin/env python import os import sys if __name__ == "__main__":    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "django_mytweets.settings")    from django.core.management import   execute_from_command_line    execute_from_command_line(sys.argv) The source code of the manage.py file will be self-explanatory once you read the following code explanation. #!/usr/bin/env python The first line is just the declaration that the following file is a Python file, followed by the import section in which os and sys modules are imported. These modules mainly contain system-related operations. import os import sys The next piece of code checks whether the file is executed by the main function, which is the first function to be executed, and then loads the Django setting module to the current path. As you are already running a virtual environment, this will set the path for all the modules to the path of the current running virtual environment. if __name__ == "__main__":    os.environ.setdefault("DJANGO_SETTINGS_MODULE",     "django_mytweets.settings") django_mytweets/ ( Inner folder) __init__.py Django projects are Python packages, and this file is required to tell Python that this folder is to be treated as a package. A package in Python's terminology is a collection of modules, and they are used to group similar files together and prevent naming conflicts. settings.py: This is the main configuration file for your Django project. In it, you can specify a variety of options, including database settings, site language(s), what Django features need to be enabled, and so on. By default, the database is configured to use SQLite Database, which is advisable to use for testing purposes. Here, we will only see how to enter the database in the settings file; it also contains the basic setting configuration, and with slight modification in the manage.py file, it can be moved to another folder, such as config or conf. To make every other third-party application a part of the project, we need to register it in the settings.py file. INSTALLED_APPS is a variable that contains all the entries about the installed application. As the project grows, it becomes difficult to manage; therefore, there are three logical partitions for the INSTALLED_APPS variable, as follows: DEFAULT_APPS: This parameter contains the default Django installed applications (such as the admin) THIRD_PARTY_APPS: This parameter contains other application like SocialAuth used for social authentication LOCAL_APPS: This parameter contains the applications that are created by you url.py: This is another configuration file. You can think of it as a mapping between URLs and the Django view functions that handle them. This file is one of Django's more powerful features. When we start writing code for our application, we will create new files inside the project's folder. So, the folder also serves as a container for our code. Now that you have a general idea of the structure of a Django project, let's configure our database system. Summary We prepared our development environment in this article, created our first project, set up the database, and learned how to launch the Django development server. We learned the best way to write code for our Django project and saw the default Django project structure. Resources for Article: Further resources on this subject: Tinkering Around in Django JavaScript Integration [article] Adding a developer with Django forms [article] So, what is Django? [article]
Read more
  • 0
  • 0
  • 8916

article-image-introducing-gamemaker
Packt
24 Mar 2015
5 min read
Save for later

Introducing GameMaker

Packt
24 Mar 2015
5 min read
In this article by Nathan Auckett, author of the book GameMaker Essentials, you will learn what GameMaker is all about, who made it, what it is used for, and more. You will then also be learning how to install GameMaker on your computer that is ready for use. (For more resources related to this topic, see here.) In this article, we will cover the following topics: Understanding GameMaker Installing GameMaker: Studio What is this article about? Understanding GameMaker Before getting started with GameMaker, it is best to know exactly what it is and what it's designed to do. GameMaker is a 2D game creation software by YoYo Games. It was designed to allow anyone to easily develop games without having to learn complex programming languages such as C++ through the use of its drag and drop functionality. The drag and drop functionality allows the user to create games by visually organizing icons on screen, which represent actions and statements that will occur during the game. GameMaker also has a built-in programming language called GameMaker Language, or GML for short. GML allows users to type out code to be run during their game. All drag and drop actions are actually made up of this GML code. GameMaker is primarily designed for 2D games, and most of its features and functions are designed for 2D game creation. However, GameMaker does have the ability to create 3D games and has a number of functions dedicated to this. GameMaker: Studio There are a number of different versions of GameMaker available, most of which are unsupported because they are outdated; however, support can still be found in the GameMaker Community Forums. GameMaker: Studio is the first version of GameMaker after GameMaker HTML5 to allow users to create games and export them for use on multiple devices and operating systems including PC, Mac, Linux, and Android, on both mobile and desktop versions. GameMaker: Studio is designed to allow one code base (GML) to run on any device with minimal changes to the base code. Users are able to export their games to run on any supported device or system such as HTML5 without changing any code to make things work. GameMaker: Studio was also the first version available for download and use through the Steam marketplace. YoYo Games took advantage of the Steam workshop and allowed Steam-based users to post and share their creations through the service. GameMaker: Studio is sold in a number of different versions, which include several enhanced features and capabilities as the price gets higher. The standard version is free to download and use. However, it lacks some advanced features included in higher versions and only allows for exporting to the Windows desktop. The professional version is the second cheapest from the standard version. It includes all features, but only has the Windows desktop and Windows app exports. Other exports can be purchased at an extra cost ranging from $99.99 to $300. The master version is the most expensive of all the options. It comes with every feature and every export, including all future export modules in version 1.x. If you already own exports in the professional version, you can get the prices of those exports taken off the price of the master version. Installing GameMaker: Studio Installing GameMaker is performed much like any other program. In this case, we will be installing GameMaker: Studio as this is the most up-to-date version at this point. You can find the download at the YoYo Games website, https://www.yoyogames.com/. From the site, you can pick the free version or purchase one of the others. All the installations are basically the same. Once the installer is downloaded, we are ready to install GameMaker: Studio. This is just like installing any other program. Just run the file, and then follow the on-screen instructions to accept the license agreement, choose an install location, and install the software. On the first run, you may see a progress bar appear at the top left of your screen. This is just GameMaker running its first time setup. It will also do this during the update process as YoYo Games releases new features. Once it is done, you should see a welcome screen and will be prompted to enter your registration code. The key should be e-mailed to you when you make an account during the purchase/download process. Enter this key and your copy of GameMaker: Studio should be registered. You may be prompted to restart GameMaker at this time. Close GameMaker and re-open it and you should see the welcome screen and be able to choose from a number of options on it: What is this article about? We now have GameMaker: Studio installed and are ready to get started with it. In this article, we will be covering the essential things to know about GameMaker: Studio. This includes everything from drag and drop actions to programming in GameMaker using GameMaker Language (GML). You will learn about how things in GameMaker are structured, and how to organize resources to keep things as clean as possible. Summary In this article, we looked into what GameMaker actually is and learned that there are different versions available. We also looked at the different types of GameMaker: Studio available for download and purchase. We then learned how to install GameMaker: Studio, which is the final step in getting ready to learn the essential skills and starting to make our very own games. Resources for Article: Further resources on this subject: Getting Started – An Introduction to GML [Article] Animating a Game Character [Article] Why should I make cross-platform games? [Article]
Read more
  • 0
  • 0
  • 8915
Modal Close icon
Modal Close icon