Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Save more on your purchases! discount-offer-chevron-icon
Savings automatically calculated. No voucher code required.
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Newsletter Hub
Free Learning
Arrow right icon
timer SALE ENDS IN
0 Days
:
00 Hours
:
00 Minutes
:
00 Seconds

How-To Tutorials

7014 Articles
article-image-swifts-core-libraries
Packt
13 Oct 2016
26 min read
Save for later

Swift's Core Libraries

Packt
13 Oct 2016
26 min read
In this article, Jon Hoffman, the author of the book Mastering Swift 3, talks about how he was really excited when Apple announced that they were going to release a version of Swift for Linux, so he could use Swift for his Linux and embedded development as well as his Mac OS and iOS development. When Apple first released Swift 2.2 for Linux, he was very excited but was also a little disappointed because he could not read/write files, access network services, or use Grand Central Dispatch (GCD or libdispatch) on Linux like he could on Apple's platforms. With the release of Swift 3, Apple has corrected this with the release of Swift's core libraries. (For more resources related to this topic, see here.) In this article, you will learn about the following topics: What are the Swift core libraries How to use Apple's URL loading system How to use the Formatter classes How to use the File Manager class The Swift Core Libraries are written to provide a rich set of APIs that are consistent across the various platforms that Swift supports. By using these libraries, developers will be able to write code that will be portable to all platforms that Swift supports. These libraries provide a higher level of functionality as compared to the Swift standard library. The core libraries provide functionality in a number of areas, such as: Networking Unit testing Scheduling and execution of work (libdispatch) Property lists, JSON parsing, and XML parsing Support for dates, times, and calendar calculations Abstraction of OS-specific behavior Interaction with the file system User preferences We are unable to cover all of the Core Libraries in this single article; however, we will look at some of the more useful ones. We will start off by looking at Apple's URL loading system that is used for network development. Apple's URL loading system Apple's URL loading system is a framework of classes available to interact with URLs. We can use these classes to communicate with services that use standard Internet protocols. The classes that we will be using in this section to connect to and retrieve information from REST services are as follows: URLSession: This is the main session object. URLSessionConfiguration: This is used to configure the behavior of the URLSession object. URLSessionTask: This is a base class to handle the data being retrieved from the URL. Apple provides three concrete subclasses of the URLSessionTask class. URL: This is an object that represents the URL to connect to. URLRequest: This class contains information about the request that we are making and is used by the URLSessionTask service to make the request. HTTPURLResponse: This class contains the response to our request. Now, let's look at each of these classes a little more in depth so that we have a basic understanding of what each does. URLSession A URLSession object provides an API for interacting with various protocols such as HTTP and HTTPS. The session object, which is an instance of the URLSession, manages this interaction. These session objects are highly configurable, which allows us to control how our requests are made and how we handle the data that is returned. Like most networking APIs, URLSession is asynchronous. This means that we have to provide a way to return the response from the service back to the code that needs it. The most popular way to return the results from a session is to pass a completion handler block (closure) to the session. This completion handler is then called when the service successfully responds or we receive an error. All of the examples in this article use completion handlers to process the data that is returned from the services. URLSessionConfiguration The URLSessionConfiguration class defines the behavior and policies to use when using the URLSession object to connect to a URL. When using the URLSession object, we usually create a URLSessionConfiguration instance first, because an instance of this class is required when we create an instance of the URLSession class. The URLSessionConfiguration class defines three session types: Default session configuration: Manages the upload and download tasks with default configurations Ephemeral session configuration: This configuration behaves similar to the default session configuration, except that it does not cache anything to disk Background session configuration: This session allows for uploads and downloads to be performed, even when the app is running in the background It is important to note that we should make sure that we configure the URLSessionConfiguration object appropriately before we use it to create an instance of the URLSession class. When the session object is created, it creates a copy of the configuration object that we provide. Any changes made to the configuration object once the session object is created are ignored by the session. If we need to make changes to the configuration, we must create another instance of the URLSession class. URLSessionTask The URLSession service uses an instance of the URLSessionTask class to make the call to the service that we are connecting to. The URLSessionTask class is a base class, and Apple has provided three concrete subclasses that we can use: URLSessionDataTask: This returns the response, in memory, directly to the application as one or more Data objects. This is the task that we generally use most often. URLSessionDownloadTask: This writes the response directly to a temporary file. URLSessionUploadTask: This is used for making requests that require a request body, such as a POST or PUT request. It is important to note that a task will not send the request to the service until we call the resume() method. URL The URL object represents the URL that we are going to connect to. The URL class is not limited to URLs that represent remote servers, but it can also be used to represent a local file on disk. In this article, we will be using the URL class exclusively to represent the URL of the remote service that we are connecting to. URLRequest We use the URLRequest class to encapsulate our URL and the request properties. It is important to understand that the URLRequest class is used to encapsulate the necessary information to make our request, but it does not make the actual request. To make the request, we use instances of the URLSession and URLSessionTask classes. HTTPURLResponse The HTTPURLResponse class is a subclass of the URLResponse class that encapsulates the metadata associated with the response to a URL request. The HTTPURLResponse class provides methods for accessing specific information associated with an HTTP response. Specifically, this class allows us to access the HTTP header fields and the response status codes. We briefly covered a number of classes in this section and it may not be clear how they all actually fit together; however, once you see the examples a little further in this article, it will become much clearer. Before we go into our examples, let's take a quick look at the type of service that we will be connecting to. REST web services REST has become one of the most important technologies for stateless communications between devices. Due to the lightweight and stateless nature of the REST-based services, its importance is likely to continue to grow as more devices are connected to the Internet. REST is an architecture style for designing networked applications. The idea behind REST is that instead of using complex mechanisms, such as SOAP or CORBA to communicate between devices, we use simple HTTP requests for the communication. While, in theory, REST is not dependent on the Internet protocols, it is almost always implemented using them. Therefore, when we are accessing REST services, we are almost always interacting with web servers in the same way that our web browsers interact with these servers. REST web services use the HTTP POST, GET, PUT, or DELETE methods. If we think about a standard Create, Read, Update, Delete (CRUD) application, we would use a POST request to create or update data, a GET request to read data, and a DELETE request to delete data. When we type a URL into our browser's address bar and hit Enter, we are generally making a GET request to the server and asking it to send us the web page associated with that URL. When we fill out a web form and click the submit button, we are generally making a POST request to the server. We then include the parameters from the web form in the body of our POST request. Now, let's look at how to make an HTTP GET request using Apple's networking API. Making an HTTP GET request In this example, we will make a GET request to Apple's iTunes search API to get a list of items related to the search term "Jimmy Buffett". Since we are retrieving data from the service, by REST standards, we should use a GET request to retrieve the data. While the REST standard is to use GET requests to retrieve data from a service, there is nothing stopping a developer of a web service from using a GET request to create or update a data object. It is not recommended to use a GET request in this manner, but just be aware that there are services out there that do not adhere to the REST standards. The following code makes a request to Apple's iTunes search API and then prints the results to the console: public typealias dataFromURLCompletionClosure = (URLResponse?, Data?) -> Void public func sendGetRequest ( _ handler: @escaping dataFromURLCompletionClosure) { let sessionConfiguration = URLSessionConfiguration.default; let urlString = "https://itunes.apple.com/search?term=jimmy+buffett" if let encodeString = urlString.addingPercentEncoding( withAllowedCharacters: CharacterSet.urlQueryAllowed), let url = URL(string: encodeString) { var request = URLRequest(url:url) request.httpMethod = "GET" let urlSession = URLSession( configuration:sessionConfiguration, delegate: nil, delegateQueue: nil) let sessionTask = urlSession.dataTask(with: request) { (data, response, error) in handler(response, data) } sessionTask.resume() } } We start off by creating a type alias named DataFromURLCompletionClosure. The DataFromURLCompletionClosure type will be used for both the GET and POST examples of this article. If you are not familiar with using a typealias object to define a closure type. We then create a function named sendGetRequest(), which will be used to make the GET request to Apple's iTunes API. This function accepts one argument named handler, which is a closure that conforms to the DataFromURLCompletionClosure type. The handler closure will be used to return the results from the request. Stating with Swift 3, the default for closure arguments to functions is not escaping, which means that, by default, the closures argument cannot escape the function body. A closure is considered to escape a function when that closure, which is passed as an argument to the function, is called after the function returns. Since the closure will be called after the function returns, we use the @escaping attribute before the parameter type to indicate it is allowed to escape. Within our sendGetRequest() method, we begin by creating an instance of the URLSessionConfiguration class using the default settings. If we need to, we can modify the session configuration properties after we create it, but in this example, the default configuration is what we want. After we create our session configuration, we create the URL string. This is the URL of the service we are connecting to. With a GET request, we put our parameters in the URL itself. In this specific example, https://itunes.apple.com/search is the URL of the web service. We then follow the web service URL with a question mark (?), which indicates that the rest of the URL string consists of parameters for the web service. The parameters take the form of key/value pairs, which means that each parameter has a key and a value. The key and value of a parameter, in a URL, are separated by an equals sign (=). In our example, the key is term and the value is jimmy+buffett. Next, we run the URL string that we just created through the addingPercentEncoding() method to make sure our URL string is encoded properly. We use the CharacterSet.urlQueryAllowed character set with this method to ensure we have a valid URL string. Next, we use the URL string that we just built to create a URL instance named url. Since we are making a GET request, this URL instance will represent both the location of the web service and the parameters that we are sending to it. We create an instance of the URLRequest class using the URL instance that we just created. In this example, we set the HTTPMethod property; however, we can also set other properties, such as the timeout interval or add items to our HTTP header. Now, we use the sessionConfiguration constant that we created at the beginning of the sendGetRequest() function to create an instance of the URLSession class. The URLSession class provides the API that we will use to connect to Apple's iTunes search API. In this example, we use the dataTask(with:) method of the URLSession instance to return an instance of the URLSessionDataTask type named sessionTask. The sessionTask instance is what makes the request to the iTunes search API. When we receive the response from the service, we use the handler callback to return both the URLResponse object and the Data object. The URLResponse contains information about the response, and the Data instance contains the body of the response. Finally, we call the resume() method of the URLSessionDataTask instance to make the request to the web service. Remember, as we mentioned earlier, a URLSessionTask instance will not send the request to the service until we call the resume() method. Now, let's look at how we would call the sendGetRequest() function. The first thing we need to do is to create a closure that will be passed to the sendGetRequest() function and called when the response from the web service is received. In this example, we will simply print the response to the console. Since the response is in the JSON format, we could use the JSONSerialization class, which we will see later in this article, to parse the response; however, since this section is on networking, we will simply print the response to the console. Here is the code: let printResultsClosure: dataFromURLCompletionClosure = { if let data = $1 { let sString = let sString = String(data: data, encoding: String.Encoding(rawValue: String.Encoding.utf8.rawValue)) print(sString) } else { print("Data is nil") } } We define this closure, named printResultsClosure, to be an instance of the DataFromURLCompletionClosure type. Within the closure, we unwrap the first parameter and set the value to a constant named data. If the first parameter is not nil, we convert the data constant to an instance of the String class, which is then printed to the console. Now, let's call the sendGetRequest() method with the following code: let aConnect = HttpConnect() aConnect.sendGetRequest(printResultsClosure) This code creates an instance of the HttpConnect class and then calls the sendGetRequest() method, passing the printResultsClosure closure as the only parameter. If we run this code while we are connected to the Internet, we will receive a JSON response that contains a list of items related to Jimmy Buffett on iTunes. Now that we have seen how to make a simple HTTP GET request, let's look at how we would make an HTTP POST request to a web service. Making an HTTP POST request Since Apple's iTunes, APIs use GET requests to retrieve data. In this section, we will use the free http://httpbin.org service to show you how to make a POST request. The POST service that http://httpbin.org provides can be found at http://httpbin.org/post. This service will echo back the parameters that it receives so that we can verify that our request was made properly. When we make a POST request, we generally have some data that we want to send or post to the server. This data takes the form of key/value pairs. These pairs are separated by an ampersand (&) symbol, and each key is separated from its value by an equals sign (=). As an example, let's say that we want to submit the following data to our service: firstname: Jon lastname: Hoffman age: 47 years The body of the POST request would take the following format: firstname=Jon&lastname=Hoffman&age=47 Once we have the data in the proper format, we will then use the dataUsingEncoding() method, as we did with the GET request to properly encode the POST data. Since the data going to the server is in the key/value format, the most appropriate way to store this data, prior to sending it to the service, is with a Dictionary object. With this in mind, we will need to create a method that will take a Dictionary object and return a string object that can be used for the POST request. The following code will do that: private func dictionaryToQueryString(_ dict: [String : String]) -> String { var parts = [String]() for (key, value) in dict { let part : String = key + "=" + value parts.append(part); } return parts.joined(separator: "&") } This function loops through each key/value pair of the Dictionary object and creates a String object that contains the key and the value separated by the equals sign (=). We then use the joinWithSeperator() function to join each item in the array, separated by the specified sting. In our case, we want to separate each string with the ampersand symbol (&). We then return this newly created string to the code that called it. Now, let's create the sendPostRequest() function that will send the POST request to the http://httpbin.org post service. We will see a lot of similarities between this sendPostRequest() function and the sendGetRequest() function, which we showed you in the Making an HTTP GET request section. Let's take a look at the following code: public func sendPostRequest(_ handler: @escaping dataFromURLCompletionClosure) { let sessionConfiguration = URLSessionConfiguration.default let urlString = "http://httpbin.org/post" if let encodeString = urlString.addingPercentEncoding( withAllowedCharacters: CharacterSet.urlQueryAllowed), let url = URL(string: encodeString) { var request = URLRequest(url:url) request.httpMethod = "POST" let params = dictionaryToQueryString(["One":"1 and 1", "Two":"2 and 2"]) request.httpBody = params.data( using: String.Encoding.utf8, allowLossyConversion: true) let urlSession = URLSession( configuration:sessionConfiguration, delegate: nil, delegateQueue: nil) let sessionTask = urlSession.dataTask(with: request) { (data, response, error) in handler(response, data) } sessionTask.resume() } } This code is very similar to the sendGetRequest() function that we saw earlier in this section. The two main differences are the httpMethod of the URLRequest is set to POST rather than GET and how we set the parameters. In this function, we set the httpBody property of the URLRequest instance to the parameters we are submitting. Now that we have seen how to use the URL Loading System, let's look at how we can use Formatters. Formatter Formatter is an abstract class that declares an interface for an object that creates, converts, or validates a human readable form of some data. The types that subclass the Formatter class are generally used when we want to take a particular object, like an instance of the Date class, and present the value in a form that the user of our application can understand. Apple has provided several concrete implementations of the Formatter class, and in this section we will look at two of them. It is important to remember that the formatters that Apple provides will provide the proper format for the default locale of the device the application is running on. We will start off by looking at the DateFormatter type. DateFormatter The DateFormatter class is a subclass of the Formatter abstract class that can be used to convert a Date object into a human readable string. It can also be used to convert a String representation of a date into a Date object. We will look at both of these use cases in this section. Let's begin by seeing how we could covert a Date object into a human readable string. The DateFormatter type has five predefined styles that we can use when we are converting a Date object to a human readable string. The following chart shows what the five styles look like for an en-US locale. DateFormatter Style Date Time .none No Format No Format .short 12/25/16 6:00 AM .medium Dec 25, 2016 6:00:00 AM .long December 25, 2016 6:00:00 AM EST .full Sunday December 25, 2016 6:00:00 AM Eastern Standard Time The following code shows how we would use the predefined DateFormatter styles: let now = Date() let formatter = DateFormatter() formatter.dateStyle = .short formatter.timeStyle = .medium let dateStr = formatter.string(from: now) We use the string(from:) method to convert the now date to a human readable string. In this example, for the en-US locale, the dateStr constant would contain text similar to "Aug 19, 2016, 6:40 PM". There are numerous times when the predefined styles do not meet our needs. For those times, we can define our own styles using a custom format string. This string is a series of characters that the DateFormatter type knows are stand-ins for the values we want to show. The DateFormatter instance will replace these stand-ins with the appropriate values. The following table shows some of the formatting values that we can use to format our Date objects: Stand-in Format Description Example output yy Two digit year 16, 14, 04 yyyy Four digit year 2016, 2014, 2004 MM Two digit month 06, 08, 12 MMM Three letter Month Jul, Aug, Dec MMMM Full month name July, August dd Two digit day 10, 11, 30 EEE Three letter Day Mon, Sat, Sun EEEE Full day Monday, Sunday a Period of day AM, PM hh Two digit hour 02, 03, 04 HH Two digit hour for 24 hour clock 11, 14, 16 mm Two digit minute 30, 35, 45 ss Two digit seconds 30, 35, 45 The following code shows how we would use these custom formatters: let now = Date() let formatter2 = DateFormatter() formatter2.dateFormat = "YYYY-MM-dd HH:mm:ss" let dateStr2 = formatter2.string(from: now) We use the string(from:) method to convert the now date to a human readable string. In this example, for the en-US locale, the dateStr2 constant would contain text similar to 2016-08-19 19:03:23. Now let's look at how we would take a formatted date string and convert it to a Date object. We will use the same format string that we used in our last example: formatter2.dateFormat = "YYYY-MM-dd HH:mm:ss" let dateStr3 = "2016-08-19 16:32:02" let date = formatter2.date(from: dateStr3) In this example, we took the human readable date string and converted it to a Date object using the date(from:) method. If the format of the human readable date string does not match the format specified in the DateFormatter instance, then the conversion will fail and return nil. Now let's take a look at the NumberFormatter type. NumberFormatter The NumberFormatter class is a subclass of the Formatter abstract class that can be used to convert a number into a human readable string with a specified format. This formatter is especially useful when we want to display a currency string since it will convert the number to the proper currency to the proper locale. Let's begin by looking at how we would convert a number into a currency string: let formatter1 = NumberFormatter() formatter1.numberStyle = .currency let num1 = formatter1.string(from: 23.99) In the previous code, we define our number style to be .currency, which tells our formatter that we want to convert our number to a currency string. We then use the string(from:) method to convert the number to a string. In this example, for the en-US locale, the num1 constant would contain the string "$23.99". Now let's see how we would round a number using the NumberFormatter type. The following code will round the number to two decimal points: let formatter2 = NumberFormatter() formatter2.numberStyle = .decimal formatter2.maximumFractionDigits = 2 let num2 = formatter2.string(from: 23.2357) In this example, we set the numberStyle property to the .decimal style and we also define the maximum number of decimal digits to be two using the maximumFractionDigits property. We use the string(from:) method to convert the number to a string. In this example, the num2 constant will contain the string "23.24". Now let's see how we can spell out a number using the NumberFormatter type. The following code will take a number and spell out the number: let formatter3 = NumberFormatter() formatter3.numberStyle = .spellOut let num3 = formatter3.string(from: 2015) In this example, we set the numberStyle property to the .spellout style. This style will spell out the number. For this example, the num3 constant will contain the string two thousand fifteen. Now let's look at how we can manipulate the file system using the FileManager class FileManager The file system is a complex topic and how we manipulate it within our applications is generally specific to the operating system that our application is running on. This can be an issue when we are trying to port code from one operating system to another. Apple has addressed this issue by putting the FileManager object in the core libraries. The FileManager object lets us examine and make changes to the file system in a uniform manner across all operating systems that Swift supports. The FileManager class provides us with a shared instance that we can use. This instance should be suitable for most of our file system related tasks. We can access this shared instance using the default property. When we use the FileManager object we can provide paths as either an instance of the URL or String types. In this section, all of our paths will be String types for consistency purposes. Let's start off by seeing how we could list the contents of a directory using the FileManager object. The following code shows how to do this: let fileManager = FileManager.default do { let path = "/Users/jonhoffman/" let dirContents = try fileManager.contentsOfDirectory(atPath: path) for item in dirContents { print(item); } } catch let error { print("Failed reading contents of directory: (error)") } We start off by getting the shared instance of the FileManager object using the default property. We will use this same shared instance for all of our examples in this section rather than redefining it for each example. Next, we define our path and use it with the contentsOfDirectory(atPath:) method. This method returns an array of String types that contains the names of the items in the path. We use a for loop to list these items. Next let's look at how we would create a directory using the file manager. The following code shows how to do this: do { let path = "/Users/jonhoffman/masteringswift/test/dir" try fileManager.createDirectory(atPath: path, withIntermediateDirectories: true) } catch let error { print("Failed creating directory, (error) ") } In this example, we use the createDirectory(atPath: withIntermediateDirectories:) method to create the directory. When we set the withIntermediateDirectories parameter to true, this method will create any parent directories that are missing. When this parameter is set to false, if any parent directories are missing the operation will fail. Now let's look at how we would copy an item from one location to another: do { let pathOrig = "/Users/jonhoffman/masteringswift/" let pathNew = "/Users/jonhoffman/masteringswift2/" try fileManager.copyItem(atPath: pathOrig, toPath: pathNew) } catch let error { print("Failed copying directory, (error) ") } In this example, we use the copyItem(atPath: toPath:) method to copy one item to another location. This method can be used to copy either directories or files. If it is used for directories, the entire path structure below the directory specified by the path is copied. The file manager also has a method that will let us move an item rather than copying it. Let's see how to do this: do { let pathOrig = "/Users/jonhoffman/masteringswift2/" let pathNew = "/Users/jonhoffman/masteringswift3/" try fileManager.moveItem(atPath: pathOrig, toPath: pathNew) } catch let error { print("Failed moving directory, (error) ") } To move an item, we use the moveItem(atPath: toPath:) method. Just like the copy example we just saw, the move method can be used for either files or directories. If the path specifies a directory, then the entire directory structure below that path will be moved. Now let's see how we can delete an item from the file system: do { let path = "/Users/jonhoffman/masteringswift/" try fileManager.removeItem(atPath: path) } catch let error { print("Failed Removing directory, (error) ") } In this example, we use the removeItem(atPath:) method to remove the item from the file system. A word of warning, once you delete something it is gone and there is no getting it back. Next let's look at how we can read permissions for items in our file system. For this we will need to create a file named test.file, which our path will point to: let path = "/Users/jonhoffman/masteringswift3/test.file" if fileManager.fileExists(atPath: path) { let isReadable = fileManager.isReadableFile(atPath: path) let isWriteable = fileManager.isWritableFile(atPath: path) let isExecutable = fileManager.isExecutableFile(atPath: path) let isDeleteable = fileManager.isDeletableFile(atPath: path) print("can read (isReadable)") print("can write (isWriteable)") print("can execute (isExecutable)") print("can delete (isDeleteable)") } In this example, we use four different methods to read the file system permissions for the file system item. These methods are: isReadableFile(atPath:): true if the file is readable isWritableFile(atPath:):true if the file is writable isExecutableFile(atPath:): true if the file is executable isDeletableFile(atPath:): true if the file is deletable If we wanted to read or write text files, we could use methods provided by the String type rather than the FileManager type. Even though we do not use the FileManager class, for this example, we wanted to show how to read and write text files. Let's see how we would write some text to a file: let filePath = "/Users/jonhoffman/Documents/test.file" let outString = "Write this text to the file" do { try outString.write(toFile: filePath, atomically: true, encoding: .utf8) } catch let error { print("Failed writing to path: (error)") } In this example, we start off by defining our path as a String type just as we did in the previous examples. We then create another instance of the String type that contains the text we want to put in our file. To write the text to the file we use the write(toFile: atomically: encoding:) method of the String instance. This method will create the file if needed and write the contents of the String instance to the file. It is just as easy to read a text file using the String type. The following example shows how to do this: let filePath = "/Users/jonhoffman/Documents/test.file" var inString = "" do { inString = try String(contentsOfFile: filePath) } catch let error { print("Failed reading from path: (error)") } print("Text Read: (inString)") In this example, we use the contentsOfFile: initiator to create an instance of the String type that contains the contents of the file specified by the file path. Now that we have seen how to use the FileManager type, let's look at how we would serialize and deserialize JSON documents using the JSONSerialization type. Summary In this article, we looked at some of the libraries that make up the Swift Core Libraries. While these libraries are only a small portion of the libraries that make up the Swift Core Libraries they are arguable some of the most useful libraries. To explore the other libraries that, you can refer to Apple's GitHub page: https://github.com/apple/swift-corelibs-foundation. Resources for Article: Further resources on this subject: Flappy Swift [article] Network Development with Swift [article] Your First Swift 2 Project [article]
Read more
  • 0
  • 0
  • 25872

article-image-solving-nlp-problem-keras-part-2
Sasank Chilamkurthy
13 Oct 2016
6 min read
Save for later

Solving an NLP Problem with Keras, Part 2

Sasank Chilamkurthy
13 Oct 2016
6 min read
In this two-part post series, we are solving a Natural Language Processing (NLP) problem with Keras. In Part 1, we covered the problem and the ATIS dataset we are using. We also went over the word embeddings (mapping words to a vector) along with Recurrent Neural Networks that solve complicated word tagging problems. We passed the word embedding sequence as input into the RNN and we then started coding that up. Now, it is time in this post to start loading the data. Loading Data Let's load the data using data.load.atisfull(). It will download the data the first time it is run. Words and labels are encoded as indexes to a vocabulary. This vocabulary is stored in w2idx and labels2idx. import numpy as np import data.load train_set, valid_set, dicts = data.load.atisfull() w2idx, labels2idx = dicts['words2idx'], dicts['labels2idx'] train_x, _, train_label = train_set val_x, _, val_label = valid_set # Create index to word/label dicts idx2w = {w2idx[k]:k for k in w2idx} idx2la = {labels2idx[k]:k for k in labels2idx} # For conlleval script words_train = [ list(map(lambda x: idx2w[x], w)) for w in train_x] labels_train = [ list(map(lambda x: idx2la[x], y)) for y in train_label] words_val = [ list(map(lambda x: idx2w[x], w)) for w in val_x] labels_val = [ list(map(lambda x: idx2la[x], y)) for y in val_label] n_classes = len(idx2la) n_vocab = len(idx2w) Let's print an example sentence and label: print("Example sentence : {}".format(words_train[0])) print("Encoded form: {}".format(train_x[0])) print() print("It's label : {}".format(labels_train[0])) print("Encoded form: {}".format(train_label[0])) Here is the output: Example sentence : ['i', 'want', 'to', 'fly', 'from', 'boston', 'at', 'DIGITDIGITDIGIT', 'am', 'and', 'arrive', 'in', 'denver', 'at', 'DIGITDIGITDIGITDIGIT', 'in', 'the', 'morning'] Encoded form: [232 542 502 196 208 77 62 10 35 40 58 234 137 62 11 234 481 321] It's label : ['O', 'O', 'O', 'O', 'O', 'B-fromloc.city_name', 'O', 'B-depart_time.time', 'I-depart_time.time', 'O', 'O', 'O', 'B-toloc.city_name', 'O', 'B-arrive_time.time', 'O', 'O', 'B-arrive_time.period_of_day'] Encoded form: [126 126 126 126 126 48 126 35 99 126 126 126 78 126 14 126 126 12] Keras model Next, we define the Keras model. Keras has an inbuilt Embedding layer for word embeddings. It expects integer indices. SimpleRNN is the recurrent neural network layer described in Part 1. We will have to use TimeDistributed to pass the output of RNN Ot At each time step: t To a fully connected layer. Otherwise, the output at the final time step will be passed on to the next layer. from keras.models import Sequential from keras.layers.embeddings import Embedding from keras.layers.recurrent import SimpleRNN from keras.layers.core import Dense, Dropout from keras.layers.wrappers import TimeDistributed from keras.layers import Convolution1D model = Sequential() model.add(Embedding(n_vocab,100)) model.add(Dropout(0.25)) model.add(SimpleRNN(100,return_sequences=True)) model.add(TimeDistributed(Dense(n_classes, activation='softmax'))) model.compile('rmsprop', 'categorical_crossentropy') Training Now, let's start training our model. We will pass each sentence as a batch to the model. We cannot use model.fit() because it expects all of the sentences to be the same size. We will therefore use model.train_on_batch(). Training is very fast, since the dataset is relatively small. Each epoch takes 20 seconds on my Macbook Air. import progressbar n_epochs = 30 for i in range(n_epochs): print("Training epoch {}".format(i)) bar = progressbar.ProgressBar(max_value=len(train_x)) for n_batch, sent in bar(enumerate(train_x)): label = train_label[n_batch] # Make labels one hot label = np.eye(n_classes)[label][np.newaxis,:] # View each sentence as a batch sent = sent[np.newaxis,:] if sent.shape[1] >1: #ignore 1 word sentences model.train_on_batch(sent, label) Evaluation To measure the accuracy of the model, we use model.predict_on_batch() and metrics.accuracy.conlleval(). from metrics.accuracy import conlleval labels_pred_val = [] bar = progressbar.ProgressBar(max_value=len(val_x)) for n_batch, sent in bar(enumerate(val_x)): label = val_label[n_batch] label = np.eye(n_classes)[label][np.newaxis,:] sent = sent[np.newaxis,:] pred = model.predict_on_batch(sent) pred = np.argmax(pred,-1)[0] labels_pred_val.append(pred) labels_pred_val = [ list(map(lambda x: idx2la[x], y)) for y in labels_pred_val] con_dict = conlleval(labels_pred_val, labels_val, words_val, 'measure.txt') print('Precision = {}, Recall = {}, F1 = {}'.format( con_dict['r'], con_dict['p'], con_dict['f1'])) With this model, I get a 92.36 F1 Score. Precision = 92.07, Recall = 92.66, F1 = 92.36 Note that for the sake of brevity, I've not shown the logging part of the code. Loggging losses and accuracies are an important part of coding up an model. An improved model (described in the next section) with logging is at main.py. You can run it as : $ python main.py Improvements One drawback with our current model is that there is no look ahead, that is, output: ot This depends only on the current and previous words, but not on the words next to it. You can imagine clues about the properties of the current word that are also held by the next word. Lookahead can easily be implemented by having a convolutional layer before RNN and word embeddings: model = Sequential() model.add(Embedding(n_vocab,100)) model.add(Convolution1D(128, 5, border_mode='same', activation='relu')) model.add(Dropout(0.25)) model.add(GRU(100,return_sequences=True)) model.add(TimeDistributed(Dense(n_classes, activation='softmax'))) model.compile('rmsprop', 'categorical_crossentropy') With this improved model, I get a 94.90F1 Score! Conclusion In this two-part post series, you learned about word embeddings and RNNs. We applied these to an NLP problem: ATIS. We also made an improvement to our model. To improve the model further, you can try using word embeddings learned on a large site like Wikipedia. Also, there are variants of RNNs such as LSTM or GRU that can be experimented with. About the author Sasank Chilamkurthy works at Fractal Analytics. His work involves deep learning  on medical images obtained from radiology and pathology. He is mainly  interested in computer vision.
Read more
  • 0
  • 0
  • 3262

article-image-server-side-swift-building-slack-bot-part-2
Peter Zignego
13 Oct 2016
5 min read
Save for later

Server-side Swift: Building a Slack Bot, Part 2

Peter Zignego
13 Oct 2016
5 min read
In Part 1 of this series, I introduced you to SlackKit and Zewo, which allows us to build and deploy a Slack bot written in Swift to a Linux server. Here in Part 2, we will finish the app, showing all of the Swift code. We will also show how to get an API token, how to test the app and deploy it on Heroku, and finally how to launch it. Show Me the Swift Code! Finally, some Swift code! To create our bot, we need to edit our main.swift file to contain our bot logic: import String importSlackKit class Leaderboard: MessageEventsDelegate { // A dictionary to hold our leaderboard var leaderboard: [String: Int] = [String: Int]() letatSet = CharacterSet(characters: ["@"]) // A SlackKit client instance let client: SlackClient // Initalize the leaderboard with a valid Slack API token init(token: String) { client = SlackClient(apiToken: token) client.messageEventsDelegate = self } // Enum to hold commands the bot knows enum Command: String { case Leaderboard = "leaderboard" } // Enum to hold logic that triggers certain bot behaviors enum Trigger: String { casePlusPlus = "++" caseMinusMinus = "--" } // MARK: MessageEventsDelegate // Listen to the messages that are coming in over the Slack RTM connection funcmessageReceived(message: Message) { listen(message: message) } funcmessageSent(message: Message){} funcmessageChanged(message: Message){} funcmessageDeleted(message: Message?){} // MARK: Leaderboard Internal Logic privatefunc listen(message: Message) { // If a message contains our bots user ID and a recognized command, handle that command if let id = client.authenticatedUser?.id, text = message.text { iftext.lowercased().contains(query: Command.Leaderboard.rawValue) &&text.contains(query: id) { handleCommand(command: .Leaderboard, channel: message.channel) } } // If a message contains a trigger value, handle that trigger ifmessage.text?.contains(query: Trigger.PlusPlus.rawValue) == true { handleMessageWithTrigger(message: message, trigger: .PlusPlus) } ifmessage.text?.contains(query: Trigger.MinusMinus.rawValue) == true { handleMessageWithTrigger(message: message, trigger: .MinusMinus) } } // Text parsing can be messy when you don't have Foundation... privatefunchandleMessageWithTrigger(message: Message, trigger: Trigger) { if let text = message.text, start = text.index(of: "@"), end = text.index(of: trigger.rawValue) { let string = String(text.characters[start...end].dropLast().dropFirst()) let users = client.users.values.filter{$0.id == self.userID(string: string)} // If the receiver of the trigger is a user, use their user ID ifusers.count> 0 { letidString = userID(string: string) initalizationForValue(dictionary: &leaderboard, value: idString) scoringForValue(dictionary: &leaderboard, value: idString, trigger: trigger) // Otherwise just store the receiver value as is } else { initalizationForValue(dictionary: &leaderboard, value: string) scoringForValue(dictionary: &leaderboard, value: string, trigger: trigger) } } } // Handle recognized commands privatefunchandleCommand(command: Command, channel:String?) { switch command { case .Leaderboard: // Send message to the channel with the leaderboard attached if let id = channel { client.webAPI.sendMessage(channel:id, text: "Leaderboard", linkNames: true, attachments: [constructLeaderboardAttachment()], success: {(response) in }, failure: { (error) in print("Leaderboard failed to post due to error:(error)") }) } } } privatefuncinitalizationForValue(dictionary: inout [String: Int], value: String) { if dictionary[value] == nil { dictionary[value] = 0 } } privatefuncscoringForValue(dictionary: inout [String: Int], value: String, trigger: Trigger) { switch trigger { case .PlusPlus: dictionary[value]?+=1 case .MinusMinus: dictionary[value]?-=1 } } // MARK: Leaderboard Interface privatefuncconstructLeaderboardAttachment() -> Attachment? { let Great! But we’ll need to replace the dummy API token with the real deal before anything will work. Getting an API Token We need to create a bot integration in Slack. You’ll need a Slack instance that you have administrator access to. If you don’t already have one of those to play with, go sign up. Slack is free for small teams: Create a new bot here. Enter a name for your bot. I’m going to use “leaderbot”. Click on “Add Bot Integration”. Copy the API token that Slack generates and replace the placeholder token at the bottom of main.swift with it. Testing 1,2,3… Now that we have our API token, we’re ready to do some local testing. Back in Xcode, select the leaderbot command-line application target and run your bot (⌘+R). When we go and look at Slack, our leaderbot’s activity indicator should show that it’s online. It’s alive! To ensure that it’s working, we should give our helpful little bot some karma points: @leaderbot++ And ask it to see the leaderboard: @leaderbot leaderboard Head in the Clouds Now that we’ve verified that our leaderboard bot works locally, it’s time to deploy it. We are deploying on Heroku, so if you don’t have an account, go and sign up for a free one. First, we need to add a Procfile for Heroku. Back in the terminal, run: echo slackbot: .build/debug/leaderbot > Procfile Next, let’s check in our code: git init git add . git commit -am’leaderbot powering up’ Finally, we’ll setup Heroku: Install the Heroku toolbelt. Log in to Heroku in your terminal: heroku login Create our application on Heroku and set our buildpack: heroku create --buildpack https://github.com/pvzig/heroku-buildpack-swift.git leaderbot Set up our Heroku remote: heroku git:remote -a leaderbot Push to master: git push heroku master Once you push to master, you’ll see Heroku going through the process of building your application. Launch! When the build is complete, all that’s left to do is to run our bot: heroku run:detached slackbot Like when we tested locally, our bot should become active and respond to our commands! You’re Done! Congratulations, you’ve successfully built and deployed a Slack bot written in Swift onto a Linux server! Built With: Jay: Pure-Swift JSON parser and formatterkylef’s Heroku buildpack for Swift Open Swift: Open source cross project standards for Swift SlackKit: A Slack client library Zewo: Open source libraries for modern server software Disclaimer The linux version of SlackKit should be considered an alpha release. It’s a fun tech demo to show what’s possible with Swift on the server, not something to be relied upon. Feel free to report issues you come across. About the author Peter Zignego is an iOS developer in Durham, North Carolina, USA. He writes at bytesized.co, tweets at @pvzig, and freelances at Launch Software.
Read more
  • 0
  • 0
  • 6085

article-image-lets-start-extending-docker
Packt
13 Oct 2016
8 min read
Save for later

Let's start with Extending Docker

Packt
13 Oct 2016
8 min read
In this article by Russ McKendrick author of the book Extending Docker, we will discuss the following topics: Why Docker has been so widely accepted by the entire industry What does a typical container's life cycle look like? What will you need for the remainder of the chapters? (For more resources related to this topic, see here.) The rise of Docker Not very often does a technology come along that is adopted so widely across an entire industry. Since its first public release in March 2013, Docker has not only gained the support of both end users, like you and I, but also industry leaders such as Amazon, Microsoft, and Google. Docker is currently using the following sentence on their website to describe why you would want to use it: Docker provides an integrated technology suite that enables development and IT operations teams to build, ship, and run distributed applications anywhere. There is a meme, based on the disaster girl photo, which sums up why such a seemingly simple explanation is actually quite important: So as simple as Docker's description sounds, it's actually a been utopia for most developers and IT operations teams for a number of years to have tool that can ensure that an application can consistently work across the following three main stages of an application's life cycle: Development Staging and Preproduction Production To illustrate why this used to be a problem before Docker arrived at the scene, let's look at how the services were traditionally configured and deployed. People tended to typically use a mixture of dedicated machines and virtual machines. So let's look at these in more detail. While this is possible using configuration management tools, such as Puppet,or orchestration tools, such as Ansible, to maintain consistency between server environments, it is difficult to enforce these across both servers and a developer's workstation. Dedicated machines Traditionally, these are a single piece of hardware that have been configured to run your application, while the applications have direct access to the hardware, you are constrained by the binaries and libraries you can install on a dedicated machine, as they have to be shared across the entire machine. To illustrate one potential problem Docker has fixed, let's say you had a single dedicated server that was running your PHP application. When you initially deployed the dedicated machine, all three of the applications, which make up your e-commerce website, worked with PHP 5.6, so there was no problem with compatibility. Your development team has been slowly working through the three PHP applications. You have deployed it on your host to make them work with PHP 7, as this will give them a good boost in performance. However, there is a single bug that they have not been able to resolve with App2, which means that it will not run under PHP 7 without crashing when a user adds an item to their shopping cart. If you have a single host running your three applications, you will not be able to upgrade from PHP 5.6 to PHP 7 until your development team has resolved the bug with App2, unless you do one of the following: Deploy a new host running PHP 7 and migrate App1 and App3 to it; this could be both time consuming and expensive Deploy a new host running PHP 5.6 and migrate App2 to it; again this could be both time consuming and expensive Wait until the bug has been fixed; the performance improvements that the upgrade from PHP 5.6 to PHP 7 bring to the application could increase the sales and there is no ETA for the fix If you go for the first two options, you also need to ensure that the new dedicated machine either matches the developer's PHP 7 environment or that a new dedicated machine is configured in exactly the same way as your existing environment; after all, you don't want to introduce further problems by having a poorly configured machine. Virtual machines One solution to the scenario detailed earlier would be to slice up your dedicated machine's resources and make them available to the application by installing a hypervisor such as the following: KVM: http://www.linux-kvm.org/ XenSource: http://www.xenproject.org/ VMware vSphere: http://www.vmware.com/uk/products/vsphere- hypervisor/ Once installed, you can then install your binaries and libraries on each of the different virtual hosts and also install your applications on each one. Going back to the scenario given in the dedicated machine section, you will be able to upgrade to PHP 7 on the virtual machines with App1 and App2 installed, while leaving App2 untouched and functional while the development work on the fix. Great, so what is the catch? From the developer's view, there is none as they have their applications running with the PHP versions, which work best for them; however, from an IT operations point of view: More CPU, RAM, and disk space: Each of the virtual machines will require additional resources as the overhead of running three guest OS, as well as the three applications have to be taken into account More management: IT operations now need to patch, monitor, and maintain four machines, the dedicated host machine along with three virtual machines, where as before they only had a single dedicated host. As earlier, you also need to ensure that the configuration of the three virtual machines that are hosting your applications match the configuration that the developers have been using during the development process; again, you do not want to introduce additional problems due to configuration and process drift between departments. Dedicated versus virtual machines The following diagram shows the how a typical dedicated and virtual machine host would be configured: As you can see, the biggest differences between the two are quite clear. You are making a trade-off between resource utilization and being able to run your applications using different binaries/libraries. Containers Now we have covered the way in which our applications have been traditionally deployed. Let's look at what Docker adds to the mix. Back to our scenario of the three applications running on a single host machine. Installing Docker on the host and then deploying each of the applications as a container on this host gives you the benefits of the virtual machine, while vastly reducing the footprint, that is, removing the need for the hypervisor and guest operating system completely, and replacing them with a SlimLine interface directly into the host machines kernel. The advantages this gives both the IT operations and development teams are as follows: Low overhead: As mentioned already, the resource and management for the IT operations team is lower Development provide the containers: Rather than relying on the IT operations team to configure each of the three applications environments to machine the development environment, they can simply pass their containers to be put into production As you can see from the following diagram, the layers between the application and host operating system have been reduced: This may seem too good to be true, and to be honest, there is a "but". For most web applications or applications that are pre-compiled static binaries, you shouldn't have a problem. However, as Docker shares resources with the underlying host machine, such as the Kernel version, if your application needs to be compiled or have a reliance on certain libraries that are only compatible with the shared resources, then you will have to deploy your containers on a like-for-like operating system, and in some cases, hardware. Docker has tried to address this issue with the acquisition of a company called Unikernel Systems in January 2016. At the time of writing this book, not a lot is known about how Docker is planning to integrate this technology into their core product, if at all. You can find out more about this technology at https://blog. docker.com/2016/01/unikernel/. Summary In this article we got to know that not very often does a technology come along that is adopted so widely across an entire industry. Since its first public release in March 2013, Docker has not only gained the support of both end users, like you and I, but also industry leaders such as Amazon, Microsoft, and Google. Docker is currently using the following sentence on their website to describe why you would want to use it: Resources for Article: Further resources on this subject: Introduction to Docker [article] CoreOS Networking and Flannel Internals [article] Virtualizing Hosts and Applications [article]
Read more
  • 0
  • 0
  • 3219

article-image-customizing-player-character
Packt
13 Oct 2016
18 min read
Save for later

Customizing the Player Character

Packt
13 Oct 2016
18 min read
One of the key features of an RPG is to be able to customize your character player. In this article by Vahe Karamian, author of the book Building an RPG with Unity 5.x, we will take a look at how we can provide a means to achieve this. (For more resources related to this topic, see here.) Once again, the approach and concept are universal, but the actual implementation might be a little different based on your model structure. Create a new scene and name it Character Customization. Create a Cube prefab and set it to the origin. Change theScaleof the cube to<5, 0.1, 5>, you can also change the name of the GameObject to Base. This will be the platform that our character model stands on while the player customizes his/her character before game play. Drag and drop thefbxfile representing your character model into theScene View. The next few steps will entirely depend on your model hierarchy and structure as designed by the modeller. To illustrate the point, I have placed the same model in the scene twice. The one on the left is the model that has been configured to display only the basics, the model on the right is the model in its original state as shown in the figure below: Notice that this particular model I am using has everything attached. These include the different types of weapons, shoes, helmets, and armour. The instantiated prefab on the left hand side has turned off all of the extras from the GameObject's hierarchy. Here is how the hierarchy looks in the Hierarchy View: The model has a veryextensivehierarchy in its structure, the figure above is a small snippet to demonstrate that you will need to navigate the structure and manually identify and enable or disable the mesh representing a particular part of the model. Customizable Parts Based on my model, I cancustomizea few things on my 3D model. I can customize the shoulder pads, I can customize the body type, I can customize the weapons and armor it has, I can customize the helmet and shoes, and finally I can also customize the skin texture to give it different looks. Let's get a listing of all the different customizable items we have for our character: Shoulder Shields:there are four types Body Type:there are three body types; skinny, buff, and chubby Armor:knee pad, leg plate Shields:there are two types of shields Boots:there are two types of boots Helmet:there are four types of helmets Weapons:there are 13 different types of weapons Skins:there are 13 different types of skins User Interface Now that we know what ouroptionsare for customizing our player character, we can start thinking about the User Interface (UI) that will be used to enable the customization of the character. To design our UI, we will need to create aCanvasGameObject, this is done by right-clicking in theHierarchy Viewand selectingCreate|UI|Canvas. This will place aCanvasGameObject and anEventSystemGameObject in theHierarchy View. It is assumed that you already know how to create a UI in Unity. I am going to use a Panel togroupthe customizable items. For the moment I will be using checkboxes for some items and scroll bars for the weapons and skin texture. The following figure will illustrate how my UI for customization looks: These UI elements will need to be integrated with Event Handlers that will perform the necessary actions for enabling or disabling certain parts of the character model. For instance, using the UI I can select Shoulder Pad 4, Buff Body Type, move the scroll bar until the Hammer weapon shows up, selecting the second Helmet checkbox, selecting Shield 1 and Boot 2, my character will look like the figure below.We need a way to refer to each one of the meshes representing the different types of customizable objects on the model. This will be done through a C# script. The script will need to keep track of all the parts we are going to be managing for customization. Some models will not have the extra meshes attached. You can always create empty GameObjects at a particular location on the model, and you can dynamically instantiate the prefab representing your custom object at the given point. This can also be done for our current model, for instance, if we have a special space weapon that somehow gets dropped by the aliens in the game world, we can attach the weapon to our model through C# code. The important thing is to understand the concept, and the rest is up to you! The Code for Character Customization Things don't happen automatically. So we need to create some C# code that will handle the customization of our character model. The script we create here will handle the UI events that will drive the enabling and disabling of different parts of the model mesh. Create a new C# script and call itCharacterCustomization.cs. This script will be attached to theBaseGameObject in the scene. Here is a listing of the script: using UnityEngine; using UnityEngine.UI; using System.Collections; using UnityEngine.SceneManagement; public class CharacterCustomization : MonoBehaviour { public GameObject PLAYER_CHARACTER; public Material[] PLAYER_SKIN; public GameObject CLOTH_01LOD0; public GameObject CLOTH_01LOD0_SKIN; public GameObject CLOTH_02LOD0; public GameObject CLOTH_02LOD0_SKIN; public GameObject CLOTH_03LOD0; public GameObject CLOTH_03LOD0_SKIN; public GameObject CLOTH_03LOD0_FAT; public GameObject BELT_LOD0; public GameObject SKN_LOD0; public GameObject FAT_LOD0; public GameObject RGL_LOD0; public GameObject HAIR_LOD0; public GameObject BOW_LOD0; // Head Equipment public GameObject GLADIATOR_01LOD0; public GameObject HELMET_01LOD0; public GameObject HELMET_02LOD0; public GameObject HELMET_03LOD0; public GameObject HELMET_04LOD0; // Shoulder Pad - Right Arm / Left Arm public GameObject SHOULDER_PAD_R_01LOD0; public GameObject SHOULDER_PAD_R_02LOD0; public GameObject SHOULDER_PAD_R_03LOD0; public GameObject SHOULDER_PAD_R_04LOD0; public GameObject SHOULDER_PAD_L_01LOD0; public GameObject SHOULDER_PAD_L_02LOD0; public GameObject SHOULDER_PAD_L_03LOD0; public GameObject SHOULDER_PAD_L_04LOD0; // Fore Arm - Right / Left Plates public GameObject ARM_PLATE_R_1LOD0; public GameObject ARM_PLATE_R_2LOD0; public GameObject ARM_PLATE_L_1LOD0; public GameObject ARM_PLATE_L_2LOD0; // Player Character Weapons public GameObject AXE_01LOD0; public GameObject AXE_02LOD0; public GameObject CLUB_01LOD0; public GameObject CLUB_02LOD0; public GameObject FALCHION_LOD0; public GameObject GLADIUS_LOD0; public GameObject MACE_LOD0; public GameObject MAUL_LOD0; public GameObject SCIMITAR_LOD0; public GameObject SPEAR_LOD0; public GameObject SWORD_BASTARD_LOD0; public GameObject SWORD_BOARD_01LOD0; public GameObject SWORD_SHORT_LOD0; // Player Character Defense Weapons public GameObject SHIELD_01LOD0; public GameObject SHIELD_02LOD0; public GameObject QUIVER_LOD0; public GameObject BOW_01_LOD0; // Player Character Calf - Right / Left public GameObject KNEE_PAD_R_LOD0; public GameObject LEG_PLATE_R_LOD0; public GameObject KNEE_PAD_L_LOD0; public GameObject LEG_PLATE_L_LOD0; public GameObject BOOT_01LOD0; public GameObject BOOT_02LOD0; // Use this for initialization void Start() { } public bool ROTATE_MODEL = false; // Update is called once per frame void Update() { if (Input.GetKeyUp(KeyCode.R)) { this.ROTATE_MODEL = !this.ROTATE_MODEL; } if (this.ROTATE_MODEL) { this.PLAYER_CHARACTER.transform.Rotate(new Vector3(0, 1, 0), 33.0f * Time.deltaTime); } if (Input.GetKeyUp(KeyCode.L)) { Debug.Log(PlayerPrefs.GetString("NAME")); } } public void SetShoulderPad(Toggle id) { switch (id.name) { case "SP-01": { this.SHOULDER_PAD_R_01LOD0.SetActive(id.isOn); this.SHOULDER_PAD_R_02LOD0.SetActive(false); this.SHOULDER_PAD_R_03LOD0.SetActive(false); this.SHOULDER_PAD_R_04LOD0.SetActive(false); this.SHOULDER_PAD_L_01LOD0.SetActive(id.isOn); this.SHOULDER_PAD_L_02LOD0.SetActive(false); this.SHOULDER_PAD_L_03LOD0.SetActive(false); this.SHOULDER_PAD_L_04LOD0.SetActive(false); PlayerPrefs.SetInt("SP-01", 1); PlayerPrefs.SetInt("SP-02", 0); PlayerPrefs.SetInt("SP-03", 0); PlayerPrefs.SetInt("SP-04", 0); break; } case "SP-02": { this.SHOULDER_PAD_R_01LOD0.SetActive(false); this.SHOULDER_PAD_R_02LOD0.SetActive(id.isOn); this.SHOULDER_PAD_R_03LOD0.SetActive(false); this.SHOULDER_PAD_R_04LOD0.SetActive(false); this.SHOULDER_PAD_L_01LOD0.SetActive(false); this.SHOULDER_PAD_L_02LOD0.SetActive(id.isOn); this.SHOULDER_PAD_L_03LOD0.SetActive(false); this.SHOULDER_PAD_L_04LOD0.SetActive(false); PlayerPrefs.SetInt("SP-01", 0); PlayerPrefs.SetInt("SP-02", 1); PlayerPrefs.SetInt("SP-03", 0); PlayerPrefs.SetInt("SP-04", 0); break; } case "SP-03": { this.SHOULDER_PAD_R_01LOD0.SetActive(false); this.SHOULDER_PAD_R_02LOD0.SetActive(false); this.SHOULDER_PAD_R_03LOD0.SetActive(id.isOn); this.SHOULDER_PAD_R_04LOD0.SetActive(false); this.SHOULDER_PAD_L_01LOD0.SetActive(false); this.SHOULDER_PAD_L_02LOD0.SetActive(false); this.SHOULDER_PAD_L_03LOD0.SetActive(id.isOn); this.SHOULDER_PAD_L_04LOD0.SetActive(false); PlayerPrefs.SetInt("SP-01", 0); PlayerPrefs.SetInt("SP-02", 0); PlayerPrefs.SetInt("SP-03", 1); PlayerPrefs.SetInt("SP-04", 0); break; } case "SP-04": { this.SHOULDER_PAD_R_01LOD0.SetActive(false); this.SHOULDER_PAD_R_02LOD0.SetActive(false); this.SHOULDER_PAD_R_03LOD0.SetActive(false); this.SHOULDER_PAD_R_04LOD0.SetActive(id.isOn); this.SHOULDER_PAD_L_01LOD0.SetActive(false); this.SHOULDER_PAD_L_02LOD0.SetActive(false); this.SHOULDER_PAD_L_03LOD0.SetActive(false); this.SHOULDER_PAD_L_04LOD0.SetActive(id.isOn); PlayerPrefs.SetInt("SP-01", 0); PlayerPrefs.SetInt("SP-02", 0); PlayerPrefs.SetInt("SP-03", 0); PlayerPrefs.SetInt("SP-04", 1); break; } } } public void SetBodyType(Toggle id) { switch (id.name) { case "BT-01": { this.RGL_LOD0.SetActive(id.isOn); this.FAT_LOD0.SetActive(false); break; } case "BT-02": { this.RGL_LOD0.SetActive(false); this.FAT_LOD0.SetActive(id.isOn); break; } } } public void SetKneePad(Toggle id) { this.KNEE_PAD_R_LOD0.SetActive(id.isOn); this.KNEE_PAD_L_LOD0.SetActive(id.isOn); } public void SetLegPlate(Toggle id) { this.LEG_PLATE_R_LOD0.SetActive(id.isOn); this.LEG_PLATE_L_LOD0.SetActive(id.isOn); } public void SetWeaponType(Slider id) { switch (System.Convert.ToInt32(id.value)) { case 0: { this.AXE_01LOD0.SetActive(false); this.AXE_02LOD0.SetActive(false); this.CLUB_01LOD0.SetActive(false); this.CLUB_02LOD0.SetActive(false); this.FALCHION_LOD0.SetActive(false); this.GLADIUS_LOD0.SetActive(false); this.MACE_LOD0.SetActive(false); this.MAUL_LOD0.SetActive(false); this.SCIMITAR_LOD0.SetActive(false); this.SPEAR_LOD0.SetActive(false); this.SWORD_BASTARD_LOD0.SetActive(false); this.SWORD_BOARD_01LOD0.SetActive(false); this.SWORD_SHORT_LOD0.SetActive(false); break; } case 1: { this.AXE_01LOD0.SetActive(true); this.AXE_02LOD0.SetActive(false); this.CLUB_01LOD0.SetActive(false); this.CLUB_02LOD0.SetActive(false); this.FALCHION_LOD0.SetActive(false); this.GLADIUS_LOD0.SetActive(false); this.MACE_LOD0.SetActive(false); this.MAUL_LOD0.SetActive(false); this.SCIMITAR_LOD0.SetActive(false); this.SPEAR_LOD0.SetActive(false); this.SWORD_BASTARD_LOD0.SetActive(false); this.SWORD_BOARD_01LOD0.SetActive(false); this.SWORD_SHORT_LOD0.SetActive(false); break; } case 2: { this.AXE_01LOD0.SetActive(false); this.AXE_02LOD0.SetActive(true); this.CLUB_01LOD0.SetActive(false); this.CLUB_02LOD0.SetActive(false); this.FALCHION_LOD0.SetActive(false); this.GLADIUS_LOD0.SetActive(false); this.MACE_LOD0.SetActive(false); this.MAUL_LOD0.SetActive(false); this.SCIMITAR_LOD0.SetActive(false); this.SPEAR_LOD0.SetActive(false); this.SWORD_BASTARD_LOD0.SetActive(false); this.SWORD_BOARD_01LOD0.SetActive(false); this.SWORD_SHORT_LOD0.SetActive(false); break; } case 3: { this.AXE_01LOD0.SetActive(false); this.AXE_02LOD0.SetActive(false); this.CLUB_01LOD0.SetActive(true); this.CLUB_02LOD0.SetActive(false); this.FALCHION_LOD0.SetActive(false); this.GLADIUS_LOD0.SetActive(false); this.MACE_LOD0.SetActive(false); this.MAUL_LOD0.SetActive(false); this.SCIMITAR_LOD0.SetActive(false); this.SPEAR_LOD0.SetActive(false); this.SWORD_BASTARD_LOD0.SetActive(false); this.SWORD_BOARD_01LOD0.SetActive(false); this.SWORD_SHORT_LOD0.SetActive(false); break; } case 4: { this.AXE_01LOD0.SetActive(false); this.AXE_02LOD0.SetActive(false); this.CLUB_01LOD0.SetActive(false); this.CLUB_02LOD0.SetActive(true); this.FALCHION_LOD0.SetActive(false); this.GLADIUS_LOD0.SetActive(false); this.MACE_LOD0.SetActive(false); this.MAUL_LOD0.SetActive(false); this.SCIMITAR_LOD0.SetActive(false); this.SPEAR_LOD0.SetActive(false); this.SWORD_BASTARD_LOD0.SetActive(false); this.SWORD_BOARD_01LOD0.SetActive(false); this.SWORD_SHORT_LOD0.SetActive(false); break; } case 5: { this.AXE_01LOD0.SetActive(false); this.AXE_02LOD0.SetActive(false); this.CLUB_01LOD0.SetActive(false); this.CLUB_02LOD0.SetActive(false); this.FALCHION_LOD0.SetActive(true); this.GLADIUS_LOD0.SetActive(false); this.MACE_LOD0.SetActive(false); this.MAUL_LOD0.SetActive(false); this.SCIMITAR_LOD0.SetActive(false); this.SPEAR_LOD0.SetActive(false); this.SWORD_BASTARD_LOD0.SetActive(false); this.SWORD_BOARD_01LOD0.SetActive(false); this.SWORD_SHORT_LOD0.SetActive(false); break; } case 6: { this.AXE_01LOD0.SetActive(false); this.AXE_02LOD0.SetActive(false); this.CLUB_01LOD0.SetActive(false); this.CLUB_02LOD0.SetActive(false); this.FALCHION_LOD0.SetActive(false); this.GLADIUS_LOD0.SetActive(true); this.MACE_LOD0.SetActive(false); this.MAUL_LOD0.SetActive(false); this.SCIMITAR_LOD0.SetActive(false); this.SPEAR_LOD0.SetActive(false); this.SWORD_BASTARD_LOD0.SetActive(false); this.SWORD_BOARD_01LOD0.SetActive(false); this.SWORD_SHORT_LOD0.SetActive(false); break; } case 7: { this.AXE_01LOD0.SetActive(false); this.AXE_02LOD0.SetActive(false); this.CLUB_01LOD0.SetActive(false); this.CLUB_02LOD0.SetActive(false); this.FALCHION_LOD0.SetActive(false); this.GLADIUS_LOD0.SetActive(false); this.MACE_LOD0.SetActive(true); this.MAUL_LOD0.SetActive(false); this.SCIMITAR_LOD0.SetActive(false); this.SPEAR_LOD0.SetActive(false); this.SWORD_BASTARD_LOD0.SetActive(false); this.SWORD_BOARD_01LOD0.SetActive(false); this.SWORD_SHORT_LOD0.SetActive(false); break; } case 8: { this.AXE_01LOD0.SetActive(false); this.AXE_02LOD0.SetActive(false); this.CLUB_01LOD0.SetActive(false); this.CLUB_02LOD0.SetActive(false); this.FALCHION_LOD0.SetActive(false); this.GLADIUS_LOD0.SetActive(false); this.MACE_LOD0.SetActive(false); this.MAUL_LOD0.SetActive(true); this.SCIMITAR_LOD0.SetActive(false); this.SPEAR_LOD0.SetActive(false); this.SWORD_BASTARD_LOD0.SetActive(false); this.SWORD_BOARD_01LOD0.SetActive(false); this.SWORD_SHORT_LOD0.SetActive(false); break; } case 9: { this.AXE_01LOD0.SetActive(false); this.AXE_02LOD0.SetActive(false); this.CLUB_01LOD0.SetActive(false); this.CLUB_02LOD0.SetActive(false); this.FALCHION_LOD0.SetActive(false); this.GLADIUS_LOD0.SetActive(false); this.MACE_LOD0.SetActive(false); this.MAUL_LOD0.SetActive(false); this.SCIMITAR_LOD0.SetActive(true); this.SPEAR_LOD0.SetActive(false); this.SWORD_BASTARD_LOD0.SetActive(false); this.SWORD_BOARD_01LOD0.SetActive(false); this.SWORD_SHORT_LOD0.SetActive(false); break; } case 10: { this.AXE_01LOD0.SetActive(false); this.AXE_02LOD0.SetActive(false); this.CLUB_01LOD0.SetActive(false); this.CLUB_02LOD0.SetActive(false); this.FALCHION_LOD0.SetActive(false); this.GLADIUS_LOD0.SetActive(false); this.MACE_LOD0.SetActive(false); this.MAUL_LOD0.SetActive(false); this.SCIMITAR_LOD0.SetActive(false); this.SPEAR_LOD0.SetActive(true); this.SWORD_BASTARD_LOD0.SetActive(false); this.SWORD_BOARD_01LOD0.SetActive(false); this.SWORD_SHORT_LOD0.SetActive(false); break; } case 11: { this.AXE_01LOD0.SetActive(false); this.AXE_02LOD0.SetActive(false); this.CLUB_01LOD0.SetActive(false); this.CLUB_02LOD0.SetActive(false); this.FALCHION_LOD0.SetActive(false); this.GLADIUS_LOD0.SetActive(false); this.MACE_LOD0.SetActive(false); this.MAUL_LOD0.SetActive(false); this.SCIMITAR_LOD0.SetActive(false); this.SPEAR_LOD0.SetActive(false); this.SWORD_BASTARD_LOD0.SetActive(true); this.SWORD_BOARD_01LOD0.SetActive(false); this.SWORD_SHORT_LOD0.SetActive(false); break; } case 12: { this.AXE_01LOD0.SetActive(false); this.AXE_02LOD0.SetActive(false); this.CLUB_01LOD0.SetActive(false); this.CLUB_02LOD0.SetActive(false); this.FALCHION_LOD0.SetActive(false); this.GLADIUS_LOD0.SetActive(false); this.MACE_LOD0.SetActive(false); this.MAUL_LOD0.SetActive(false); this.SCIMITAR_LOD0.SetActive(false); this.SPEAR_LOD0.SetActive(false); this.SWORD_BASTARD_LOD0.SetActive(false); this.SWORD_BOARD_01LOD0.SetActive(true); this.SWORD_SHORT_LOD0.SetActive(false); break; } case 13: { this.AXE_01LOD0.SetActive(false); this.AXE_02LOD0.SetActive(false); this.CLUB_01LOD0.SetActive(false); this.CLUB_02LOD0.SetActive(false); this.FALCHION_LOD0.SetActive(false); this.GLADIUS_LOD0.SetActive(false); this.MACE_LOD0.SetActive(false); this.MAUL_LOD0.SetActive(false); this.SCIMITAR_LOD0.SetActive(false); this.SPEAR_LOD0.SetActive(false); this.SWORD_BASTARD_LOD0.SetActive(false); this.SWORD_BOARD_01LOD0.SetActive(false); this.SWORD_SHORT_LOD0.SetActive(true); break; } } } public void SetHelmetType(Toggle id) { switch (id.name) { case "HL-01": { this.HELMET_01LOD0.SetActive(id.isOn); this.HELMET_02LOD0.SetActive(false); this.HELMET_03LOD0.SetActive(false); this.HELMET_04LOD0.SetActive(false); break; } case "HL-02": { this.HELMET_01LOD0.SetActive(false); this.HELMET_02LOD0.SetActive(id.isOn); this.HELMET_03LOD0.SetActive(false); this.HELMET_04LOD0.SetActive(false); break; } case "HL-03": { this.HELMET_01LOD0.SetActive(false); this.HELMET_02LOD0.SetActive(false); this.HELMET_03LOD0.SetActive(id.isOn); this.HELMET_04LOD0.SetActive(false); break; } case "HL-04": { this.HELMET_01LOD0.SetActive(false); this.HELMET_02LOD0.SetActive(false); this.HELMET_03LOD0.SetActive(false); this.HELMET_04LOD0.SetActive(id.isOn); break; } } } public void SetShieldType(Toggle id) { switch (id.name) { case "SL-01": { this.SHIELD_01LOD0.SetActive(id.isOn); this.SHIELD_02LOD0.SetActive(false); break; } case "SL-02": { this.SHIELD_01LOD0.SetActive(false); this.SHIELD_02LOD0.SetActive(id.isOn); break; } } } public void SetSkinType(Slider id) { this.SKN_LOD0.GetComponent<Renderer>().material = this.PLAYER_SKIN[System.Convert.ToInt32(id.value)]; this.FAT_LOD0.GetComponent<Renderer>().material = this.PLAYER_SKIN[System.Convert.ToInt32(id.value)]; this.RGL_LOD0.GetComponent<Renderer>().material = this.PLAYER_SKIN[System.Convert.ToInt32(id.value)]; } public void SetBootType(Toggle id) { switch (id.name) { case "BT-01": { this.BOOT_01LOD0.SetActive(id.isOn); this.BOOT_02LOD0.SetActive(false); break; } case "BT-02": { this.BOOT_01LOD0.SetActive(false); this.BOOT_02LOD0.SetActive(id.isOn); break; } } } } This is a long script but it is straightforward. At the top of the script we have defined all of the variables that will be referencing the different meshes in our model character. All variables are of type GameObject with the exception of thePLAYER_SKINvariable which is an array ofMaterialdata type. The array is used to store the different types of texture created for the character model. There are a few functions defined that are called by the UI event handler. These functions are:SetShoulderPad(Toggle id); SetBodyType(Toggle id); SetKneePad(Toggle id); SetLegPlate(Toggle id); SetWeaponType(Slider id); SetHelmetType(Toggle id); SetShieldType(Toggle id); SetSkinType(Slider id);All of the functions take a parameter that identifies which specific type is should enable or disable. A BIG NOTE HERE! You can also use the system we just built to create all of the different variations of your Non-Character Player models and store them as prefabs! Wow! This will save you so much time and effort in creating your characters representing different barbarians!!! Preserving Our Character State Now that we have spent the time to customize our character, we need to preserve our character and use it in our game. In Unity, there is a function calledDontDestroyOnLoad(). This is a great function that can be utilized at this time. What does it do? It keeps the specified GameObject in memory going from one scene to the next. We can use these mechanisms for now, eventually though, you will want to create a system that can save and load your user data. Go ahead and create a new C# script and call itDoNotDestroy.cs. This script is going to be very simple. Here is the listing: using UnityEngine; using System.Collections; public class DoNotDestroy : MonoBehaviour { // Use this for initialization void Start() { DontDestroyOnLoad(this); } // Update is called once per frame void Update() { } } After you create the script go ahead and attach it to your character model prefab in the scene. Not bad, let's do a quick recap of what we have done so far. Recap By now you should have three scenes that are functional. We have our scene that represents the main menu, we have our scene that represents our initial level, and we just created a scene that is used for character customization. Here is the flow of our game thus far: We start the game, see the main menu, select theStart Gamebutton to enter the character customization scene, do our customization, and when we click theSavebutton we loadlevel 1. For this to work, we have created the following C# scripts: GameMaster.cs:used as the main script to keep track of our game state CharacterCustomization.cs:used exclusively for customizing our character DoNotDestroy.cs:used to save the state of a given object CharacterController.cs:used to control the motion of our character IKHandle.cs:used to implement inverse kinematics for the foot When you combine all of this together you now have a good framework and flow that can be extended and improved as we go along. Summary We covered some very important topics and concepts in the article that can be used and enhanced for your games. We started the article by looking into how to customize your player character. The concepts you take away from the article can be applied to a wide variety of scenarios. We look at how to understand the structure of your character model so that you can better determine the customization methods. These are the different types of weapons, clothing, armour, shields and so on... We then looked at how to create a user interface to help enable us with the customization of our player character during gameplay. We also learned that the tool we developed can be used to quickly create several different character models (customized) and store them as Prefabs for later use! Great time saver!!! We also learned how to preserve the state of our player character after customization for gameplay. You should now have an idea of how to approach your project. Resources for Article: Further resources on this subject: Animations Sprites [article] Development Tricks with Unreal Engine 4 [article] The Game World [article]
Read more
  • 0
  • 0
  • 44983

article-image-offloading-work-ui-thread-android
Packt
13 Oct 2016
8 min read
Save for later

Offloading work from the UI Thread on Android

Packt
13 Oct 2016
8 min read
In this article by Helder Vasconcelos, the author of Asynchronous Android Programming book - Second Edition, will present the most common asynchronous techniques techniques used on Android to develop an application, that using the multicore CPUs available on the most recent devices, is able to deliver up to date results quickly, respond to user interactions immediately and produce smooth transitions between the different UI states without draining the device battery. (For more resources related to this topic, see here.) Several reports have shown that an efficient application that provides a great user experience have better review ratings, higher user engagement and are able to achieve higher revenues. Why do we need Asynchronous Programming on Android? The Android system, by default, executes the UI rendering pipeline, base components (Activity, Fragment, Service, BroadcastReceiver, ...) lifecycle callback handling and UI interaction processing on a single thread, sometimes known as UI thread or main thread. The main thread handles his work sequentially collecting its work from a queue of tasks (Message Queue) that are queued to be processed by a particular application component. When any unit of work, such as a I/O operation, takes a significant period of time to complete, it will block and prevent the main thread from handling the next tasks waiting on the main thread queue to processed. Besides that, most Android devices refresh the screen 60 times per second, so every 16 milliseconds (1s/60 frames) a UI rendering task has to be processed by the main thread in order to draw and update the device screen. The next figure shows up a typical main thread timeline that runs the UI rendering task every 16ms. When a long lasting operation, prevents the main thread from executing frame rendering in time, the current frame drawing is deferred or some frame drawings are missed generating a UI glitch noticeable to the application user. A typical long lasting operation could be: Network data communication HTTP REST Request SOAP Service Access File Upload or Backup Reading or writing of files to the filesystem Shared Preferences Files File Cache Access Internal Database reading or writing Camera, Image, Video, Binary file processing. A user Interface glitch produced by dropped frames on Android is known on Android as jank. The Android SDK command systrace (https://developer.android.com/studio/profile/systrace.html) comes with the ability to measure the performance of UI rendering and then diagnose and identify problems that may arrive from various threads that are running on the application process. In the next image we illustrate a typical main thread timeline when a blocking operation dominates the main thread for more than 2 frame rendering windows: As you can perceive, 2 frames are dropped and the 1 UI Rendering frame was deferred because our blocking operation took approximately 35ms, to finish: When the long running operation that runs on the main thread does not complete within 5 seconds, the Android System displays an “Application not Responding” (ANR) dialog to the user giving him the option to close the application. Hence, in order to execute compute-intensive or blocking I/O operations without dropping a UI frame, generate UI glitches or degrade the application responsiveness, we have to offload the task execution to a background thread, with less priority, that runs concurrently and asynchronously in an independent line of execution, like shown on the following picture. Although, the use of asynchronous and multithreaded techniques always introduces complexity to the application, Android SDK and some well known open source libraries provide high level asynchronous constructs that allow us to perform reliable asynchronous work that relieve the main thread from the hard work. Each asynchronous construct has advantages and disadvantages and by choosing the right construct for your requirements can make your code more reliable, simpler, easier to maintain and less error prone. Let’s enumerate the most common techniques that are covered in detail in the “Asynchronous Android Programming” book. AsyncTask AsyncTask is simple construct available on the Android platform since Android Cupcake (API Level 3) and is the most widely used asynchronous construct. The AsyncTask was designed to run short-background operations that once finished update the UI. The AsyncTask construct performs the time consuming operation, defined on the doInBackground function, on a global static thread pool of background threads. Once doInBackground terminates with success, the AsyncTask construct switches back the processing to the main thread (onPostExecute) delivering the operation result for further processing. This technique if it is not used properly can lead to memory leaks or inconsistent results. HandlerThread The HandlerThread is a Threat that incorporates a message queue and an Android Looper that runs continuously waiting for incoming operations to execute. To submit new work to the Thread we have to instantiate a Handler that is attached to HandlerThread Looper. public class DownloadImageTask extends AsyncTask<URL, Integer, Bitmap> { protected Long doInBackground(URL... urls) {} protected void onProgressUpdate(Integer... progress) {} protected void onPostExecute(Bitmap image) {} } HandlerThread handlerThread = new HandlerThread("MyHandlerThread"); handlerThread.start(); Looper looper = handlerThread.getLooper(); Handler handler = new Handler(looper); handler.post(new Runnable(){ @Override public void run() {} }); The Handler interface allow us to submit a Message or a Runnable subclass object that could aggregate data and a chunk of work to be executed. Loader The Loader construct allow us to run asynchronous operations that load content from a content provider or a data source, such as an Internal Database or a HTTP service. The API can load data asynchronously, detect data changes, cache data and is aware of the Fragment and Activity lifecycle. The Loader API was introduced to the Android platform at API level 11, but are available for backwards compatibility through the Android Support libraries. public static class TextLoader extends AsyncTaskLoader<String> { @Override public String loadInBackground() { // Background work } @Override public void deliverResult(String text) {} @Override protected void onStartLoading() {} @Override protected void onStopLoading() {} @Override public void onCanceled(String text) {} @Override protected void onReset() {} } IntentService The IntentService class is a specialized subclass of Service that implements a background work queue using a single HandlerThread. When work is submitted to an IntentService, it is queued for processing by a HandlerThread, and processed in order of submission. public class BackupService extends IntentService { @Override protected void onHandleIntent(Intent workIntent) { // Background Work } } JobScheduler The JobScheduler API allow us to execute jobs in background when several prerequisites are fulfilled and taking into the account the energy and network context of the device. This technique allows us to defer and batch job executions until the device is charging or an unmetered network is available. JobScheduler scheduler = (JobScheduler) getSystemService( Context.JOB_SCHEDULER_SERVICE ); JobInfo.Builder builder = new JobInfo.Builder(JOB_ID, serviceComponent); builder.setRequiredNetworkType(JobInfo.NETWORK_TYPE_UNMETERED); builder.setRequiresCharging(true); scheduler.schedule(builder.build()); RxJava RxJava is an implementation of the Reactive Extensions (ReactiveX) on the JVM, that was developed by Netflix, used to compose asynchronous event processing that react to an observable source of events. The framework extends the Observer pattern by allowing us to create a stream of events, that could be intercepted by operators (input/output) that modify the original stream of events and deliver the result or an error to a final Observer. The RxJava Schedulers allow us to control in which thread our Observable will begin operating on and in which thread the event is delivered to the final Observer or Subscriber. Subscription subscription = getPostsFromNetwork() .map(new Func1<Post, Post>() { @Override public Post call(Post Post) { ... return result; } }) .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(new Observer<Post>() { @Override public void onCompleted() {} @Override public void onError() {} @Override public void onNext(Post post) { // Process the result } }); Summary As you have seen, there are several high level asynchronous constructs available to offload the long computing or blocking tasks from the UI thread and it is up to developer to choose right construct for each situation because there is not an ideal choice that could be applied everywhere. Asynchronous multithreaded programming, that produces reliable results, is difficult and error prone task so using a high level technique tends to simplify the application source code and the multithreading processing logic required to scale the application over the available CPU cores. Remember that keeping your application responsive and smooth is essential to delight your users and increase your chances to create a notorious mobile application. The techniques and asynchronous constructs summarized on the previous paragraphs are covered in detail in the Asynchronous Android Programming book published by Packt Publishing. Resources for Article: Further resources on this subject: Getting started with Android Development [article] Practical How-To Recipes for Android [article] Speeding up Gradle builds for Android [article]
Read more
  • 0
  • 0
  • 14408
Unlock access to the largest independent learning library in Tech for FREE!
Get unlimited access to 7500+ expert-authored eBooks and video courses covering every tech area you can think of.
Renews at $19.99/month. Cancel anytime
article-image-iot-and-decision-science
Packt
13 Oct 2016
10 min read
Save for later

IoT and Decision Science

Packt
13 Oct 2016
10 min read
In this article by Jojo Moolayil, author of the book Smarter Decisions - The Intersection of Internet of Things and Decision Science, you will learn that the Internet of Things (IoT) and Decision Science have been among the hottest topics in the industry for a while now. You would have heard about IoT and wanted to learn more about it, but unfortunately you would have come across multiple names and definitions over the Internet with hazy differences between them. Also, Decision Science has grown from a nascent domain to become one of the fastest and most widespread horizontal in the industry in the recent years. With the ever-increasing volume, variety, and veracity of data, decision science has become more and more valuable for the industry. Using data to uncover latent patterns and insights to solve business problems has made it easier for businesses to take actions with better impact and accuracy. (For more resources related to this topic, see here.) Data is the new oil for the industry, and with the boom of IoT, we are in a world where more and more devices are getting connected to the Internet with sensors capturing more and more vital granular dimensions details that had never been touched earlier. The IoT is a game changer with a plethora of devices connected to each other; the industry is eagerly attempting to untap the huge potential that it can deliver. The true value and impact of IoT is delivered with the help of Decision Science. IoT has inherently generated an ocean of data where you can swim to gather insights and take smarter decisions with the intersection of Decision Science and IoT. In this book, you will learn about IoT and Decision Science in detail by solving real-life IoT business problems using a structured approach. In this article, we will begin by understanding the fundamental basics of IoT and Decision Science problem solving. You will learn the following concepts: Understanding IoT and demystifying Machine to Machine (M2M), IoT, Internet of Everything (IoE), and Industrial IoT (IIoT) Digging deeper into the logical stack of IoT Studying the problem life cycle Exploring the problem landscape The art of problem solving The problem solving framework It is highly recommended that you explore this article in depth. It focuses on the basics and concepts required to build problems and use cases. Understanding the IoT To get started with the IoT, lets first try to understand it using the easiest constructs. Internet and Things; we have two simple words here that help us understand the entire concept. So what is the Internet? It is basically a network of computing devices. Similarly, what is a Thing? It could be any real-life entity featuring Internet connectivity. So now, what do we decipher from IoT? It is a network of connected Things that can transmit and receive data from other things once connected to the network. This is how we describe the Internet of Things in a nutshell. Now, let's take a glance at the definition. IoT can be defined as the ever-growing network of Things (entities) that feature Internet connectivity and the communication that occurs between them and other Internet-enabled devices and systems. The Things in IoT are enabled with sensors that capture vital information from the device during its operations, and the device features Internet connectivity that helps it transfer and communicate to other devices and the network. Today, when we discuss about IoT, there are so many other similar terms that come into the picture, such as Industrial Internet, M2M, IoE, and a few more, and we find it difficult to understand the differences between them. Before we begin delineating the differences between these hazy terms and understand how IoT evolved in the industry, lets first take a simple real-life scenario to understand how exactly IoT looks like. IoT in a real-life scenario Let's take a simple example to understand how IoT works. Consider a scenario where you are a father in a family with a working mother and 10-year old son studying in school. You and your wife work in different offices. Your house is equipped with quite a few smart devices, say, a smart microwave, smart refrigerator, and smart TV. You are currently in office and you get notified on your smartphone that your son, Josh, has reached home from school. (He used his personal smart key to open the door.) You then use your smartphone to turn on the microwave at home to heat the sandwiches kept in it. Your son gets notified on the smart home controller that you have hot sandwiches ready for him. He quickly finishes them and starts preparing for a math test at school and you resume your work. After a while, you get notified again that your wife has also reached home (She also uses a similar smart key.) and you suddenly realize that you need to reach home to help your son with his math test. You again use your smartphone and change the air conditioner settings for three people and set the refrigerator to defrost using the app. In another 15 minutes, you are home and the air conditioning temperature is well set for three people. You then grab a can of juice from the refrigerator and discuss some math problems with your son on the couch. Intuitive, isnt it? How did it his happen and how did you access and control everything right from your phone? Well, this is how IoT works! Devices can talk to each other and also take actions based on the signals received: The IoT scenario Lets take a closer look at the same scenario. You are sitting in office and you could access the air conditioner, microwave, refrigerator, and home controller through your smartphone. Yes, the devices feature Internet connectivity and once connected to the network, they can send and receive data from other devices and take actions based on signals. A simple protocol helps these devices understand and send data and signals to a plethora of heterogeneous devices connected to the network. We will get into the details of the protocol and how these devices talk to each other soon. However, before that, we will get into some details of how this technology started and why we have so many different names today for IoT. Demystifying M2M, IoT, IIoT, and IoE So now that we have a general understanding about what is IoT, lets try to understand how it all started. A few questions that we will try to understand are: Is IoT very new in the market?, When did this start?, How did this start?, Whats the difference between M2M, IoT, IoE, and all those different names?, and so on. If we try to understand the fundamentals of IoT, that is, machines or devices connected to each other in a network, which isn't something really new and radically challenging, then what is this buzz all about? The buzz about machines talking to each other started long before most of us thought of it, and back then it was called Machine to Machine Data. In early 1950, a lot of machinery deployed for aerospace and military operations required automated communication and remote access for service and maintenance. Telemetry was where it all started. It is a process in which a highly automated communication was established from which data is collected by making measurements at remote or inaccessible geographical areas and then sent to a receiver through a cellular or wired network where it was monitored for further actions. To understand this better, lets take an example of a manned space shuttle sent for space exploration. A huge number of sensors are installed in such a space shuttle to monitor the physical condition of astronauts, the environment, and also the condition of the space shuttle. The data collected through these sensors is then sent back to the substation located on Earth, where a team would use this data to analyze and take further actions. During the same time, industrial revolution peaked and a huge number of machines were deployed in various industries. Some of these industries where failures could be catastrophic also saw the rise in machine-to-machine communication and remote monitoring: Telemetry Thus, machine-to-machine data a.k.a. M2M was born and mainly through telemetry. Unfortunately, it didnt scale to the extent that it was supposed to and this was largely because of the time it was developed in. Back then, cellular connectivity was not widespread and affordable, and installing sensors and developing the infrastructure to gather data from them was a very expensive deal. Therefore, only a small chunk of business and military use cases leveraged this. As time passed, a lot of changes happened. The Internet was born and flourished exponentially. The number of devices that got connected to the Internet was colossal. Computing power, storage capacities, and communication and technology infrastructure scaled massively. Additionally, the need to connect devices to other devices evolved, and the cost of setting up infrastructure for this became very affordable and agile. Thus came the IoT. The major difference between M2M and IoT initially was that the latter used the Internet (IPV4/6) as the medium whereas the former used cellular or wired connection for communication. However, this was mainly because of the time they evolved in. Today, heavy engineering industries have machinery deployed that communicate over the IPV4/6 network and is called Industrial IoT or sometimes M2M. The difference between the two is bare minimum and there are enough cases where both are used interchangeably. Therefore, even though M2M was actually the ancestor of IoT, today both are pretty much the same. M2M or IIoT are nowadays aggressively used to market IoT disruptions in the industrial sector. IoE or Internet of Everything was a term that surfaced on the media and Internet very recently. The term was coined by Cisco with a very intuitive definition. It emphasizes Humans as one dimension in the ecosystem. It is a more organized way of defining IoT. The IoE has logically broken down the IoT ecosystem into smaller components and simplified the ecosystem in an innovative way that was very much essential. IoE divides its ecosystem into four logical units as follows: People Processes Data Devices Built on the foundation of IoT, IoE is defined as The networked connection of People, Data, Processes, and Things. Overall, all these different terms in the IoT fraternity have more similarities than differences and, at the core, they are the same, that is, devices connecting to each other over a network. The names are then stylized to give a more intrinsic connotation of the business they refer to, such as Industrial IoT and Machine to Machine for (B2B) heavy engineering, manufacturing and energy verticals, Consumer IoT for the B2C industries, and so on. Summary In this article we learnt how to start with the IoT. It is basically a network of computing devices. Similarly, what is a Thing? It could be any real-life entity featuring Internet connectivity. So now, what do we decipher from IoT? It is a network of connected Things that can transmit and receive data from other things once connected to the network. This is how we describe the Internet of Things in a nutshell. Resources for Article: Further resources on this subject: Machine Learning Tasks [article] Welcome to Machine Learning Using the .NET Framework [article] Why Big Data in the Financial Sector? [article]
Read more
  • 0
  • 0
  • 1789

article-image-reconstructing-3d-scenes
Packt
13 Oct 2016
25 min read
Save for later

Reconstructing 3D Scenes

Packt
13 Oct 2016
25 min read
In this article by Robert Laganiere, the author of the book OpenCV 3 Computer Vision Application Programming Cookbook Third Edition, has covered the following recipes: Calibrating a camera Recovering camera pose (For more resources related to this topic, see here.) Digital image formation Let us now redraw a new version of the figure describing the pin-hole camera model. More specifically, we want to demonstrate the relation between a point in 3D at position (X,Y,Z) and its image (x,y), on a camera specified in pixel coordinates. Note the changes that have been made to the original figure. First, a reference frame was positioned at the center of the projection; then, the Y-axis was alligned to point downwards to get a coordinate system that is compatible with the usual convention, which places the image origin at the upper-left corner of the image. Finally, we have identified a special point on the image plane, by considering the line coming from the focal point that is orthogonal to the image plane. The point (u0,v0) is the pixel position at which this line pierces the image plane and is called the principal point. It would be logical to assume that this principal point is at the center of the image plane, but in practice, this one might be off by few pixels depending on the precision of the camera. Since we are dealing with digital images, the number of pixels on the image plane (its resolution) is another important characteristic of a camera. We learned previously that a 3D point (X,Y,Z) will be projected onto the image plane at (fX/Z,fY/Z). Now, if we want to translate this coordinate into pixels, we need to divide the 2D image position by the pixel width (px) and then the height (py). Note that by dividing the focal length given in world units (generally given in millimeters) by px, we obtain the focal length expressed in (horizontal) pixels. We will then define this term as fx. Similarly, fy =f/py is defined as the focal length expressed in vertical pixel unit. The complete projective equation is therefore as shown: We know that (u0,v0) is the principal point that is added to the result in order to move the origin to the upper-left corner of the image. Also, the physical size of a pixel can be obtained by dividing the size of the image sensor (generally in millimeters) by the number of pixels (horizontally or vertically). In modern sensor, pixels are generally of square shape, that is, they have the same horizontal and vertical size. The preceding equations can be rewritten in matrix form. Here is the complete projective equation in its most general form: Calibrating a camera Camera calibration is the process by which the different camera parameters (that is, the ones appearing in the projective equation) are obtained. One can obviously use the specifications provided by the camera manufacturer, but for some tasks, such as 3D reconstruction, these specifications are not accurate enough. However, accurate calibration information can be obtained by undertaking an appropriate camera calibration step. An active camera calibration procedure will proceed by showing known patterns to the camera and analyzing the obtained images. An optimization process will then determine the optimal parameter values that explain the observations. This is a complex process that has been made easy by the availability of OpenCV calibration functions. How to do it... To calibrate a camera, the idea is to show it a set of scene points for which the 3D positions are known. Then, you need to observe where these points project on the image. With the knowledge of a sufficient number of 3D points and associated 2D image points, the exact camera parameters can be inferred from the projective equation. Obviously, for accurate results, we need to observe as many points as possible. One way to achieve this would be to take a picture of a scene with known 3D points, but in practice, this is rarely feasible. A more convenient way is to take several images of a set of 3D points from different viewpoints. This approach is simpler, but it requires you to compute the position of each camera view in addition to the computation of the internal camera parameters, which is fortunately feasible. OpenCV proposes that you use a chessboard pattern to generate the set of 3D scene points required for calibration. This pattern creates points at the corners of each square, and since this pattern is flat, we can freely assume that the board is located at Z=0, with the X and Y axes well-aligned with the grid. In this case, the calibration process simply consists of showing the chessboard pattern to the camera from different viewpoints. The following is an example of a calibration pattern image made of 7x5 inner corners as captured during the calibration step: The good thing is that OpenCV has a function that automatically detects the corners of this chessboard pattern. You simply provide an image and the size of the chessboard used (the number of horizontal and vertical inner corner points). The function will return the position of these chessboard corners on the image. If the function fails to find the pattern, then it simply returns false, as shown: //output vectors of image points std::vector<cv::Point2f> imageCorners; //number of inner corners on the chessboard cv::Size boardSize(7,5); //Get the chessboard corners bool found = cv::findChessboardCorners(image, // image of chessboard pattern boardSize, // size of pattern imageCorners); // list of detected corners The output parameter, imageCorners, will simply contain the pixel coordinates of the detected inner corners of the shown pattern. Note that this function accepts additional parameters if you need to tune the algorithm, which are not discussed here. There is also a special function that draws the detected corners on the chessboard image, with lines connecting them in a sequence: //Draw the corners cv::drawChessboardCorners(image, boardSize, imageCorners, found); // corners have been found The following image is obtained: The lines that connect the points show the order in which the points are listed in the vector of detected image points. To perform a calibration, we now need to specify the corresponding 3D points. You can specify these points in the units of your choice (for example, in centimeters or in inches); however, the simplest is to assume that each square represents one unit. In that case, the coordinates of the first point would be (0,0,0) (assuming that the board is located at a depth of Z=0), the coordinates of the second point would be (1,0,0), and so on, the last point being located at (6,4,0). There are a total of 35 points in this pattern, which is too less to obtain an accurate calibration. To get more points, you need to show more images of the same calibration pattern from various points of view. To do so, you can either move the pattern in front of the camera or move the camera around the board; from a mathematical point of view, this is completely equivalent. The OpenCV calibration function assumes that the reference frame is fixed on the calibration pattern and will calculate the rotation and translation of the camera with respect to the reference frame. Let‘s now encapsulate the calibration process in a CameraCalibrator class. The attributes of this class are as follows: // input points: // the points in world coordinates // (each square is one unit) std::vector<std::vector<cv::Point3f>> objectPoints; // the image point positions in pixels std::vector<std::vector<cv::Point2f>> imagePoints; // output Matrices cv::Mat cameraMatrix; cv::Mat distCoeffs; // flag to specify how calibration is done int flag; Note that the input vectors of the scene and image points are in fact made of std::vector of point instances; each vector element is a vector of the points from one view. Here, we decided to add the calibration points by specifying a vector of the chessboard image filename as input, the method will take care of extracting the point coordinates from the images: // Open chessboard images and extract corner points int CameraCalibrator::addChessboardPoints(const std::vector<std::string>& filelist, // list of filenames cv::Size & boardSize) { // calibration noard size // the points on the chessboard std::vector<cv::Point2f> imageCorners; std::vector<cv::Point3f> objectCorners; // 3D Scene Points: // Initialize the chessboard corners // in the chessboard reference frame // The corners are at 3D location (X,Y,Z)= (i,j,0) for (int i=0; i<boardSize.height; i++) { for (int j=0; j<boardSize.width; j++) { objectCorners.push_back(cv::Point3f(i, j, 0.0f)); } } // 2D Image points: cv::Mat image; // to contain chessboard image int successes = 0; // for all viewpoints for (int i=0; i<filelist.size(); i++) { // Open the image image = cv::imread(filelist[i],0); // Get the chessboard corners bool found = cv::findChessboardCorners(image, //image of chessboard pattern boardSize, // size of pattern imageCorners); // list of detected corners // Get subpixel accuracy on the corners if (found) { cv::cornerSubPix(image, imageCorners, cv::Size(5, 5), // half size of serach window cv::Size(-1, -1), cv::TermCriteria(cv::TermCriteria::MAX_ITER + cv::TermCriteria::EPS,30,// max number of iterations 0.1)); // min accuracy // If we have a good board, add it to our data if (imageCorners.size() == boardSize.area()) { // Add image and scene points from one view addPoints(imageCorners, objectCorners); successes++; } } //If we have a good board, add it to our data if (imageCorners.size() == boardSize.area()) { // Add image and scene points from one view addPoints(imageCorners, objectCorners); successes++; } } return successes; } The first loop inputs the 3D coordinates of the chessboard and the corresponding image points are the ones provided by the cv::findChessboardCorners function; this is done for all the available viewpoints. Moreover, in order to obtain a more accurate image point location, the cv::cornerSubPix function can be used, and as the name suggests, the image points will then be localized at subpixel accuracy. The termination criterion that is specified by the cv::TermCriteria object defines the maximum number of iterations and the minimum accuracy in subpixel coordinates. The first of these two conditions that is reached will stop the corner refinement process. When a set of chessboard corners have been successfully detected, these points are added to the vectors of the image and scene points using our addPoints method. Once a sufficient number of chessboard images have been processed (and consequently, a large number of 3D scene point / 2D image point correspondences are available), we can initiate the computation of the calibration parameters as shown: // Calibrate the camera // returns the re-projection error double CameraCalibrator::calibrate(cv::Size &imageSize){ //Output rotations and translations std::vector<cv::Mat> rvecs, tvecs; // start calibration return calibrateCamera(objectPoints, // the 3D points imagePoints, // the image points imageSize, // image size cameraMatrix, // output camera matrix distCoeffs, // output distortion matrix rvecs, tvecs, // Rs, Ts flag); // set options } In practice, 10 to 20 chessboard images are sufficient, but these must be taken from different viewpoints at different depths. The two important outputs of this function are the camera matrix and the distortion parameters. These will be described in the next section. How it works... In order to explain the result of the calibration, we need to go back to the projective equation presented in the introduction of this article. This equation describes the transformation of a 3D point into a 2D point through the successive application of two matrices. The first matrix includes all of the camera parameters, which are called the intrinsic parameters of the camera. This 3x3 matrix is one of the output matrices returned by the cv::calibrateCamera function. There is also a function called cv::calibrationMatrixValues that explicitly returns the value of the intrinsic parameters given by a calibration matrix. The second matrix is there to have the input points expressed into camera-centric coordinates. It is composed of a rotation vector (a 3x3 matrix) and a translation vector (a 3x1 matrix). Remember that in our calibration example, the reference frame was placed on the chessboard. Therefore, there is a rigid transformation (made of a rotation component represented by the matrix entries r1 to r9 and a translation represented by t1, t2, and t3) that must be computed for each view. These are in the output parameter list of the cv::calibrateCamera function. The rotation and translation components are often called the extrinsic parameters of the calibration and they are different for each view. The intrinsic parameters remain constant for a given camera/lens system. The calibration results provided by the cv::calibrateCamera are obtained through an optimization process. This process aims to find the intrinsic and extrinsic parameters that minimizes the difference between the predicted image point position, as computed from the projection of the 3D scene points, and the actual image point position, as observed on the image. The sum of this difference for all the points specified during the calibration is called the re-projection error. The intrinsic parameters of our test camera obtained from a calibration based on the 27 chessboard images are fx=409 pixels; fy=408 pixels; u0=237; and v0=171. Our calibration images have a size of 536x356 pixels. From the calibration results, you can see that, as expected, the principal point is close to the center of the image, but yet off by few pixels. The calibration images were taken using a Nikon D500 camera with an 18mm lens. Looking at the manufacturer specifitions, we find that the sensor size of this camera is 23.5mm x 15.7mm which gives us a pixel size of 0.0438mm. The estimated focal length is expressed in pixels, so multiplying the result by the pixel size gives us an estimated focal length of 17.8mm, which is consistent with the actual lens we used. Let us now turn our attention to the distortion parameters. So far, we have mentioned that under the pin-hole camera model, we can neglect the effect of the lens. However, this is only possible if the lens that is used to capture an image does not introduce important optical distortions. Unfortunately, this is not the case with lower quality lenses or with lenses that have a very short focal length. Even the lens we used in this experiment introduced some distortion, that is, the edges of the rectangular board are curved in the image. Note that this distortion becomes more important as we move away from the center of the image. This is a typical distortion observed with a fish-eye lens and is called radial distortion. It is possible to compensate for these deformations by introducing an appropriate distortion model. The idea is to represent the distortions induced by a lens by a set of mathematical equations. Once established, these equations can then be reverted in order to undo the distortions visible on the image. Fortunately, the exact parameters of the transformation, which will correct the distortions, can be obtained together with the other camera parameters during the calibration phase. Once this is done, any image from the newly calibrated camera will be undistorted. Therefore, we have added an additional method to our calibration class. //remove distortion in an image (after calibration) cv::Mat CameraCalibrator::remap(const cv::Mat &image) { cv::Mat undistorted; if (mustInitUndistort) { //called once per calibration cv::initUndistortRectifyMap(cameraMatrix, // computed camera matrix distCoeffs, // computed distortion matrix cv::Mat(), // optional rectification (none) cv::Mat(), // camera matrix to generate undistorted image.size(), // size of undistorted CV_32FC1, // type of output map map1, map2); // the x and y mapping functions mustInitUndistort= false; } // Apply mapping functions cv::remap(image, undistorted, map1, map2, cv::INTER_LINEAR); // interpolation type return undistorted; } Running this code on one of our calibration image results in the following undistorted image: To correct the distortion, OpenCV uses a polynomial function that is applied to the image points in order to move them at their undistorted position. By default, five coefficients are used; a model made of eight coefficients is also available. Once these coefficients are obtained, it is possible to compute two cv::Mat mapping functions (one for the x coordinate and one for the y coordinate) that will give the new undistorted position of an image point on a distorted image. This is computed by the cv::initUndistortRectifyMap function, and the cv::remap function remaps all the points of an input image to a new image. Note that because of the nonlinear transformation, some pixels of the input image now fall outside the boundary of the output image. You can expand the size of the output image to compensate for this loss of pixels, but you now obtain output pixels that have no values in the input image (they will then be displayed as black pixels). There‘s more... More options are available when it comes to camera calibration. Calibration with known intrinsic parameters When a good estimate of the camera’s intrinsic parameters is known, it could be advantageous to input them in the cv::calibrateCamera function. They will then be used as initial values in the optimization process. To do so, you just need to add the cv::CALIB_USE_INTRINSIC_GUESS flag and input these values in the calibration matrix parameter. It is also possible to impose a fixed value for the principal point (cv::CALIB_FIX_PRINCIPAL_POINT), which can often be assumed to be the central pixel. You can also impose a fixed ratio for the focal lengths fx and fy (cv::CALIB_FIX_RATIO); in which case, you assume that the pixels have a square shape. Using a grid of circles for calibration Instead of the usual chessboard pattern, OpenCV also offers the possibility to calibrate a camera by using a grid of circles. In this case, the centers of the circles are used as calibration points. The corresponding function is very similar to the function we used to locate the chessboard corners, for example: cv::Size boardSize(7,7); std::vector<cv::Point2f> centers; bool found = cv:: findCirclesGrid(image, boardSize, centers); See also The A flexible new technique for camera calibration article by Z. Zhang  in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no 11, 2000, is a classic paper on the problem of camera calibration Recovering camera pose When a camera is calibrated, it becomes possible to relate the captured with the outside world. If the 3D structure of an object is known, then one can predict how the object will be imaged on the sensor of the camera. The process of image formation is indeed completely described by the projective equation that was presented at the beginning of this article. When most of the terms of this equation are known, it becomes possible to infer the value of the other elements (2D or 3D) through the observation of some images. In this recipe, we will look at the camera pose recovery problem when a known 3D structure is observed. How to do it... Lets consider a simple object here, a bench in a park. We took an image of it using the camera/lens system calibrated in the previous recipe. We have manually identified 8 distinct image points on the bench that we will use for our camera pose estimation. Having access to this object makes it possible to make some physical measurements. The bench is composed of a seat of size 242.5cmx53.5cmx9cm and a back of size 242.5cmx24cmx9cm that is fixed 12cm over the seat. Using this information, we can then easily derive the 3D coordinates of the eight identified points in an object-centric reference frame (here we fixed the origin at the left extremity of the intersection between the two planes). We can then create a vector of cv::Point3f containing these coordinates. //Input object points std::vector<cv::Point3f> objectPoints; objectPoints.push_back(cv::Point3f(0, 45, 0)); objectPoints.push_back(cv::Point3f(242.5, 45, 0)); objectPoints.push_back(cv::Point3f(242.5, 21, 0)); objectPoints.push_back(cv::Point3f(0, 21, 0)); objectPoints.push_back(cv::Point3f(0, 9, -9)); objectPoints.push_back(cv::Point3f(242.5, 9, -9)); objectPoints.push_back(cv::Point3f(242.5, 9, 44.5)); objectPoints.push_back(cv::Point3f(0, 9, 44.5)); The question now is where the camera was with respect to these points when the shown picture was taken. Since the coordinates of the image of these known points on the 2D image plane are also known, it becomes easy to answer this question using the cv::solvePnP function. Here, the correspondence between the 3D and the 2D points has been established manually, but as a reader of this book, you should be able to come up with methods that would allow you to obtain this information automatically. //Input image points std::vector<cv::Point2f> imagePoints; imagePoints.push_back(cv::Point2f(136, 113)); imagePoints.push_back(cv::Point2f(379, 114)); imagePoints.push_back(cv::Point2f(379, 150)); imagePoints.push_back(cv::Point2f(138, 135)); imagePoints.push_back(cv::Point2f(143, 146)); imagePoints.push_back(cv::Point2f(381, 166)); imagePoints.push_back(cv::Point2f(345, 194)); imagePoints.push_back(cv::Point2f(103, 161)); // Get the camera pose from 3D/2D points cv::Mat rvec, tvec; cv::solvePnP(objectPoints, imagePoints, // corresponding 3D/2D pts cameraMatrix, cameraDistCoeffs, // calibration rvec, tvec); // output pose // Convert to 3D rotation matrix cv::Mat rotation; cv::Rodrigues(rvec, rotation); This function computes the rigid transformation (rotation and translation) that brings the object coordinates in the camera-centric reference frame (that is, the ones that has its origin at the focal point). It is also important to note that the rotation computed by this function is given in the form of a 3D vector. This is a compact representation in which the rotation to apply is described by a unit vector (an axis of rotation) around which the object is rotated by a certain angle. This axis-angle representation is also called the Rodrigues’ rotation formula. In OpenCV, the angle of rotation corresponds to the norm of the output rotation vector, which is later aligned with the axis of rotation. This is why the cv::Rodrigues function is used to obtain the 3D matrix of rotation that appears in our projective equation. The pose recovery procedure described here is simple, but how do we know we obtained the right camera/object pose information. We can visually assess the quality of the results using the cv::viz module that gives us the ability to visualize 3D information. The use of this module is explained in the last section of this recipe. For now, lets display a simple 3D representation of our object and the camera that captured it: It might be difficult to judge of the quality of the pose recovery just by looking at this image but if you test the example of this recipe on your computer, you will have the possibility to move this representation in 3D using your mouse which should give you a better sense of the solution obtained. How it works... In this recipe, we assumed that the 3D structure of the object was known as well as the correspondence between sets of object points and image points. The camera’s intrinsic parameters were also known through calibration. If you look at our projective equation presented at the end of the Digital image formation section of the introduction of this article, this means that we have points for which coordinates (X,Y,Z) and (x,y) are known. We also have the elements of first matrix known (the intrinsic parameters). Only the second matrix is unknown; this is the one that contains the extrinsic parameters of the camera that is the camera/object pose information. Our objective is to recover these unknown parameters from the observation of 3D scene points. This problem is known as the Perspective-n-Point problem or PnP problem. Rotation has three degrees of freedom (for example, angle of rotation around the three axes) and translation also has three degrees of freedom. We therefore have a total of 6 unknowns. For each object point/image point correspondence, the projective equation gives us three algebraic equations but since the projective equation is up to a scale factor, we only have 2 independent equations. A minimum of three points is therefore required to solve this system of equations. Obviously, more points provide a more reliable estimate. In practice, many different algorithms have been proposed to solve this problem and OpenCV proposes a number of different implementation in its cv::solvePnP function. The default method consists in optimizing what is called the reprojection error. Minimizing this type of error is considered to be the best strategy to get accurate 3D information from camera images. In our problem, it corresponds to finding the optimal camera position that minimizes the 2D distance between the projected 3D points (as obtained by applying the projective equation) and the observed image points given as input. Note that OpenCV also has a cv::solvePnPRansac function. As the name suggest, this function uses the RANSAC algorithm in order to solve the PnP problem. This means that some of the object points/image points correspondences may be wrong and the function will returns which ones have been identified as outliers. This is very useful when these correspondences have been obtained through an automatic process that can fail for some points. There‘s more... When working with 3D information, it is often difficult to validate the solutions obtained. To this end, OpenCV offers a simple yet powerful visualization module that facilitates the development and debugging of 3D vision algorithms. It allows inserting points, lines, cameras, and other objects in a virtual 3D environment that you can interactively visualize from various points of views. cv::Viz, a 3D Visualizer module cv::Viz is an extra module of the OpenCV library that is built on top of the VTK open source library. This Visualization Tooolkit (VTK) is a powerful framework used for 3D computer graphics. With cv::viz, you create a 3D virtual environment to which you can add a variety of objects. A visualization window is created that displays the environment from a given point of view. You saw in this recipe an example of what can be displayed in a cv::viz window. This window responds to mouse events that are used to navigate inside the environment (through rotations and translations). This section describes the basic use of the cv::viz module. The first thing to do is to create the visualization window. Here we use a white background: // Create a viz window cv::viz::Viz3d visualizer(“Viz window“); visualizer.setBackgroundColor(cv::viz::Color::white()); Next, you create your virtual objects and insert them into the scene. There is a variety of predefined objects. One of them is particularly useful for us; it is the one that creates a virtual pin-hole camera: // Create a virtual camera cv::viz::WCameraPosition cam(cMatrix, // matrix of intrinsics image, // image displayed on the plane 30.0, // scale factor cv::viz::Color::black()); // Add the virtual camera to the environment visualizer.showWidget(“Camera“, cam); The cMatrix variable is a cv::Matx33d (that is,a cv::Matx<double,3,3>) instance containing the intrinsic camera parameters as obtained from calibration. By default this camera is inserted at the origin of the coordinate system. To represent the bench, we used two rectangular cuboid objects. // Create a virtual bench from cuboids cv::viz::WCube plane1(cv::Point3f(0.0, 45.0, 0.0), cv::Point3f(242.5, 21.0, -9.0), true, // show wire frame cv::viz::Color::blue()); plane1.setRenderingProperty(cv::viz::LINE_WIDTH, 4.0); cv::viz::WCube plane2(cv::Point3f(0.0, 9.0, -9.0), cv::Point3f(242.5, 0.0, 44.5), true, // show wire frame cv::viz::Color::blue()); plane2.setRenderingProperty(cv::viz::LINE_WIDTH, 4.0); // Add the virtual objects to the environment visualizer.showWidget(“top“, plane1); visualizer.showWidget(“bottom“, plane2); This virtual bench is also added at the origin; it then needs to be moved at its camera-centric position as found from our cv::solvePnP function. It is the responsibility of the setWidgetPose method to perform this operation. This one simply applies the rotation and translation components of the estimated motion. cv::Mat rotation; // convert vector-3 rotation // to a 3x3 rotation matrix cv::Rodrigues(rvec, rotation); // Move the bench cv::Affine3d pose(rotation, tvec); visualizer.setWidgetPose(“top“, pose); visualizer.setWidgetPose(“bottom“, pose); The final step is to create a loop that keeps displaying the visualization window. The 1ms pause is there to listen to mouse events. // visualization loop while(cv::waitKey(100)==-1 && !visualizer.wasStopped()) { visualizer.spinOnce(1, // pause 1ms true); // redraw } This loop will stop when the visualization window is closed or when a key is pressed over an OpenCV image window. Try to apply inside this loop some motion on an object (using setWidgetPose); this is how animation can be created. See also Model-based object pose in 25 lines of code by D. DeMenthon and L. S. Davis, in European Conference on Computer Vision, 1992, pp.335–343 is a famous method for recovering camera pose from scene points. Summary This article teaches us how, under specific conditions, the 3D structure of the scene and the 3D pose of the cameras that captured it can be recovered. We have seen how a good understanding of projective geometry concepts allows to devise methods enabling 3D reconstruction. Resources for Article: Further resources on this subject: OpenCV: Image Processing using Morphological Filters [article] Learn computer vision applications in Open CV [article] Cardboard is Virtual Reality for Everyone [article]
Read more
  • 0
  • 0
  • 7668

article-image-spark-beginners
Packt
13 Oct 2016
30 min read
Save for later

Spark for Beginners

Packt
13 Oct 2016
30 min read
In this article by Rajanarayanan Thottuvaikkatumana, author of the book Apache Spark 2 for Beginners, you will get an overview of Spark. By exampledata is one of the most important assets of any organization. The scale at which data is being collected and used in organizations is growing beyond imagination. The speed at which data is being ingested, the variety of the data types in use, and the amount of data that is being processed and stored are breaking all time records every moment. It is very common these days, even in small scale organizations, the data is growing from gigabytes to terabytes to petabytes. Just because of the same reason, the processing needs are also growing that asks for capability to process data at rest as well as data on the move. (For more resources related to this topic, see here.) Take any organization, its success depends on the decisions made by its leaders and for taking sound decisions, you need the backing of good data and the information generated by processing the data. This poses a big challenge on how to process the data in a timely and cost-effective manner so that right decisions can be made. Data processing techniques have evolved since the early days of computers. Countless data processing products and frameworks came into the market and disappeared over these years. Most of these data processing products and frameworks were not general purpose in nature. Most of the organizations relied on their own bespoke applications for their data processing needs in a silo way or in conjunction with specific products. Large-scale Internet applications popularly known as Internet of Things (IoT) applications heralded the common need to have open frameworks to process huge amount of data ingested at great speed dealing with various types of data. Large scale websites, media streaming applications, and huge batch processing needs of the organizations made the need even more relevant. The open source community is also growing considerably along with the growth of Internet delivering production quality software supported by reputed software companies. A huge number of companies started using open source software and started deploying them in their production environments. Apache Spark Spark is a Java Virtual Machine (JVM) based distributed data processing engine that scales, and it is fast as compared to many other data processing frameworks. Spark was born out of University of California, Berkeley, and later became one of the top projects in Apache. The research paper Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center talks about the philosophy behind the design of Spark. The research paper says: "To test the hypothesis that simple specialized frameworks provide value, we identified one class of jobs that were found to perform poorly on Hadoop by machine learning researchers at our lab: iterative jobs, where a dataset is reused across a number of iterations. We built a specialized framework called Spark optimized for these workloads." The biggest claim from Spark on the speed is Run programs up to 100x faster than Hadoop MapReduce in memory, or 10x faster on disk. Spark could make this claim because Spark does the processing in the main memory of the worker nodes andprevents the unnecessary I/O operations with the disks. The other advantage Spark offers is the ability to chain the tasks even at an application programming level without writing onto the disks at all or minimizing the number of writes to the disks. The Spark programming paradigm is very powerful and exposes a uniform programming model supporting the application development in multiple programming languages. Spark supports programming in Scala, Java, Python, and R even though there is no functional parity across all the programming languages supported. Apart from writing Spark applications in these programming languages, Spark has an interactive shell with Read, Evaluate, Print, and Loop (REPL) capabilities for the programming languages Scala, Python, and R. At this moment, there is no REPL support for Java in Spark. The Spark REPL is a very versatile tool that can be used to try and test Spark application code in an interactive fashion. The Spark REPL enables easy prototyping, debugging, and much more. In addition to the core data processing engine, Spark comes with a powerful stack of domain-specific libraries that use the core Spark libraries and provide various functionalities useful for various big data processing needs. The following list gives the supported libraries: Library Use Supported Languages Spark SQL Enables the use of SQL statements or DataFrame API inside Spark applications Scala, Java, Python, and R Spark Streaming Enables processing of live data streams Scala, Java, and Python Spark MLlib Enables development of machine learning applications Scala, Java, Python, and R Spark GraphX Enables graph processing and supports a growing library of graph algorithms Scala Understanding the Spark programming model Spark became an instant hit in the market because of its ability to process a huge amount of data types and growing number of data sources and data destinations. The most important and basic data abstraction Spark provides is the resilient distributed dataset (RDD). Spark supports distributed processing on a cluster of nodes. The moment there is a cluster of nodes, there are good chances that when the data processing is going on, some of the nodes can die. When such failures happen, the framework should be capable of coming out of such failures. Spark is designed to do that and that is what the resilient part in the RDD signifies. If there is a huge amount of data to be processed and there are nodes available in the cluster, the framework should have the capability to split the big dataset into smaller chunks and distribute them to be processed on more than one node in a cluster in parallel. Spark is capable of doing that and that is what the distributed part in the RDD signifies. In other words, Spark is designed from ground up to have its basic dataset abstraction capable of getting split into smaller pieces deterministically and distributed to more than one nodes in the cluster for parallel processing while elegantly handling the failures in the nodes. Spark RDD is immutable. Once an RDD is created, intentionally or unintentionally, it cannot be changed. This gives another insight into the construction of an RDD. There are some strong rules based on which an RDD is created. Because of that, when the nodes processing some part of an RDD die, the driver program can recreate those parts and assign the task of processing it to another node and ultimately completing the data processing job successfully. Since the RDD is immutable, splitting a big one to smaller ones, distributing them to various worker nodes for processing and finally compiling the results to produce the final result can be done safely without worrying about the underlying data getting changed. Spark RDD is distributable. If Spark is run in a cluster mode where there are multiple worker nodes available to take the tasks, all these nodes are having different execution contexts. The individual tasks are distributed and run on different JVMs. All these activities of a big RDD getting divided into smaller chunks, getting distributed for processing to the worker nodes and finally assembling the results back are completely hidden from the users. Spark has its on mechanism from recovering from the system faults and other forms of errors happening during the data processing.Hence this data abstraction is highly resilient. Spark RDD lives in memory (most of the time). Spark does keep all the RDDs in the memory as much as it can. Only in rare situations where Spark is running out of memory or if the data size is growing beyond the capacity, it is written into disk. Most of the processing on RDD happens in the memory and that is the reason why Spark is able to process the data in a lightning fast speed. Spark RDD is strongly typed. Spark RDD can be created using any supported data types. These data types can be Scala/Java supported intrinsic data types or custom created data types such as your own classes. The biggest advantage coming out of this design decision is the freedom from runtime errors. If it is going to break because of a data type issue, it will break during the compile time. Spark does the data processing using the RDDs. From the relevant data source such as text files, and NoSQL data stores, data is read to form the RDDs. On such an RDD, various data transformations are performed and finally the result is collected. To be precise, Spark comes with Spark Transformations and Spark Actions that act upon RDDs.Whenever a Spark Transformation is done on an RDD, a new RDD gets created. This is because RDDs are inherently immutable. These RDDs that are getting created at the end of each Spark Transformation can be saved for future reference or they will go out of scope eventually. The Spark Actions are used to return the computed values to the driver program. The process of creating one or more RDDs, apply transformations and actions on them is a very common usage pattern seen ubiquitously in Spark applications. Spark SQL Spark SQL is a library built on top of Spark. It exposes SQL interface, and DataFrame API. DataFrame API supports programming languages Scala, Java, Python and R. In programming languages such as R, there is a data frame abstraction used to store data tables in memory. The Python data analysis library named Pandas also has a similar data frame concept. Once that data structure is available in memory, the programs can extract the data, slice and dice the way as per the need. The same data table concept is extended to Spark known as DataFrame built on top of RDD and there is a very comprehensive API known as DataFrame API in Spark SQL to process the data in the DataFrame. An SQL-like query language is also developed on top of the DataFrame abstraction catering to the needs of the end users to query and process the underlying structured data. In summary, DataFrame is a distributed data table organized in rows and columns having names for each column. There is no doubt that SQL is the lingua franca for doing data analysis and Spark SQL is the answer from the Spark family of toolsets to do data analysis. So what it provides? It provides the ability to run SQL on top of Spark. Whether the data is coming from CSV, Avro, Parquet, Hive, NoSQL data stores such as Cassandra, or even RDBMS, Spark SQL can be used to analyze data and mix in with Spark programs. Many of the data sources mentioned here are supported intrinsically by Spark SQL and many others are supported by external packages. The most important aspect to highlight here is the ability of Spark SQL to deal with data from a very wide variety of data sources.Once it is available as a DataFrame in Spark, Spark SQL can process them in a completely distributed way, combine the DataFrames coming from various data sources to process, and query as if the entire dataset is coming from a single source. In the previous section, the RDD was discussed and introduced as the Spark programming model. Are the DataFrames API and the usage of SQL dialects in Spark SQL replacing RDD-based programming model? Definitely not! The RDD-based programming model is the generic and the basic data processing model in Spark. RDD-based programming requires the need to use real programming techniques. The Spark Transformations and Spark Actions use a lot of functional programming constructs. Even though the amount code that is required to be written in RDD-based programming model is less as compared to Hadoop MapReduce or any other paradigm, still there is a need to write some amount of functional code. The is is a barrier to entry enter for many data scientists, data analysts, and business analysts who may perform major exploratory kind of data analysis or doing some prototyping with the data. Spark SQL completely removes those constraints. Simple and easy-to-use domain-specific language (DSL) based methods to read and write data from data sources, SQL-like language to select, filter, aggregate, and capability to read data from a wide variety of data sources makes it easy for anybody who knows the data structure to use it. Which is the best use case to use RDD and which is the best use case to use Spark SQL? The answer is very simple. If the data is structured, it can be arranged in tables, and if each column can be given a name, then use Spark SQL. This doesn't mean that the RDD and DataFrame are two disparate entities. They interoperate very well. Conversions from RDD to DataFrame and vice versa are very much possible. Many of the Spark Transformations and Spark Actions that are typically applied on RDDs can also be applied on DataFrames. Interaction with Spark SQL library is done mainly through two methods. One is through SQL-like queries and the other is through DataFrame API. The Spark programming paradigm has many abstractions to choose from when it comes to developing data processing applications. The fundamentals of Spark programming starts with RDDs that can easily deal with unstructured, semi-structured, and structured data. The Spark SQL library offers highly optimized performance when processing structured data. This makes the basic RDDs look inferior in terms of performance. To fill this gap, from Spark 1.6 onwards, a new abstraction named Dataset was introduced that compliments the RDD-based Spark programming model. It works pretty much the same way as RDD when it comes to Spark Transformations and Spark Actions at the same time it is highly optimized like the Spark SQL. Dataset API provides strong compile-time type safety when it comes to writing programs and because of that the Dataset API is available only in Scala and Java. Too many choices confuses everybody. Here in the Spark programming model also the same problem is seen. But it is not as confusing as in many other programming paradigms. Whenever there is a need to process any kind of data with very high flexibility in terms of the data processing requirements and having the lowest API level control such as library development, RDD-based programming model is ideal. Whenever there is a need to process structured data with flexibility for accessing and processing data with optimized performance across all the supported programming languages, DataFrame-based Spark SQL programming model is ideal. Whenever there is a need to process unstructured data with optimized performance requirements as well as compile-time type safety but not very complex Spark Transformations and Spark Actions usage requirements, dataset-based programming model is ideal. At a data processing application development level, if the programming language of choice permits, it is better to use Dataset and DataFrame to have better performance. R on Spark A base R installation cannot interact with Spark. The SparkR package popularly known as R on Spark exposes all the required objects, and functions for R to talk to the Spark ecosystem. As compared to Scala, Java, and Python, the Spark programming in R is different and the SparkR package mainly exposes R API for DataFrame-based Spark SQL programming. At this moment, R cannot be used to manipulate the RDDs of Spark directly. So for all practical purposes, the R API for Spark has access to only Spark SQL abstractions. How SparkR is going to help the data scientists to do better data processing? The base R installation mandates that all the data to be stored (or accessible) on the computer where R is installed. The data processing happen on the single computer on which the R installation is available. More over if the data size is more than the main memory available on the computer, R will not be able to do the required processing. With SparkR package, there is an access to a whole new world of a cluster of nodes for data storage and for carrying out data processing. With the help of SparkR package, R can be used to access the Spark DataFrames as well as R DataFrames. It is very important to have a distinction of the two types of data frames. R DataFrame is completely local and a data structure of the R language. Spark DataFrame is a parallel collection of structured data managed by the Spark infrastructure. An R DataFrame can be converted to a Spark DataFrame. A Spark DataFrame can be converted to an R DataFrame. When a Spark DataFrame is converted to R DataFrame, it should fit in the available memory of the computer. This conversion is a great feature. By converting an R DataFrame to Spark DataFrame, the data can be distributed and processed in parallel. By converting a Spark DataFrame to an R DataFrame, many computations, charting and plotting that is done by other R functions can be done. In a nutshell, the SparkR package brings in the power of distributed and parallel computing capabilities to R. Many times when doing data processing with R, because of the sheer size of the data and the need to fit it into the main memory of the computer, the data processing is done in multiple batches and the results are consolidated to compute the final results. This kind of multibatch processing can be completely avoided if Spark with R is used to process the data. Many times reporting, charting, and plotting are done on the aggregated and summarized raw data. The raw data size can be huge and need not fit into one computer. In such cases, Spark with R can be used to process the entire raw data and finally the aggregated and summarized data can be used to produce the reports, charts, or plots. Because of the inability to process huge amount of data and for carrying data analysis with R, many times ETL tools are made to use for doing the preprocessing or transformations on the raw data.Only in the final stage the data analysis is done using R. Because of Spark's ability to process data at scale, Spark with R can replace the entire ETL pipeline and do the desired data analysis with R. SparkR package is yet another R package but that is not stopping anybody from using any of the R packages that are already being used. At the same time, it supplements the data processing capability of R manifold by making use of the huge data processing capabilities of Spark. Spark data analysis with Python The ultimate goal of processing data is to use the results for answering business questions. It is very important to understand the data that is being used to answer the business questions. To understand the data better, various tabulation methods, charting and plotting techniques are used. Visual representation of the data reinforces the understanding of the underlying data. Because of this, data visualization is used extensively in data analysis. There are different terms that are being used in various publications to mean the analysis of data for answering business questions. Data analysis, data analytics, business intelligence, and so on, are some of the ubiquitous terms floating around. This section is not going to delve into the discussion on the meaning, similarities or differences of these terms. On the other hand, the focus is going to be on how to bridge the gap of two major activities typically done by data scientists or data analysts. The first one being the data processing. The second one being the use of the processed data to do analysis with the help of charting and plotting. Data analysis is the forte of data analysts and data scientists. This book focuses on the usage of Spark and Python to process the data and produce charts and plots. In many data analysis use cases, a super set of data is processed and the reduced resultant dataset is used for the data analysis. This is specifically valid in the case of big data analysis, where a small set of processed data is used for analysis. Depending on the use case, for various data analysis needs an appropriate data processing is to be done as a prerequisite. Most of the use cases that are going to be covered in this book falls into this model where the first step deals with the necessary data processing and the second step deals with the charting and plotting required for the data analysis. In typical data analysis use cases, the chain of activities involves an extensive and multi staged Extract-Transform-Load (ETL) pipeline ending with a data analysis platform or application. The end result of this chain of activities include but not limited to tables of summary data and various visual representations of the data in the form of charts and plots. Since Spark can process data from heterogeneous distributed data sources very effectively, the huge ETL pipeline that existed in legacy data analysis applications can be consolidated into self contained applications that do the data processing and data analysis. Process data using Spark, analyze using Python Python is a programming language heavily used by the data analysts and data scientists these days. There are numerous scientific and statistical data processing libraries as well as charting and plotting libraries available that can be used in Python programs. It is also a widely used programming language to develop data processing applications in Spark. This brings in a great flexibility to have a unified data processing and data analysis framework with Spark, Python,and Python libraries, enabling to carry out scientific, and statistical processing, charting and plotting. There are numerous such libraries that work with Python. Out of all those, the NumPy and SciPylibraries are being used here to do numerical, statistical, and scientific data processing. The library matplotlib is being used here to carry out charting and plotting that produces 2D images. Processed data is used for data analysis. It requires deep understanding of the processed data. Charts and plots enhance the understanding of the characteristics of the underlying data. In essence, for a data analysis application, data processing, charting and plotting are essential. This book covers the usage of Spark with Python in conjunction with Python charting and plotting libraries for developing data analysis applications. Spark Streaming Data processing use cases can be mainly divided into two types. The first type is the use cases where the data is static and processing is done in its entirety as one unit of work or by dividing that into smaller batches. While doing the data processing, neither the underlying dataset changes nor new datasets get added to the processing units. This is batch processing. The second type is the use cases where the data is getting generated like a stream, and the processing is done as and when the data is generated. This is stream processing. Data sources generate data like a stream and many real-world use cases require them to be processed in a real-time fashion. The meaning of real-time can change from use case to use case. The main parameter that defines what is meant by realtime for a given use case is how soon the ingested data needs to be processed. Or the frequent interval in which all the data ingested since the last interval needs to be processed. For example, when a major sports event is happening, the application that consumes the score events and sending it to the subscribed users should be processing the data as fast as it can. The faster they can be sent, the better it is. But what is the definition of fast here? Is it fine to process the score data say after an hour of the score event happened? Probably not. Is it fine to process the data say after a minute of the score event happened? It is definitely better than processing after an hour. Is it fine to process the data say after a second of the score event happened? Probably yes, and much better than the earlier data processing time intervals. In any data stream processing use cases, this time interval is very important. The data processing framework should have the capability to process the data stream in an appropriate time interval of choice to deliver good business value. When processing stream data in regular intervals of choice, the data is collected from the beginning of the time interval to the end of the time interval, grouped them in a micro batch and data processing is done on that batch of data. Over an extended period of time, the data processing application would have processed many such micro batches of data. In this type of processing, the data processing application will have visibility to only the specific micro batch that is getting processed at a given point in time. In other words, the application will not have any visibility or access to the already processed micro batches of data. Now, there is another dimension to this type of processing. Suppose a given use case mandates the need to process the data every minute, but at the same time, while processing the data of a given micro batch, there is a need to peek into the data that was already processed in the last 15 minutes. A fraud detection module of a retail banking transaction processing application is a good example of this particular business requirement. There is no doubt that the retail banking transactions are to be processed within milliseconds of its occurrence. When processing an ATM cash withdrawal transaction, it is a good idea to see whether somebody is trying to continuously withdraw cash in quick succession and if found, send proper alerting. For this, when processing a given cash withdrawal transaction, check whether there are any other cash withdrawals from the same ATM using the same card happened in the last 15 minutes. The business rule is to alert when such transactions are more than two in the last 15 minutes. In this use case, the fraud detection application should have the visibility to all the transactions happened in a window of 15 minutes. A good stream data processing framework should have the capability to process the data in any given interval of time with ability to peek into the data ingested within a sliding window of time. The Spark Streaming library that is working on top of Spark is one of the best data stream processing framework that has both of these capabilities. Spark machine learning Calculations based on formulae or algorithms were very common since ancient times to find the output for a given input. But without knowing the formulae or algorithms, computer scientists and mathematicians devised methods to generate formulae or algorithms based on an existing set of input, output dataset and predict the output of a new input data based on the generated formulae or algorithms. Generally, this process of 'learning' from a dataset and doing predictions based on the 'learning' is known as Machine Learning. It has its origin from the study of artificial intelligence in computer science. Practical machine learning has numerous applications that are being consumed by the laymen on a daily basis. YouTube users now get suggestions for the next items to be played in the playlist based on the video they are currently viewing. Popular movie rating sites are giving ratings and recommendations based on the user preferences. Social media websites, such as Facebook, suggest a list of names of the users' friends for easy tagging of pictures. What Facebook is doing here is that, it is classifying the pictures by name that is already available in the albums and checking whether the newly added picture has any similarity with the existing ones. If it finds a similarity, it suggests the name.The applications of this kind of picture identification are many. The way all these applications are working is based on the huge amount of input, output dataset that is already collected and the learning done based on that dataset. When a new input dataset comes, a prediction is made by making use of the 'learning' that the computer or machine already did. In traditional computing, input data is fed to a program to generate output. But in machine learning, input data and output data are fed to a machine learning algorithm to generate a function or program that can be used to predict the output of an input according to the 'learning' done on the input, output dataset fed to the machine learning algorithm. The data available in the wild may be classified into groups, or it may form clusters or may fit into certain relationships. These are different kinds of machine learning problems. For example, if there is a databank of preowned car sale prices with its associated attributes or features, it is possible to predict the fair price of a car just by knowing the associated attributes or features. Regression algorithms are used to solve these kinds of problems. If there is a databank of spam and non-spam e-mails, then when a new mail comes, it is possible to predict whether the new mail is a spam or non-spam.Classification algorithms are used to solve these kind of problems. These are just a few machine learning algorithm types. But in general, when using a bank of data, if there is a need to apply a machine learning algorithm and using that model predictions are to be done, then the data should be divided into features and outputs. So whichever may be the machine learning algorithm that is being used, there will be a set of features and one or more output(s). Many books and publications use the term label for output. In other words, features are the input and label is the output. Data comes in various shapes and forms. Depending on the machine learning algorithm used, the training data has to be preprocessed to have the features and labels in the right format to be fed to the machine learning algorithm. That in turn generates the appropriate hypothesis function, which takes the features as the input and produces the predicted label. Why Spark for machine learning? Spark Machine learning library uses many Spark core functionalities as well as the Spark libraries such as Spark SQL. The Spark machine learning library makes the machine learning application development easy by combining data processing and machine learning algorithm implementations in a unified framework with ability to do data processing on a cluster of nodes combined with ability to read and write data to a variety of data formats. Spark comes with two flavors of the machine learning library. They are spark.mllib and spark.ml. The first one is developed on top of Spark's RDD abstraction and the second one is developed on top of Spark's DataFrame abstraction. It is recommended to use the spark.ml library for any future machine learning application developments. Spark graph processing Graph is a mathematical concept and a data structure in computer science. It has huge applications in many real-world use cases. It is used to model pair-wise relationship between entities. The entity here is known as Vertex and two vertices are connected by an Edge. A graph comprises of a collection of vertices and edges connecting them. Conceptually, it is a deceptively simple abstraction but when it comes to processing a huge number of vertices and edges, it is computationally intensive and consumes lots of processing time and computing resources. There are numerous application constructs that can be modeled as graph. In a social networking application, the relationship between users can be modeled as a graph in which the users form the vertices of the graph and the the relationship between users form the edges of the graph. In a multistage job scheduling application, the individual tasks form the vertices of the graph and the sequencing of the tasks forms the edges. In a road traffic modeling system, the towns form the vertices of the graph and the roads connecting the towns form the edges. The edges of a given graph have a very important property, namely, the direction of the connection. In many use cases, the direction of connection doesn't matter. In the case of connectivity between the cities by roads is one such example. But if the use case is to produce driving directions within a city, the connectivity between traffic-junctions has a direction. Take any two traffic-junctions and there will be a road connectivity, but it is possible that it is a oneway. So, it depends on in which direction the traffic is flowing. If the road is open for traffic from traffic-junction J1 to J2 but closed from J2 to J1, then the graph of driving directions will have a connectivity from J1 to J2 and not from J2 to J1. In such cases, the edge connecting J1 and J2 has a direction. If the traffic between J2 and J3 are open in both ways, then the the edge connecting J2 and J3 has no direction. A graph with all the edges having direction is called a directed graph. For graph processing, so many libraries are available in the open source world itself. Giraph, Pregel, GraphLab, and Spark GraphX are some of them. The Spark GraphX is one of the recent entrants into this space. What is so special about Spark GraphX? It is a graph processing library built on top of the Spark data processing framework. Compared to the other graph processing libraries, Spark GraphX has a real advantage. It can make use of all the data processing capabilities of Spark. In reality, the performance of graph processing algorithms is not the only one aspect that needs consideration. In many of the applications, the data that needs to be modeled as graph does not exist in that form naturally. In many use cases more than the graph processing, lot of processor time and other computing resources are expended to get the data in the right format so that the graph processing algorithms can be applied. This is the sweet spot where the combination of Spark data processing framework and Spark GraphX library delivering its most value. The data processing jobs to make the data ready to be consumed by the Spark GraphX can be easily done using the plethora of tools available in the Spark toolkit. In summary, the Spark GraphX library, which is part of the Spark family combines the power of the core data processing capabilities of Spark and a very easy to use graph processing library. The biggest limitation of Spark GraphX library is that its API is not currently supported with programming languages such as Python and R. But there is an external Spark package named GraphFrames that solves this limitation. Since GraphFrames is a DataFrame-based library, once it is matured, it will enable the graph processing in all the programming languages supported by DataFrames. This Spark external package is definitely a potential candidate to be included as part of the Spark itself. Summary Any technology learned or taught has to be concluded with an application developed covering its salient features. Spark is no different. This book, accomplishes an end-to-end application developed using Lambda Architecture using Spark as the data processing platform and its family of libraries built on top of it. Resources for Article: Further resources on this subject: Setting up Spark [article] Machine Learning Using Spark MLlib [article] Holistic View on Spark [article]
Read more
  • 0
  • 0
  • 2463

article-image-basics-image-histograms-opencv
Packt
12 Oct 2016
11 min read
Save for later

Basics of Image Histograms in OpenCV

Packt
12 Oct 2016
11 min read
In this article by Samyak Datta, author of the book Learning OpenCV 3 Application Development we are going to focus our attention on a different style of processing pixel values. The output of the techniques, which would comprise our study in the current article, will not be images, but other forms of representation for images, namely image histograms. We have seen that a two-dimensional grid of intensity values is one of the default forms of representing images in digital systems for processing as well as storage. However, such representations are not at all easy to scale. So, for an image with a reasonably low spatial resolution, say 512 x 512 pixels, working with a two-dimensional grid might not pose any serious issues. However, as the dimensions increase, the corresponding increase in the size of the grid may start to adversely affect the performance of the algorithms that work with the images. A primary advantage that an image histogram has to offer is that the size of a histogram is a constant that is independent of the dimensions of the image. As a consequence of this, we are guaranteed that irrespective of the spatial resolution of the images that we are dealing with, the algorithms that power our solutions will have to deal with a constant amount of data if they are working with image histograms. (For more resources related to this topic, see here.) Each descriptor captures some particular aspects or features of the image to construct its own form of representation. One of the common pitfalls of using histograms as a form of image representation as compared to its native form of using the entire two-dimensional grid of values is loss of information. A full-fledged image representation using pixel intensity values for all pixel locations naturally consists of all the information that you would need to reconstruct a digital image. However, the same cannot be said about histograms. When we study about image histograms in detail, we'll get to see exactly what information do we stand to lose. And this loss in information is prevalent across all forms of image descriptors. The basics of histograms At the outset, we will briefly explain the concept of a histogram. Most of you might already know this from your lessons on basic statistics. However, we will reiterate this for the sake of completeness. Histogram is a form of data representation technique that relies on an aggregation of data points. The data is aggregated into a set of predefined bins that are represented along the x axis, and the number of data points that fall within each of the bins make up the corresponding counts on the y axis. For example, let's assume that our data looks something like the following: D={2,7,1,5,6,9,14,11,8,10,13} If we define three bins, namely Bin_1 (1 - 5), Bin_2 (6 - 10), and Bin_3 (11 - 15), then the histogram corresponding to our data would look something like this: Bins Frequency Bin_1 (1 - 5) 3 Bin_2 (6 - 10) 5 Bin_3 (11 - 15) 3 What this histogram data tells us is that we have three values between 1 and 5, five between 6 and 10, and three again between 11 and 15. Note that it doesn't tell us what the values are, just that some n values exist in a given bin. A more familiar visual representation of the histogram in discussion is shown as follows: As you can see, the bins have been plotted along the x axis and their corresponding frequencies along the y axis. Now, in the context of images, how is a histogram computed? Well, it's not that difficult to deduce. Since the data that we have comprise pixel intensity values, an image histogram is computed by plotting a histogram using the intensity values of all its constituent pixels. What this essentially means is that the sequence of pixel intensity values in our image becomes the data. Well, this is in fact the simplest kind of histogram that you can compute using the information available to you from the image. Now, coming back to image histograms, there are some basic terminologies (pertaining to histograms in general) that you need to be aware of before you can dip your hands into code. We have explained them in detail here: Histogram size: The histogram size refers to the number of bins in the histogram. Range: The range of a histogram is the range of data that we are dealing with. The range of data as well as the histogram size are both important parameters that define a histogram. Dimensions: Simply put, dimensions refer to the number of the type of items whose values we aggregate in the histogram bins. For example, consider a grayscale image. We might want to construct a histogram using the pixel intensity values for such an image. This would be an example of a single-dimensional histogram because we are just interested in aggregating the pixel intensity values and nothing else. The data, in this case, is spread over a range of 0 to 255. On account of being one-dimensional, such histograms can be represented graphically as 2D plots—one-dimensional data (pixel intensity values) being plotted on the x axis (in the form of bins) along with the corresponding frequency counts along the y axis. We have already seen an example of this before. Now, imagine a color image with three channels: red, green, and blue. Let's say that we want to plot a histogram for the intensities in the red and green channels combined. This means that our data now becomes a pair of values (r, g). A histogram that is plotted for such data will have a dimensionality of 2. The plot for such a histogram will be a 3D plot with the data bins covering the x and y axes and the frequency counts plotted along the z axis. Now that we have discussed the theoretical aspects of image histograms in detail, let's start thinking along the lines of code. We will start with the simplest (and in fact the most ubiquitous) design of image histograms. The range of our data will be from 0 to 255 (both inclusive), which means that all our data points will be integers that fall within the specified range. Also, the number of data points will equal the number of pixels that make up our input image. The simplicity in design comes from the fact that we fix the size of the histogram (the number of bins) as 256. Now, take a moment to think about what this means. There are 256 different possible values that our data points can take and we have a separate bin corresponding to each one of those values. So such an image histogram will essentially depict the 256 possible intensity values along with the counts of the number of pixels in the image that are colored with each of the different intensities. Before taking a peek at what OpenCV has to offer, let's try to implement such a histogram on our own! We define a function named computeHistogram() that takes the grayscale image as an input argument and returns the image histogram. From our earlier discussions, it is evident that the histogram must contain 256 entries (for the 256 bins): one for each integer between 0 and 255. The value stored in the histogram corresponding to each of the 256 entries will be the count of the image pixels that have a particular intensity value. So, conceptually, we can use an array for our implementation such that the value stored in the histogram [ i ] (for 0≤i≤255) will be the count of the number of pixels in the image having the intensity of i. However, instead of using a C++ array, we will comply with the rules and standards followed by OpenCV and represent the histogram as a Mat object. We have already seen that a Mat object is nothing but a multidimensional array store. The implementation is outlined in the following code snippet: Mat computeHistogram(Mat input_image) { Mat histogram = Mat::zeros(256, 1, CV_32S); for (int i = 0; i < input_image.rows; ++i) { for (int j = 0; j < input_image.cols; ++j) { int binIdx = (int) input_image.at<uchar>(i, j); histogram.at<int>(binIdx, 0) += 1; } } return histogram; } As you can see, we have chosen to represent the histogram as a 256-element-column-vector Mat object. We iterate over all the pixels in the input image and keep on incrementing the corresponding counts in the histogram (which had been initialized to 0). As per our description of the image histogram properties, it is easy to see that the intensity value of any pixel is the same as the bin index that is used to index into the appropriate histogram bin to increment the count. Having such an implementation ready, let's test it out with the help of an actual image. The following code demonstrates a main() function that reads an input image, calls the computeHistogram() function that we have defined just now, and displays the contents of the histogram that is returned as a result: int main() { Mat input_image = imread("/home/samyak/Pictures/lena.jpg", IMREAD_GRAYSCALE); Mat histogram = computeHistogram(input_image); cout << "Histogram...n"; for (int i = 0; i < histogram.rows; ++i) cout << i << " : " << histogram.at<int>(i, 0) << "n"; return 0; } We have used the fact that the histogram that is returned from the function will be a single column Mat object. This makes the code that displays the contents of the histogram much cleaner. Histograms in OpenCV We have just seen the implementation of a very basic and minimalistic histogram using the first principles in OpenCV. The image histogram was basic in the sense that all the bins were uniform in size and comprised only a single pixel intensity. This made our lives simple when we designed our code for the implementation; there wasn't any need to explicitly check the membership of a data point (the intensity value of a pixel) with all the bins of our histograms. However, we know that a histogram can have bins whose sizes span more than one. Can you think of the changes that we might need to make in the code that we had written just now to accommodate for bin sizes larger than 1? If this change seems doable to you, try to figure out how to incorporate the possibility of non-uniform bin sizes or multidimensional histograms. By now, things might have started to get a little overwhelming to you. No need to worry. As always, OpenCV has you covered! The developers at OpenCV have provided you with a calcHist() function whose sole purpose is to calculate the histograms for a given set of arrays. By arrays, we refer to the images represented as Mat objects, and we use the term set because the function has the capability to compute multidimensional histograms from the given data: Mat computeHistogram(Mat input_image) { Mat histogram; int channels[] = { 0 }; int histSize[] = { 256 }; float range[] = { 0, 256 }; const float* ranges[] = { range }; calcHist(&input_image, 1, channels, Mat(), histogram, 1, histSize, ranges, true, false); return histogram; } Before we move on to an explanation of the different parameters involved in the calcHist() function call, I want to bring your attention to the abundant use of arrays in the preceding code snippet. Even arguments as simple as histogram sizes are passed to the function in the form of arrays rather than integer values, which at first glance seem quite unnecessary and counter-intuitive. The usage of arrays is due to the fact that the implementation of calcHist() is equipped to handle multidimensional histograms as well, and when we are dealing with such multidimensional histogram data, we require multiple parameters to be passed, one for each dimension. This would become clearer once we demonstrate an example of calculating multidimensional histograms using the calcHist() function. For the time being, we just wanted to clear the immediate confusion that might have popped up in your minds upon seeing the array parameters. Here is a detailed list of the arguments in the calcHist() function call: Source images Number of source images Channel indices Mask Dimensions (dims) Histogram size Ranges Uniform flag Accumulate flag The last couple of arguments (the uniform and accumulate flags) have default values of true and false, respectively. Hence, the function call that you have seen just now can very well be written as follows: calcHist(&input_image, 1, channels, Mat(), histogram, 1, histSize, ranges); Summary Thus in this article we have successfully studied fundamentals of using histograms in OpenCV for image processing. Resources for Article: Further resources on this subject: Remote Sensing and Histogram [article] OpenCV: Image Processing using Morphological Filters [article] Learn computer vision applications in Open CV [article]
Read more
  • 0
  • 0
  • 22139
article-image-moving-windows-appliance
Packt
12 Oct 2016
7 min read
Save for later

Moving from Windows to Appliance

Packt
12 Oct 2016
7 min read
In this article by Daniel Langenhan, author of VMware vRealize Orchestrator Cookbook, Second Edition, will show you how to move an existing Windows Orchestrator installation to the appliance. With vRO 7 the Windows install of Orchestrator doesn't exist anymore. (For more resources related to this topic, see here.) Getting ready We need an Orchestrator installed on windows. Download the same version of the Orchestrator appliance as you have installed in the Windows version. If needed upgrade the Windows version to the latest possible one. How to do it... There are three ways, using the migration tool, repointing to an external database or export/import the packages. Migration tool There is a migration tool that comes with vRO7 that allows you to pack up your vRO5.5 or 6.x install and deploy it into a vRO7. The migration tool works on Windows and Linux. It collects the configuration, the plug-ins as well as their configuration certificates, and licensing into a file. Follow these steps to use the migration tool: Deploy a new vRO7 Appliance. Log in to your Windows Orchestrator OS. Stop the VMware vCenter Orchestrator Service (Windows services). Open a Web browser and log in to your new vRO7 - control center and then go to Export/Import Configuration. Select Migrate Configuration and click on the here link. The link points to: https://[vRO7]:8283/vco-controlcenter/api/server/migration-tool . Stop the vRO7 Orchestrator service. Unzip the migration-tool.zip and copy the subfolder called migration‑cli into the Orchestrator director, for example, C:Program FilesVMwareInfrastructureOrchestratormigration-clibin. Open a command prompt. If you have a Java install, make sure your path points to it. Try java ‑version. If that works continue, if not, do the following: Set the PATH environment variable to the Java install that comes with Orchestrator, set PATH=%PATH%;C:Program FilesVMwareInfrastructureOrchestratorUninstall_vCenter Orchestratoruninstall-jrebin CD to the directory ..Orchestratormigration-clibin. Execute the command vro-migrate.bat export. There may be errors showing about SLF4J, you can ignore those. In the main directory (..Orchestrator) you should now find an orchestrator‑config‑export‑VC55‑[date].zip file. Go back to the Web browser and upload the ZIP file into Migration Configuration by clicking on Browse and select the file. Click on Import. You now see what can be imported. You can unselect item you wish not to migrate. Click Finish Migration. Restart the Orchestrator service. Check the settings. External database If you have an external database things are pretty easy. For using the initial internal database please see the additional steps in the There's more section of this recipe. Backup the external database. Connect to the Windows Orchestrator Configurator. Write down all the plugins you have installed as well as their version. Shutdown the Windows version and deploy the Appliance, this way you can use the same IP and hostname if you want. Login to the Appliance version's Configurator. Stop the Orchestrator service Install all plugins you had in the Windows version. Attach the external database. Make sure that all trusted SSL certificates are still there, such as vCenter and SSO. Check the authentication if it is still working. Use the test login. Check your licensing. Force a plugin reinstall (Troubleshooting | Reinstall the plug-ins when the server starts). Start the Orchestrator service and try to log in. Make a complete sanity check. Package transfer This is the method that will only pull your packages across. This the only easy method to use when you are transitioning between different databases, such as between MS SQL and PostgreSQL. Connect to your Windows version Create a package of all the workflows, action, and other items you need. Shutdown Windows and deploy the Appliance. Configure the Appliance with DB, Authentication and all the plugins you previously had. Import the package. How it works... Moving from the Windows version of Orchestrator to the Appliance version isn't such a big thing. Worst case scenario is using the packaging transfer. The only really important thing is to use the same version of the Windows Orchestrator as the Appliance version. You can download a lot of old versions including 5.5 from http://www.vmware.com/in.html. If you can't find the same version, upgrade your existing vCenter Orchestrator to one you can download. After you transferred the data to the appliance you need to make sure that everything works correctly and then you can upgrade to vRO7. There's more... When you just run Orchestrator from your Windows vCenter installation and didn't configure an external database then Orchestrator uses the vCenter database and mixes the Orchestrator tables with the vCenter tables. In order to only export the Orchestrator ones, we will use the MS SQL Server Management Studio (free download from www.microsoft.com called Microsoft SQL Server 2008 R2 RTM). To transfer the only the Orchestrator database tables from the vCenter MS-SQL to an external SQL do the following: Stop the VMware vCenter Orchestrator Service (Windows Services) on your Windows Orchestrator. Start the SQL Server Management Studio on your external SQL server. Connect to the vCenter DB. For SQL Express use: [vcenter]VIM_SQLEXP with Windows Authentication. Right-click on your vCenter Database (SQL Express: VIM_VCDB) and select Tasks | Export Data. In the wizard, select your source, which should be the correct one already and click Next. Choose SQL Server Native Client 10.0 and enter the name of your new SQL server. Click on New to create a new database on that SQL server (or use an empty one you created already. Click Next. Select Copy data from one or more tables or views and click Next. Now select every database which starts with VMO_ and then click Next. Select Run immediately and click Finish. Now you have the Orchestrator database extracted as an external database. You still need to configure a user and rights. Then proceed with the section External database in this recipe. Orchestrator client and 4K display scaling This recipe shows a hack how to make the Orchestrator client scale on 4K displays. Getting ready We need to download the program Resource Tuner (http://www.restuner.com/) the trial version will work, however, consider buying it if it works for you. You need to know the path to your Java install, this should be something like: C:Program Files (x86)Javajre1.x.xxbin How to do it... Before you start…. Please be careful as this impacts your whole Java environment. This worked for me very well with Java 1.8.0_91-b14. Download and install Resource Tuner. Run Resource Tuner as administrator. Open the file javaws.exe in your Java directory. Expand manifest and the click on the first entry (the name can change due to localization). Look for the line <dpiAware>true</dpiAware>. Exchange the true for a false Save end exit. Repeat the same for all the other java*.exe in the same directory as well as j2launcher.exe. Start the Client.jnlp (the file that downloads when you start the web application). How it works... In Windows 10 you can set the scaling of applications when you are using high definition monitors (4K displays). What you are doing is telling Java that it is not DPI aware, meaning it the will use the Windows 10 default scaler, instead of an internal scaler. There's more... For any other application such as Snagit or Photoshop I found that this solution works quite well: http://www.danantonielli.com/adobe-app-scaling-on-high-dpi-displays-fix/. Summary In this article we discussed about moving an existing Windows Orchestrator installation to the appliance and a hack on how to make the Orchestrator client scale on 4K displays. Resources for Article: Further resources on this subject: Working with VMware Infrastructure [article] vRealize Automation and the Deconstruction of Components [article] FAQ on Virtualization and Microsoft App-V [article]
Read more
  • 0
  • 0
  • 7598

article-image-asynchronous-programming-f
Packt
12 Oct 2016
15 min read
Save for later

Asynchronous Programming in F#

Packt
12 Oct 2016
15 min read
In this article by Alfonso Garcia Caro Nunez and Suhaib Fahad, author of the book Mastering F#, sheds light on how writing applications that are non-blocking or reacting to events is increasingly becoming important in this cloud world we live in. A modern application needs to carry out a rich user interaction, communicate with web services, react to notifications, and so on; the execution of reactive applications is controlled by events. Asynchronous programming is characterized by many simultaneously pending reactions to internal or external events. These reactions may or may not be processed in parallel. (For more resources related to this topic, see here.) In .NET, both C# and F# provide asynchronous programming experience through keywords and syntaxes. In this article, we will go through the asynchronous programming model in F#, with a bit of cross-referencing or comparison drawn with the C# world. In this article, you will learn about asynchronous workflows in F# Asynchronous workflows in F# Asynchronous workflows are computation expressions that are setup to run asynchronously. It means that the system runs without blocking the current computation thread when a sleep, I/O, or other asynchronous process is performed. You may be wondering why do we need asynchronous programming and why can't we just use the threading concepts that we did for so long. The problem with threads is that the operation occupies the thread for the entire time that something happens or when a computation is done. On the other hand, asynchronous programming will enable a thread only when it is required, otherwise it will be normal code. There is also lot of marshalling and unmarshalling (junk) code that we will write around to overcome the issues that we face when directly dealing with threads. Thus, asynchronous model allows the code to execute efficiently whether we are downloading a page 50 or 100 times using a single thread or if we are doing some I/O operation over the network and there are a lot of incoming requests from the other endpoint. There is a list of functions that the Async module in F# exposes to create or use these asynchronous workflows to program. The asynchronous pattern allows writing code that looks like it is written for a single-threaded program, but in the internals, it uses async blocks to execute. There are various triggering functions that provide a wide variety of ways to create the asynchronous workflow, which is either a background thread, a .NET framework task object, or running the computation in the current thread itself. In this article, we will use the example of downloading the content of a webpage and modifying the data, which is as follows: let downloadPage (url: string) = async { let req = HttpWebRequest.Create(url) use! resp = req.AsyncGetResponse() use respStream = resp.GetResponseStream() use sr = new StreamReader(respStream) return sr.ReadToEnd() } downloadPage("https://www.google.com") |> Async.RunSynchronously The preceding function does the following: The async expression, { … }, generates an object of type Async<string> These values are not actual results; rather, they are specifications of tasks that need to run and return a string Async.RunSynchronously takes this object and runs synchronously We just wrote a simple function with asynchronous workflows with relative ease and reason about the code, which is much better than using code with Begin/End routines. One of the most important point here is that the code is never blocked during the execution of the asynchronous workflow. This means that we can, in principle, have thousands of outstanding web requests—the limit being the number supported by the machine, not the number of threads that host them. Using let! In asynchronous workflows, we will use let! binding to enable execution to continue on other computations or threads, while the computation is being performed. After the execution is complete, the rest of the asynchronous workflow is executed, thus simulating a sequential execution in an asynchronous way. In addition to let!, we can also use use! to perform asynchronous bindings; basically, with use!, the object gets disposed when it loses the current scope. In our previous example, we used use! to get the HttpWebResponse object. We can also do as follows: let! resp = req.AsyncGetResponse() // process response We are using let! to start an operation and bind the result to a value, do!, which is used when the return of the async expression is a unit. do! Async.Sleep(1000) Understanding asynchronous workflows As explained earlier, asynchronous workflows are nothing but computation expressions with asynchronous patterns. It basically implements the Bind/Return pattern to implement the inner workings. This means that the let! expression is translated into a call to async. The bind and async.Return function are defined in the Async module in the F# library. This is a compiler functionality to translate the let! expression into computation workflows and, you, as a developer, will never be required to understand this in detail. The purpose of explaining this piece is to understand the internal workings of an asynchronous workflow, which is nothing but a computation expression. The following listing shows the translated version of the downloadPage function we defined earlier: async.Delay(fun() -> let req = HttpWebRequest.Create(url) async.Bind(req.AsyncGetResponse(), fun resp -> async.Using(resp, fun resp -> let respStream = resp.GetResponseStream() async.Using(new StreamReader(respStream), fun sr " -> reader.ReadToEnd() ) ) ) ) The following things are happening in the workflow: The Delay function has a deferred lambda that executes later. The body of the lambda creates an HttpWebRequest and is forwarded in a variable req to the next segment in the workflow. The AsyncGetResponse function is called and a workflow is generated, where it knows how to execute the response and invoke when the operation is completed. This happens internally with the BeginGetResponse and EndGetResponse functions already present in the HttpWebRequest class; the AsyncGetResponse is just a wrapper extension present in the F# Async module. The Using function then creates a closure to dispose the object with the IDisposable interface once the workflow is complete. Async module The Async module has a list of functions that allows writing or consuming asynchronous code. We will go through each function in detail with an example to understand it better. Async.AsBeginEnd It is very useful to expose the F# workflow functionality out of F#, say if we want to use and consume the API's in C#. The Async.AsBeginEnd method gives the possibility of exposing the asynchronous workflows as a triple of methods—Begin/End/Cancel—following the .NET Asynchronous Programming Model (APM). Based on our downloadPage function, we can define the Begin, End, Cancel functions as follows: type Downloader() = let beginMethod, endMethod, cancelMethod = " Async.AsBeginEnd downloadPage member this.BeginDownload(url, callback, state : obj) = " beginMethod(url, callback, state) member this.EndDownload(ar) = endMethod ar member this.CancelDownload(ar) = cancelMethod(ar) Async.AwaitEvent The Async.AwaitEvent method creates an asynchronous computation that waits for a single invocation of a .NET framework event by adding a handler to the event. type MyEvent(v : string) = inherit EventArgs() member this.Value = v; let testAwaitEvent (evt : IEvent<MyEvent>) = async { printfn "Before waiting" let! r = Async.AwaitEvent evt printfn "After waiting: %O" r.Value do! Async.Sleep(1000) return () } let runAwaitEventTest () = let evt = new Event<Handler<MyEvent>, _>() Async.Start <| testAwaitEvent evt.Publish System.Threading.Thread.Sleep(3000) printfn "Before raising" evt.Trigger(null, new MyEvent("value")) printfn "After raising" > runAwaitEventTest();; > Before waiting > Before raising > After raising > After waiting : value The testAwaitEvent function listens to the event using Async.AwaitEvent and prints the value. As the Async.Start will take some time to start up the thread, we will simply call Thread.Sleep to wait on the main thread. This is for example purpose only. We can think of scenarios where a button-click event is awaited and used inside an async block. Async.AwaitIAsyncResult Creates a computation result and waits for the IAsyncResult to complete. IAsyncResult is the asynchronous programming model interface that allows us to write asynchronous programs. It returns true if IAsyncResult issues a signal within the given timeout. The timeout parameter is optional, and its default value is -1 of Timeout.Infinite. let testAwaitIAsyncResult (url: string) = async { let req = HttpWebRequest.Create(url) let aResp = req.BeginGetResponse(null, null) let! asyncResp = Async.AwaitIAsyncResult(aResp, 1000) if asyncResp then let resp = req.EndGetResponse(aResp) use respStream = resp.GetResponseStream() use sr = new StreamReader(respStream) return sr.ReadToEnd() else return "" } > Async.RunSynchronously (testAwaitIAsyncResult "https://www.google.com") We will modify the downloadPage example with AwaitIAsyncResult, which allows a bit more flexibility where we want to add timeouts as well. In the preceding example, the AwaitIAsyncResult handle will wait for 1000 milliseconds, and then it will execute the next steps. Async.AwaitWaitHandle Creates a computation that waits on a WaitHandle—wait handles are a mechanism to control the execution of threads. The following is an example with ManualResetEvent: let testAwaitWaitHandle waitHandle = async { printfn "Before waiting" let! r = Async.AwaitWaitHandle waitHandle printfn "After waiting" } let runTestAwaitWaitHandle () = let event = new System.Threading.ManualResetEvent(false) Async.Start <| testAwaitWaitHandle event System.Threading.Thread.Sleep(3000) printfn "Before raising" event.Set() |> ignore printfn "After raising" The preceding example uses ManualResetEvent to show how to use AwaitHandle, which is very similar to the event example that we saw in the previous topic. Async.AwaitTask Returns an asynchronous computation that waits for the given task to complete and returns its result. This helps in consuming C# APIs that exposes task based asynchronous operations. let downloadPageAsTask (url: string) = async { let req = HttpWebRequest.Create(url) use! resp = req.AsyncGetResponse() use respStream = resp.GetResponseStream() use sr = new StreamReader(respStream) return sr.ReadToEnd() } |> Async.StartAsTask let testAwaitTask (t: Task<string>) = async { let! r = Async.AwaitTask t return r } > downloadPageAsTask "https://www.google.com" |> testAwaitTask |> Async.RunSynchronously;; The preceding function is also downloading the web page as HTML content, but it starts the operation as a .NET task object. Async.FromBeginEnd The FromBeginEnd method acts as an adapter for the asynchronous workflow interface by wrapping the provided Begin/End method. Thus, it allows using large number of existing components that support an asynchronous mode of work. The IAsyncResult interface exposes the functions as Begin/End pattern for asynchronous programming. We will look at the same download page example using FromBeginEnd: let downloadPageBeginEnd (url: string) = async { let req = HttpWebRequest.Create(url) use! resp = Async.FromBeginEnd(req.BeginGetResponse, req.EndGetResponse) use respStream = resp.GetResponseStream() use sr = new StreamReader(respStream) return sr.ReadToEnd() } The function accepts two parameters and automatically identifies the return type; we will use BeginGetResponse and EndGetResponse as our functions to call. Internally, Async.FromBeginEnd delegates the asynchronous operation and gets back the handle once the EndGetResponse is called. Async.FromContinuations Creates an asynchronous computation that captures the current success, exception, and cancellation continuations. To understand these three operations, let's create a sleep function similar to Async.Sleep using timer: let sleep t = Async.FromContinuations(fun (cont, erFun, _) -> let rec timer = new Timer(TimerCallback(callback)) and callback state = timer.Dispose() cont(()) timer.Change(t, Timeout.Infinite) |> ignore ) let testSleep = async { printfn "Before" do! sleep 5000 printfn "After 5000 msecs" } Async.RunSynchronously testSleep The sleep function takes an integer and returns a unit; it uses Async.FromContinuations to allow the flow of the program to continue when a timer event is raised. It does so by calling the cont(()) function, which is a continuation to allow the next step in the asynchronous flow to execute. If there is any error, we can call erFun to throw the exception and it will be handled from the place we are calling this function. Using the FromContinuation function helps us wrap and expose functionality as async, which can be used inside asynchronous workflows. It also helps to control the execution of the programming with cancelling or throwing errors using simple APIs. Async.Start Starts the asynchronous computation in the thread pool. It accepts an Async<unit> function to start the asynchronous computation. The downloadPage function can be started as follows: let asyncDownloadPage(url) = async { let! result = downloadPage(url) printfn "%s" result"} asyncDownloadPage "http://www.google.com" |> Async.Start   We wrap the function to another async function that returns an Async<unit> function so that it can be called by Async.Start. Async.StartChild Starts a child computation within an asynchronous workflow. This allows multiple asynchronous computations to be executed simultaneously, as follows: let subTask v = async { print "Task %d started" v Thread.Sleep (v * 1000) print "Task %d finished" v return v } let mainTask = async { print "Main task started" let! childTask1 = Async.StartChild (subTask 1) let! childTask2 = Async.StartChild (subTask 5) print "Subtasks started" let! child1Result = childTask1 print "Subtask1 result: %d" child1Result let! child2Result = childTask2 print "Subtask2 result: %d" child2Result print "Subtasks completed" return () } Async.RunSynchronously mainTask Async.StartAsTask Executes a computation in the thread pool and returns a task that will be completed in the corresponding state once the computation terminates. We can use the same example of starting the downloadPage function as a task. let downloadPageAsTask (url: string) = async { let req = HttpWebRequest.Create(url) use! resp = req.AsyncGetResponse() use respStream = resp.GetResponseStream() use sr = new StreamReader(respStream) return sr.ReadToEnd() } |> Async.StartAsTask let task = downloadPageAsTask("http://www.google.com") prinfn "Do some work" task.Wait() printfn "done"   Async.StartChildAsTask Creates an asynchronous computation from within an asynchronous computation, which starts the given computation as a task. let testAwaitTask = async { print "Starting" let! child = Async.StartChildAsTask <| async { // " Async.StartChildAsTask shall be described later print "Child started" Thread.Sleep(5000) print "Child finished" return 100 } print "Waiting for the child task" let! result = Async.AwaitTask child print "Child result %d" result } Async.StartImmediate Runs an asynchronous computation, starting immediately on the current operating system thread. This is very similar to the Async.Start function we saw earlier: let asyncDownloadPage(url) = async { let! result = downloadPage(url) printfn "%s" result"} asyncDownloadPage "http://www.google.com" |> Async.StartImmediate Async.SwitchToNewThread Creates an asynchronous computation that creates a new thread and runs its continuation in it: let asyncDownloadPage(url) = async { do! Async.SwitchToNewThread() let! result = downloadPage(url) printfn "%s" result"} asyncDownloadPage "http://www.google.com" |> Async.Start   Async.SwitchToThreadPool Creates an asynchronous computation that queues a work item that runs its continuation, as follows: let asyncDownloadPage(url) = async { do! Async.SwitchToNewThread() let! result = downloadPage(url) do! Async.SwitchToThreadPool() printfn "%s" result"} asyncDownloadPage "http://www.google.com" |> Async.Start   Async.SwitchToContext Creates an asynchronous computation that runs its continuation in the Post method of the synchronization context. Let's assume that we set the text from the downloadPage function to a UI textbox, then we will do it as follows: let syncContext = System.Threading.SynchronizationContext() let asyncDownloadPage(url) = async { do! Async.SwitchToContext(syncContext) let! result = downloadPage(url) textbox.Text <- result"} asyncDownloadPage "http://www.google.com" |> Async.Start   Note that in the console applications, the context will be null. Async.Parallel The Parallel function allows you to execute individual asynchronous computations queued in the thread pool and uses the fork/join pattern. let parallel_download() = let sites = ["http://www.bing.com"; "http://www.google.com"; "http://www.yahoo.com"; "http://www.search.com"] let htmlOfSites = Async.Parallel [for site in sites -> downloadPage site ] |> Async.RunSynchronously printfn "%A" htmlOfSites   We will use the same example of downloading HTML content in a parallel way. The preceding example shows the essence of parallel I/O computation The async function, { … }, in the downloadPage function shows the asynchronous computation These are then composed in parallel using the fork/join combinator In this sample, the composition executed waits synchronously for overall result Async.OnCancel A cancellation interruption in the asynchronous computation when a cancellation occurs. It returns an asynchronous computation trigger before being disposed. // This is a simulated cancellable computation. It checks the " token source // to see whether the cancel signal was received. let computation " (tokenSource:System.Threading.CancellationTokenSource) = async { use! cancelHandler = Async.OnCancel(fun () -> printfn " "Canceling operation.") // Async.Sleep checks for cancellation at the end of " the sleep interval, // so loop over many short sleep intervals instead of " sleeping // for a long time. while true do do! Async.Sleep(100) } let tokenSource1 = new " System.Threading.CancellationTokenSource() let tokenSource2 = new " System.Threading.CancellationTokenSource() Async.Start(computation tokenSource1, tokenSource1.Token) Async.Start(computation tokenSource2, tokenSource2.Token) printfn "Started computations." System.Threading.Thread.Sleep(1000) printfn "Sending cancellation signal." tokenSource1.Cancel() tokenSource2.Cancel() The preceding example implements the Async.OnCancel method to catch or interrupt the process when CancellationTokenSource is cancelled. Summary In this article, we went through detail, explanations about different semantics in asynchronous programming with F#, used with asynchronous workflows. We saw a number of functions from the Async module. Resources for Article: Further resources on this subject: Creating an F# Project [article] Asynchronous Control Flow Patterns with ES2015 and beyond [article] Responsive Applications with Asynchronous Programming [article]
Read more
  • 0
  • 0
  • 13428

article-image-sending-notifications-using-raspberry-pi-zero
Packt
12 Oct 2016
6 min read
Save for later

Sending Notifications using Raspberry Pi Zero

Packt
12 Oct 2016
6 min read
In this article by Marco Schwartz, author of Building Smart Homes with Raspberry Pi Zero, we are going to start diving into a very interesting field that will change the way we interact with our environment: the Internet of Things (IoT). The IoT basically proposes to connect every device around us to the Internet, so we can interact with them from anywhere in the world. (For more resources related to this topic, see here.) Within this context, a very important application is to receive notifications from your devices when they detect something in your home, for example a motion in your home or the current temperature. This is exactly what we are going to do in this article, we are going to learn how to make your Raspberry Pi Zero board send you notifications via text message, email, and push notifications. Let's start! Hardware and software requirements As always, we are going to start with the list of required hardware and software components for the project. Except Raspberry Pi Zero, you will need some additional components for each of the sections in this article. For the project of this article, we are going to use a simple PIR motion sensor to detect motion from your Pi. Then, for the last two projects of the article, we'll use the DHT11 sensor. Finally, you will need the usual breadboard and jumper wires. This is the list of components that you will need for this whole article, not including the Raspberry Pi Zero: PIR motion sensor (https://www.sparkfun.com/products/13285) DHT11 sensor + 4.7k Ohm resistor (https://www.adafruit.com/products/386) Breadboard (https://www.adafruit.com/products/64) Jumper wires (https://www.adafruit.com/products/1957) On the software side, you will need to create an account on IFTTT, which we will use in all the projects of this article. For that, simply go to: https://ifttt.com/ You should be redirected to the main page of IFTTT where you'll be able to create an account: Making a motion sensor that sends text messages For the first project of this chapter, we are going to attach a motion sensor to the Raspberry Pi board and make the Raspberry Pi Zero send us a text message whenever motion is detected. For that, we are going to use IFTTT to make the link between our Raspberry Pi and our phone. Indeed, whenever IFTTT will receive a trigger from the Raspberry Pi, it will automatically send us a text message. Lets first connect the PIR motion sensor to the Raspberry Pi. For that, simply connect the VCC pin of the sensor to a 3.3V pin of the Raspberry Pi, GND to GND, and the OUT pin of the sensor to GPIO18 of the Raspberry Pi. This is the final result: Let's now add our first channel to IFTTT, which will allow us later to interact with the Raspberry Pi and with web services. You can easily add new channels by clicking on the corresponding tab on the IFTTT website. First, add the Maker channel to your account: This will basically give you a key that you will need when writing the code for this project: After that, add the SMS channel to your IFTTT account. Now, you can actually create your first recipe. Select the Maker channel as the trigger channel: Then, select Receive a web request: As the name of this request, enter motion_detected: As the action channel, which is the channel that will be executed when a trigger is received, choose the SMS channel: For the action, choose Send me an SMS: You can now enter the message you want to see in the text messages: Finally, confirm the creation of the recipe: Now that our recipe is created and active, we can move on to actually configuring Raspberry Pi so it sends alerts whenever a motion is detected. As usual, we'll use Node.js to code this program. After a minute, pass your hand in front of the sensor: your Raspberry Pi should immediately send a command to IFTTT and after some seconds you should be able to receive a message on your mobile phone: Congratulations, you can now use your Raspberry Pi Zero to send important notifications, on your mobile phone! Note that for an actual use of this project in your home, you might want to limit the number of messages you are sending as IFTTT has a limit on the number of messages you can send (check the IFTTT website for the current limit). For example, you could use this for only very important alerts, like in case of an intruder coming in your home in your absence. Sending temperature alerts through email In the second project of the article, we are going to learn how to send automated email alerts based on data measured by the Raspberry Pi. Let's first assemble the project. Place the DHT11 sensor on the breadboard and then place the 4.7k Ohm resistor between pin 1 and 2 of the sensor. Then, connect pin 1 of the sensor to the 3.3V pin of the Raspberry Pi, pin 2 to GPIO18, and pin 4 to GND. This is the final result: Let us now see how to configure the project. Go over to IFTTT and create add the Email Channel to your account: After that, create a new recipe by choosing the Maker channel as the trigger: For the event, enter temperature_alert and then choose Email as the action channel: You will then be able to customize the text and subject of the email sent to Pi. As we want to send the emails whenever the temperature in your home gets too low, you can use a similar message: You can now finalize the creation of the recipe and close IFTTT. Let's now see how to configure the Raspberry Pi Zero. As the code for this project is quite similar to the one we saw in the previous section, I will only highlight the main differences here. It starts by including the required components: var request = require('request'); var sensorLib = require('node-dht-sensor');   Then, give the correct name to the event we'll use in the project: var eventName = 'temperature_low'; We also define the pin on which the sensor is connected: var sensorPin = 18; As we want to send alerts based on the measured temperature, we need to define a threshold. As it was quite warm when I made this project, I have assigned a high threshold at 30 degrees Celsius, but you can, of course, modify it: var threshold = 30; Summary In this article, we learned all the basics about sending automated notifications from your Raspberry Pi. We learned, for example, how to send notifications via email, text messages, and push notifications. This is really important to build a smart home, as you want to be able to get alerts in real-time from what's going on inside your home and also receive regular reports about the current status of your home. Resources for Article: Further resources on this subject: The Raspberry Pi and Raspbian [article] Building Our First Poky Image for the Raspberry Pi [article] Raspberry Pi LED Blueprints [article]
Read more
  • 0
  • 0
  • 17109
article-image-create-user-profile-system-and-use-null-coalesce-operator
Packt
12 Oct 2016
15 min read
Save for later

Create a User Profile System and use the Null Coalesce Operator

Packt
12 Oct 2016
15 min read
In this article by Jose Palala and Martin Helmich, author of PHP 7 Programming Blueprints, will show you how to build a simple profiles page with listed users which you can click on, and create a simple CRUD-like system which will enable us to register new users to the system, and delete users for banning purposes. (For more resources related to this topic, see here.) You will learn to use the PHP 7 null coalesce operator so that you can show data if there is any, or just display a simple message if there isn’t any. Let's create a simple UserProfile class. The ability to create classes has been available since PHP 5. A class in PHP starts with the word class, and the name of the class: class UserProfile { private $table = 'user_profiles'; } } We've made the table private and added a private variable, where we define which table it will be related to. Let's add two functions, also known as a method, inside the class to simply fetch the data from the database: function fetch_one($id) { $link = mysqli_connect(''); $query = "SELECT * from ". $this->table . " WHERE `id` =' " . $id "'"; $results = mysqli_query($link, $query); } function fetch_all() { $link = mysqli_connect('127.0.0.1', 'root','apassword','my_dataabase' ); $query = "SELECT * from ". $this->table . "; $results = mysqli_query($link, $query); } The null coalesce operator We can use PHP 7's null coalesce operator to allow us to check whether our results contain anything, or return a defined text which we can check on the views—this will be responsible for displaying any data. Lets put this in a file which will contain all the define statements, and call it: //definitions.php define('NO_RESULTS_MESSAGE', 'No results found'); require('definitions.php'); function fetch_all() { …same lines ... $results = $results ?? NO_RESULTS_MESSAGE; return $message; } On the client side, we'll need to come up with a template to show the list of user profiles. Let’s create a basic HTML block to show that each profile can be a div element with several list item elements to output each table. In the following function, we need to make sure that all values have been filled in with at least the name and the age. Then we simply return the entire string when the function is called: function profile_template( $name, $age, $country ) { $name = $name ?? null; $age = $age ?? null; if($name == null || $age === null) { return 'Name or Age need to be set'; } else { return '<div> <li>Name: ' . $name . ' </li> <li>Age: ' . $age . '</li> <li>Country: ' . $country . ' </li> </div>'; } } Separation of concerns In a proper MVC architecture, we need to separate the view from the models that get our data, and the controllers will be responsible for handling business logic. In our simple app, we will skip the controller layer since we just want to display the user profiles in one public facing page. The preceding function is also known as the template render part in an MVC architecture. While there are frameworks available for PHP that use the MVC architecture out of the box, for now we can stick to what we have and make it work. PHP frameworks can benefit a lot from the null coalesce operator. In some codes that I've worked with, we used to use the ternary operator a lot, but still had to add more checks to ensure a value was not falsy. Furthermore, the ternary operator can get confusing, and takes some getting used to. The other alternative is to use the isSet function. However, due to the nature of the isSet function, some falsy values will be interpreted by PHP as being a set. Creating views Now that we have our  model complete, a template render function, we just need to create the view with which we can look at each profile. Our view will be put inside a foreach block, and we'll use the template we wrote to render the right values: //listprofiles.php <html> <!doctype html> <head> <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/css/bootstrap.min.css"> </head> <body> <?php foreach($results as $item) { echo profile_template($item->name, $item->age, $item->country; } ?> </body> </html> Let's put the code above into index.php. While we may install the Apache server, configure it to run PHP, install new virtual hosts and the other necessary featuress, and put our PHP code into an Apache folder, this will take time, so, for the purposes of testing this out, we can just run PHP's server for development. To  run the built-in PHP server (read more at http://php.net/manual/en/features.commandline.webserver.php) we will use the folder we are running, inside a terminal: php -S localhost:8000 If we open up our browser, we should see nothing yet—No results found. This means we need to populate our database. If you have an error with your database connection, be sure to replace the correct database credentials we supplied into each of the mysql_connect calls that we made. To supply data into our database, we can create a simple SQL script like this: INSERT INTO user_profiles ('Chin Wu', 30, 'Mongolia'); INSERT INTO user_profiles ('Erik Schmidt', 22, 'Germany'); INSERT INTO user_profiles ('Rashma Naru', 33, 'India'); Let's save it in a file such as insert_profiles.sql. In the same directory as the SQL file, log on to the MySQL client by using the following command: mysql -u root -p Then type use <name of database>: mysql> use <database>; Import the script by running the source command: mysql> source insert_profiles.sql Now our user profiles page should show the following: Create a profile input form Now let's create the HTML form for users to enter their profile data. Our profiles app would be no use if we didn't have a simple way for a user to enter their user profile details. We'll create the profile input form  like this: //create_profile.php <html> <body> <form action="post_profile.php" method="POST"> <label>Name</label><input name="name"> <label>Age</label><input name="age"> <label>Country</label><input name="country"> </form> </body> </html> In this profile post, we'll need to create a PHP script to take care of anything the user posts. It will create an SQL statement from the input values and output whether or not they were inserted. We can use the null coalesce operator again to verify that the user has inputted all values and left nothing undefined or null: $name = $_POST['name'] ?? ""; $age = $_POST['country'] ?? ""; $country = $_POST['country'] ?? ""; This prevents us from accumulating errors while inserting data into our database. First, let's create a variable to hold each of the inputs in one array: $input_values = [ 'name' => $name, 'age' => $age, 'country' => $country ]; The preceding code is a new PHP 5.4+ way to write arrays. In PHP 5.4+, it is no longer necessary to put an actual array(); the author personally likes the new syntax better. We should create a new method in our UserProfile class to accept these values: Class UserProfile { public function insert_profile($values) { $link = mysqli_connect('127.0.0.1', 'username','password', 'databasename'); $q = " INSERT INTO " . $this->table . " VALUES ( '". $values['name']."', '".$values['age'] . "' ,'". $values['country']. "')"; return mysqli_query($q); } } Instead of creating a parameter in our function to hold each argument as we did with our profile template render function, we can simply use an array to hold our values. This way, if a new field needs to be inserted into our database, we can just add another field to the SQL insert statement. While we are at it, let's create the edit profile section. For now, we'll assume that whoever is using this edit profile is the administrator of the site. We'll need to create a page where, provided the $_GET['id'] or has been set, that the user that we will be fetching from the database and displaying on the form. <?php require('class/userprofile.php');//contains the class UserProfile into $id = $_GET['id'] ?? 'No ID'; //if id was a string, i.e. "No ID", this would go into the if block if(is_numeric($id)) { $profile = new UserProfile(); //get data from our database $results = $user->fetch_id($id); if($results && $results->num_rows > 0 ) { while($obj = $results->fetch_object()) { $name = $obj->name; $age = $obj->age; $country = $obj->country; } //display form with a hidden field containing the value of the ID ?> <form action="post_update_profile.php" method="post"> <label>Name</label><input name="name" value="<?=$name?>"> <label>Age</label><input name="age" value="<?=$age?>"> <label>Country</label><input name="country" value="<?=country?>"> </form> <?php } else { exit('No such user'); } } else { echo $id; //this should be No ID'; exit; } Notice that we're using what is known as the shortcut echo statement in the form. It makes our code simpler and easier to read. Since we're using PHP 7, this feature should come out of the box. Once someone submits the form, it goes into our $_POST variable and we'll create a new Update function in our UserProfile class. Admin system Let's finish off by creating a simple grid for an admin dashboard portal that will be used with our user profiles database. Our requirement for this is simple: We can just set up a table-based layout that displays each user profile in rows. From the grid, we will add the links to be able to edit the profile, or  delete it, if we want to. The code to display a table in our HTML view would look like this: <table> <tr> <td>John Doe</td> <td>21</td> <td>USA</td> <td><a href="edit_profile.php?id=1">Edit</a></td> <td><a href="profileview.php?id=1">View</a> <td><a href="delete_profile.php?id=1">Delete</a> </tr> </table> This script to this is the following: //listprofiles.php $sql = "SELECT * FROM userprofiles LIMIT $start, $limit "; $rs_result = mysqli_query ($sql); //run the query while($row = mysqli_fetch_assoc($rs_result) { ?> <tr> <td><?=$row['name'];?></td> <td><?=$row['age'];?></td> <td><?=$row['country'];?></td> <td><a href="edit_profile.php?id=<?=$id?>">Edit</a></td> <td><a href="profileview.php?id=<?=$id?>">View</a> <td><a href="delete_profile.php?id=<?=$id?>">Delete</a> </tr> <?php } There's one thing that we haven't yet created: A delete_profile.php page. The view and edit pages - have been discussed already. Here's how the delete_profile.php page would look: <?php //delete_profile.php $connection = mysqli_connect('localhost','<username>','<password>', '<databasename>'); $id = $_GET['id'] ?? 'No ID'; if(is_numeric($id)) { mysqli_query( $connection, "DELETE FROM userprofiles WHERE id = '" .$id . "'"); } else { echo $id; } i(!is_numeric($id)) { exit('Error: non numeric $id'); } else { echo "Profile #" . $id . " has been deleted"; ?> Of course, since we might have a lot of user profiles in our database, we have to create a simple pagination. In any pagination system, you just need to figure out the total number of rows, and how many rows you want displayed per page. We can create a function that will be able to return a URL that contains the page number and how many to view per page. From our queries database, we first create a new function for us to select only up to the total number of items in our database: class UserProfile{ // …. Etc … function count_rows($table) { $dbconn = new mysqli('localhost', 'root', 'somepass', 'databasename'); $query = $dbconn->query("select COUNT(*) as num from '". $table . "'"); $total_pages = mysqli_fetch_array($query); return $total_pages['num']; //fetching by array, so element 'num' = count } For our pagination, we can create a simple paginate function which accepts the base_url of the page where we have pagination, the rows per page — also known as the number of records we want each page to have — and the total number of records found: require('definitions.php'); require('db.php'); //our database class Function paginate ($baseurl, $rows_per_page, $total_rows) { $pagination_links = array(); //instantiate an array to hold our html page links //we can use null coalesce to check if the inputs are null ( $total_rows || $rows_per_page) ?? exit('Error: no rows per page and total rows); //we exit with an error message if this function is called incorrectly $pages = $total_rows % $rows_per_page; $i= 0; $pagination_links[$i] = "<a href="http://". $base_url . "?pagenum=". $pagenum."&rpp=".$rows_per_page. ">" . $pagenum . "</a>"; } return $pagination_links; } This function will help display the above page links in a table: function display_pagination($links) { $display = ' <div class="pagination">'; <table><tr>'; foreach ($links as $link) { echo "<td>" . $link . "</td>"; } $display .= '</tr></table></div>'; return $display; } Notice that we're following the principle that there should rarely be any echo statements inside a function. This is because we want to make sure that other users of these functions are not confused when they debug some mysterious output on their page. By requiring the programmer to echo out whatever the functions return, it becomes easier to debug our program. Also, we're following the separation of concerns—our code doesn't output the display, it just formats the display. So any future programmer can just update the function's internal code and return something else. It also makes our function reusable; imagine that in the future someone uses our function—this way, they won't have to double check that there's some misplaced echo statement within our functions. A note on alternative short tags As you know, another way to echo is to use  the <?= tag. You can use it like so: <?="helloworld"?>.These are known as short tags. In PHP 7, alternative PHP tags have been removed. The RFC states that <%, <%=, %> and <script language=php>  have been deprecated. The RFC at https://wiki.php.net/rfc/remove_alternative_php_tags says that the RFC does not remove short opening tags (<?) or short opening tags with echo (<?=). Since we have laid out the groundwork of creating paginate links, we now just have to invoke our functions. The following script is all that is needed to create a paginated page using the preceding function: $mysqli = mysqli_connect('localhost','<username>','<password>', '<dbname>'); $limit = $_GET['rpp'] ?? 10; //how many items to show per page default 10; $pagenum = $_GET['pagenum']; //what page we are on if($pagenum) $start = ($pagenum - 1) * $limit; //first item to display on this page else $start = 0; //if no page var is given, set start to 0 /*Display records here*/ $sql = "SELECT * FROM userprofiles LIMIT $start, $limit "; $rs_result = mysqli_query ($sql); //run the query while($row = mysqli_fetch_assoc($rs_result) { ?> <tr> <td><?php echo $row['name']; ?></td> <td><?php echo $row['age']; ?></td> <td><?php echo $row['country']; ?></td> </tr> <?php } /* Let's show our page */ /* get number of records through */ $record_count = $db->count_rows('userprofiles'); $pagination_links = paginate('listprofiles.php' , $limit, $rec_count); echo display_pagination($paginaiton_links); The HTML output of our page links in listprofiles.php will look something like this: <div class="pagination"><table> <tr> <td> <a href="listprofiles.php?pagenum=1&rpp=10">1</a> </td> <td><a href="listprofiles.php?pagenum=2&rpp=10">2</a> </td> <td><a href="listprofiles.php?pagenum=3&rpp=10">2</a> </td> </tr> </table></div> Summary As you can see, we have a lot of use cases for the null coalesce. We learned how to make a simple user profile system, and how to use PHP 7's null coalesce feature when fetching data from the database, which returns null if there are no records. We also learned that the null coalesce operator is similar to a ternary operator, except this returns null by default if there is no data. Resources for Article: Further resources on this subject: Running Simpletest and PHPUnit [article] Mapping Requirements for a Modular Web Shop App [article] HTML5: Generic Containers [article]
Read more
  • 0
  • 0
  • 8131

article-image-solving-nlp-problem-keras-part-1
Sasank Chilamkurthy
12 Oct 2016
5 min read
Save for later

Solving an NLP Problem with Keras, Part 1

Sasank Chilamkurthy
12 Oct 2016
5 min read
In a previous two-part post series on Keras, I introduced Convolutional Neural Networks(CNNs) and the Keras deep learning framework. We used them to solve a Computer Vision (CV) problem involving traffic sign recognition. Now, in this two-part post series, we will solve a Natural Language Processing (NLP) problem with Keras. Let’s begin. The Problem and the Dataset The problem we are going to tackle is Natural Language Understanding. The aim is to extract the meaning of speech utterances. This is still an unsolved problem. Therefore, we can break this problem into a solvable practical problem of understanding the speaker in a limited context. In particular, we want to identify the intent of a speaker asking for information about flights. The dataset we are going to use is Airline Travel Information System (ATIS). This dataset was collected by DARPA in the early 90s. ATIS consists of spoken queries on flight related information. An example utterance is I want to go from Boston to Atlanta on Monday. Understanding this is then reduced to identifying arguments like Destination and Departure Day. This task is called slot-filling. Here is an example sentence and its labels. You will observe that labels are encoded in an Inside Outside Beginning (IOB) representation. Let’s look at the dataset: |Words | Show | flights | from | Boston | to | New | York| today| |Labels| O | O | O |B-dept | O|B-arr|I-arr|B-date| The ATIS official split contains 4,978/893 sentences for a total of 56,590/9,198 words (average sentence length is 15) in the train/test set. The number of classes (different slots) is 128, including the O label (NULL). Unseen words in the test set are encoded by the <UNK> token, and each digit is replaced with string DIGIT;that is,20 is converted to DIGITDIGIT. Our approach to the problem is to use: Word embeddings Recurrent neural networks I'll talk about these briefly in the following sections. Word Embeddings Word embeddings map words to a vector in a high-dimensional space. These word embeddings can actually learn the semantic and syntactic information of words. For instance, they can understand that similar words are close to each other in this space and dissimilar words are far apart. This can be learned either using large amounts of text like Wikipedia, or specifically for a given problem. We will take the second approach for this problem. As an illustation, I have shown here the nearest neighbors in the word embedding space for some of the words. This embedding space was learned by the model that we’ll define later in the post: sunday delta california boston august time car wednesday continental colorado nashville september schedule rental saturday united florida toronto july times limousine friday american ohio chicago june schedules rentals monday eastern georgia phoenix december dinnertime cars tuesday northwest pennsylvania cleveland november ord taxi thursday us north atlanta april f28 train wednesdays nationair tennessee milwaukee october limo limo saturdays lufthansa minnesota columbus january departure ap sundays midwest michigan minneapolis may sfo later Recurrent Neural Networks Convolutional layers can be a great way to pool local information, but they do not really capture the sequentiality of data. Recurrent Neural Networks (RNNs) help us tackle sequential information like natural language. If we are going to predict properties of the current word, we better remember the words before it too. An RNN has such an internal state/memory that stores the summary of the sequence it has seen so far. This allows us to use RNNs to solve complicated word tagging problems such as Part Of Speech (POS) tagging or slot filling, as in our case. The following diagram illustrates the internals of RNN:  Source: Nature RNN Let's briefly go through the diagram: Is the input to the RNN.   x_1,x_2,...,x_(t-1),x_t,x_(t+1)... Is the hidden state of the RNN at the step.  st This is computed based on the state at the step. t-1 As st=f(Uxt+Ws(t-1)) Here f is a nonlinearity such astanh or ReLU. ot Is the output at the step. t Computed as:ot=f(Vst)U,V,W Are the learnable parameters of RNN. For our problem, we will pass a word embeddings’ sequence as the input to the RNN. Putting it all together Now that we've setup the problem and have an understanding of the basic blocks, let's code it up. Since we are using the IOB representation for labels, it's not simpleto calculate the scores of our model. We therefore use the conlleval perl script to compute the F1 Scores. I've adapted the code from here for the data preprocessing and score calculation. The complete code is available at GitHub: $ git clone https://github.com/chsasank/ATIS.keras.git $ cd ATIS.keras I recommend using jupyter notebook to run and experiment with the snippets from the tutorial. $ jupyter notebook Conclusion In part 2, we will load the data using data.load.atisfull(). We will also define the Keras model, and then we will train the model. To measure the accuracy of the model, we’ll use model.predict_on_batch() and metrics.accuracy.conlleval(). And finally, we will improve our model to achieve better results. About the author Sasank Chilamkurthy works at Fractal Analytics. His work involves deep learning on medical images obtained from radiology and pathology. He is mainly interested in computer vision.
Read more
  • 0
  • 0
  • 4638
Modal Close icon
Modal Close icon