Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Save more on your purchases! discount-offer-chevron-icon
Savings automatically calculated. No voucher code required.
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Events
Videos
Audiobooks
Packt Hub
Free Learning
Arrow right icon
timer SALE ENDS IN
0 Days
:
00 Hours
:
00 Minutes
:
00 Seconds

How-To Tutorials

7019 Articles
article-image-crud-operations-rest
Packt
16 Sep 2015
11 min read
Save for later

CRUD Operations in REST

Packt
16 Sep 2015
11 min read
In this article by Ludovic Dewailly, the author of Building a RESTful Web Service with Spring, we will learn how requests to retrieve data from a RESTful endpoint, created to access the rooms in a sample property management system, are typically mapped to the HTTP GET method in RESTful web services. We will expand on this by implementing some of the endpoints to support all the CRUD (Create, Read, Update, Delete) operations. In this article, we will cover the following topics: Mapping the CRUD operations to the HTTP methods Creating resources Updating resources Deleting resources Testing the RESTful operations Emulating the PUT and DELETE methods (For more resources related to this topic, see here.) Mapping the CRUD operations[km1]  to HTTP [km2] [km3] methods The HTTP 1.1 specification defines the following methods: OPTIONS: This method represents a request for information about the communication options available for the requested URI. This is, typically, not directly leveraged with REST. However, this method can be used as a part of the underlying communication. For example, this method may be used when consuming web services from a web page (as a part of the C[km4] ross-origin resource sharing mechanism). GET: This method retrieves the information identified by the request URI. In the context of the RESTful web services, this method is used to retrieve resources. This is the method used for read operations (the R in CRUD). HEAD: The HEAD requests are semantically identical to the GET requests except the body of the response is not transmitted. This method is useful for obtaining meta-information about resources. Similar to the OPTIONS method, this method is not typically used directly in REST web services. POST: This method is used to instruct the server to accept the entity enclosed in the request as a new resource. The create operations are typically mapped to this HTTP method. PUT: This method requests the server to store the enclosed entity under the request URI. To support the updating of REST resources, this method can be leveraged. As per the HTTP specification, the server can create the resource if the entity does not exist. It is up to the web service designer to decide whether this behavior should be implemented or resource creation should only be handled by POST requests. DELETE: The last operation not yet mapped is for the deletion of resources. The HTTP specification defines a DELETE method that is semantically aligned with the deletion of RESTful resources. TRACE: This method is used to perform actions on web servers. These actions are often aimed to aid development and the testing of HTTP applications. The TRACE requests aren't usually mapped to any particular RESTful operations. CONNECT: This HTTP method is defined to support HTTP tunneling through a proxy server. Since it deals with transport layer concerns, this method has no natural semantic mapping to the RESTful operations. The RESTful architecture does not mandate the use of HTTP as a communication protocol. Furthermore, even if HTTP is selected as the underlying transport, no provisions are made regarding the mapping of the RESTful operations to the HTTP method. Developers could feasibly support all operations through POST requests. This being said, the following CRUD to HTTP method mapping is commonly used in REST web services: Operation HTTP method Create POST Read GET Update PUT Delete DELETE Our sample web service will use these HTTP methods to support CRUD operations. The rest of this article will illustrate how to build such operations. Creating r[km5] esources The inventory component of our sample property management system deals with rooms. If we have already built an endpoint to access the rooms. Let's take a look at how to define an endpoint to create new resources: @RestController @RequestMapping("/rooms") public class RoomsResource { @RequestMapping(method = RequestMethod.POST) public ApiResponse addRoom(@RequestBody RoomDTO room) { Room newRoom = createRoom(room); return new ApiResponse(Status.OK, new RoomDTO(newRoom)); } } We've added a new method to our RoomsResource class to handle the creation of new rooms. @RequestMapping is used to map requests to the Java method. Here we map the POST requests to addRoom(). Not specifying a value (that is, path) in @RequestMapping is equivalent to using "/". We pass the new room as @RequestBody. This annotation instructs Spring to map the body of the incoming web request to the method parameter. Jackson is used here to convert the JSON request body to a Java object. With this new method, the POSTing requests to http://localhost:8080/rooms with the following JSON body will result in the creation of a new room: { name: "Cool Room", description: "A room that is very cool indeed", room_category_id: 1 } Our new method will return the newly created room: { "status":"OK", "data":{ "id":2, "name":"Cool Room", "room_category_id":1, "description":"A room that is very cool indeed" } } We can decide to return only the ID of the new resource in response to the resource creation. However, since we may sanitize or otherwise manipulate the data that was sent over, it is a good practice to return the full resource. Quickly testing endpoints[km6]  For the purpose of quickly testing our newly created endpoint, let's look at testing the new rooms created using Postman. Postman (https://www.getpostman.com) is a Google Chrome plugin extension that provides tools to build and test web APIs. This following screenshot illustrates how Postman can be used to test this endpoint: In Postman, we specify the URL to send the POST request to http://localhost:8080/rooms, with the "[km7] application/json" content type header and the body of the request. Sending this requesting will result in a new room being created and returned as shown in the following: We have successfully added a room to our inventory service using Postman. It is equally easy to create incomplete requests to ensure our endpoint performs any necessary sanity checks before persisting data into the database. JSON versus[km8]  form data Posting forms is the traditional way of creating new entities on the web and could easily be used to create new RESTful resources. We can change our method to the following: @RequestMapping(method = RequestMethod.POST, consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE) public ApiResponse addRoom(String name, String description, long roomCategoryId) { Room room = createRoom(name, description, roomCategoryId); return new ApiResponse(Status.OK, new RoomDTO(room)); } The main difference with the previous method is that we tell Spring to map form requests (that is, with application/x-www-form-urlencoded the content type) instead of JSON requests. In addition, rather than expecting an object as a parameter, we receive each field individually. By default, Spring will use the Java method attribute names to map incoming form inputs. Developers can change this behavior by annotating attribute with @RequestParam("…") to specify the input name. In situations where the main web service consumer is a web application, using form requests may be more applicable. In most cases, however, the former approach is more in line with RESTful principles and should be favored. Besides, when complex resources are handled, form requests will prove cumbersome to use. From a developer standpoint, it is easier to delegate object mapping to a third-party library such as Jackson. Now that we have created a new resource, let's see how we can update it. Updating r[km9] esources Choosing URI formats is an important part of designing RESTful APIs. As seen previously, rooms are accessed using the /rooms/{roomId} path and created under /rooms. You may recall that as per the HTTP specification, PUT requests can result in creation of entities, if they do not exist. The decision to create new resources on update requests is up to the service designer. It does, however, affect the choice of path to be used for such requests. Semantically, PUT requests update entities stored under the supplied request URI. This means the update requests should use the same URI as the GET requests: /rooms/{roomId}. However, this approach hinders the ability to support resource creation on update since no room identifier will be available. The alternative path we can use is /rooms with the room identifier passed in the body of the request. With this approach, the PUT requests can be treated as POST requests when the resource does not contain an identifier. Given the first approach is semantically more accurate, we will choose not to support resource create on update, and we will use the following path for the PUT requests: /rooms/{roomId} Update endpoint[km10]  The following method provides the necessary endpoint to modify the rooms: @RequestMapping(value = "/{roomId}", method = RequestMethod.PUT) public ApiResponse updateRoom(@PathVariable long roomId, @RequestBody RoomDTO updatedRoom) { try { Room room = updateRoom(updatedRoom); return new ApiResponse(Status.OK, new RoomDTO(room)); } catch (RecordNotFoundException e) { return new ApiResponse(Status.ERROR, null, new ApiError(999, "No room with ID " + roomId)); } } As discussed in the beginning of this article, we map update requests to the HTTP PUT verb. Annotating this method with @RequestMapping(value = "/{roomId}", method = RequestMethod.PUT) instructs Spring to direct the PUT requests here. The room identifier is part of the path and mapped to the first method parameter. In fashion similar to the resource creation requests, we map the body to our second parameter with the use of @RequestBody. Testing update requests[km11]  With Postman, we can quickly create a test case to update the room we created. To do so, we send a PUT request with the following body: { id: 2, name: "Cool Room", description: "A room that is really very cool indeed", room_category_id: 1 } The resulting response will be the updated room, as shown here: { "status": "OK", "data": { "id": 2, "name": "Cool Room", "room_category_id": 1, "description": "A room that is really very cool indeed." } } Should we attempt to update a nonexistent room, the server will generate the following response: { "status": "ERROR", "error": { "error_code": 999, "description": "No room with ID 3" } } Since we do not support resource creation on update, the server returns an error indicating that the resource cannot be found. Deleting resources[km12]  It will come as no surprise that we will use the DELETE verb to delete REST resources. Similarly, the reader will have already figured out that the path to delete requests will be /rooms/{roomId}. The Java method that deals with room deletion is as follows: @RequestMapping(value = "/{roomId}", method = RequestMethod.DELETE) public ApiResponse deleteRoom(@PathVariable long roomId) { try { Room room = inventoryService.getRoom(roomId); inventoryService.deleteRoom(room.getId()); return new ApiResponse(Status.OK, null); } catch (RecordNotFoundException e) { return new ApiResponse(Status.ERROR, null, new ApiError( 999, "No room with ID " + roomId)); } } By declaring the request mapping method to be RequestMethod.DELETE, Spring will make this method handle the DELETE requests. Since the resource is deleted, returning it in the response would not make a lot of sense. Service designers may choose to return a boolean flag to indicate the resource was successfully deleted. In our case, we leverage the status element of our response to carry this information back to the consumer. The response to deleting a room will be as follows: { "status": "OK" } With this operation, we have now a full-fledged CRUD API for our Inventory Service. Before we conclude this article, let's discuss how REST developers can deal with situations where not all HTTP verbs can be utilized. HTTP method override In certain situations (for example, when the service or its consumers are behind an overzealous corporate firewall, or if the main consumer is a web page), only the GET and POST HTTP methods might be available. In such cases, it is possible to emulate the missing verbs by passing a customer header in the requests. For example, resource updates can be handle using POST requests by setting a customer header (for example, X-HTTP-Method-Override) to PUT to indicate that we are emulating a PUT request via a POST request. The following method will handle this scenario: @RequestMapping(value = "/{roomId}", method = RequestMethod.POST, headers = {"X-HTTP-Method-Override=PUT"}) public ApiResponse updateRoomAsPost(@PathVariable("roomId") long id, @RequestBody RoomDTO updatedRoom) { return updateRoom(id, updatedRoom); } By setting the headers attribute on the mapping annotation, Spring request routing will intercept the POST requests with our custom header and invoke this method. Normal POST requests will still map to the Java method we had put together to create new rooms. Summary In this article, we've performed the implementation of our sample RESTful web service by adding all the CRUD operations necessary to manage the room resources. We've discussed how to organize URIs to best embody the REST principles and looked at how to quickly test endpoints using Postman. Now that we have a fully working component of our system, we can take some time to discuss performance. Resources for Article: Further resources on this subject: Introduction to Spring Web Application in No Time[article] Aggregators, File exchange Over FTP/FTPS, Social Integration, and Enterprise Messaging[article] Time Travelling with Spring[article]
Read more
  • 0
  • 0
  • 20530

article-image-groovy-closures
Packt
16 Sep 2015
9 min read
Save for later

Groovy Closures

Packt
16 Sep 2015
9 min read
In this article by Fergal Dearle, the author of the book Groovy for Domain-Specific Languages - Second Edition, we will focus exclusively on closures. We will take a close look at them from every angle. Closures are the single most important feature of the Groovy language. Closures are the special seasoning that helps Groovy stand out from Java. They are also the single most powerful feature that we will use when implementing DSLs. In the article, we will discuss the following topics: We will start by explaining just what a closure is and how we can define some simple closures in our Groovy code We will look at how many of the built-in collection methods make use of closures for applying iteration logic, and see how this is implemented by passing a closure as a method parameter We will look at the various mechanisms for calling closures A handy reference that you might want to consider having at hand while you read this article is GDK Javadocs, which will give you full class descriptions of all of the Groovy built-in classes, but of particular interest here is groovy.lang.Closure. (For more resources related to this topic, see here.) What is a closure Closures are such an unfamiliar concept to begin with that it can be hard to grasp initially. Closures have characteristics that make them look like a method in so far as we can pass parameters to them and they can return a value. However, unlike methods, closures are anonymous. A closure is just a snippet of code that can be assigned to a variable and executed later: def flintstones = ["Fred","Barney"] def greeter = { println "Hello, ${it}" } flintstones.each( greeter ) greeter "Wilma" greeter = { } flintstones.each( greeter ) greeter "Wilma" Because closures are anonymous, they can easily be lost or overwritten. In the preceding example, we defined a variable greeter to contain a closure that prints a greeting. After greeter is overwritten with an empty closure, any reference to the original closure is lost. It's important to remember that greeter is not the closure. It is a variable that contains a closure, so it can be supplanted at any time. Because greeter has a dynamic type, we could have assigned any other object to it. All closures are a subclass of the type groovy.lang.Closure. Because groovy.lang is automatically imported, we can refer to Closure as a type within our code. By declaring our closures explicitly as Closure, we cannot accidentally assign a non-closure to them: Closure greeter = { println it } For each closure that is declared in our code, Groovy generates a Closure class for us, which is a subclass of groovy.lang.Closure. Our closure object is an instance of this class. Although we cannot predict what exact type of closure is generated, we can rely on it being a subtype of groovy.lang.Closure. Closures and collection methods We will encounter Groovy lists and see some of the iteration functions, such as the each method: def flintstones = ["Fred","Barney"] flintstones.each { println "Hello, ${it}" } This looks like it could be a specialized control loop similar to a while loop. In fact, it is a call to the each method of Object. The each method takes a closure as one of its parameters, and everything between the curly braces {} defines another anonymous closure. Closures defined in this way can look quite similar to code blocks, but they are not the same. Code defined in a regular Java or Groovy style code block is executed as soon as it is encountered. With closures, the block of code defined in the curly braces is not executed until the call() method of the closure is made: println "one" def two = { println "two" } println "three" two.call() println "four" Will print the following: one three two four Let's dig a bit deeper into the structure of the each of the calls shown in the preceding code. I refer to each as a call because that's what it is—a method call. Groovy augments the standard JDK with numerous helper methods. This new and improved JDK is referred to as the Groovy JDK, or GDK for short. In the GDK, Groovy adds the each method to the java.lang.Object class. The signature of the each method is as follows: Object each(Closure closure) The java.lang.Object class has a number of similar methods such as each, find, every, any, and so on. Because these methods are defined as part of Object, you can call them on any Groovy or Java object. They make little sense on most objects, but they do something sensible if not very useful: given: "an Integer" def number = 1 when: "we call the each method on it" number.each { println it } then: "just the object itself gets passed into the Closure" "1" == output() These methods all have specific implementations for all of the collection types, including arrays, lists, ranges, and maps. So, what is actually happening when we see the call to flintstones.each is that we are calling the list's implementation of the each method. Because each takes a Closure as its last and only parameter, the following code block is interpreted by Groovy as an anonymous Closure object to be passed to the method. The actual call to the closure passed to each is deferred until the body of the each method itself is called. The closure may be called multiple times—once for every element in the collection. Closures as method parameters We already know that parentheses around method parameters are optional, so the previous call to each can also be considered equivalent to: flintstones.each ({ println "Hello, ${it}") Groovy has a special handling for methods whose last parameter is a closure. When invoking these methods, the closure can be defined anonymously after the method call parenthesis. So, yet another legitimate way to call the preceding line is: flintstones.each() { println "hello, ${it}" } The general convention is not to use parentheses unless there are parameters in addition to the closure: given: def flintstones = ["Fred", "Barney", "Wilma"] when: "we call findIndexOf passing int and a Closure" def result = flintstones.findIndexOf(0) { it == 'Wilma'} then: result == 2 The signature of the GDK findIndexOf method is: int findIndexOf(int, Closure) We can define our own methods that accept closures as parameters. The simplest case is a method that accepts only a single closure as a parameter: def closureMethod(Closure c) { c.call() } when: "we invoke a method that accepts a closure" closureMethod { println "Closure called" } then: "the Closure passed in was executed" "Closure called" == output() Method parameters as DSL This is an extremely useful construct when we want to wrap a closure in some other code. Suppose we have some locking and unlocking that needs to occur around the execution of a closure. Rather than the writer of the code to locking via a locking API call, we can implement the locking within a locker method that accepts the closure: def locked(Closure c) { callToLockingMethod() c.call() callToUnLockingMethod() } The effect of this is that whenever we need to execute a locked segment of code, we simply wrap the segment in a locked closure block, as follows: locked { println "Closure called" } In a small way, we are already writing a mini DSL when we use these types on constructs. This call to the locked method looks, to all intents and purposes, like a new language construct, that is, a block of code defining the scope of a locking operation. When writing methods that take other parameters in addition to a closure, we generally leave the Closure argument to last. As already mentioned in the previous section, Groovy has a special syntax handling for these methods, and allows the closure to be defined as a block after the parameter list when calling the method: def closureMethodInteger(Integer i, Closure c) { println "Line $i" c.call() } when: "we invoke a method that accepts an Integer and a Closure" closureMethodInteger(1) { println "Line 2" } then: "the Closure passed in was executed with the parameter" """Line 1 Line 2""" == output() Forwarding parameters Parameters passed to the method may have no impact on the closure itself, or they may be passed to the closure as a parameter. Methods can accept multiple parameters in addition to the closure. Some may be passed to the closure, while others may not: def closureMethodString(String s, Closure c) { println "Greet someone" c.call(s) } when: "we invoke a method that accepts a String and a Closure" closureMethodString("Dolly") { name -> println "Hello, $name" } then: "the Closure passed in was executed with the parameter" """Greet someone Hello, Dolly""" == output() This construct can be used in circumstances where we have a look-up code that needs to be executed before we have access to an object. Say we have customer records that need to be retrieved from a database before we can use them: def withCustomer (id, Closure c) { def cust = getCustomerRecord(id) c.call(cust) } withCustomer(12345) { customer -> println "Found customer ${customer.name}" } We can write an updateCustomer method that saves the customer record after the closure is invoked, and amend our locked method to implement transaction isolation on the database, as follows: class Customer { String name } def locked (Closure c) { println "Transaction lock" c.call() println "Transaction release" } def update (customer, Closure c) { println "Customer name was ${customer.name}" c.call(customer) println "Customer name is now ${customer.name}" } def customer = new Customer(name: "Fred") At this point, we can write code that nests the two method calls by calling update as follows: locked { update(customer) { cust -> cust.name = "Barney" } } This outputs the following result, showing how the update code is wrapped by updateCustomer, which retrieves the customer object and subsequently saves it. The whole operation is wrapped by locked, which includes everything within a transaction: Transaction lock Customer name was Fred Customer name is now Barney Transaction release Summary In this article, we covered closures in some depth. We explored the various ways to call a closure and the means of passing parameters. We saw how we can pass closures as parameters to methods, and how this construct can allow us to appear to add mini DSL syntax to our code. Closures are the real "power" feature of Groovy, and they form the basis of most of the DSLs. Resources for Article: Further resources on this subject: Using Groovy Closures Instead of Template Method [article] Metaprogramming and the Groovy MOP [article] Clojure for Domain-specific Languages - Design Concepts with Clojure [article]
Read more
  • 0
  • 0
  • 3406

article-image-remote-desktop-your-pi-everywhere
Packt
16 Sep 2015
6 min read
Save for later

Remote Desktop to Your Pi from Everywhere

Packt
16 Sep 2015
6 min read
In this article by Gökhan Kurt, author of the book Raspberry Pi Android Projects, we will make a gentle introduction to both Pi and Android platforms to warm us up. Many users of the Pi face similar problems when they wish to administer it. You have to be near your Pi and connect a screen and a keyboard to it. We will solve this everyday problem by remotely connecting to our Pi desktop interface. The article covers the following topics: Installing necessary components in the Pi and Android Connecting the Pi and Android (For more resources related to this topic, see here.) Installing necessary components in the Pi and Android The following image shows you that the LXDE desktop manager comes with an initial setup and a few preinstalled programs: LXDE desktop management environment By clicking on the screen image on the tab bar located at top, you will be able to open a terminal screen that we will use to send commands to the Pi. The next step is to install a component called x11vnc. This is a VNC server for X, the window management component of Linux. Issue following command on the terminal: sudo apt-get install x11vnc This will download and install x11vnc to the Pi. We can even set a password to be used by VNC clients that will remote desktop to this Pi using the following command and provide a password to be used later on: x11vnc –storepasswd Next, we can get the x11vnc server running whenever the Pi is rebooted and the LXDE desktop manager starts. This can be done through the following steps: Go into the .config directory on the Pi user's home directory located at /home/pi:cd /home/pi/.config Make a subdirectory here named autostart:mkdir autostart Go into the autostart directory:cd autostart Start editing a file named x11vnc.desktop. As a terminal editor, I am using nano, which is the easiest one to use on the Pi for novice users, but there are more exciting alternatives, such as vi: nano x11vnc.desktop Add the following content into this file: [Desktop Entry] Encoding=UTF-8 Type=Application Name=X11VNC Comment= Exec=x11vnc -forever -usepw -display :0 -ultrafilexfer StartupNotify=false Terminal=false Hidden=false Save and exit using (Ctrl+X, Y, <Enter>) in order if you are using nano as the editor of your choice. Now you should reboot the Pi to get the server running using the following command: sudo reboot After rebooting, we can now find out what IP address our Pi has been given in the terminal window by issuing the ifconfig command. The IP address assigned to your Pi is to be found under the eth0 entry and is given after the inet addr keyword. Write this address down: Example output from ifconfig command The next step is to download a VNC client to your Android device.In this project, we will use a freely available client for Android, namely androidVNC or as it is named in the Play Store—VNC Viewer for Android by androidVNC team + antlersoft. The latest version in use at the writing of this book was 0.5.0. Note that in order to be able to connect your Android VNC client to the Pi, both the Pi and the Android device should be connected to the same network. Android through Wi-Fi and Pi through its Ethernet port. Connecting the Pi and Android Install and open androidVNC on your device. You will be presented with a first activity user interface asking for the details of the connection. Here, you should provide Nickname for the connection, Password you enter when you run the x11vnc –storepasswd command, and the IP Address of the Pi that you have found out using the ifconfig command. Initiate the connection by pressing the Connect button, and you should now be able to see the Pi desktop on your Android device. In androidVNC, you should be able to move the mouse pointer by clicking on the screen and under the options menu in the androidVNC app, you will find out how to send text and keys to the Pi with the help of Enter and Backspace. You may even find it convenient to connect to the Pi from another computer. I recommend using RealVNC for this purpose, which is available on Windows, Linux, and Mac OS. What if I want to use Wi-Fi on the Pi? In order to use a Wi-Fi dongle on the Pi, first of all, open the wpa-supplicant configuration file using the nano editor with the following command: sudo nano /etc/wpa_supplicant/wpa_supplicant.conf Add the following to the end of this file: network={ ssid="THE ID OF THE NETWORK YOU WANT TO CONNECT" psk="PASSWORD OF YOUR WIFI" } I assume that you have set up your wireless home network to use WPA-PSK as the authentication mechanism. If you have another mechanism, you should refer to the wpa_supplicant documentation. LXDE provides even better ways to connect to Wi-Fi networks through a GUI. It can be found on the upper-right corner of the desktop environment on the Pi. Connecting from everywhere Now, we have connected to the Pi from our device, which we need to connect to the same network as the Pi. However, most of us would like to connect to the Pi from around the world as well. To do this, first of all, we need to now the IP address of the home network assigned to us by our network provider. By going to http://whatismyipaddress.com URL, we can figure out what our home network's IP address is. The next step is to log in to our router and open up requests to the Pi from around the world. For this purpose, we will use a functionality found on most modern routers called port forwarding. Be aware of the risks contained in port forwarding. You are opening up access to your Pi from all around the world, even to malicious ones. I strongly recommend that you change the default password of the user pi before performing this step. You can change passwords using the passwd command. By logging onto a router's management portal and navigating to the Port Forwarding tab, we can open up requests to the Pi's internal network IP address, which we have figured out previously, and the default port of the VNC server, which is 5900. Now, we can provide our external IP address to androidVNC from anywhere around the world instead of an internal IP address that works only if we are on the same network as the Pi. Port forwarding settings on Netgear router administration page Refer to your router's user manual to see how to change the Port Forwarding settings. Most routers require you to connect through the Ethernet port in order to access the management portal instead of Wi-Fi. Summary In this article, we installed Raspbian, warmed up with the Pi, and connected the Pi using an Android device. Resources for Article:   Further resources on this subject: Raspberry Pi LED Blueprints [article] Color and motion finding [article] From Code to the Real World [article]
Read more
  • 0
  • 0
  • 13338

article-image-implementing-microsoft-dynamics-ax
Packt
16 Sep 2015
6 min read
Save for later

Implementing Microsoft Dynamics AX

Packt
16 Sep 2015
6 min read
 In this article by Yogesh Kasat and JJ Yadav, authors of the book Microsoft Dynamics AX Implementation Guide, you will learn one of the important topic in Microsoft Dynamics AX implementation process—configuration data management. (For more resources related to this topic, see here.) The configuration of an ERP system is one of the most important parts of the process. Configuration means setting up the base data and parameters to enable your product features such as financial, shipping, sales tax, and so on. Microsoft Dynamics AX has been developed based on the generic requirements of various organizations and contains the business processes belonging to diverse business segments. It is a very configurable product that allows the implementation team to configure features based on specific business needs. During the project, the implementation team identifies the relevant components of the system and sets up and aligns these components to meet the specific business requirements. This process starts in the analysis phase of the project carrying on through the design, development, and deployment phases. Configuration management is different from data migration. Data migration broadly covers the transactional data of the legacy system and core data such as Opening balances, Open AR, Open AP, customers, vendors, and so on. When we talk about configuration management, we are referring to items like fiscal years and periods, chart of accounts, segments, and defining applicable rules, journal types, customer groups, terms of payments, module-based parameters, workflows, number sequences, and the like. In a broader sense, configuration covers the basic parameters, setup data, and reference data which you configure for the different modules in Dynamics AX. The following diagram shows the different phases of configuration management: In any ERP implementation project, you deal with multiple environments. For example, you start with CRP, after the development you move to the test environment, and then training, UAT, and production, as shown in the following diagram: One of the biggest challenges that an implementation team faces is moving the configuration from one environment to another. If configurations keep changing in every environment, it becomes more difficult to manage them. Similar to code promotion and release management across environments, configuration changes need to be tracked through a change-control process across environments to ensure that you are testing with a consistent set of configurations. The objective is to keep track of all the configuration changes and make sure that they make it to the final cut in the production environment. The following sections outline some approaches used for configuration data management in the Dynamics AX project. The golden environment An environment that is pristine without any transactions—the golden environment—is sometimes referred to as a stage or pre-prod environment. Create the configurations from scratch and/or use various tools to create and update the configuration data. Develop a process to update the configuration in the golden environment once it has been changed and approved in the test environments. The golden environment can be turned into a production environment or the data can be copied over to the production environment using database restore. The golden environment database can be used as a starting point for every run of data migration. For example, if you are preparing for UAT, use the golden environment database as a starting point. Copy to UAT and perform data migration in your UAT environment. This would ensure time you are testing with the golden configurations (If the configuration is missing in the golden environment, you would be able to catch it during testing and fix your UAT and the golden environment too). The pros of the golden environment are given as follows: The golden environment is a single environment for controlling the configuration data It uses all the tools available for the initial configuration There are less number of chances for corruption of the configuration data The cons of the golden environment are given as follows: There is a risk of missing configuration updates due to not following the processes (as the configuration updates are made directly in the testing and UAT environments). There are chances of migrating the revision data into the production environment like workflow history, address revisions, and policies versions. There is a risk of migrating environment-specific data from the golden environment to the production environment. This is not useful for a project going live in multiple phases, as you will not be able to transfer the incremental configuration data using database restore. You must keep the environment in sync with the latest code. Copying the template company In this approach, the implementation team typically defines a template legal entity and configures the template company from scratch. Once completed, the template company's configuration data is copied over to the actual legal entity using the data export/import process. This approach is useful for projects going live in multiple phases, where a global template is created and used across different legal entities. Whereas, in AX 2012, a lot configuration data is shared and it makes it almost impossible to copy the company data. Building configuration templates In this approach, the implementation team typically builds a repository of all the configurations done in a file, imports them in each subsequent environment, and finally, in the production environment. The pros of building configuration templates are as follows: It is a clean approach. You can version-control the configuration file. This approach is very useful for projects going live in multiple phases, as you can import the incremental configuration data in the subsequent releases. This approach may need significant development efforts to create the X+ scripts or DIXF custom entities to import all the required configurations. Summary Clearly there are several options to choose from for configuration data management but they have their own pros and cons. While building configuration template is ideal solution for configuration data management it could be costly as it may need significant development effort to build custom entity to export and import data across environments. The golden environment process is widely used on the implementation projects as it’s easy to manage and require minimal development team involvement. Resources for Article: Further resources on this subject: Web Services and Forms[article] Setting Up and Managing E-mails and Batch Processing[article] Integrating Microsoft Dynamics GP Business Application fundamentals[article]
Read more
  • 0
  • 0
  • 4251

article-image-building-wpf-net-client
Packt
16 Sep 2015
8 min read
Save for later

Building a WPF .NET Client

Packt
16 Sep 2015
8 min read
In this article by Einar Ingebrigtsen, author of the book SignalR: Real-time Application Development - Second Edition we will bring the full feature set of what we've built so far for the web onto the desktop through a WPF .NET client. There are quite a few ways of developing Windows client solutions, and WPF was introduced back in 2005 and has become one of the most popular ways of developing software for Windows. In WPF, we have something called XAML, which is what Windows Phone development supports and is also the latest programming model in Windows 10. In this chapter, the following topics will be covered: MVVM Brief introduction to the SOLID principles XAML WPF (For more resources related to this topic, see here.) Decoupling it all So you might be asking yourself, what is MVVM? It stands for Model View ViewModel: a pattern for client development that became very popular in the XAML stack, enabled by Microsoft based on Martin Fowlers presentation model (http://martinfowler.com/eaaDev/PresentationModel.html). Its principle is that you have a ViewModel that holds the state and exposes behavior that can be utilized from a view. The view observes any changes of the state the ViewModel exposes, making the ViewModel totally unaware that there is a View. The ViewModel is decoupled and can be put in isolation and is perfect for automated testing. As part of the state that the ViewModel typically holds is the model part, which is something it usually gets from the server, and a SignalR hub is the perfect transport to get this. It boils down to recognizing the different concerns that make up the frontend and separating it all. This gives us the following diagram: Decoupling – the next level In this chapter, one of the things we will brush up is the usage of the Dependency Inversion Principle, the D of SOLID. Let's start with the first principle: the S in SOLID of Single Responsibility Principle, which states that a method or a class should only have one reason to change and only have one responsibility. With this, we can't have our units take on more than one responsibility and need help from collaborators to do the entire job. These collaborators are things we now depend on and we should represent these dependencies clearly to our units so that anyone or anything instantiating it knows what we are depending on. We have now flipped around the way in which we get dependencies. Instead of the unit trying to instantiate everything itself, we now clearly state what we need as collaborators, opening up for the calling code to decide what implementations of these dependencies you want to pass on. Also, this is an important aspect; typically, you'd want the dependencies expressed in the form of interfaces, yielding flexibility for the calling code. Basically, what this all means is that instead of a unit or system instantiating and managing its dependencies, we decouple and let something called as the Inversion of Control container deal with this. In the sample, we will use an IoC (Inversion of Control) container called Ninject that will deal with this for us. What it basically does is manage what implementations to give to the dependency specified on the constructor. Often, you'll find that the dependencies are interfaces in C#. This means one is not coupled to a specific implementation and has the flexibility of changing things at runtime based on configuration. Another role of the IOC container is to govern the life cycle of the dependencies. It is responsible for knowing when to create new instances and when to reuse an instance. For instance, in a web application, there are some systems that you want to have a life cycle of per request, meaning that we will get the same instance for the lifetime of a web request. The life cycle is configurable in what is known as a binding. When you explicitly set up the relationship between a contract (interface) and its implementation, you can choose to set up the life cycle behavior as well. Building for the desktop The first thing we will need is a separate project in our solution: Let's add it by right-clicking on the solution in Solution Explorer and navigating to Add | New Project: In the Add New Project dialog box, we want to make sure the .NET Framework 4.5.1 is selected. We could have gone with 4.5, but some of the dependencies that we're going to use have switched to 4.5.1. This is the latest version of the .NET Framework at the time of writing, so if you can, use it. Make sure to select Windows Desktop and then select WPF Application. Give the project the name SignalRChat.WPF and then click on the OK button: Setting up the packages We will need some packages to get started properly. This process is described in detail in Chapter 1, The Primer. Let's start off by adding SignalR, which is our primary framework that we will be working with to move on. We will be pulling this using NuGet, as described in Chapter 1, The Primer: Right-click on the References in Solution Explorer and select Manage NuGet Packages, and type Microsoft.AspNet.SignalR.Client in the Search dialog box. Select it and click on Install. Next, we're going to pull down something called as Bifrost. Bifrost is a library that helps us build MVVM-based solutions on WPF; there are a few other solutions out there, but we'll focus on Bifrost. Add a package called Bifrost.Client. Then, we need the package that gives us the IOC container called Ninject, working together with Bifrost. Add a package called Bifrost.Ninject. Observables One of the things that is part of WPF and all other XAML-based platforms is the notion of observables; be it in properties or collections that will notify when they change. The notification is done through well-known interfaces for this, such as INotifyPropertyChanged or INotifyCollectionChanged. Implementing these interfaces quickly becomes tedious all over the place where you want to notify everything when there are changes. Luckily, there are ways to make this pretty much go away. We can generate the code for this instead, either at runtime or at build time. For our project, we will go for a build-time solution. To accomplish this, we will use something called as Fody and a plugin for it called PropertyChanged. Add another NuGet package called PropertyChanged.Fody. If you happen to get problems during compiling, it could be the result of the dependency to a package called Fody not being installed. This happens for some versions of the package in combination with the latest Roslyn compiler. To fix this, install the NuGet package called Fody explicitly. Now that we have all the packages, we will need some configuration in code: Open the App.xam.cs file and add the following statement: using Bifrost.Configuration; The next thing we will need is a constructor for the App class: public App() { Configure.DiscoverAndConfigure(); } This will tell Bifrost to discover the implementations of the well-known interfaces to do the configuration. Bifrost uses the IoC container internally all the time, so the next thing we will need to do is give it an implementation. Add a class called ContainerCreator at the root of the project. Make it look as follows: using Bifrost.Configuration; using Bifrost.Execution; using Bifrost.Ninject; using Ninject; namespace SignalRChat.WPF { public class ContainerCreator : ICanCreateContainer { public IContainer CreateContainer() { var kernel = new StandardKernel(); var container = new Container(kernel); return container; } } } We've chosen Ninject among others that Bifrost supports, mainly because of familiarity and habit. If you happen to have another favorite, Bifrost supports a few. It's also fairly easy to implement your own support; just go to the source at http://github.com/dolittle/bifrost to find reference implementations. In order for Bifrost to be targeting the desktop, we need to tell it through configuration. Add a class called Configurator at the root of the project. Make it look as follows: using Bifrost.Configuration; namespace SignalRChat.WPF { public class Configurator : ICanConfigure { public void Configure(IConfigure configure) { configure.Frontend.Desktop(); } } } Summary Although there are differences between creating a web solution and a desktop client, the differences have faded over time. We can apply the same principles across the different environments; it's just different programming languages. The SignalR API adds the same type of consistency in thinking, although not as matured as the JavaScript API with proxy generation and so on; still the same ideas and concepts are found in the underlying API. Resources for Article: Further resources on this subject: The Importance of Securing Web Services [article] Working with WebStart and the Browser Plugin [article] Microsoft Azure – Developing Web API for Mobile Apps [article]
Read more
  • 0
  • 0
  • 7972

article-image-straight-blender
Packt
16 Sep 2015
18 min read
Save for later

Straight into Blender!

Packt
16 Sep 2015
18 min read
 In this article by Romain Caudron and Pierre-Armand Nicq, the authors of Blender 3D By Example, you will start getting familiar with Blender. (For more resources related to this topic, see here.) Here, navigation within the interface will be presented. Its approach is atypical in comparison to other 3D software, such as Autodesk Maya® or Autodesk 3DS Max®, but once you get used to this, it will be extremely effective. If you have had the opportunity to use Blender before, it is important to note that the interface went through changes during the evolution of the software (especially since version 2.5). We will give you an idea of the possibilities that this wonderful free and open source software gives by presenting different workflows. You will learn some vocabulary and key concepts of 3D creation so that you will not to get lost during your learning. Finally, you will have a brief introduction to the projects that we will carry out throughout this book. Let's dive into the third dimension! The following topics will be covered in this article: Learning some theory and vocabulary Navigating the 3D viewport How to set up preferences Using keyboard shortcuts to save time An overview of the 3D workflow Before learning how to navigate the Blender interface, we will give you a short introduction to the 3D workflow. An anatomy of a 3D scene To start learning about Blender, you need to understand some basic concepts. Don't worry, there is no need to have special knowledge in mathematics or programming to create beautiful 3D objects; it only requires curiosity. Some artistic notions are a plus. All 3D elements, which you will handle, will evolve in to a scene. There is a three-dimensional space with a coordinate system composed of three axes. In Blender, the x axis shows the width, y axis shows the depth, and the z axis shows the height. Some softwares use a different approach and reverses the y and z axes. These axes are color-coded, we advise you to remember them: the x axis in red, the y axis in green and the z axis in blue. A scene may have the scale you want and you can adjust it according to your needs. This looks like a film set for a movie. A scene can be populated by one or more cameras, lights, models, rigs, and many other elements. You will have the control of their placement and their setup. A 3D scene looks like a film set. A mesh is made of vertices, edges, and faces. The vertices are some points in the scene space that are placed at the end of the edges. They could be thought of as 3D points in space and the edges connect them. Connected together, the edges and the vertices form a face, also called a polygon. It is a geometric plane, which has several sides as its name suggests. In 3D software, a polygon is constituted of at least three sides. It is often essential to favor four-sided polygons during modeling for a better result. You will have an opportunity to see this in more detail later. Your actors and environments will be made of polygonal objects, or more commonly called as meshes. If you have played old 3D games, you've probably noticed the very angular outline of the characters; it was, in fact, due to a low count of polygons. We must clarify that the orientation of the faces is important for your polygon object to be illuminated. Each face has a normal. This is a perpendicular vector that indicates the direction of the polygon. In order for the surface to be seen, it is necessary that the normals point to the outside of the model. Except in special cases where the interior of a polygonal object is empty and invisible. You will be able to create your actors and environment as if you were handling virtual clay to give them the desired shape. Anatomy of a 3D Mesh To make your characters presentable, you will have to create their textures, which are 2D images that will be mapped to the 3D object. UV coordinates will be necessary in order to project the texture onto the mesh. Imagine an origami paper cube that you are going to unfold. This is roughly the same. These details are contained in a square space with the representation of the mesh laid flat. You can paint the texture of your model in your favorite software, even in Blender. This is the representation of the UV mapping process. The texture on the left is projected to the 3D model on the right. After this, you can give the illusion of life to your virtual actors by animating them. For this, you will need to place animation keys spaced on the timeline. If you change the state of the object between two keyframes, you will get the illusion of movement—animation. To move the characters, there is a very interesting process that uses a bone system, mimicking the mechanism of a real skeleton. Your polygon object will be then attached to the skeleton with a weight assigned to the vertices on each bone, so if you animate the bones, the mesh components will follow them. Once your characters, props, or environment are ready, you will be able to choose a focal length and an adequate framework for your camera. In order to light your scene, the choice of the render engine will be important for the kind of lamps to use, but usually there are three types of lamps as used in cinema productions. You will have to place them carefully. There are directional lights, which behave like the sun and produce hard shadows. There are omnidirectional lights, which will allow you to simulate diffuse light, illuminating everything around it and casting soft shadows. There are also spots that will simulate a conical shape. As in the film industry or other imaging creation fields, good lighting is a must-have in order to sell the final picture. Lighting is an expressive and narrative element that can magnify your models, or make them irrelevant. Once everything is in place, you are going to make a render. You will have a choice between a still image and an animated sequence. All the given parameters with the lights and materials will be calculated by the render engine. Some render engines offer an approach based on physics with rays that are launched from the camera. Cycles is a good example of this kind of engine and succeeds in producing very realistic renders. Others will have a much simpler approach, but none less technically based on visible elements from the camera. All of this is an overview of what you will be able to achieve while reading this book and following along with Blender. What can you do with Blender? In addition to being completely free and open source, Blender is a powerful tool that is stable and with an integral workflow that will allow you to understand your learning of 3D creation with ease. Software updates are very frequent; they fix bugs and, more importantly, add new features. You will not feel alone as Blender has an active and passionate community around it. There are many sites providing tutorials, and an official documentation detailing the features of Blender. You will be able to carry out everything you need in Blender, including things that are unusual for a 3D package such as concept art creation, sculpting, or digital postproduction, which we have not yet discussed, including compositing and video editing. This is particularly interesting in order to push the aesthetics of your future images and movies to another level. It is also possible to make video games. Also, note that the Blender game engine is still largely unknown and underestimated. Although this aspect of the software is not as developed as other specialized game engines, it is possible to make good quality games without switching to another software. You will realize that the possibilities are enormous, and you will be able to adjust your workflow to suit your needs and desires. Software of this type could scare you by its unusual handling and its complexity, but you'll realize that once you have learned its basics, it is really intuitive in many ways. Getting used to the navigation in Blender Now that you have been introduced to the 3D workflow, you will learn how to navigate the Blender interface, starting with the 3D viewport. An introduction to the navigation of the 3D Viewport It is time to learn how to navigate in the Blender viewport. The viewport represents the 3D space, in which you will spend most of your time. As we previously said, it is defined by three axes (x, y, and z). Its main goal is to display the 3D scene from a certain point of view while you're working on it. The Blender 3D Viewport When you are navigating through this, it will be as if you were a movie director but with special powers that allow you to film from any point of view. The navigation is defined by three main actions: pan, orbit, and zoom. The pan action means that you will move horizontally or vertically according to your current point of view. If we connect that to our cameraman metaphor, it's like if you were moving laterally to the left, or to the right, or moving up or down with a camera crane. By default, in Blender the shortcut to pan around is to press the Shift button and the Middle Mouse Button (MMB), and drag the mouse. The orbit action means that you will rotate around the point that you are focusing on. For instance, imagine that you are filming a romantic scene of two actors and that you rotate around them in a circular manner. In this case, the couple will be the main focus. In a 3D scene, your main focus would be a 3D character, a light, or any other 3D object. To orbit around in the Blender viewport, the default shortcut is to press the MMB and then drag the mouse. The last action that we mentioned is zoom. The zoom action is straightforward. It is the action of moving our point of view closer to an element or further away from an element. In Blender, you can zoom in by scrolling your mouse wheel up and zoom out by scrolling your mouse wheel down. To gain time and precision, Blender proposes some predefined points of view. For instance, you can quickly go in a top view by pressing the numpad 7, you can also go in a front view by pressing the numpad 1, you can go in a side view by pressing the numpad 3, and last but not least, the numpad 0 allows you to go in Camera view, which represents the final render point of the view of your scene. You can also press the numpad 5 in order to activate or deactivate the orthographic mode. The orthographic mode removes perspective. It is very useful if you want to be precise. It feels as if you were manipulating a blueprint of the 3D scene. The difference between Perspective (left) and Orthographic (right) If you are lost, you can always look at the top left corner of the viewport in order to see in which view you are, and whether the orthographic mode is on or off. Try to learn by heart all these shortcuts; you will use them a lot. With repetition, this will become a habit. What are editors? In Blender, the interface is divided into subpanels that we call editors; even the menu bar where you save your file is an editor. Each editor gives you access to tools categorized by their functionality. You have already used an editor, the 3D view. Now it's time to learn more about the editor's anatomy. In this picture, you can see how Blender is divided into editors The anatomy of an editor There are 17 different editors in Blender and they all have the same base. An editor is composed of a Header, which is a menu that groups different options related to the editor. The first button of the header is to switch between other editors. For instance, you can replace the 3D view by the UV Image Editor by clicking on it. You can easily change its place by right-clicking on it in an empty space and by choosing the Flip to Top/Bottom option. The header can be hidden by selecting its top edge and by pulling it down. If you want to bring it back, press the little plus sign at the far right. The header of the 3D viewport. The first button is for switching between editors, and also, we can choose between different options in the menu In some editors, you can get access to hidden panels that give you other options. For instance, in the 3D view you can press the T key or the N key to toggle them on or off. As in the header, if a sub panel of an editor is hidden, you can click on the little plus sign to display it again. Split, Join, and Detach Blender offers you the possibility of creating editors where you want. To do this, you need to right-click on the border of an editor and select Split Area in order to choose where to separate them. Right-click on the border of an editor to split it into two editors The current editor will then be split in two editors. Now you can switch to any other editor that you desire by clicking on the first button of the header bar. If you want to merge two editors into one, you can right-click on the border that separates them and select the Join Area button. You will then have to click on the editor that you want to erase by pointing the arrow on it. Use the Join Area option to join two editors together You then have to choose which editor you want to remove by pointing and clicking on it. We are going to see another method of splitting editors that is nice. You can drag the top right corner of an editor and another editor will magically appear! If you want to join back two editors together, you will have to drag the top right corner in the direction of the editor that you want to remove. The last manipulation can be tricky at first, but with a little bit of practice, you will also be able to do it with closed eyes! The top right corner of an editor If you have multiple monitors, it could be a great idea to detach some editors in a separated window. With this, you could gain space and won't be overwhelmed by a condensed interface. In order to do this, you will need to press the Shift key and drag the top right corner of the editor with the Left Mouse Button (LMB). Some useful layout presets Blender offers you many predefined layouts that depend on the context of your creation. For instance, you can select the Animation preset in order to have all the major animation tools, or you can use the UV Editing preset in order to prepare your texturing. To switch between the presets, go to the top of the interface (in the Info editor, near the Help menu) and click on the drop-down menu. If you want, you can add new presets by clicking on the plus sign or delete presets by clicking on the X button. If you want to rename a preset, simply enter a new name in the corresponding text field. The following screenshot shows the Layout presets drop-down menu: The layout presets drop-down menu Setting up your preferences When we start learning new software, it's good to know how to set up your preferences. Blender has a large number of options, but we will show you just the basic ones in order to change the default navigation style or to add new tools that we call add-ons in Blender. An introduction to the Preferences window The preferences window can be opened by navigating to the File menu and selecting the User Preferences option. If you want, you can use the Ctrl + Alt + U shortcut or the Cmd key and comma key on a Mac system. There are seven tabs in this window as shown here: The different tabs that compose the Preferences window A nice thing that Blender offers is the ability to change its default theme. For this, you can go to the Themes tab and choose between different presets or even change the aspect of each interface elements. Another useful setting to change is the number of undo that is 32 steps, by default. To change this number, go to the Editing tab and under the Undo label, slide the Steps to the desired value. Customizing the default navigation style We will now show you how to use a different style of navigation in the viewport. In many other 3D programs, such as Autodesk Maya®, you can use the Alt key in order to navigate in the 3D view. In order to activate this in Blender, navigate to the Input tab, and under the Mouse section, check the Emulate 3 Button Mouse option. Now if you want to use this navigation style in the viewport, you can press Alt and LMB to orbit around, Ctrl + Alt and the LMB to zoom, and Alt + Shift and the LMB to pan. Remember these shortcuts as they will be very useful when we enter the sculpting mode while using a pen tablet. The Emulate 3 Button Mouse checkbox is shown as follows: The Emulate 3 Button Mouse will be very useful when sculpting using a pen tablet Another useful setting is the Emulate Numpad. It allows you to use the numeric keys that are above the QWERTY keys in addition to the numpad keys. This is very useful for changing the views if you have a laptop without a numpad, or if you want to improve your workflow speed. The Emulate Numpad allows you to use the numeric keys above the QWERTY keys in order to switch views or toggle the perspective on or off Improving Blender with add-ons If you want even more tools, you can install what is called as add-ons on your copy of Blender. Add-ons, also called Plugins or Scripts, are Python files with the .py extension. By default, Blender comes with many disabled add-ons ordered by category. We will now activate two very useful add-ons that will improve our speed while modeling. First, go to the Add-ons tab, and click on the Mesh button in the category list at the left. Here, you will see all the default mesh add-ons available. Click on the check-boxes at the left of the Mesh: F2 and Mesh: LoopTools subpanels in order to activate these add-ons. If you know the name of the add-on you want to activate, you can try to find it by typing its name in the search bar. There are many websites where you can download free add-ons, starting from the official Blender website. If you want to install a script, you can click on the Install from File button and you will be asked to select the corresponding Python file. The official Blender Add-ons Catalog You can find it at http://wiki.blender.org/index.php/Extensions:2.6/Py/Scripts. The following screenshot shows the steps for activating the add-ons: Steps for Add-ons activation Where are the add-ons on the hard-disk? All the scripts are placed in the add-ons folder that is located wherever you have installed Blender on your hard disk. This folder will usually be at Your Installation PathBlender FoundationBlender2.VersionNumberscriptsaddons. If you find it easier, you can drop the Python files here instead of at the standard installation. Don't forget to click on the Save User Settings button in order to save all your changes! Summary In this article, you have learned the steps behind 3D creations. You know what a mesh is and what it is composed of. Then you have been introduced to navigation in Blender by manipulating the 3D viewport and going through the user preference menu. In the later sections, you configured some preferences and extended Blender by activating some add-ons. Resources for Article: Further resources on this subject: Editing the UV islands[article] Working with Blender[article] Designing Objects for 3D Printing [article]
Read more
  • 0
  • 0
  • 3159
Unlock access to the largest independent learning library in Tech for FREE!
Get unlimited access to 7500+ expert-authored eBooks and video courses covering every tech area you can think of.
Renews at $19.99/month. Cancel anytime
article-image-building-solutions-using-patterns
Packt
16 Sep 2015
6 min read
Save for later

Building Solutions Using Patterns

Packt
16 Sep 2015
6 min read
In this article by Mark Brummel, the author of Learning Dynamics NAV Patterns, we will learn how to create an application using Dynamics Nav. While creating an application, we can apply patterns and coding concepts into a new module that is recognizable for the users to be as a Microsoft Dynamics NAV application, and is easy to understand and maintain by other developers. The solution that we will make is for a small bed and breakfast (B&B), allowing them to manage their rooms and reservations. This can be integrated into the financial part of Dynamics NAV. It is not the intention of this article to make a full-featured finished product. We will discuss the basic design principles, and the decision making processes. Therefore, we simplify the functional process. One of the restrictions in our application is that we rent rooms per night. This article will be covering the following topics: Building blocks Creating the Table objects (For more resources related to this topic, see here.) Building blocks We borrowed the term classes from the object-oriented programming as a collection of things that belong together. Classes can be tables or code units in Microsoft Dynamics NAV. The first step in our process is to define the classes. These will be created as tables or code units, following the patterns that we have learned: Setup This is the generic set of parameters for the application. Guest This is the person who stays at our B&B. This can be one or two persons, or a group (family). Room Our B&B has a number of rooms with different attributes that determine the price, together with the season. Season This is the time of the year. Price This is the price for one night in a room. Reservation Rooms can be reserved on a daily basis with a starting and ending date. Stay This is the set of one or more consecutive nights at our B&B. Check-In This is the start of a stay, checking in for reservation. Check-Out At the end of a stay, we would like to send a bill. Clean Whenever a room is cleaned, we would like to register this. Evaluation Each stay can be evaluated by a customer. Invoice This generate a Sales Invoice for a Stay. Apply Architectural Patterns The second step is to decide per class which Architectural Patterns we can use. In some special cases, we might need to write down new patterns, based on the data structures that are not used in the standard application. Setup For the application setup, we will use the Singleton pattern. This allows us to define a single set of values for the entire application that is kept in memory during the lifetime of the system. Guest To register our guests, we will use the standard Customer table in Dynamics NAV. This has pros and cons. The good thing about doing this is the ability to use all the standard analysis options in the application for our customers without reinventing the wheel. Some B&B users might decide to also sell souvenirs or local products so that they can use items and the standard trade part of Dynamics NAV. We can also use the campaigns in the Relationship Management module. The bad part, or challenge, is upgradability. If we were to add fields to the customer table, or modify the standard page elements, we will have to merge these into the application each time we get a new version of the product, which is once per month. We will use the new delta file, as well as the testability framework to challenge this. Room The architectural pattern for a room is a tough decision. Most users of our system run a small B&B, so we can consider rooms to be the setup data. Number Series is not a required pattern. We will therefore decide to implement a Supplemental Table. Season Each B&B can setup their own seasons. They are used to determine price, but when not used, the system will have to work too. We implement a Supplemental Table too. Price Rooms can have a default price, or a price per season and a guest. Based on this requirement, we will implement the Rules Pattern that allows us a complex array of setup values. Reservation We want to carefully trace reservations and cancellations per room and per guest. We would like to analyze the data based on the season. For this feature, we will implement the Journal-Batch-Line pattern and introduce an Entry table that is managed by the Journal. Stay We would like to register each unique stay in our system rather than individual nights. This allows us to easily combine parameters, and generate a total price. We will implement this as a Master Data, based on the requirement to be able to use number series. The Stay does not have requirements for a lines table, nor does it represent a document in our organization. Check-In When a guest checks in to the bed and breakfast, we can check a reservation and apply the reservation to the Stay. Check-Out When a guest leaves, we would like to setup the final bill, and ask to evaluate the stay. This process will be a method on the Stay class with encapsulated functions, creating the sales invoice, and generating an evaluation document. Clean Rooms have to be cleaned each day when a guest stays, but at least once a week when the room is empty. We will use the entry pattern without a journal. Clean will be a method on the Room class. Each day we will generate entries using the Job Queue Entry pattern. The Room will also have a method that indicates if a room has been cleaned. Evaluation A Stay in our B&B can be evaluated by our guests. The evaluation has a different criteria. We will use the Document Pattern. Invoice We can create the method as an encapsulated method of the Stay class. In order to link the Sales Invoice to the Stay, we will add the Stay No. field to the Sales Header, the Sales Invoice Header, and the Sales Cr.Memo Header tables. Creating the Table Objects Based on the Architectural Patterns, we can define a set of objects that we can start working with, which is as follows: Object names are limited to 30 characters, which is challenging for naming them. The Bed and Breakfast name illustrates this challenge. Only use abbreviation when the limitation of length is a problem. Summary In this article, you learned how to define classes for building an application. You have also learned about the kinds of architectural patterns that will be involved in creating the classes in your application. Resources for Article: Further resources on this subject: Performance by Design [article] Advanced Data Access Patterns [article] Formatting Report Items and Placeholders [article]
Read more
  • 0
  • 0
  • 991

article-image-virtualization-0
Packt
16 Sep 2015
16 min read
Save for later

Virtualization

Packt
16 Sep 2015
16 min read
This article by Skanda Bhargav, the author of Troubleshooting Ubuntu Server, deals with virtualization techniques—why virtualization is important and how administrators can install and serve users with services via virtualization. We will learn about KVM, Xen, and Qemu. So sit back and let's take a spin into the virtual world of Ubuntu. (For more resources related to this topic, see here.) What is virtualization? Virtualization is a technique by which you can convert a set of files into a live running machine with an OS. It is easy to set up one machine and much easier to clone and replicate the same machine across hardware. Also, each of the clones can be customized based on requirements. We will look at setting up a virtual machine using Kernel-based Virtual Machine, Xen, and Qemu in the sections that follow. Today, people are using the power of virtualization in different situations and environments. Developers use virtualization in order to have an independent environment in which to safely test and develop applications without affecting other working environments. Administrators are using virtualization to separate services and also commission or decommission services as and when required or requested. By default, Ubuntu supports the Kernel-based Virtual Machine (KVM), which has built-in extensions for AMD and Intel-based processors. Xen and Qemu are the options suggested where you have hardware that does not have extensions for virtualization. libvirt The libvirt library is an open source library that is helpful for interfacing with different virtualization technologies. One small task before starting with libvirt is to check your hardware support extensions for KVM. The command to do so is as follows: kvm-ok You will see a message stating whether or not your CPU supports hardware virtualization. An additional task would be to verify the BIOS settings for virtualization and activate it. Installation Use the following command to install the package for libvirt: sudo apt-get install kvm libvirt-bin Next, you will need to add the user to the group libvirt. This will ensure that user gets additional options for networking. The command is as follows: sudo adduser $USER libvirtd We are now ready to install a guest OS. Its installation is very similar to that of installing a normal OS on the hardware. If your virtual machine needs a graphical user interface (GUI), you can make use of an application virt-viewer and connect using VNC to the virtual machine's console. We will be discussing the virt-viewer and its uses in the later sections of this article. virt-install virt-install is a part of the python-virtinst package. The command to install this package is as follows: sudo apt-get install python-virtinst One of the ways of using virt-install is as follows: sudo virt-install -n new_my_vm -r 256 -f new_my_vm.img -s 4 -c jeos.iso --accelerate --connect=qemu:///system --vnc --noautoconsole -v Let's understand the preceding command part by part: -n: This specifies the name of virtual machine that will be created -r: This specifies the RAM amount in MBs -f: This is the path for the virtual disk -s: This specifies the size of the virtual disk -c: This is the file to be used as virtual CD, but it can be an .iso file as well --accelerate: This is used to make use of kernel acceleration technologies --vnc: This exports the guest console via vnc --noautoconsole: This disables autoconnect for the virtual machine console -v: This creates a fully virtualized guest Once virt-install is launched, you may connect to console with virt-viewer utility from remote connections or locally using GUI. Use to wrap long text to next line. virt-clone One of the applications to clone a virtual machine to another is virt-clone. Cloning is a process of creating an exact replica of the virtual machine that you currently have. Cloning is helpful when you need a lot of virtual machines with same configuration. Here is an example of cloning a virtual machine: sudo virt-clone -o my_vm -n new_vm_clone -f /path/to/ new_vm_clone.img --connect=qemu:///sys Let's understand the preceding command part by part: -o: This is the original virtual machine that you want to clone -n: This is the new virtual machine name -f: This is the new virtual machine's file path --connect: This specifies the hypervisor to be used Managing the virtual machine Let's see how to manage the virtual machine we installed using virt. virsh Numerous utilities are available for managing virtual machines and libvirt; virsh is one such utility that can be used via command line. Here are a few examples: The following command lists the running virtual machines: virsh -c qemu:///system list The following command starts a virtual machine: virsh -c qemu:///system start my_new_vm The following command starts a virtual machine at boot: virsh -c qemu:///system autostart my_new_vm The following command restarts a virtual machine: virsh -c qemu:///system reboot my_new_vm You can save the state of virtual machine in a file. It can be restored later. Note that once you save the virtual machine, it will not be running anymore. The following command saves the state of the virtual machine: virsh -c qemu://system save my_new_vm my_new_vm-290615.state The following command restores a virtual machine from saved state: virsh -c qemu:///system restore my_new_vm-290615.state The following command shuts down a virtual machine: virsh -c qemu:///system shutdown my_new_vm The following command mounts a CD-ROM in the virtual machine: virsh -c qemu:///system attach-disk my_new_vm /dev/cdrom /media/cdrom The virtual machine manager A GUI-type utility for managing virtual machines is virt-manager. You can manage both local and remote virtual machines. The command to install the package is as follows: sudo apt-get install virt-manager The virt-manager works on a GUI environment. Hence, it is advisable to install it on a remote machine other than the production cluster, as production cluster should be used for doing the main tasks. The command to connect the virt-manager to a local server running libvirt is as follows: virt-manager -c qemu:///system If you want to connect the virt-manager from a different machine, then first you need to have SSH connectivity. This is required as libvirt will ask for a password on the machine. Once you have set up passwordless authentication, use the following command to connect manager to server: virt-manager -c qemu+ssh://virtnode1.ubuntuserver.com/system Here, the virtualization server is identified with the hostname ubuntuserver.com. The virtual machine viewer A utility for connecting to your virtual machine's console is virt-viewer. This requires a GUI to work with the virtual machine. Use the following command to install virt-viewer: sudo apt-get install virt-viewer Now, connect to your virtual machine console from your workstation using the following command: virt-viewer -c qemu:///system my_new_vm You may also connect to a remote host using SSH passwordless authentication by using the following command: virt-viewer -c qemu+ssh://virtnode4.ubuntuserver.com/system my_new_vm JeOS JeOS, short for Just Enough Operation System, is pronounced as "Juice" and is an operating system in the Ubuntu flavor. It is specially built for running virtual applications. JeOS is no longer available as a downloadable ISO CD-ROM. However, you can pick up any of the following approaches: Get a server ISO of the Ubuntu OS. While installing, hit F4 on your keyboard. You will see a list of items and select the one that reads Minimal installation. This will install the JeOS variant. Build your own copy with vmbuilder from Ubuntu. The kernel of JeOS is specifically tuned to run in virtual environments. It is stripped off of the unwanted packages and has only the base ones. JeOS takes advantage of the technological advancement in VMware products. A powerful combination of limited size with performance optimization is what makes JeOS a preferred OS over a full server OS in a large virtual installation. Also, with this OS being so light, the updates and security patches will be small and only limited to this variant. So, the users who are running their virtual applications on the JeOS will have less maintenance to worry about compared to a full server OS installation. vmbuilder The second way of getting the JeOS is by building your own copy of Ubuntu; you need not download any ISO from the Internet. The beauty of vmbuilder is that it will get the packages and tools based on your requirements. Then, build a virtual machine with these and the whole process is quick and easy. Essentially, vmbuilder is a script that will automate the process of creating a virtual machine, which can be easily deployed. Currently, the virtual machines built with vmbuilder are supported on KVM and Xen hypervisors. Using command-line arguments, you can specify what additional packages you require, remove the ones that you feel aren't necessary for your needs, select the Ubuntu version, and do much more. Some developers and admins contributed to the vmbuilder and changed the design specifics, but kept the commands same. Some of the goals were as follows: Reusability by other distributions Plugin feature added for interactions, so people can add logic for other environments A web interface along with CLI for easy access and maintenance Setup Firstly, we will need to set up libvirt and KVM before we use vmbuilder. libvirt was covered in the previous section. Let's now look at setting up KVM on your server. We will install some additional packages along with the KVM package, and one of them is for enabling X server on the machine. The command that you will need to run on your Ubuntu server is as follows: sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils The output of this command will be as follows: Let's look at what each of the packages mean: libvirt-bin: This is used by libvirtd for administration of KVM and Qemu qemu-kvm: This runs in the background ubuntu-vm-builder: This is a tool for building virtual machines from the command line bridge-utils: This enables networking for various virtual machines Adding users to groups You will have to add the user to the libvirtd command; this will enable them to run virtual machines. The command to add the current user is as follows: sudo adduser `id -un` libvirtd The output is as follows:   Installing vmbuilder Download the latest vmbuilder called python-vm-builder. You may also use the older ubuntu-vm-builder, but there are slight differences in the syntax. The command to install python-vm-builder is as follows: sudo apt-get install python-vm-builder The output will be as follows:   Defining the virtual machine While defining the virtual machine that you want to build, you need to take care of the following two important points: Do not assume that the enduser will know the technicalities of extending the disk size of virtual machine if the need arises. Either have a large virtual disk so that the application can grow or document the process to do so. However, it would be better to have your data stored in an external storage device. Allocating RAM is fairly simple. But remember that you should allocate your virtual machine an amount of RAM that is safe to run your application. To check the list of parameters that vmbuilder provides, use the following command: vmbuilder ––help   The two main parameters are virtualization technology, also known as hypervisor, and targeted distribution. The distribution we are using is Ubuntu 14.04, which is also known as trusty because of its codename. The command to check the release version is as follows: lsb_release -a The output is as follows:   Let's build a virtual machine on the same version of Ubuntu. Here's an example of building a virtual machine with vmbuilder: sudo vmbuilder kvm ubuntu --suite trusty --flavour virtual --arch amd64 -o --libvirt qemu:///system Now, we will discuss what the parameters mean: --suite: This specifies which Ubuntu release we want the virtual machine built on --flavour: This specifies which virtual kernel to use to build the JeOS image --arch: This specifies the processor architecture (64 bit or 32 bit) -o: This overwrites the previous version of the virtual machine image --libvirt: This adds the virtual machine to the list of available virtual machines Now that we have created a virtual machine, let's look at the next steps. JeOS installation We will examine the settings that are required to get our virtual machine up and running. IP address A good practice for assigning IP address to the virtual machines is to set a fixed IP address, usually from the private pool. Then, include this info as part of the documentation. We will define an IP address with following parameters: --ip (address): This is the IP address in dotted form --mask (value): This is the IP mask in dotted form (default is 255.255.255.0) --net (value): This is the IP net address (default is X.X.X.0) --bcast (value): This is the IP broadcast (default is X.X.X.255) --gw (address): This is the gateway address (default is X.X.X.1) --dns (address): This is the name server address (default is X.X.X.1) Our command looks like this now: sudo vmbuilder kvm ubuntu --suite trusty --flavour virtual --arch amd64 -o --libvirt qemu:///system --ip 192.168.0.10 You may have noticed that we have assigned only the IP, and all others will take the default value. Enabling the bridge We will have to enable the bridge for our virtual machines, as various remote hosts will have to access the applications. We will configure libvirt and modify the vmbuilder template to do so. First, create the template hierarchy and copy the default template into this folder: mkdir -p VMBuilder/plugins/libvirt/templates cp /etc/vmbuilder/libvirt/* VMBuilder/plugins/libvirt/templates/ Use your favorite editor and modify the following lines in the VMBuilder/plugins/libvirt/templates/libvirtxml.tmpl file: <interface type='network'> <source network='default'/> </interface> Replace these lines with the following lines: <interface type='bridge'> <source bridge='br0'/> </interface>   Partitions You have to allocate partitions to applications for their data storage and working. It is normal to have a separate storage space for each application in /var. The command provided by vmbuilder for this is --part: --part PATH vmbuilder will read the file from the PATH parameter and consider each line as a separate partition. Each line has two entries, mountpoint and size, where size is defined in MBs and is the maximum limit defined for that mountpoint. For this particular exercise, we will create a new file with name vmbuilder.partition and enter the following lines for creating partitions: root 6000 swap 4000 --- /var 16000 Also, please note that different disks are identified by the delimiter ---. Now, the command should be like this: sudo vmbuilder kvm ubuntu --suite trusty --flavour virtual --arch amd64 -o --libvirt qemu:///system --ip 192.168.0.10 --part vmbuilder.partition Use to wrap long text to the next line. Setting the user and password We have to define a user and a password in order for the user to log in to the virtual machine after startup. For now, let's use a generic user identified as user and the password password. We can ask user to change the password after first login. The following parameters are used to set the username and password: --user (username): This sets the username (default is ubuntu) --name (fullname): This sets a name for the user (default is ubuntu) --pass (password): This sets the password for the user (default is ubuntu) So, now our command will be as follows: sudo vmbuilder kvm ubuntu --suite trusty --flavour virtual --arch amd64 -o --libvirt qemu:///system --ip 192.168.0.10 --part vmbuilder.partition --user user --name user --pass password Final steps in the installation – first boot There are certain things that will need to be done at the first boot of a machine. We will install openssh-server at first boot. This will ensure that each virtual machine has a key, which is unique. If we had done this earlier in the setup phase, all virtual machines would have been given the same key; this might have posed a security issue. Let's create a script called first_boot.sh and run it at the first boot of every new virtual machine: # This script will run the first time the virtual machine boots # It is run as root apt-get update apt-get install -qqy --force-yes openssh-server Then, add the following line to the command line: --firstboot first_boot.sh Final steps in the installation – first login Remember we had specified a default password for the virtual machine. This means all the machines where this image will be used for installation will have the same password. We will prompt the user to change the password at first login. For this, we will use a shell script named first_login.sh. Add the following lines to the file: # This script is run the first time a user logs in. echo "Almost at the end of setting up your machine" echo "As a security precaution, please change your password" passwd Then, add the parameter to your command line: --firstlogin first_login.sh Auto updates You can make your virtual machine update itself at regular intervals. To enable this feature, add a package named unattended-upgrades to the command line: --addpkg unattended-upgrades ACPI handling ACPI handling will enable your virtual machine to take care of shutdown and restart events that are received from a remote machine. We will install the acipd package for the same: --addpkg acipd The complete command So, the final command with the parameters that we discussed previously would look like this: sudo vmbuilder kvm ubuntu --suite trusty --flavour virtual --arch amd64 -o --libvirt qemu:///system --ip 192.168.0.10 --part vmbuilder.partition --user user --name user --pass password --firstboot first_boot.sh --firstlogin first_login.sh --addpkg unattended-upgrades --addpkg acipd   Summary In this article, we discussed various virtualization techniques. We discussed virtualization as well as the tools and packages that help in creating and running a virtual machine. Also, you learned about the ways we can view, manage, connect to, and make use of the applications running on the virtual machine. Then, we saw the lightweight version of Ubuntu that is fine-tuned to run virtualization and applications on a virtual platform. At the later stages of this article, we covered how to build a virtual machine from a command line, how to add packages, how to set up user profiles, and the steps for first boot and first login. Resources for Article: Further resources on this subject: Introduction to OpenVPN [article] Speeding up Gradle builds for Android [article] Installing Red Hat CloudForms on Red Hat OpenStack [article]
Read more
  • 0
  • 0
  • 6715

article-image-recommender-systems
Packt
16 Sep 2015
6 min read
Save for later

Recommender Systems

Packt
16 Sep 2015
6 min read
In this article by Suresh K Gorakala and Michele Usuelli, authors of the book Building a Recommendation System with R, we will learn how to prepare relevant data by covering the following topics: Selecting the most relevant data Exploring the most relevant data Normalizing the data Binarizing the data (For more resources related to this topic, see here.) Data preparation Here, we show how to prepare the data to be used in recommender models. These are the steps: Select the relevant data. Normalize the data. Selecting the most relevant data On exploring the data, you will notice that the table contains: Movies that have been viewed only a few times; their rating might be biased because of lack of data Users that rated only a few movies; their rating might be biased We need to determine the minimum number of users per movie and vice versa. The correct solution comes from an iteration of the entire process of preparing the data, building a recommendation model, and validating it. Since we are implementing the model for the first time, we can use a rule of thumb. After having built the models, we can come back and modify the data preparation. We define ratings_movies containing the matrix that we will use. It takes the following into account: Users who have rated at least 50 movies Movies that have been watched at least 100 times The following code shows this: ratings_movies <- MovieLense[rowCounts(MovieLense) > 50, colCounts(MovieLense) > 100] ratings_movies ## 560 x 332 rating matrix of class 'realRatingMatrix' with 55298 ratings. ratings_movies contains about half the number of users and a fifth of the number of movies that MovieLense has. Exploring the most relevant data Let's visualize the top 2 percent of users and movies of the new matrix: # visualize the top matrix min_movies <- quantile(rowCounts(ratings_movies), 0.98) min_users <- quantile(colCounts(ratings_movies), 0.98) Let's build the heat-map: image(ratings_movies[rowCounts(ratings_movies) > min_movies, colCounts(ratings_movies) > min_users], main = ""Heatmap of the top users and movies"") As you have already noticed, some rows are darker than the others. This might mean that some users give higher ratings to all the movies. However, we have visualized the top movies only. In order to have an overview of all the users, let's take a look at the distribution of the average rating by users: average_ratings_per_user <- rowMeans(ratings_movies) Let's visualize the distribution: qplot(average_ratings_per_user) + stat_bin(binwidth = 0.1) + ggtitle(""Distribution of the average rating per user"") As suspected, the average rating varies a lot across different users. Normalizing the data Users that give high (or low) ratings to all their movies might bias the results. We can remove this effect by normalizing the data in such a way that the average rating of each user is 0. The prebuilt normalize function does it automatically: ratings_movies_norm <- normalize(ratings_movies) Let's take a look at the average rating by user. sum(rowMeans(ratings_movies_norm) > 0.00001) ## [1] 0 As expected, the mean rating of each user is 0 (apart from the approximation error). We can visualize the new matrix using an image. Let's build the heat-map: # visualize the normalised matrix image(ratings_movies_norm[rowCounts(ratings_movies_norm) > min_movies,colCounts(ratings_movies_norm) > min_users],main = ""Heatmap of the top users and movies"") The first difference that we can notice are the colors, and it's because the data is continuous. Previously, the rating was an integer number between 1 and 5. After normalization, the rating can be any number between -5 and 5. There are still some lines that are more blue and some that are more red. The reason is that we are visualizing only the top movies. We already checked that the average rating is 0 for each user. Binarizing the data A few recommendation models work on binary data, so we might want to binarize our data, that is, define a table containing only 0s and 1s. The 0s will be treated as either missing values or bad ratings. In our case, we can do either of the following: Define a matrix that has 1 if the user rated the movie and 0 otherwise. In this case, we are losing the information about the rating. Define a matrix that has 1 if the rating is more than or equal to a definite threshold (for example 3) and 0 otherwise. In this case, giving a bad rating to a movie is equivalent to not rating it. Depending on the context, one choice is more appropriate than the other. The function to binarize the data is binarize. Let's apply it to our data. First, let's define a matrix equal to 1 if the movie has been watched, that is, if its rating is at least 1. ratings_movies_watched <- binarize(ratings_movies, minRating = 1) Let's take a look at the results. In this case, we will have black-and-white charts, so we can visualize a bigger portion of users and movies, for example, 5 percent. Similar to what we did earlier, let's select the 5 percent using quantile. The row and column counts are the same as the original matrix, so we can still apply rowCounts and colCounts on ratings_movies: min_movies_binary <- quantile(rowCounts(ratings_movies), 0.95) min_users_binary <- quantile(colCounts(ratings_movies), 0.95) Let's build the heat-map: image(ratings_movies_watched[rowCounts(ratings_movies) > min_movies_binary, colCounts(ratings_movies) > min_users_binary],main = ""Heatmap of the top users and movies"") Only a few cells contain non-watched movies. This is just because we selected the top users and movies. Let's use the same approach to compute and visualize the other binary matrix. Now, each cell is one if the rating is above a threshold, for example 3, and 0 otherwise. ratings_movies_good <- binarize(ratings_movies, minRating = 3) Let's build the heat-map: image(ratings_movies_good[rowCounts(ratings_movies) > min_movies_binary, colCounts(ratings_movies) > min_users_binary], main = ""Heatmap of the top users and movies"") As expected, we have more white cells now. Depending on the model, we can leave the ratings matrix as it is or normalize/binarize it. Summary In this article, you learned about data preparation and how you should select, explore, normalize, and binarize the data. Resources for Article: Further resources on this subject: Structural Equation Modeling and Confirmatory Factor Analysis [article] Warming Up [article] https://www.packtpub.com/books/content/supervised-learning [article]
Read more
  • 0
  • 0
  • 2624

article-image-working-user-interface
Packt
16 Sep 2015
17 min read
Save for later

Working on the User Interface

Packt
16 Sep 2015
17 min read
In this article by Fabrizio Caldarelli, author of the book Yii By Example, will cover the following topics related to the user interface in this article: Customizing JavaScript and CSS Using AJAX Using the Bootstrap widget Viewing multiple models in the same view Saving linked models in the same view It is now time for you to learn what Yii2 supports in order to customize the JavaScript and CSS parts of web pages. A recurrent use of JavaScript is to handle AJAX calls, that is, to manage widgets and compound controls (such as a dependent drop-down list) from jQuery and Bootstrap. Finally, we will employ jQuery to dynamically create more models from the same class in the form, which will be passed to the controller in order to be validated and saved. (For more resources related to this topic, see here.) Customize JavaScript and CSS Using JavaScript and CSS is fundamental to customize frontend output. Differently from Yii1, where calling JavaScript and CSS scripts and files was done using the Yii::app() singleton, in the new framework version, Yii2, this task is part of the yiiwebView class. There are two ways to call JavaScript or CSS: either directly passing the code to be executed or passing the path file. The registerJs() function allows us to execute the JavaScript code with three parameters: The first parameter is the JavaScript code block to be registered The second parameter is the position where the JavaScript tag should be inserted (the header, the beginning of the body section, the end of the body section, enclosed within the jQuery load() method, or enclosed within the jQuery document.ready() method, which is the default) The third and last parameter is a key that identifies the JavaScript code block (if it is not provided, the content of the first parameter will be used as the key) On the other hand, the registerJsFile() function allows us to execute a JavaScript file with three parameters: The first parameter is the path file of the JavaScript file The second parameter is the HTML attribute for the script tag, with particular attention given to the depends and position values, which are not treated as tag attributes The third parameter is a key that identifies the JavaScript code block (if it's not provided, the content of the first parameter will be used as the key) CSS, similar to JavaScript, can be executed using the code or by passing the path file. Generally, JavaScript or CSS files are published in the basic/web folder, which is accessible without restrictions. So, when we have to use custom JavaScript or CSS files, it is recommended to put them in a subfolder of the basic/web folder, which can be named as css or js. In some circumstances, we might be required to add a new CSS or JavaScript file for all web application pages. The most appropriate place to put these entries is AppAsset.php, a file located in basic/assets/AppAsset.php. In it, we can add CSS and JavaScript entries required in web applications, even using dependencies if we need to. Using AJAX Yii2 provides appropriate attributes for some widgets to make AJAX calls; sometimes, however, writing a JavaScript code in these attributes will make code hard to read, especially if we are dealing with complex codes. Consequently, to make an AJAX call, we will use external JavaScript code executed by registerJs(). This is a template of the AJAX class using the GET or POST method: <?php $this->registerJs( <<< EOT_JS // using GET method $.get({ url: url, data: data, success: success, dataType: dataType }); // using POST method $.post({ url: url, data: data, success: success, dataType: dataType }); EOT_JS ); ?> An AJAX call is usually the effect of a user interface event (such as a click on a button, a link, and so on). So, it is directly connected to jQuery .on() event on element. For this reason, it is important to remember how Yii2 renders the name and id attributes of input fields. When we call Html::activeTextInput($model, $attribute) or in the same way use <?= $form->field($model, $attribute)->textInput() ?>. The name and id attributes of the input text field will be rendered as follows: id : The model class name separated with a dash by the attribute name in lowercase; for example, if the model class name is Room and the attribute is floor, the id attribute will be room-floor name: The model class name that encloses the attribute name, for example, if the model class name is Reservation and the attribute is price_per_day, the name attribute will be Reservation[price_per_day]; so every field owned by the Reservation model will be enclosed all in a single array In this example, there are two drop-down lists and a detail box. The two drop-down lists refer to customers and reservations; when user clicks on a customer list item, the second drop-down list of reservations will be filled out according to their choice. Finally, when a user clicks on a reservation list item, a details box will be filled out with data about the selected reservation. In an action named actionDetailDependentDropdown():   public function actionDetailDependentDropdown() { $showDetail = false; $model = new Reservation(); if(isset($_POST['Reservation'])) { $model->load( Yii::$app->request->post() ); if(isset($_POST['Reservation']['id'])&& ($_POST['Reservation']['id']!=null)) { $model = Reservation::findOne($_POST['Reservation']['id']); $showDetail = true; } } return $this->render('detailDependentDropdown', [ 'model' => $model, 'showDetail' => $showDetail ]); } In this action, we will get the customer_id and id parameters from a form based on the Reservation model data and if it are filled out, the data will be used to search for the correct reservation model to be passed to the view. There is a flag called $showDetail that displays the reservation details content if the id attribute of the model is received. In Controller, there is also an action that will be called using AJAX when the user changes the customer selection in the drop-down list:   public function actionAjaxDropDownListByCustomerId($customer_id) { $output = ''; $items = Reservation::findAll(['customer_id' => $customer_id]); foreach($items as $item) { $content = sprintf('reservation #%s at %s', $item->id, date('Y-m-d H:i:s', strtotime($item- >reservation_date))); $output .= yiihelpersHtml::tag('option', $content, ['value' => $item->id]); } return $output; } This action will return the <option> HTML tags filled out with reservations data filtered by the customer ID passed as a parameter. If the customer drop-down list is changed, the detail div will be hidden, an AJAX call will get all the reservations filtered by customer_id, and the result will be passed as content to the reservations drop-down list. If the reservations drop-down list is changed, a form will be submitted. Next in the form declaration, we can find the first of all the customer drop-down list and then the reservations list, which use a closure to get the value from the ArrayHelper::map() methods. We could add a new property in the Reservation model by creating a function starting with the prefix get, such as getDescription(), and put in it the content of the closure, or rather: public function getDescription() { $content = sprintf('reservation #%s at %s', $this>id, date('Y-m-d H:i:s', strtotime($this>reservation_date))); return $content; } Or we could use a short syntax to get data from ArrayHelper::map() in this way: <?= $form->field($model, 'id')->dropDownList(ArrayHelper::map( $reservations, 'id', 'description'), [ 'prompt' => '--- choose' ]); ?> Finally, if $showDetail is flagged, a simple details box with only the price per day of the reservation will be displayed. Using the Bootstrap widget Yii2 supports Bootstrap as a core feature. Bootstrap framework CSS and JavaScript files are injected by default in all pages, and we could use this feature even to only apply CSS classes or call our own JavaScript function provided by Bootstrap. However, Yii2 embeds Bootstrap as a widget, and we can access this framework's capabilities like any other widget. The most used are: Class name Description yiibootstrapAlert This class renders an alert Bootstrap component yiibootstrapButton This class renders a Bootstrap button yiibootstrapDropdown This class renders a Bootstrap drop-down menu component yiibootstrapNav This class renders a nav HTML component yiibootstrapNavBar This class renders a navbar HTML component For example, yiibootstrapNav and yiibootstrapNavBar are used in the default main template.   <?php NavBar::begin([ 'brandLabel' => 'My Company', 'brandUrl' => Yii::$app->homeUrl, 'options' => [ 'class' => 'navbar-inverse navbar-fixed-top', ], ]); echo Nav::widget([ 'options' => ['class' => 'navbar-nav navbar- right'], 'items' => [ ['label' => 'Home', 'url' => ['/site/index']], ['label' => 'About', 'url' => ['/site/about']], ['label' => 'Contact', 'url' => ['/site/contact']], Yii::$app->user->isGuest ? ['label' => 'Login', 'url' => ['/site/login']] : ['label' => 'Logout (' . Yii::$app->user- >identity->username . ')', 'url' => ['/site/logout'], 'linkOptions' => ['data-method' => 'post']], ], ]); NavBar::end(); ?> Yii2 also supports, by itself, many jQuery UI widgets through the JUI extension for Yii 2, yii2-jui. If we do not have the yii2-jui extension in the vendor folder, we can get it from Composer using this command: php composer.phar require --prefer-dist yiisoft/yii2-jui In this example, we will discuss the two most used widgets: datepicker and autocomplete. Let's have a look at the datepicker widget. This widget can be initialized using a model attribute or by filling out a value property. The following is an example made using a model instance and one of its attributes: echo DatePicker::widget([ 'model' => $model, 'attribute' => 'from_date', //'language' => 'it', //'dateFormat' => 'yyyy-MM-dd', ]); And, here is a sample of the value property's use: echo DatePicker::widget([ 'name' => 'from_date', 'value' => $value, //'language' => 'it', //'dateFormat' => 'yyyy-MM-dd', ]); When data is sent via POST, the date_from and date_to fields will be converted from the d/m/y to the y-m-d format to make it possible for the database to save data. Then, the model object is updated through the save() method. Using the Bootstrap widget, an alert box will be displayed in the view after updating the model. Create the datePicker view: <?php use yiihelpersHtml; use yiiwidgetsActiveForm; use yiijuiDatePicker; ?> <div class="row"> <div class="col-lg-6"> <h3>Date Picker from Value<br />(using MM/dd/yyyy format and English language)</h3> <?php $value = date('Y-m-d'); echo DatePicker::widget([ 'name' => 'from_date', 'value' => $value, 'language' => 'en', 'dateFormat' => 'MM/dd/yyyy', ]); ?> </div> <div class="col-lg-6"> <?php if($reservationUpdated) { ?> <?php echo yiibootstrapAlert::widget([ 'options' => [ 'class' => 'alert-success', ], 'body' => 'Reservation successfully updated', ]); ?> <?php } ?> <?php $form = ActiveForm::begin(); ?> <h3>Date Picker from Model<br />(using dd/MM/yyyy format and italian language)</h3> <br /> <label>Date from</label> <?php // First implementation of DatePicker Widget echo DatePicker::widget([ 'model' => $reservation, 'attribute' => 'date_from', 'language' => 'it', 'dateFormat' => 'dd/MM/yyyy', ]); ?> <br /> <br /> <?php // Second implementation of DatePicker Widget echo $form->field($reservation, 'date_to')- >widget(yiijuiDatePicker::classname(), [ 'language' => 'it', 'dateFormat' => 'dd/MM/yyyy', ]) ?> <?php echo Html::submitButton('Send', ['class' => 'btn btn- primary']) ?> <?php $form = ActiveForm::end(); ?> </div> </div> The view is split into two columns, left and right. The left column simply displays a DataPicker example from the value (fixed to the current date). The right column displays an alert box if the $reservation model has been updated, and the next two kinds of widget declaration too; the first one without using $form and the second one using $form, both outputting the same HTML code. In either case, the DatePicker date output format is set to dd/MM/yyyy through the dateFormat property and the language is set to Italian through the language property. Multiple models in the same view Often, we can find many models of same or different class in a single view. First of all, remember that Yii2 encapsulates all the views' form attributes in the same container, named the same as the model class name. Therefore, when the controller receives the data, these will all be organized in a key of the $_POST array named the same as the model class name. If the model class name is Customer, every form input name attribute will be Customer[attributeA_of_model] This is built with: $form->field($model, 'attributeA_of_model')->textInput(). In the case of multiple models of the same class, the container will again be named as the model class name, but every attribute of each model will be inserted in an array, such as: Customer[0][attributeA_of_model_0] Customer[0][attributeB_of_model_0] … … … Customer[n][attributeA_of_model_n] Customer[n][attributeB_of_model_n] These are built with: $form->field($model, '[0]attributeA_of_model')->textInput(); $form->field($model, '[0]attributeB_of_model')->textInput(); … … … $form->field($model, '[n]attributeA_of_model')->textInput(); $form->field($model, '[n]attributeB_of_model')->textInput(); Notice that the array key information is inserted in the attribute name! So when data is passed to the controller, $_POST['Customer'] will be an array composed by the Customer models and every key of this array, for example, $_POST['Customer'][0] is a model of the Customer class. Let's see now how to save three customers at once. We will create three containers, one for each model class that will contain some fields of the Customer model. Create a view containing a block of input fields repeated for every model passed from the controller: <?php use yiihelpersHtml; use yiiwidgetsActiveForm; /* @var $this yiiwebView */ /* @var $model appmodelsRoom */ /* @var $form yiiwidgetsActiveForm */ ?> <div class="room-form"> <?php $form = ActiveForm::begin(); ?> <div class="model"> <?php for($k=0;$k<sizeof($models);$k++) { ?> <?php $model = $models[$k]; ?> <hr /> <label>Model #<?php echo $k+1 ?></label> <?= $form->field($model, "[$k]name")->textInput() ?> <?= $form->field($model, "[$k]surname")->textInput() ?> <?= $form->field($model, "[$k]phone_number")- >textInput() ?> <?php } ?> </div> <hr /> <div class="form-group"> <?= Html::submitButton('Save', ['class' => 'btn btn- primary']) ?> </div> <?php ActiveForm::end(); ?> </div> For each model, all the fields will have the same validator rules of the Customer class, and every single model object will be validated separately. Saving linked models in the same view It could be convenient to save different kind of models in the same view. This approach allows us to save time and to navigate from every single detail until a final item that merges all data is created. Handling different kind of models linked to each other it is not so different from what we have seen so far. The only point to take care of is the link (foreign keys) between models, which we must ensure is valid. Therefore, the controller action will receive the $_POST data encapsulated in the model's class name container; if we are thinking, for example, of the customer and reservation models, we will have two arrays in the $_POST variable, $_POST['Customer'] and $_POST['Reservation'], containing all the fields about the customer and reservation models. Then, all data must be saved together. It is advisable to use a database transaction while saving data because the action can be considered as ended only when all the data has been saved. Using database transactions in Yii2 is incredibly simple! A database transaction starts with calling beginTransaction() on the database connection object and finishes with calling the commit() or rollback() method on the database transaction object created by beginTransaction(). To start a transaction: $dbTransaction = Yii::$app->db->beginTransaction(); Commit transaction, to save all the database activities: $dbTransaction->commit(); Rollback transaction, to clear all the database activities: $dbTransaction->rollback(); So, if a customer was saved and the reservation was not (for any possible reason), our data would be partial and incomplete. Using a database transaction, we will avoid this danger. We now want to create both the customer and reservation models in the same view in a single step. In this way, we will have a box containing the customer model fields and a box with the reservation model fields in the view. Create a view the fields from the customer and reservation models: <?php use yiihelpersHtml; use yiiwidgetsActiveForm; use yiihelpersArrayHelper; use appmodelsRoom; ?> <div class="room-form"> <?php $form = ActiveForm::begin(); ?> <div class="model"> <?php echo $form->errorSummary([$customer, $reservation]); ?> <h2>Customer</h2> <?= $form->field($customer, "name")->textInput() ?> <?= $form->field($customer, "surname")->textInput() ?> <?= $form->field($customer, "phone_number")->textInput() ?> <h2>Reservation</h2> <?= $form->field($reservation, "room_id")- >dropDownList(ArrayHelper::map(Room::find()->all(), 'id', function($room, $defaultValue) { return sprintf('Room n.%d at floor %d', $room- >room_number, $room->floor); })); ?> <?= $form->field($reservation, "price_per_day")->textInput() ?> <?= $form->field($reservation, "date_from")->textInput() ?> <?= $form->field($reservation, "date_to")->textInput() ?> </div> <div class="form-group"> <?= Html::submitButton('Save customer and room', ['class' => 'btn btn-primary']) ?> </div> <?php ActiveForm::end(); ?> </div> We have created two blocks in the form to fill out the fields for the customer and the reservation. Now, create a new action named actionCreateCustomerAndReservation in ReservationsController in basic/controllers/ReservationsController.php:   public function actionCreateCustomerAndReservation() { $customer = new appmodelsCustomer(); $reservation = new appmodelsReservation(); // It is useful to set fake customer_id to reservation model to avoid validation error (because customer_id is mandatory) $reservation->customer_id = 0; if( $customer->load(Yii::$app->request->post()) && $reservation->load(Yii::$app->request->post()) && $customer->validate() && $reservation->validate() ) { $dbTrans = Yii::$app->db->beginTransaction(); $customerSaved = $customer->save(); if($customerSaved) { $reservation->customer_id = $customer->id; $reservationSaved = $reservation->save(); if($reservationSaved) { $dbTrans->commit(); } else { $dbTrans->rollback(); } } else { $dbTrans->rollback(); } } return $this->render('createCustomerAndReservation', [ 'customer' => $customer, 'reservation' => $reservation ]); } Summary In this article, we saw how to embed JavaScript and CSS in a layout and views, with file content or an inline block. This was applied to an example that showed you how to change the number of columns displayed based on the browser's available width; this is a typically task for websites or web apps that display advertising columns. Again on the subject of JavaScript, you learned how implement direct AJAX calls, taking an example where the reservation detail was dynamically loaded from the customers dropdown list. Next we looked at Yii's core user interface library, which is built on Bootstrap and we illustrated how to use the main Bootstrap widgets natively, together with DatePicker, probably the most commonly used jQuery UI widget. Finally, the last topics covered were multiple models of the same and different classes. We looked at two examples on these topics: the first one to save multiple customers at the same time and the second to create a customer and reservation in the same view. Resources for Article: Further resources on this subject: Yii: Adding Users and User Management to Your Site [article] Meet Yii [article] Creating an Extension in Yii 2 [article]
Read more
  • 0
  • 0
  • 2869
article-image-identifying-best-places
Packt
16 Sep 2015
9 min read
Save for later

Identifying the Best Places

Packt
16 Sep 2015
9 min read
In this article by Ben Mearns, author of the book QGIS Blueprints, we will take a look at how the raster data can be analyzed, enhanced, and used for map production. Specifically, you will learn to produce a grid of the suitable locations based on the criteria values in other grids using raster analysis and map algebra. Then, using the grid, we will produce a simple click-based map. The end result will be a site suitability web application with click-based discovery capabilities. We'll be looking at the suitability for the farmland preservation selection. In this article, we will cover the following topics: Vector data ETL for raster analysis Batch processing Leaflet map application publication with QGIS2Leaf (For more resources related to this topic, see here.) Vector data Extract, Transform, and Load Our suitability analysis uses map algebra and criteria grids to give us a single value for the suitability for some activity in every place. This requires that the data be expressed in the raster (grid) format. So, let's perform the other necessary ETL steps and then convert our vector data to raster. We will perform the following actions: Ensure that our data has identical spatial reference systems. For example, we may be using a layer of the roads maintained by the state department of transportation and a layer of land use maintained by the department of natural resources. These layers must have identical spatial reference systems or be transformed to have identical systems. Extract geographic objects according to their classes as defined in some attribute table field if we want to operate on them while they're still in the vector form. If no further analysis is necessary, convert to raster. Loading data and establishing the CRS conformity It is important for the layers in this project to be transformed or projected into the same geographic or projected coordinate system. This is necessary for an accurate analysis and for publication to the web formats. Perform the following steps for this: Disable 'on the fly' projection if it is turned on. Otherwise, 'on the fly' will automatically project your data again to display it with the layers that are already in the Canvas. Navigate to Setting | Options and perform the settings shown in the following screenshot: Add the project layers: Navigate to Layer | Add Layer | Vector Layer. Add the following layers from within c2/data. ApplicantsCountyEasementsLanduseRoads You can select multiple layers to add by pressing Shift and clicking on the contiguous files or pressing Ctrl and clicking on the noncontiguous files. Import the Digital Elevation Model from c2/data/dem/dem.tif. Navigate to Layer | Add Layer | Raster Layer. From the dem directory, select dem.tif and then click on Open. Even though the layers are in a different CRS, QGIS does not warn us in this case. You must discover the issue by checking each layer individually. Check the CRS of the county layer and one other layer: Highlight the county layer in the Layers panel. Navigate to Layer | Properties. The CRS is displayed under the General tab in the Coordinate reference system section: Note that the county layer is in EPSG: 26957, while the others are in EPSG: 2776. We will transform the county layer from EPSG:26957 to EPSG:2776. Navigate to Layer | Save As | Select CRS. We will save all the output from this article in c2/output. To prepare the layers for conversion to raster, we will add a new generic column to all the layers populated with the number 1. This will be translated to a Boolean type raster, where the presence of the object that the raster represents (for example, roads) is indicated by a cell of 1 and all others with a zero. Follow these steps for the applicants, easements, and roads: Navigate to Layer | Toggle Editing. Then, navigate to Layer | Open Attribute Table. Add a column with the button at the top of the Attribute table dialog. Use value as the name for the new column and the following data format options: Select the new column from the dropdown in the Attribute table and enter 1 into the value box: Click on Update All. Navigate to Layer | Toggle Editing. Finally, save. The extracting (filtering) features Let's suppose that our criteria includes only a subset of the features in our roads layer—major unlimited access roads (but not freeways), a subset of the features as determined by a classification code (CFCC). To temporarily extract this subset, we will do a layer query by performing the following steps: Filter the major roads from the roads layer. Highlight the roads layer. Navigate to Layer | Query. Double-click on CFCC to add it to the expression. Click on the = operator to add to the expression Under the Values section, click on All to view all the unique values in the CFCC field. Double-click on A21 to add this to the expression. Do this for all the codes less than A36. Include A63 for highway on-ramps. You selection code will look similar to this: "CFCC" = 'A21' OR "CFCC" = 'A25' OR "CFCC" = 'A31' OR "CFCC" = 'A35' OR "CFCC" = 'A63' Click on OK, as shown in the following screenshot: Create a new c2/output directory. Save the roads layer as a new layer with only the selected features (major_roads) in this directory. To clear a layer filter, return to the query dialog on the applied layer (highlight it in the Layers pane; navigate to Layer | Query and click on Clear). Repeat these steps for the developed (LULC1 = 1) and agriculture (LULC1 = 2) landuses (separately) from the landuse layer. Converting to raster In this section, we will convert all the needed vector layers to raster. We will be doing this in batch, which will allow us to repeat the same operation many times over multiple layers. Doing more at once—working in batch The QGIS Processing Framework provides capabilities to run the same operation many times on different data. This is called batch processing. A batch process is invoked from an operation's context menu in the Processing Toolbox. The batch dialog requires that the parameters for each layer be populated for every iteration. Convert the vector layers to raster. Navigate to Processing Toolbox. Select Advanced Interface from the dropdown at the bottom of Processing Toolbox (if it is not selected, it will show as Simple Interface). Type rasterize to search for the Rasterize tool. Right-click on the Rasterize tool and select Execute as batch process: Fill in the Batch Processing dialog, making sure to specify the parameters as follows: Parameter Value Input layer (For example, roads) Attribute field value Output raster size Output resolution in map units per pixel Horizontal 30 Vertical 30 Raster type Int16 Output layer (For example, roads) The following images show how this will look in QGIS: Scroll to the right to complete the entry of parameter values.   Organize the new layers (optional step).    Batch sometimes gives unfriendly names based on some bug in the dialog box.    Change the layer names by doing the following for each layer created by batch:    Highlight the layer.    Navigate to Layer | Properties.    Change the layer name to the name of the vector layer from which this was created (for example, applicants). You should be able to find a hint for this value in the layer properties in the layer source (name of the .tif file).    Group the layers.    Press Shift + click on all the layers created by batch and the previous roads raster.    Navigate to Right click | Group selected. Publishing the results as a web application Now that we have completed our modeling for the site selection of a farmland for conservation, let's take steps to publish this for the Web. QGIS2leaf QGIS2leaf allows us to export our QGIS map to web map formats (JavaScript, HTML, and CSS) using the Leaflet map API. Leaflet is a very lightweight, extensible, and responsive (and trendy) web mapping interface. QGIS2Leaf converts all our vector layers to GeoJSON, which is the most common textual way to express the geographic JavaScript objects. As our operational layer is in GeoJSON, Leaflet's click interaction is supported, and we can access the information in the layers by clicking. It is a fully editable HTML and JavaScript file. You can customize and upload it to an accessible web location. QGIS2leaf is very simple to use as long as the layers are prepared properly (for example, with respect to CRS) up to this point. It is also very powerful in creating a good starting application including GeoJSON, HTML, and JavaScript for our Leaflet web map. Make sure to install the QGIS2Leaf plugin if you haven't already. Navigate to Web | QGIS2leaf | Exports a QGIS Project to a working Leaflet webmap. Click on the Get Layers button to add the currently displayed layers to the set that QGIS2leaf will export. Choose a basemap and enter the additional details if so desired. Select Encode to JSON. These steps will produce a map application similar to the following one. We'll take a look at how to restore the labels: Summary In this article, using the site selection example, we covered basic vector data ETL, raster analysis, and web map creation. We started with vector data, and after unifying CRS, we prepared the attribute tables. We then filtered and converted it to raster grids using batch processing. Finally, we published the prepared vector output with QGIS2Leaf as a simple Leaflet web map application with a strong foundation for extension. Resources for Article:   Further resources on this subject: Style Management in QGIS [article] Preparing to Build Your Own GIS Application [article] Geocoding Address-based Data [article]
Read more
  • 0
  • 0
  • 2074

article-image-performance-design
Packt
15 Sep 2015
9 min read
Save for later

Performance by Design

Packt
15 Sep 2015
9 min read
In this article by Shantanu Kumar, author of the book, Clojure High Performance Programming - Second Edition, we learn how Clojure is a safe, functional programming language that brings great power and simplicity to the user. Clojure is also dynamically and strongly typed, and has very good performance characteristics. Naturally, every activity performed on a computer has an associated cost. What constitutes acceptable performance varies from one use-case and workload to another. In today's world, performance is even the determining factor for several kinds of applications. We will discuss Clojure (which runs on the JVM (Java Virtual Machine)), and its runtime environment in the light of performance, which is the goal of the book. In this article, we will study the basics of performance analysis, including the following: A whirlwind tour of how the application stack impacts performance Classifying the performance anticipations by the use cases types (For more resources related to this topic, see here.) Use case classification The performance requirements and priority vary across the different kinds of use cases. We need to determine what constitutes acceptable performance for the various kinds of use cases. Hence, we classify them to identify their performance model. When it comes to details, there is no sure shot performance recipe of any kind of use case, but it certainly helps to study their general nature. Note that in real life, the use cases listed in this section may overlap with each other. The user-facing software The performance of user facing applications is strongly linked to the user's anticipation. Having a difference of a good number of milliseconds may not be perceptible for the user but at the same time, a wait for more than a few seconds may not be taken kindly. One important element to normalize the anticipation is to engage the user by providing a duration-based feedback. A good idea to deal with such a scenario would be to start the task asynchronously in the background, and poll it from the UI layer to generate duration-based feedback for the user. Another way could be to incrementally render the results to the user to even out the anticipation. Anticipation is not the only factor in user facing performance. Common techniques like staging or precomputation of data, and other general optimization techniques can go a long way to improve the user experience with respect to performance. Bear in mind that all kinds of user facing interfaces fall into this use case category—the Web, mobile web, GUI, command line, touch, voice-operated, gesture...you name it. Computational and data-processing tasks Non-trivial compute intensive tasks demand a proportional amount of computational resources. All of the CPU, cache, memory, efficiency and the parallelizability of the computation algorithms would be involved in determining the performance. When the computation is combined with distribution over a network or reading from/staging to disk, I/O bound factors come into play. This class of workloads can be further subclassified into more specific use cases. A CPU bound computation A CPU bound computation is limited by the CPU cycles spent on executing it. Arithmetic processing in a loop, small matrix multiplication, determining whether a number is a Mersenne prime, and so on, would be considered CPU bound jobs. If the algorithm complexity is linked to the number of iterations/operations N, such as O(N), O(N2) and more, then the performance depends on how big N is, and how many CPU cycles each step takes. For parallelizable algorithms, performance of such tasks may be enhanced by assigning multiple CPU cores to the task. On virtual hardware, the performance may be impacted if the CPU cycles are available in bursts. A memory bound task A memory bound task is limited by the availability and bandwidth of the memory. Examples include large text processing, list processing, and more. For example, specifically in Clojure, the (reduce f (pmap g coll)) operation would be memory bound if coll is a large sequence of big maps, even though we parallelize the operation using pmap here. Note that higher CPU resources cannot help when memory is the bottleneck, and vice versa. Lack of availability of memory may force you to process smaller chunks of data at a time, even if you have enough CPU resources at your disposal. If the maximum speed of your memory is X and your algorithm on single the core accesses the memory at speed X/3, the multicore performance of your algorithm cannot exceed three times the current performance, no matter how many CPU cores you assign to it. The memory architecture (for example, SMP and NUMA) contributes to the memory bandwidth in multicore computers. Performance with respect to memory is also subject to page faults. A cache bound task A task is cache bound when its speed is constrained by the amount of cache available. When a task retrieves values from a small number of repeated memory locations, for example, a small matrix multiplication, the values may be cached and fetched from there. Note that CPUs (typically) have multiple layers of cache, and the performance will be at its best when the processed data fits in the cache, but the processing will still happen, more slowly, when the data does not fit into the cache. It is possible to make the most of the cache using cache-oblivious algorithms. A higher number of concurrent cache/memory bound threads than CPU cores is likely to flush the instruction pipeline, as well as the cache at the time of context switch, likely leading to a severely degraded performance. An input/output bound task An input/output (I/O) bound task would go faster if the I/O subsystem, that it depends on, goes faster. Disk/storage and network are the most commonly used I/O subsystems in data processing, but it can be serial port, a USB-connected card reader, or any I/O device. An I/O bound task may consume very few CPU cycles. Depending on the speed of the device, connection pooling, data compression, asynchronous handling, application caching, and more, may help in performance. One notable aspect of I/O bound tasks is that performance is usually dependent on the time spent waiting for connection/seek, and the amount of serialization that we do, and hardly on the other resources. In practice, many data processing workloads are usually a combination of CPU bound, memory bound, cache bound, and I/O bound tasks. The performance of such mixed workloads effectively depends on the even distribution of CPU, cache, memory, and I/O resources over the duration of the operation. A bottleneck situation arises only when one resource gets too busy to make way for another. Online transaction processing The online transaction processing (OLTP) systems process the business transactions on demand. It can sit behind systems such as a user-facing ATM machine, point-of-sale terminal, a network-connected ticket counter, ERP systems, and more. The OLTP systems are characterized by low latency, availability, and data integrity. They run day-to-day business transactions. Any interruption or outage is likely to have a direct and immediate impact on the sales or service. Such systems are expected to be designed for resiliency rather than the delayed recovery from failures. When the performance objective is unspecified, you may like to consider graceful degradation as a strategy. It is a common mistake to ask the OLTP systems to answer analytical queries; something that they are not optimized for. It is desirable of an informed programmer to know the capability of the system, and suggest design changes as per the requirements. Online analytical processing The online analytical processing (OLAP) systems are designed to answer analytical queries in short time. They typically get data from the OLTP operations, and their data model is optimized for querying. They basically provide for consolidation (roll-up), drill-down and slicing, and dicing of data for analytical purposes. They often use specialized data stores that can optimize the ad-hoc analytical queries on the fly. It is important for such databases to provide pivot-table like capability. Often, the OLAP cube is used to get fast access to the analytical data. Feeding the OLTP data into the OLAP systems may entail workflows and multistage batch processing. The performance concern of such systems is to efficiently deal with large quantities of data, while also dealing with inevitable failures and recovery. Batch processing Batch processing is automated execution of predefined jobs. These are typically bulk jobs that are executed during off-peak hours. Batch processing may involve one or more stages of job processing. Often batch processing is clubbed with work-flow automation, where some workflow steps are executed offline. Many of the batch processing jobs work on staging of data, and on preparing data for the next stage of processing to pick up. Batch jobs are generally optimized for the best utilization of the computing resources. Since there is little to moderate the demand to lower the latencies of some particular subtasks, these systems tend to optimize for throughput. A lot of batch jobs involve largely I/O processing and are often distributed over a cluster. Due to distribution, the data locality is preferred when processing the jobs; that is, the data and processing should be local in order to avoid network latency in reading/writing data. Summary We learned about the basics of what it is like to think more deeply about performance. The performance of Clojure applications depend on various factors. For a given application, understanding its use cases, design and implementation, algorithms, resource requirements and alignment with the hardware, and the underlying software capabilities, is essential. Resources for Article: Further resources on this subject: Big Data [article] The Observer Pattern [article] Working with Incanter Datasets [article]
Read more
  • 0
  • 0
  • 12069

article-image-using-3d-objects
Packt
15 Sep 2015
11 min read
Save for later

Using 3D Objects

Packt
15 Sep 2015
11 min read
In this article by Liz Staley, author of the book Manga Studio EX 5 Cookbook, you will learn the following topics: Adding existing 3D objects to a page Importing a 3D object from another program Manipulating 3D objects Adjusting the 3D camera (For more resources related to this topic, see here.) One of the features of Manga Studio 5 that people ask me about all the time is 3D objects. Manga Studio 5 comes with a set of 3D assets: characters, poses, and a few backgrounds and small objects. These can be added directly to your page, posed and positioned, and used in your artwork. While I usually use these 3D poses as a reference (much like the wooden drawing dolls that you can find in your local craft store), you can conceivably use 3D characters and imported 3D assets from programs such as Poser to create entire comics. Let's get into the third dimension now, and you will learn how to use these assets in Manga Studio 5. Adding existing 3D objects to a page Manga Studio 5 comes with many 3D objects present in the materials library. This is the fastest way to get started with using the 3D features. Getting ready You must have a page open in order to add a 3D object. Open a page of any size to start the recipes covered here. How to do it… The following steps will show us how to add an existing 3D material to a page: Open the materials library. This can be done by going to Window | Material | Material [3D]. Select a category of 3D material from the list on the left-hand side of the library, or scroll down the Material library preview window to browse all the available materials. Select a material to add to the page by clicking on it to highlight it. In this recipe, we are choosing the School girl B 02 character material. It is highlighted in the following screenshot: Hold the left mouse button down on the selected material and drag it onto the page, releasing the mouse button once the cursor is over the page, to display the material. Alternately, you can click on the Paste selected material to canvas icon at the bottom of the Material library menu. The selected 3D material will be added to the page. The School girl B 02 material is shown in this default character pose: Importing a 3D object from another program You don't have to use only the default 3D models included in Manga Studio 5. The process of importing a model is very easy. The types of files that can be imported into Manga Studio 5 are c2fc, c2fr, fbx, 1wo, 1ws, obj, 6kt, and 6kh. Getting ready You must have a page open in order to add a 3D object. Open a page of any size to start this recipe. For this recipe, you will also need a model to import into the program. These can be found on numerous websites, including my.smithmicro.com, under the Poser tab. How to do it… The following steps will walk us through the simple process of importing a 3D model into Manga Studio 5: Open the location where the 3D model you wish to import has been saved. If you have downloaded the 3D model from the Internet, it may be in the Downloads folder on your PC. Arrange the windows on your computer screen so that the location of the 3D model and Manga Studio 5 are both visible, as shown in the following screenshot: Click on the 3D model file and hold down the mouse button. While still holding down the mouse button, drag the 3D model file into the Manga Studio 5 window. Release the mouse button. The 3D model will be imported into the open page, as shown in this screenshot: Manipulating 3D objects You've learned how to add a 3D object to our project. But how can you pose it the way you want it to look for your scene? With a little time and patience, you'll be posing characters like a pro in no time! Getting ready Follow the directions in the Adding existing 3D objects to a page recipe before following the steps in this recipe. How to do it… This recipe will walk us through moving a character into a custom pose: Be sure that the Object tool under Operation is selected. Click on the 3D object to manipulate, if it is not already selected. To move the entire object up, down, left, or right, hover the mouse cursor over the fourth icon in the top-left corner of the box around the selected object. Click and hold the left mouse button; then, drag to move the object in the desired direction. The following screenshot shows the location of the icon used to move the object up, down, left, or right. It is highlighted in pink and also shown over the 3D character. If your models are moving very slowly, you may need to allocate more memory to Manga Studio EX 5. This can be done by going to File | Preferences | Performance. To rotate the object along the y axis (or the horizon line), hover the mouse cursor over the fifth icon in the top-left corner of the box around the selected object. Click on it, hold the left mouse button, and drag. The object will rotate along the y axis, as shown in this screenshot: To rotate the object along the x axis (straight up and down vertically), hover the mouse cursor over the sixth icon in the top-left corner of the box around the selected object. Click and drag. The object will rotate vertically around its center, , as shown in the following screenshot: To move the object back and forth in 3D space, hover the mouse cursor over the seventh icon in the top-left corner of the box around the selected object. Click and hold the left mouse button; then drag it. The icon is shown as follows, highlighted in pink, and the character has been moved back—away from the camera: To move one part of a character, click on the part to be moved. For this recipe, we'll move the character's arm down. To do this, we'll click on the upper arm portion of the character to select it. When a portion of the character is selected, a sphere with three lines circling it will appear. Each of these three lines represents one axis (x, y, and z) and controls the rotation of that portion of the character. This set of lines is shown here: Use the lines of the sphere to rotate the part of the character to the desired position. For a more precise movement, the scroll wheel on the mouse can be used as well. In the following screenshot, the arm has been rotated so that it is down at the character's side: Do you keep accidentally moving a part of the model that you don't want to move? Put the cursor over the part of the model that you'd like to keep in place, and then right-click. A blue box will appear on that part of the model, and the piece will be locked in to place. Right-click again to unlock the part. How it works… In this recipe, we covered how to move and rotate a 3D object and portions of 3D characters. This is the start of being able to create your own custom poses and saving them for reuse. It's also the way to pose the drawing doll models in Manga Studio to make pose references for your comic artwork. In the 3D-Body Type folder of the materials library, you will find Female and Male drawing dolls that can be posed just as the premade characters can. These generic dolls are great for getting that difficult pose down. Then use the next recipe, Adjusting the 3D camera, to get the angle you need, and draw away! The following screenshot shows a drawing doll 3D object that has been posed in a custom stance. The preceding pose was relatively easy to achieve. The figure was rotated along the x axis, and then the head and neck joints were both rotated individually so that the doll looked toward the camera. Both its arms were rotated down and then inward. The hands were posed. The ankle joints were selected and the feet were rotated so that the toes were pointed. Then the knee of the near leg was rotated to bend it. The hip of the near leg was also rotated so that the leg was lifted slightly, giving a "cutesy" look to the pose. Having trouble posing a character's hands exactly the way you want them? Then open the Sub Tool Detail palette and click on Pose in the left-hand-side menu. In this area, you will find a menu with a picture of a hand. This is a quick controller for the fingers. Select the hand that you wish to pose. Along the bottom of the menu are some preset hand poses for things such as closed fists. At the top of each finger on this menu is an icon that looks like chain links. Click on one of them to lock the finger that it is over and prevent it from moving. The triangle area over the large blue hand symbol controls how open and closed the fingers are. You will find this menu much easier than rotating each joint individually—I'm sure! Adjusting the 3D camera In addition to manipulating 3D objects or characters, you can also change the position of the 3D camera to get the composition that you desire for your work. Think of the 3D camera just like a camera on a movie set. It can be rotated or moved around to frame the actors (3D characters) and scenery just the way the director wants! Not sure whether you moved the character or the camera? Take a look at the ground plane, which is the "checkerboard" floor area underneath the characters and objects. If the character is standing straight up and down on the ground plane, it means that the camera was moved. If the character is floating above or below the ground plane, or part of the way through it, it means that the character or object was moved. Getting ready Follow the directions given in the Adding existing 3D objects to a page recipe before following the steps in this recipe. How to do it… To rotate the camera around an object (the object will remain stationary), hover the mouse cursor over the first icon in the top-left corner of the box around the selected object. Click and hold the left mouse button, and then drag. The icon and the camera rotation are shown in the following screenshot: To move the camera up, down, left, or right, hover the mouse cursor over the second icon in the top-left corner of the box around the selected object. Click and hold the left mouse button, and then drag. The icon and camera movement are shown in this screenshot: To move the camera back and forth in the 3D space, hover the mouse cursor over the third icon in the top-left corner of the box around the selected object. Again, click and hold the left mouse button, and then drag. The next screenshot shows the zoom icon in pink at the top and the overlay on top of the character. Note how the hand of the character and the top of the head are now out of the page, since the camera is closer to her and she appears larger on the canvas. Summary In this article, we have studied to add existing 3D objects to a page using Manga Studio 5 in detail. After adding the existing object, we saw steps to add the 3D object from another program. Then, there are steps to manipulate these 3D objects along the co-ordinate system by using tools available in Manga Studio 5. Finally, we learnt to position the 3D camera, by rotating it around an object. Resources for Article: Further resources on this subject: Ink Slingers [article] Getting Familiar with the Story Features [article] Animating capabilities of Cinema 4D [article]
Read more
  • 0
  • 0
  • 3947
article-image-hello-pong
Packt
15 Sep 2015
19 min read
Save for later

Hello, Pong!

Packt
15 Sep 2015
19 min read
In this article written by Alejandro Rodas de Paz and Joseph Howse, authors of the book Python Game Programming By Example, we learn how game development is a highly evolving software development process, and it how has improved continuously since the appearance of the first video games in the 1950s. Nowadays, there is a wide variety of platforms and engines, and this process has been facilitated with the arrival of open source tools. Python is a free high-level programming language with a design intended to write readable and concise programs. Thanks to its philosophy, we can create our own games from scratch with just a few lines of code. There are a plenty of game frameworks for Python, but for our first game, we will see how we can develop it without any third-party dependency. We will be covering the following topics: Installation of the required software Overview of Tkinter, a GUI library included in the Python standard library Applying object-oriented programming to encapsulate the logic of our game Basic collision and input detection Drawing game objects without external assets (For more resources related to this topic, see here.) Installing Python You will need Python 3.4 with Tcl / Tk 8.6 installed on your computer. The latest branch of this version is Python 3.4.3, which can be downloaded from https://www.python.org/downloads/. Here, you can find the official binaries for the most popular platforms, such as Windows and Mac OS. During the installation process, make sure that you check the Tcl/Tk option to include the library. The code examples included in the book have been tested against Windows 8 and Mac, but can be run on Linux without any modification. Note that some distributions may require you to install the appropriate package for Python 3. For instance, on Ubuntu, you need to install the python3-tk package. Once you have Python installed, you can verify the version by opening Command Prompt or a terminal and executing these lines: $ python –-version Python 3.4.3 After this check, you should be able to start a simple GUI program: $ python >>> from tkinter import Tk >>> root = Tk() >>> root.title('Hello, world!') >>> root.mainloop() These statements create a window, change its title, and run indefinitely until the window is closed. Do not close the new window that is displayed when the second statement is executed. Otherwise, it will raise an error because the application has been destroyed. We will use this library in our first game, and the complete documentation of the module can be found at https://docs.python.org/3/library/tkinter.html. Tkinter and Python 2 The Tkinter module was renamed to tkinter in Python 3. If you have Python 2 installed, simply change the import statement with Tkinter in uppercase, and the program should run as expected. Overview of Breakout The Breakout game starts with a paddle and a ball at the bottom of the screen and some rows of bricks at the top. The player must eliminate all the bricks by hitting them with the ball, which rebounds against the borders of the screen, the bricks, and the bottom paddle. As in Pong, the player controls the horizontal movement of the paddle. The player starts the game with three lives, and if she or he misses the ball's rebound and it reaches the bottom border of the screen, one life is lost. The game is over when all the bricks are destroyed, or when the player loses all their lives. This is a screenshot of the final version of our game: Basic GUI layout We will start out game by creating a top-level window as in the simple program we ran previously. However, this time, we will use two nested widgets: a container frame and the canvas where the game objects will be drawn, as shown here: With Tkinter, this can easily be achieved using the following code: import tkinter as tk lives = 3 root = tk.Tk() frame = tk.Frame(root) canvas = tk.Canvas(frame, width=600, height=400, bg='#aaaaff') frame.pack() canvas.pack() root.title('Hello, Pong!') root.mainloop() Through the tk alias, we access the classes defined in the tkinter module, such as Tk, Frame, and Canvas. Notice the first argument of each constructor call which indicates the widget (the child container), and the required pack() calls for displaying the widgets on their parent container. This is not necessary for the Tk instance, since it is the root window. However, this approach is not exactly object-oriented, since we use global variables and do not define any new class to represent our new data structures. If the code base grows, this can lead to poorly organized projects and highly coupled code. We can start encapsulating the pieces of our game in this way: import tkinter as tk class Game(tk.Frame): def __init__(self, master): super(Game, self).__init__(master) self.lives = 3 self.width = 610 self.height = 400 self.canvas = tk.Canvas(self, bg='#aaaaff', width=self.width, height=self.height,) self.canvas.pack() self.pack() if __name__ == '__main__': root = tk.Tk() root.title('Hello, Pong!') game = Game(root) game.mainloop() Our new type, called Game, inherits from the Frame Tkinter class. The class Game(tk.Frame): definition specifies the name of the class and the superclass between parentheses. If you are new to object-oriented programming with Python, this syntax may not sound familiar. In our first look at classes, the most important concepts are the __init__ method and the self variable: The __init__ method is a special method that is invoked when a new class instance is created. Here, we set the object attributes, such as the width, the height, and the canvas widget. We also call the parent class initialization with the super(Game, self).__init__(master) statement, so the initial state of the Frame is properly initialized. The self variable refers to the object, and it should be the first argument of a method if you want to access the object instance. It is not strictly a language keyword, but the Python convention is to call it self so that other Python programmers won't be confused about the meaning of the variable. In the preceding snippet, we introduced the if __name__ == '__main__' condition, which is present in many Python scripts. This snippet checks the name of the current module that is being executed, and will prevent starting the main loop where this module was being imported from another script. This block is placed at the end of the script, since it requires that the Game class be defined. New- and old-style classes You may see the MySuperClass.__init__(self, arguments) syntax in some Python 2 examples, instead of the super call. This is the old-style syntax, the only flavor available up to Python 2.1, and is maintained in Python 2 for backward compatibility. The super(MyClass, self).__init__(arguments) is the new-class style introduced in Python 2.2. It is the preferred approach, and we will use it throughout this book. Since no external assets are needed, you can place the set of code files given along with the book(Chapter1_01.Py) in any directory and execute it from the python command line by running the file. The main loop will run indefinitely until you click on the close button of the window, or if you kill the process from the command line. This is the starting point of our game, so let's start diving into the Canvas widget and see how we can draw and animate items in it. Diving into the Canvas widget So far, we have the window set up and now we can start drawing items on the canvas. The canvas widget is two-dimensional and uses the Cartesian coordinate system. The origin—the (0, 0) ordered pair—is placed at the top-left corner, and the axis can be represented as shown in the following screenshot: Keeping this layout in mind, we can use two methods of the Canvas widget to draw the paddle, the bricks, and the ball: canvas.create_rectangle(x0, y0, x1, y1, **options) canvas.create_oval(x0, y0, x1, y1, **options) Each of these calls returns an integer, which identifies the item handle. This reference will be used later to manipulate the position of the item and its options. The **options syntax represents a key/value pair of additional arguments that can be passed to the method call. In our case, we will use the fill and the tags option. The x0 and y0 coordinates indicate the top-left corner of the previous screenshot, and x1 and y1 are indicated in the bottom-right corner. For instance, we can call canvas.create_rectangle(250, 300, 330, 320, fill='blue', tags='paddle') to create a player's paddle, where: The top-left corner is at the coordinates (250, 300). The bottom-right corner is at the coordinates (300, 320). The fill='blue' means that the background color of the item is blue. The tags='paddle' means that the item is tagged as a paddle. This string will be useful later to find items in the canvas with specific tags. We will invoke other Canvas methods to manipulate the items and retrieve widget information. This table gives the references to the Canvas widget that will be used here: Method Description canvas.coords(item) Returns the coordinates of the bounding box of an item. canvas.move(item, x, y) Moves an item by a horizontal and a vertical offset. canvas.delete(item) Deletes an item from the canvas. canvas.winfo_width() Retrieves the canvas width. canvas.itemconfig(item, **options) Changes the options of an item, such as the fill color or its tags. canvas.bind(event, callback) Binds an input event with the execution of a function. The callback handler receives one parameter of the type Tkinter event. canvas.unbind(event) Unbinds the input event so that there is no callback function executed when the event occurs. canvas.create_text(*position, **opts) Draws text on the canvas. The position and the options arguments are similar to the ones passed in canvas.create_rectangle and canvas.create_oval. canvas.find_withtag(tag) Returns the items with a specific tag. canvas.find_overlapping(*position) Returns the items that overlap or are completely enclosed by a given rectangle. You can check out a complete reference of the event syntax as well as some practical examples at http://effbot.org/tkinterbook/tkinter-events-and-bindings.htm#events. Basic game objects Before we start drawing all our game items, let's define a base class with the functionality that they will have in common—storing a reference to the canvas and its underlying canvas item, getting information about its position, and deleting the item from the canvas: class GameObject(object): def __init__(self, canvas, item): self.canvas = canvas self.item = item def get_position(self): return self.canvas.coords(self.item) def move(self, x, y): self.canvas.move(self.item, x, y) def delete(self): self.canvas.delete(self.item) Assuming that we have created a canvas widget as shown in our previous code samples, a basic usage of this class and its attributes would be like this: item = canvas.create_rectangle(10,10,100,80, fill='green') game_object = GameObject(canvas,item) #create new instance print(game_object.get_position()) # [10, 10, 100, 80] game_object.move(20, -10) print(game_object.get_position()) # [30, 0, 120, 70] game_object.delete() In this example, we created a green rectangle and a GameObject instance with the resulting item. Then we retrieved the position of the item within the canvas, moved it, and calculated the position again. Finally, we deleted the underlying item. The methods that the GameObject class offers will be reused in the subclasses that we will see later, so this abstraction avoids unnecessary code duplication. Now that you have learned how to work with this basic class, we can define separate child classes for the ball, the paddle, and the bricks. The Ball class The Ball class will store information about the speed, direction, and radius of the ball. We will simplify the ball's movement, since the direction vector will always be one of the following: [1, 1] if the ball is moving towards the bottom-right corner [-1, -1] if the ball is moving towards the top-left corner [1, -1] if the ball is moving towards the top-right corner [-1, 1] if the ball is moving towards the bottom-left corner Representation of the possible direction vectors Therefore, by changing the sign of one of the vector components, we will change the ball's direction by 90 degrees. This will happen when the ball bounces with the canvas border, or when it hits a brick or the player's paddle: class Ball(GameObject): def __init__(self, canvas, x, y): self.radius = 10 self.direction = [1, -1] self.speed = 10 item = canvas.create_oval(x-self.radius, y-self.radius, x+self.radius, y+self.radius, fill='white') super(Ball, self).__init__(canvas, item)   For now, the object initialization is enough to understand the attributes that the class has. We will cover the ball rebound logic later, when the other game objects are defined and placed in the game canvas. The Paddle class The Paddle class represents the player's paddle and has two attributes to store the width and height of the paddle. A set_ball method will be used store a reference to the ball, which can be moved with the ball before the game starts: class Paddle(GameObject): def __init__(self, canvas, x, y): self.width = 80 self.height = 10 self.ball = None item = canvas.create_rectangle(x - self.width / 2, y - self.height / 2, x + self.width / 2, y + self.height / 2, fill='blue') super(Paddle, self).__init__(canvas, item) def set_ball(self, ball): self.ball = ball def move(self, offset): coords = self.get_position() width = self.canvas.winfo_width() if coords[0] + offset >= 0 and coords[2] + offset <= width: super(Paddle, self).move(offset, 0) if self.ball is not None: self.ball.move(offset, 0) The move method is responsible for the horizontal movement of the paddle. Step by step, the following is the logic behind this method: The self.get_position() calculates the current coordinates of the paddle The self.canvas.winfo_width() retrieves the canvas width If both the minimum and maximum x-axis coordinates plus the offset produced by the movement are inside the boundaries of the canvas, this is what happens: The super(Paddle, self).move(offset, 0) calls the method with same name in the Paddle class's parent class, which moves the underlying canvas item If the paddle still has a reference to the ball (this happens when the game has not been started), the ball is moved as well This method will be bound to the input keys so that the player can use them to control the paddle's movement. We will see later how we can use Tkinter to process the input key events. For now, let's move on to the implementation of the last one of our game's components. The Brick class Each brick in our game will be an instance of the Brick class. This class contains the logic that is executed when the bricks are hit and destroyed: class Brick(GameObject): COLORS = {1: '#999999', 2: '#555555', 3: '#222222'} def __init__(self, canvas, x, y, hits): self.width = 75 self.height = 20 self.hits = hits color = Brick.COLORS[hits] item = canvas.create_rectangle(x - self.width / 2, y - self.height / 2, x + self.width / 2, y + self.height / 2, fill=color, tags='brick') super(Brick, self).__init__(canvas, item) def hit(self): self.hits -= 1 if self.hits == 0: self.delete() else: self.canvas.itemconfig(self.item, fill=Brick.COLORS[self.hits]) As you may have noticed, the __init__ method is very similar to the one in the Paddle class, since it draws a rectangle and stores the width and the height of the shape. In this case, the value of the tags option passed as a keyword argument is 'brick'. With this tag, we can check whether the game is over when the number of remaining items with this tag is zero. Another difference from the Paddle class is the hit method and the attributes it uses. The class variable called COLORS is a dictionary—a data structure that contains key/value pairs with the number of hits that the brick has left, and the corresponding color. When a brick is hit, the method execution occurs as follows: The number of hits of the brick instance is decreased by 1 If the number of hits remaining is 0, self.delete() deletes the brick from the canvas Otherwise, self.canvas.itemconfig() changes the color of the brick. For instance, if we call this method for a brick with two hits left, we will decrease the counter by 1 and the new color will be #999999, which is the value of Brick.COLORS[1]. If the same brick is hit again, the number of remaining hits will become zero and the item will be deleted. Adding the Breakout items Now that the organization of our items is separated into these top-level classes, we can extend the __init__ method of our Game class: class Game(tk.Frame): def __init__(self, master): super(Game, self).__init__(master) self.lives = 3 self.width = 610 self.height = 400 self.canvas = tk.Canvas(self, bg='#aaaaff', width=self.width, height=self.height) self.canvas.pack() self.pack() self.items = {} self.ball = None self.paddle = Paddle(self.canvas, self.width/2, 326) self.items[self.paddle.item] = self.paddle for x in range(5, self.width - 5, 75): self.add_brick(x + 37.5, 50, 2) self.add_brick(x + 37.5, 70, 1) self.add_brick(x + 37.5, 90, 1) self.hud = None self.setup_game() self.canvas.focus_set() self.canvas.bind('<Left>', lambda _: self.paddle.move(-10)) self.canvas.bind('<Right>', lambda _: self.paddle.move(10)) def setup_game(self): self.add_ball() self.update_lives_text() self.text = self.draw_text(300, 200, 'Press Space to start') self.canvas.bind('<space>', lambda _: self.start_game()) This initialization is more complex that what we had at the beginning of the article. We can divide it into two sections: Game object instantiation, and their insertion into the self.items dictionary. This attribute contains all the canvas items that can collide with the ball, so we add only the bricks and the player's paddle to it. The keys are the references to the canvas items, and the values are the corresponding game objects. We will use this attribute later in the collision check, when we will have the colliding items and will need to fetch the game object. Key input binding, via the Canvas widget. The canvas.focus_set() call sets the focus on the canvas, so the input events are directly bound to this widget. Then we bind the left and right keys to the paddle's move() method and the spacebar to trigger the game start. Thanks to the lambda construct, we can define anonymous functions as event handlers. Since the callback argument of the bind method is a function that receives a Tkinter event as an argument, we define a lambda that ignores the first parameter—lambda _: <expression>. Our new add_ball and add_brick methods are used to create game objects and perform a basic initialization. While the first one creates a new ball on top of the player's paddle, the second one is a shorthand way of adding a Brick instance:   def add_ball(self): if self.ball is not None: self.ball.delete() paddle_coords = self.paddle.get_position() x = (paddle_coords[0] + paddle_coords[2]) * 0.5 self.ball = Ball(self.canvas, x, 310) self.paddle.set_ball(self.ball) def add_brick(self, x, y, hits): brick = Brick(self.canvas, x, y, hits) self.items[brick.item] = brick The draw_text method will be used to display text messages in the canvas. The underlying item created with canvas.create_text() is returned, and it can be used to modify the information:   def draw_text(self, x, y, text, size='40'): font = ('Helvetica', size) return self.canvas.create_text(x, y, text=text, font=font) The update_lives_text method displays the number of lives left and changes its text if the message is already displayed. It is called when the game is initialized—this is when the text is drawn for the first time—and it is also invoked when the player misses a ball rebound:    def update_lives_text(self): text = 'Lives: %s' % self.lives if self.hud is None: self.hud = self.draw_text(50, 20, text, 15) else: self.canvas.itemconfig(self.hud, text=text) We leave start_game unimplemented for now, since it triggers the game loop, and this logic will be added in the next section. Since Python requires a code block for each method, we use the pass statement. This does not execute any operation, and it can be used as a placeholder when a statement is required syntactically: def start_game(self): pass If you execute this script, it will display a Tkinter window like the one shown in the following figure. At this point, we can move the paddle horizontally, so we are ready to start the game and hit some bricks! Summary We covered the basics of the control flow and the class syntax. We used Tkinter widgets, especially the Canvas widget and its methods, to achieve the functionality needed to develop a game based on collisions and simple input detection. Our Breakout game can be customized as we want. Feel free to change the color defaults, the speed of the ball, or the number of rows of bricks. However, GUI libraries are very limited, and more complex frameworks are required to achieve a wider range of capabilities. Resources for Article: Further resources on this subject: Introspecting Maya, Python, and PyMEL [article] Understanding the Python regex engine [article] Ten IPython essentials [article]
Read more
  • 0
  • 1
  • 7371

article-image-analyzing-financial-data-qlikview
Packt
15 Sep 2015
8 min read
Save for later

Analyzing Financial Data in QlikView

Packt
15 Sep 2015
8 min read
In this article by Diane Blackwood, author of the book QlikView for Finance, the author talks about how QlikView is an easy-to-use business intelligence product designed to facilitate ad hoc relationship analysis. However, it can also be used in formal corporate performance applications by a financial user. It is designed to use a methodology of direct discovery to analyze data from multiple sources. QlikView is designed to allow you to do your own business discovery, take you out of the data management stage and into the data relationship investigation stage. Investigating relationships and outliers in financial data more can lead to effective management. (For more resources related to this topic, see here.) You could use QlikView when you wish to analyze and quickly see trends and exceptions that — with normal financial application-oriented BI products—would not be readily apparent without days of consultant and technology department setup. With QlikView, you can also analyze data relationships that are not measured in monetary units. Certainly, QlikView can be used to analyze sales trends and stock performance, but other relationships soon become apparent when you start using QlikView. Also, with the free downloadable personal edition of QlikView, you can start analyzing your own data right away. QlikView consists of two parts: The sheet: This can contain sheet objects, such as charts or list boxes, which show clickable information. The load script: This stores information about the data and the data sources that the data is coming from. Financial professionals are always using Excel to examine their data, and we can load data from an Excel sheet into QlikView. This can also help you to create a basic document sheet containing a chart. The newest version of QlikView comes with a sample Sales Order data that can be used to investigate and create sheet objects. In order to use data from other file types, you can use the File Wizard (Type) that you start from the Edit Script dialog by clicking on the Table Files button. Using the Edit Script dialog, you can view your data script and edit it in the script and add other data sources. You can also reload your data by clicking on the Reload button. If you just want to analyze data from an existing QlikView file and analyze the information in it, you do not need to work with the script at all. We will use some sample financial data that was downloaded from an ERP system to Excel in order to demonstrate how an analysis might work. Our QlikView Financial Analysis of Cheyenne Company will appear as follows: Figure 1: Our Financial Analysis QlikView Application When we create objects for analysis purposes in QlikView, the drop-down menu shows that there are multiple sheet object types to choose from, such as List Box, Statistics Box, Chart, Input Box, Current Selections Box, MultiBox, Table Box, Button, Text Object, Line/Arrow Object, Slider/Calendar Object, and Bookmark Object. In our example, we chose the Statistic Box Sheet object to add the grand total to our analysis. From this, we can see that the total company is out of balance by $1.59. From an auditor’s point of view, this amount is probably small enough to be immaterial, but, from our point of view as financial professionals, we want to know where our books are falling out of balance. To make our investigation easier, we should add one additional sheet object: a List Box for Company. This is done by right-clicking on the context menu and selecting New Sheet object and then List Box. Figure 2: Added Company List Box We can now see that we are actually out of balance in three companies. Cheyenne Co. L.P. is a company out by $1.59, but Cheyenne Holding and Cheyenne National Inc. seem to have balancing entries that balance at the total companies’ level, but these companies don’t balance at the individual company level. We can analyze our data using the list boxes just by selecting a Company and viewing the Account Groups and Cost Centers that are included (white) and excluded (gray). This is the standard color scheme usage of QlikView. Our selected company is shown in green and in the Current Selection Box. By selecting Cheyenne Holding, we would be able to verify that it is indeed a holding company, does not have any manufacturing or sales accounting groups, or cost centers. Alternatively, if we choose Provo, we can see that it is in balance. To load more than one spreadsheet or load from a different data source, we must edit load script. From the Edit Script interface, we can modify and execute a script that connects the QlikView document to an ODBC data source or to data files of different type and grab the data source information as well. Our first script was generated automatically, but scripts can be typed manually, or automatically generated scripts can be modified. Complex script statements must, at least partially, be entered manually. The Edit Script dialog uses autocomplete, so when typing, the program tries to predict what is wanted in the script without having to type it completely. The predictions include words that are part of the script syntax. The script is also color coded by syntax components. The Edit Script interface and behavior may be customized to your preferences by selecting Tools and Editor Preferences. A menu bar is found at the top of the Edit Script dialog with various script-related commands. The most frequently used commands also appear in the toolbar. In the toolbar, there is also a drop-down list for the tabs of the Edit Script wizard. The first script in the Edit Script interface is the automatically generated one that was created by the wizard when we started the QlikView file. The automatically generated script picks up the column names from the Excel file and puts in some default formatting scripting. The language selection that we made during the initial installation of QlikView determines the defaults assigned to this portion of the script. We can add data from multiple sources, such as ODBC links, additional Excel files, sources from the Web, FTP, and even other QlikView files. Our first Excel file, which we used to create the initial QlikView document, is already in our script. It happened to be October 2013 data, but suppose we wanted to add another month such as November data to our analysis? We would just navigate to the Edit Script interface from the File menu and then click on the script itself. Make sure that our cursor is at the bottom of the script after the first Excel file path and description. If you do not position your cursor where you want your additional script information to populate, you may generate your new script code in the middle of your existing script code. If you make a mistake, click on CANCEL and start over. After navigating to the script location where you want to add your new code, click on the Table Files button after the script and towards the center right first button in the column. Click on NEXT through the next four screens unless you need to add column labels. Comments can be added to scripts using // for a single line or by surrounding the comment by a beginning /* and an ending */ and comments show up as green. After clicking on the OK button to get out of Script Editor, there is another File menu item that can be used to verify that QlikView has correctly interpreted the joins. This is the Table Viewer menu item. You cannot edit in the Table view, but it is convenient to visualize how the table fields are interacting. Save the changes to the script by clicking on the OK button in the lower-right corner. Now, with the File menu, navigate to Edit Script and then to the Reload menu item and click on it to reload your data; otherwise, your new month of data will not be loaded. If you receive any error messages, the solutions can be researched in QlikView Help. In this case, the column headers were the same, so QlikView knew to add the data from the two spreadsheets together into one table. However, because of this, if we look at our Company List Box and Amount Statistics Box, we see everything added together. Figure 3: Data Doubled after Reload with Additional File The reason this data is doubled is that we do not have any way to split the months or only select October or November. Now that we have more than one month of data, we can add another List Box with the months. This will automatically link up to our Chart and Straight Table Sheet objects to separate our monthly data. Once added, from our new List Box, we can select OCTOBER or NOVEMBER, and our sheet object automatically shows the correct sum of the individual months. We can then use the List Box and linked objects to further analyze our financial data. Summary You can find further find books on QlikView published by Packt on the Packt website http://www.packtpub.com. Some of them are listed as follows: Learning QlikView Data Visualization by Karl Pover Predictive Analytics using Rattle and QlikView by Ferran Garcia Pagans Resources for Article: Further resources on this subject: Common QlikView script errors [article] Securing QlikView Documents [article] Conozca QlikView [article]
Read more
  • 0
  • 0
  • 6439
Modal Close icon
Modal Close icon