How-To Tutorials

25 Feb 2015

20 min read

Christmas Light Sequencer

25 Feb 2015

0
0
9493

How-To Tutorials

Packt

25 Feb 2015

14 min read

The Raspberry Pi and Raspbian

Packt

25 Feb 2015

14 min read

0
0
9681

How-To Tutorials

Packt

25 Feb 2015

29 min read

Static Data Management

Packt

25 Feb 2015

29 min read

0
0
6168

How-To Tutorials

Packt

25 Feb 2015

18 min read

Agile data modeling with Neo4j

Packt

25 Feb 2015

18 min read

0
0
2654

article-image-introducing-web-application-development-rails

Packt

25 Feb 2015

8 min read

Introducing Web Application Development in Rails

Packt

25 Feb 2015

8 min read

In this article by Syed Fazle Rahman, author of the book Bootstrap for Rails,we will learn how to present your application in the best possible way, which has been the most important factor for every web developer for ages. In this mobile-first generation, we are forced to go with the wind and make our application compatible with mobiles, tablets, PCs, and every possible display on Earth. Bootstrap is the one stop solution for all woes that developers have been facing. It creates beautiful responsive designs without any extra efforts and without any advanced CSS knowledge. It is a true boon for every developer. we will be focusing on how to beautify our Rails applications through the help of Bootstrap. We will create a basic Todo application with Rails. We will explore the folder structure of a Rails application and analyze which folders are important for templating a Rails Application. This will be helpful if you want to quickly revisit Rails concepts. We will also see how to create views, link them, and also style them. The styling in this article will be done traditionally through the application's default CSS files. Finally, we will discuss how we can speed up the designing process using Bootstrap. In short, we will cover the following topics: Why Bootstrap with Rails? Setting up a Todo Application in Rails Analyzing folder structure of a Rails application Creating views Styling views using CSS Challenges in traditionally styling a Rails Application (For more resources related to this topic, see here.) Why Bootstrap with Rails? Rails is one the most popular Ruby frameworks which is currently at its peak, both in terms of demand and technology trend. With more than 3,100 members contributing to its development, and tens of thousands of applications already built using it, Rails has created a standard for every other framework in the Web today. Rails was initially developed by David Heinemeier Hansson in 2003 to ease his own development process in Ruby. Later, he became generous enough to release Rails to the open source community. Today, it is popularly known as Ruby on Rails. Rails shortens the development life cycle by moving the focus from reinventing the wheel to innovating new features. It is based on the convention of the configurations principle, which means that if you follow the Rails conventions, you would end up writing much less code than you would otherwise write. Bootstrap, on the other hand, is one of the most popular frontend development frameworks. It was initially developed at Twitter for some of its internal projects. It makes the life of a novice web developer easier by providing most of the reusable components that are already built and are ready to use. Bootstrap can be easily integrated with a Rails development environment through various methods. We can directly use the .css files provided by the framework, or can extend it through its Sass version and let Rails compile it. Sass is a CSS preprocessor that brings logic and functionality into CSS. It includes features like variables, functions, mixins, and others. Using the Sass version of Bootstrap is a recommended method in Rails. It gives various options to customize Bootstrap's default styles easily. Bootstrap also provides various JavaScript components that can be used by those who don't have any real JavaScript knowledge. These components are required in almost every modern website being built today. Bootstrap with Rails is a deadly combination. You can build applications faster and invest more time to think about functionality, rather than rewrite codes. Setting up a Todo application in Rails I assume that you already have basic knowledge of Rails development. You should also have Rails and Ruby installed in your machine to start with. Let's first understand what this Todo application will do. Our application will allow us to create, update, and delete items from the Todo list. We will first analyze the folders that are created while scaffolding this application and which of them are necessary for templating the application. So, let's dip our feet into the water: First, we need to select our workspace, which can be any folder inside your system. Let's create a folder named Bootstrap_Rails_Project. Now, open the terminal and navigate to this folder. It's time to create our Todo application. Write the following command to create a Rails application named TODO: rails new TODO This command will execute a series of various other commands that are necessary to create a Rails application. So, just wait for sometime before it stops executing all the codes. If you are using a newer version of Rails, then this command will also execute bundle install command at the end. Bundle install command is used to install other dependencies. The output for the preceding command is as follows: Now, you should have a new folder inside Bootstrap_Rails_Project named TODO, which was created by the preceding code. Here is the output: Analyzing folder structure of a Rails application Let's navigate to the TODO folder to check how our application's folder structure looks like: Let me explain to you some of the important folders here: The first one is the app folder. The assets folder inside the app folder is the location to store all the static files like JavaScript, CSS, and Images. You can take a sneak peek inside them to look at the various files. The controllers folder handles various requests and responses of the browser. The helpers folder contains various helper methods both for the views and controllers. The next folder mailers, contains all the necessary files to send an e-mail. The models folder contains files that interact with the database. Finally, we have the views folder, which contains all the .erb files that will be complied to HTML files. So, let's start the Rails server and check out our application on the browser: Navigate to the TODO folder in the terminal and then type the following command to start a server: rails server You can also use the following command: rails s You will see that the server is deployed under the port 3000. So, type the following URL to view the application: http://localhost:3000. You can also use the following URL: http://0.0.0.0:3000. If your application is properly set up, you should see the default page of Rails in the browser: Creating views We will be using Rails' scaffold method to create models, views, and other necessary files that Rails needs to make our application live. Here's the set of tasks that our application should perform: It should list out the pending items Every task should be clickable, and the details related to that item should be seen in a new view We can edit that item's description and some other details We can delete that item The task looks pretty lengthy, but any Rails developer would know how easy it is to do. We don't actually have to do anything to achieve it. We just have to pass a single scaffold command, and the rest will be taken care of. Close the Rails server using Ctrl + C keys and then proceed as follows: First, navigate to the project folder in the terminal. Then, pass the following command: rails g scaffold todo title:string description:text completed:boolean This will create a new model called todo that has various fields like title, description, and completed. Each field has a type associated with it. Since we have created a new model, it has to be reflected in the database. So, let's migrate it: rake db:create db:migrate The preceding code will create a new table inside a new database with the associated fields. Let's analyze what we have done. The scaffold command has created many HTML pages or views that are needed for managing the todo model. So, let's check out our application. We need to start our server again: rails s Go to the localhost page http://localhost:3000 at port number 3000. You should still see the Rails' default page. Now, type the URL: http://localhost:3000/todos. You should now see the application, as shown in the following screenshot: Click on New Todo, you will be taken to a form which allows you to fill out various fields that we had earlier created. Let's create our first todo and click on submit. It will be shown on the listing page: It was easy, wasn't it? We haven't done anything yet. That's the power of Rails, which people are crazy about. Summary This article was to brief you on how to develop and design a simple Rails application without the help of any CSS frontend frameworks. We manually styled the application by creating an external CSS file styles.css and importing it into the application using another CSS file application.css. We also discussed the complexities that a novice web designer might face on directly styling the application. Resources for Article: Further resources on this subject: Deep Customization of Bootstrap [article] The Bootstrap grid system [article] Getting Started with Bootstrap [article]

0
0
11788

How-To Tutorials

article-image-url-routing-and-template-rendering

Packt

24 Feb 2015

11 min read

URL Routing and Template Rendering

Packt

24 Feb 2015

11 min read

In this article by Ryan Baldwin, the author of Clojure Web Development Essentials, however, we will start building our application, creating actual endpoints that process HTTP requests, which return something we can look at. We will: Learn what the Compojure routing library is and how it works Build our own Compojure routes to handle an incoming request What this chapter won't cover, however, is making any of our HTML pretty, client-side frameworks, or JavaScript. Our goal is to understand the server-side/Clojure components and get up and running as quickly as possible. As a result, our templates are going to look pretty basic, if not downright embarrassing. (For more resources related to this topic, see here.) What is Compojure? Compojure is a small, simple library that allows us to create specific request handlers for specific URLs and HTTP methods. In other words, "HTTP Method A requesting URL B will execute Clojure function C. By allowing us to do this, we can create our application in a sane way (URL-driven), and thus architect our code in some meaningful way. For the studious among us, the Compojure docs can be found at https://github.com/weavejester/compojure/wiki. Creating a Compojure route Let's do an example that will allow the awful sounding tech jargon to make sense. We will create an extremely basic route, which will simply print out the original request map to the screen. Let's perform the following steps: Open the home.clj file. Alter the home-routes defroute such that it looks like this: (defroutes home-routes (GET "/" [] (home-page)) (GET "/about" [] (about-page)) (ANY "/req" request (str request))) Start the Ring Server if it's not already started. Navigate to http://localhost:3000/req. It's possible that your Ring Server will be serving off a port other than 3000. Check the output on lein ring server for the serving port if you're unable to connect to the URL listed in step 4. You should see something like this: Using defroutes Before we dive too much into the anatomy of the routes, we should speak briefly about what defroutes is. The defroutes macro packages up all of the routes and creates one big Ring handler out of them. Of course, you don't need to define all the routes for an application under a single defroutes macro. You can, and should, spread them out across various namespaces and then incorporate them into the app in Luminus' handler namespace. Before we start making a bunch of example routes, let's move the route we've already created to its own namespace: Create a new namespace hipstr.routes.test-routes (/hipstr/routes/test_routes.clj) . Ensure that the namespace makes use of the Compojure library: (ns hipstr.routes.test-routes (:require [compojure.core :refer :all])) Next, use the defroutes macro and create a new set of routes, and move the /req route we created in the hipstr.routes.home namespace under it: (defroutes test-routes (ANY "/req" request (str request))) Incorporate the new test-routes route into our application handler. In hipstr.handler, perform the following steps: Add a requirement to the hipstr.routes.test-routes namespace: (:require [compojure.core :refer [defroutes]] [hipstr.routes.home :refer [home-routes]] [hipstr.routes.test-routes :refer [test-routes]] …) Finally, add the test-routes route to the list of routes in the call to app-handler: (def app (app-handler ;; add your application routes here [home-routes test-routes base-routes] We've now created a new routing namespace. It's with this namespace where we will create the rest of the routing examples. Anatomy of a route So what exactly did we just create? We created a Compojure route, which responds to any HTTP method at /req and returns the result of a called function, in our case a string representation of the original request map. Defining the method The first argument of the route defines which HTTP method the route will respond to; our route uses the ANY macro, which means our route will respond to any HTTP method. Alternatively, we could have restricted which HTTP methods the route responds to by specifying a method-specific macro. The compojure.core namespace provides macros for GET, POST, PUT, DELETE, HEAD, OPTIONS, and PATCH. Let's change our route to respond only to requests made using the GET method: (GET "/req" request (str request)) When you refresh your browser, the entire request map is printed to the screen, as we'd expect. However, if the URL and the method used to make the request don't match those defined in our route, the not-found route in hipstr.handler/base-routes is used. We can see this in action by changing our route to listen only to the POST methods: (POST "/req" request (str request)) If you try and refresh the browser again, you'll notice we don't get anything back. In fact, an "HTTP 404: Page Not Found" response is returned to the client. If we POST to the URL from the terminal using curl, we'll get the following expected response: # curl -d {} http://localhost:3000/req {:ssl-client-cert nil, :go-bowling? "YES! NOW!", :cookies {}, :remote-addr "0:0:0:0:0:0:0:1", :params {}, :flash nil, :route-params {}, :headers {"user-agent" "curl/7.37.1", "content-type" "application/x-www-form-urlencoded", "content-length" "2", "accept" "*/*", "host" "localhost:3000"}, :server-port 3000, :content-length 2, :form-params {}, :session/key nil, :query-params {}, :content-type "application/x-www-form-urlencoded", :character-encoding nil, :uri "/req", :server-name "localhost", :query-string nil, :body #<HttpInput org.eclipse.jetty.server.HttpInput@38dea1>, :multipart-params {}, :scheme :http, :request-method :post, :session {}} Defining the URL The second component of the route is the URL on which the route is served. This can be anything we want and as long as the request to the URL matches exactly, the route will be invoked. There are, however, two caveats we need to be aware of: Routes are tested in order of their declaration, so order matters. The trailing slash isn't handled well. Compojure will always strip the trailing slash from the incoming request but won't redirect the user to the URL without the trailing slash. As a result an HTTP 404: Page Not Found response is returned. So never base anything off a trailing slash, lest ye peril in an ocean of confusion. Parameter destructuring In our previous example we directly refer to the implicit incoming request and pass that request to the function constructing the response. This works, but it's nasty. Nobody ever said, I love passing around requests and maintaining meaningless code and not leveraging URLs, and if anybody ever did, we don't want to work with them. Thankfully, Compojure has a rather elegant destructuring syntax that's easier to read than Clojure's native destructuring syntax. Let's create a second route that allows us to define a request map key in the URL, then simply prints that value in the response: (GET "/req/:val" [val] (str val)) Compojure's destructuring syntax binds HTTP request parameters to variables of the same name. In the previous syntax, the key :val will be in the request's :params map. Compojure will automatically map the value of {:params {:val...}} to the symbol val in [val]. In the end, you'll get the following output for the URL http://localhost:3000/req/holy-moly-molly: That's pretty slick but what if there is a query string? For example, http://localhost:3000/req/holy-moly-molly!?more=ThatsAHotTomalle. We can simply add the query parameter more to the vector, and Compojure will automatically bring it in: (GET "/req/:val" [val more] (str val "<br>" more)) Destructuring the request What happens if we still need access to the entire request? It's natural to think we could do this: (GET "/req/:val" [val request] (str val "<br>" request)) However, request will always be nil because it doesn't map back to a parameter key of the same name. In Compojure, we can use the magical :as key: (GET "/req/:val" [val :as request] (str val "<br>" request)) This will now result in request being assigned the entire request map, as shown in the following screenshot: Destructuring unbound parameters Finally, we can bind any remaining unbound parameters into another map using &. Take a look at the following example code: (GET "/req/:val/:another-val/:and-another" [val & remainders] (str val "<br>" remainders)) Saving the file and navigating to http://localhost:3000/req/holy-moly-molly!/what-about/susie-q will render both val and the map with the remaining unbound keys :another-val and :and-another, as seen in the following screenshot: Constructing the response The last argument in the route is the construction of the response. Whatever the third argument resolves to will be the body of our response. For example, in the following route: (GET "/req/:val" [val] (str val)) The third argument, (str val), will echo whatever the value passed in on the URL is. So far, we've simply been making calls to Clojure's str but we can just as easily call one of our own functions. Let's add another route to our hipstr.routes.test-routes, and write the following function to construct its response: (defn render-request-val [request-map & [request-key]] "Simply returns the value of request-key in request-map, if request-key is provided; Otherwise return the request-map. If request-key is provided, but not found in the request-map, a message indicating as such will be returned." (str (if request-key (if-let [result ((keyword request-key) request-map)] result (str request-key " is not a valid key.")) request-map))) (defroutes test-routes (POST "/req" request (render-request-val request)) ;no access to the full request map (GET "/req/:val" [val] (str val)) ;use :as to get access to full request map (GET "/req/:val" [val :as full-req] (str val "<br>" full-req)) ;use :as to get access to the remainder of unbound symbols (GET "/req/:val/:another-val/:and-another" [val & remainders] (str val "<br>" remainders)) ;use & to get access to unbound params, and call our route ;handler function (GET "/req/:key" [key :as request] (render-request-val request key))) Now when we navigate to http://localhost:3000/req/server-port, we'll see the value of the :server-port key in the request map… or wait… we should… what's wrong? If this doesn't seem right, it's because it isn't. Why is our /req/:val route getting executed? As stated earlier, the order of routes is important. Because /req/:val with the GET method is declared earlier, it's the first route to match our request, regardless of whether or not :val is in the HTTP request map's parameters. Routes are matched on URL structure, not on parameters keys. As it stands right now, our /req/:key will never get matched. We'll have to change it as follows: ;use & to get access to unbound params, and call our route handler function (GET "/req/:val/:another-val/:and-another" [val & remainders] (str val "<br>" remainders)) ;giving the route a different URL from /req/:val will ensure its execution (GET "/req/key/:key" [key :as request] (render-request-val request key))) Now that our /req/key/:key route is logically unique, it will be matched appropriately and render the server-port value to screen. Let's try and navigate to http://localhost:3000/req/key/server-port again: Generating complex responses What if we want to create more complex responses? How might we go about doing that? The last thing we want to do is hardcode a whole bunch of HTML into a function, it's not 1995 anymore, after all. This is where the Selmer library comes to the rescue. Summary In this article we have learnt what Compojure is, what a Compojure routing library is and how it works. You have also learnt to build your own Compojure routes to handle an incoming request, within which you learnt how to use defroutes, the anatomy of a route, destructuring parameter and how to define the URL. Resources for Article: Further resources on this subject: Vmware Vcenter Operations Manager Essentials - Introduction To Vcenter Operations Manager [article] Websockets In Wildfly [article] Clojure For Domain-Specific Languages - Design Concepts With Clojure [article]

0
0
2297

How-To Tutorials

Packt

23 Feb 2015

7 min read

Actors and Pawns

Packt

23 Feb 2015

7 min read

In this article by William Sherif, author of the book Learning C++ by Creating Games with UE4, we will really delve into UE4 code. At first, it is going to look daunting. The UE4 class framework is massive, but don't worry. The framework is massive, so your code doesn't have to be. You will find that you can get a lot done and a lot onto the screen using relatively less code. This is because the UE4 engine code is so extensive and well programmed that they have made it possible to get almost any game-related task done easily. Just call the right functions, and voila, what you want to see will appear on the screen. The entire notion of a framework is that it is designed to let you get the gameplay you want, without having to spend a lot of time in sweating out the details. (For more resources related to this topic, see here.) Actors versus pawns A Pawn is an object that represents something that you or the computer's Artificial Intelligence (AI) can control on the screen. The Pawn class derives from the Actor class, with the additional ability to be controlled by the player directly or by an AI script. When a pawn or actor is controlled by a controller or AI, it is said to be possessed by that controller or AI. Think of the Actor class as a character in a play. Your game world is going to be composed of a bunch of actors, all acting together to make the gameplay work. The game characters, Non-player Characters (NPCs), and even treasure chests will be actors. Creating a world to put your actors in Here, we will start from scratch and create a basic level into which we can put our game characters. The UE4 team has already done a great job of presenting how the world editor can be used to create a world in UE4. I want you to take a moment to create your own world. First, create a new, blank UE4 project to get started. To do this, in the Unreal Launcher, click on the Launch button beside your most recent engine installation, as shown in the following screenshot: That will launch the Unreal Editor. The Unreal Editor is used to visually edit your game world. You're going to spend a lot of time in the Unreal Editor, so please take your time to experiment and play around with it. I will only cover the basics of how to work with the UE4 editor. You will need to let your creative juices flow, however, and invest some time in order to become familiar with the editor. To learn more about the UE4 editor, take a look at the Getting Started: Introduction to the UE4 Editor playlist, which is available at https://www.youtube.com/playlist?list=PLZlv_N0_O1gasd4IcOe9Cx9wHoBB7rxFl. Once you've launched the UE4 editor, you will be presented with the Projects dialog. The following screenshot shows the steps to be performed with numbers corresponding to the order in which they need to be performed: Perform the following steps to create a project: Select the New Project tab at the top of the screen. Click on the C++ tab (the second subtab). Then select Basic Code from the available projects listing. Set the directory where your project is located (mine is Y:Unreal Projects). Choose a hard disk location with a lot of space (the final project will be around 1.5 GB). Name your project. I called mine GoldenEgg. Click on Create Project to finalize project creation. Once you've done this, the UE4 launcher will launch Visual Studio. There will only be a couple of source files in Visual Studio, but we're not going to touch those now. Make sure that Development Editor is selected from the Configuration Manager dropdown at the top of the screen, as shown in the following screenshot: Now launch your project by pressing Ctrl + F5 in Visual Studio. You will find yourself in the Unreal Engine 4 editor, as shown in the following screenshot: The UE4 editor We will explore the UE4 editor here. We'll start with the controls since it is important to know how to navigate in Unreal. Editor controls If you've never used a 3D editor before, the controls can be quite hard to learn. These are the basic navigation controls while in edit mode: Use the arrow keys to move around in the scene Press Page Up or Page Down to go up and down vertically Left mouse click + drag it left or right to change the direction you are facing Left mouse click + drag it up or down to dolly (move the camera forward and backward, same as pressing up/down arrow keys) Right mouse click + drag to change the direction you are facing Middle mouse click + drag to pan the view Right mouse click and the W, A, S, and D keys to move around the scene Play mode controls Click on the Play button in the bar at the top, as shown in the following screenshot. This will launch the play mode. Once you click on the Play button, the controls change. In play mode, the controls are as follows: The W, A, S, and D keys for movement The left or right arrow keys to look toward the left and right, respectively The mouse's motion to change the direction in which you look The Esc key to exit play mode and return to edit mode What I suggest you do at this point is try to add a bunch of shapes and objects into the scene and try to color them with different materials. Adding objects to the scene Adding objects to the scene is as easy as dragging and dropping them in from the Content Browser tab. The Content Browser tab appears, by default, docked at the left-hand side of the window. If it isn't seen, simply select Window and navigate to Content Browser in order to make it appear. Make sure that the Content Browser is visible in order to add objects to your level Next, select the Props folder on the left-hand side of the Content Browser. Drag and drop things from the Content Browser into your game world To resize an object, press R on your keyboard. The manipulators around the object will appear as boxes, which denotes resize mode. Press R on your keyboard to resize an object In order to change the material that is used to paint the object, simply drag and drop a new material from the Content Browser window inside the Materials folder. Drag and drop a material from the Content Browser's Materials folder to color things with a new color Materials are like paints. You can coat an object with any material you want by simply dragging and dropping the material you desire onto the object you desire it to be coated on. Materials are only skin-deep: they don't change the other properties of an object (such as weight). Starting from scratch If you want to start creating a level from scratch, simply click on File and navigate to New Level..., as shown here: You can then select between Default and Empty Level. I think selecting Empty Level is a good idea, for the reasons that are mentioned later. The new level will be completely black in color to start with. Try dragging and dropping some objects from the Content Browser tab again. This time, I added a resized shapes / box for the ground plane and textured it with moss, a couple of Props / SM_Rocks, Particles / P_Fire, and most importantly, a light source. Be sure to save your map. Here's a snapshot of my map (how does yours look?): Summary In this article, we reviewed how the realistic environments are created with actors and monsters which are part of the game and also seen how the various kind of levels are created from the scratch. Resources for Article: Further resources on this subject: Program structure, execution flow, and runtime objects [article] What is Quantitative Finance? [article] Creating and Utilizing Custom Entities [article]

0
0
2928

Packt

23 Feb 2015

8 min read

Selenium Testing Tools

Packt

23 Feb 2015

8 min read

In this article by Raghavendra Prasad MG, author of the book Learning Selenium Testing Tools, Third Edition you will be introduced to the Selenium IDE WebDriver by demonstrating its installation, basic features, its advanced features, and implementation of automation framework with programming language basics required for automation. Automation being a key point of success of any software organization, everybody is looking at the freeware and huge community supported tool like Selenium. Anybody who is willing to learn and work on automation with Selenium has an opportunity to learn the tool from basic to advanced stage with this book and the book would become a life time reference for the reader. (For more resources related to this topic, see here.) Key features of the book Following are the key features of the book: The book contain the information from basic level to advanced levels and user need not have know anything as a pre requisite. The book contains real time examples which make the reader to co-relate with there real time scenarios. The book contains basics of Java which is required for Selenium automation, hence reader need to go through other books exclusively which would contain the information which is no where required for the Selenium automation. The book contains the concept of automation framework design and implementation, which would definitely help reader to build and implement his/her own automation framework. What you will learn from this book? There are a lot of things you will learn from the book. A few of them are mentioned as follows: History of Selenium and its evolution Working with previous versions of Selenium that is, Selenium IDE Selenium WebDriver – basic to advanced state Basics of Java (only what is required for Selenium automation) WebElement handling with Selenium WebDriver Page Object Factory implementation Automation frameworks types, design and implementation Utilities building for automation Who this book is for? The book is for manual testers. Even any software professionals and the ones who wants to make their carrier in Selenium automation testing can use this book. This book also helps automation testers / automation architects who want to build or implement the automation / automation frameworks on Selenium automation tool. What this book covers The book covers the following major topics: Selenium IDE Selenium IDE is a Firefox add-on developed originally by Shinya Kasatani as a way to use the original Selenium Core code without having to copy Selenium Core onto the server. Selenium Core is the key JavaScript modules that allows Selenium to drive the browser. It has been developed using JavaScript so that it can interact with the DOM (Document Object Model) using native JavaScript calls. Selenium IDE has been developed to allow testers and developers to record their actions as they follow the workflow that they need to test. Locators Locators shows how we can find elements on the page to be used in our tests. We will use XPath, CSS, link text, and ID to find elements on the page so that we can interact with them. Locators allow us to find elements on a page that can be used in our tests. In the last chapter we managed to work against a page which had decent locators. In HTML, it is seen as a good practice to make sure that every element you need to interact with has an ID attribute and a name attribute. Unfortunately, following best practices can be extremely difficult, especially when building the HTML dynamically on the server before sending it back to the browser. Following are the locators used in Selenium IDE: Locators Description Example ID This element identifies an ID attribute on the page id=inputButton name This element identifies name attribute on the page name=buttonFind link This element identifies links by the text link=index XPath This element identifies by XPath xpath=//div[@class='classname'] CSS This element identifies by CSS css=#divinthecenter DOM This element identifies by DOM dom=document.getElementById("inputButton") Selenium WebDriver The primary feature of the Selenium WebDriver is the integration of the WebDriver API and its design to provide a simpler, more concise programming interface in addition to addressing some limitations in the Selenium-RC API. Selenium WebDriver was developed to better support dynamic web pages where elements of a page may change without the page itself being reloaded. WebDriver's goal is to supply a well designed object-oriented API that provides improved support for modern advanced web application testing problems. Finding elements When working with WebDriver on a web application, we will need to find elements on the page. This is the Core to being able to work. All the methods for performing actions to the web application, like typing and clicking require that we search the element first. Finding an element on the page by its ID The first item that we are going to look at is finding an element by ID. Searching elements by ID is one of the easiest ways to find an element. We start with findElementByID(). This method is a helper method that sets an argument for a more generic findElement call. We will see now how we can use it in action. The method's signature looks like the following line of code: findElementById(String using); The using variable takes the ID of the element that you wish to look for. It will return a WebElement object that we can then work with. Using findElementById() We find an element on the page by using the findElementById() method that is on each of the Browser Driver classes. findElement calls will return a WebElement object that we can perform actions on. Follow these steps to see how it works: Open your Java IDE. (IntelliJ or Eclipse are the one's that are mostly used) We are going to use the following command: WebElement element = ((FindsById)driver). findElementById("verifybutton"); Run the test from the IDE. It will look like the following screenshot: Page Objects In this section of the article, we are going to have a look at how we can apply some best practices to tests. You will learn how to make maintainable test suites that will allow you to update tests in seconds. We will have a look at creating your own DSL so that people can see intent. We will create tests using the Page Object pattern. Working with FirefoxDriver FirefoxDriver is the easiest driver to use, since everything that we need to use is all bundled with the Java client bindings. We do the basic task of loading the browser and type into the page as follows: Update the setUp() method to load the FirefoxDriver(); driver = new FirefoxDriver(); Now we need to find an element. We will find the one with the ID nextBid: WebElement element = driver.findElement(By.id("nextBid")); Now we need to type into that element as follows: element.sendKeys("100"); Run your test and it should look like the following: import org.openqa.selenium.*; import org.openqa.selenium.firefox.*; import org.testng.annotations.*; public class TestChapter6 { WebDriver driver; @BeforeTest public void setUp(){ driver = new FirefoxDriver(); driver.get ("http://book.theautomatedtester.co.uk/chapter4"); } @AfterTest public void tearDown(){ driver.quit(); } @Test public void testExamples(){ WebElement element = driver.findElement(By.id("nextBid")); element.sendKeys("100"); } } We are currently witnessing an explosion of mobile devices in the market. A lot of them are more powerful than your average computer was just over a decade ago. This means, that in addition to having nice clean, responsive, and functional desktop applications, we are starting to have to make sure the same basic functionality is available to mobile devices. We are going to be looking at how we can set up mobile devices to be used with Selenium WebDriver. We will learn the following topics: How to use the stock browser on Android How to test with Opera Mobile How to test on iOS Understanding Selenium Grid Selenium Grid is a version of Selenium that allows teams to set up a number of Selenium instances and then have one central point to send your Selenium commands to. This differs from what we saw in Selenium Remote WebDriver where we always had to explicitly say where the Selenium Server is as well as know what browsers that server can handle. With Selenium Grid we just ask for a specific browser, and then the hub that is part of Selenium Grid will route all the Selenium commands through to the Remote Control you want. Summary We have understood and learnt what is Selenium and its evolutions from IDE to WebDriver and Grid as well. In addition we have learnt how to identify the WebElements using WebDriver, and its design pattern and locators through WebDriver. And we learnt on the automation framework design and implementation, mobile application automation on Android and iOS. Finally we understood the concept of Selenium Grid. Resources for Article: Further resources on this subject: Quick Start into Selenium Tests [article] Getting Started With Selenium Webdriver and Python [article] Exploring Advanced Interactions of WebDriver [article]

0
0
4957

article-image-customizing-app-controller

Packt

23 Feb 2015

15 min read

Customizing App Controller

Packt

23 Feb 2015

15 min read

0
0
4782

Packt

20 Feb 2015

6 min read

Sprites in Action

Packt

20 Feb 2015

6 min read

In this article by Milcho G. Milchev, author of the book SFML Essentials, we will see how we can use SFML to create a customized animation using a sequence of images. We will also see how SFML renders an animation. Animation exists in many forms. The traditional approach to animation is drawing a sequence of images which differ slightly from each other, and showing them on a screen one after the other. Even though this approach is still widely used, there are more elegant alternatives. For example, drawing (or modelling in 3D) only the limbs of a character and then animating how they move relative to time is a technique that saves a lot of time for artists. It also creates smoother results because not every frame of the animation has to be redrawn. In this book, we are going to explore only the traditional approach, since it is the simpler solution for programmers, and in many cases it is enough to bring life to any sprite. (For more resources related to this topic, see here.) The setup As we established earlier, the traditional approach involves a set of images that need to change over time. For our example, we will use a crystal, which rotates around its centre. Typically, an animation is kept in a single file (a sprite sheet), where each frame of the animation is stored, and in most cases, each frame is the same size—the size of the object. In our example, the sprite is, 32 x 32 pixels and has eight frames, which play for one second. Here is what the sprite sheet looks like: The following screenshot shows our animation setup in code: First of all, note that we are using the AssetManager class to load our sprite sheet. The next line sets the texture rectangle of the sprite to target the first image in our sprite sheet. Here is what this means in terms of the sprite sheet texture: Next, we will move this texture rectangle once in a while to simulate a rotating crystal. In the previous code, we set the number of frames to eight (as many as there are in the sprite sheet), and set the time of the animation to one second in total, which means that each frame stays for about 0.125 seconds (the animation duration is divided by the number of frames) at a time. We know what we want to do now, so let's do it: In the code, we first measure the delta time since the last frame and add it to the accumulated time. The last two lines of the code actually do all the work. The first one looks intimidating at first glance, but it is simply a way to choose the correct frame, based on how much time has passed and how long the animation is. The formula timeAsSeconds / animationDuration gives us the time relative to the animation duration. So let's say that 0.4 seconds have passed and our animation duration is 1 second. This leaves us with 0.4 seconds in local animation time. Multiply this 0.4 seconds by the number of frames, and we get the following result: 0.4 * 8 = 3.2 This gives us which frame we should be on at the moment, and how long we have been there. The current frame index is the whole part of 3.2 (which is three), and the fraction part (0.2) is how long we have been on that frame. In this case, we are only interested in the current frame so we will take that by casting the whole expression to int. This rounds the number down if the number is positive (which it always is in this case). The last part, % frameNum is there to restart the animation when it reaches beyond its last frame. So in the case where 2.3 seconds have passed, we have the following result: 2.3 * 8 = 18.4 We do not have a 19th frame to show, so we show the frame which corresponds to that in our local scale [0…7]. In this case: 18 / 8 = 2 (and 2 remainder) Since the % operator takes the remainder of a division, we are left to show the frame with the index two, which is the third frame. (We start counting from zero as programmers, remember?) The last line of the code sets the texture rectangle to the current frame. The process is quite straightforward—since we only have frames on the x axis, we do not need to worry about the y coordinate of the rectangle, and so we will set it to zero. The x is computed by animFrame * spriteSize.x, which multiplies the current frame by the width of the frame. In the case, the current frame is two and the frame's width is 32, so we get: 2 * 32 = 64 Here is what the texture rectangle will look like: The last thing we need to do is render the sprite inside the render frame and we are done. If everything goes smoothly, we should have a rotating crystal on the screen with eight frames. With this technique, we can animate sprites of all kinds no matter how many frames they have or how long the animation is. There are problems with the current approach though - the code looks messy, and it is only useful for a single animation. What if we want multiple animations for a sprite (rotating the crystal in a vertical direction as well), and we want to be able to switch between them? Currently, we would have to duplicate all our code for each animation and each animated sprite. In the next section, we will talk about how to avoid these issues by building a fully featured animation system that requires as little code duplication as possible. Summary Sprite animations seem quite easy now, don't they? Just keep in mind that there is a lot more to explore when it comes to animation. Not only are there different techniques to doing them, but also perfecting what we've developed so far might take some time. Fortunately, what we have so far will work as is in the majority of cases so I would say that you are pretty much set to go. If you want to dig deeper, buy the book and read SFML Essentials in a simple step-by-step fashion by using SFML library to create realistic looking animations as well as to develop 2D and 3D games using the SFML. Resources for Article: Further resources on this subject: Vmware Vcenter Operations Manager Essentials - Introduction To Vcenter Operations Manager [article] Translating a File In Sdl Trados Studio [article] Adding Finesse To Your Game [article]

0
0
26484

Packt

20 Feb 2015

18 min read

Puppet Language and Style

Packt

20 Feb 2015

18 min read

In this article by Thomas Uphill, author of the book Puppet Cookbook Third Edition, we will cover the following recipes: Installing a package before starting a service Installing, configuring, and starting a service Using regular expressions in if statements Using selectors and case statements Creating a centralized Puppet infrastructure Creating certificates with multiple DNS names (For more resources related to this topic, see here.) Installing a package before starting a service To show how ordering works, we'll create a manifest that installs httpd and then ensures the httpd package service is running. How to do it... We start by creating a manifest that defines the service: service {'httpd': ensure => running, require => Package['httpd'], } The service definition references a package resource named httpd; we now need to define that resource: package {'httpd': ensure => 'installed', } How it works... In this example, the package will be installed before the service is started. Using require within the definition of the httpd service ensures that the package is installed first, regardless of the order within the manifest file. Capitalization Capitalization is important in Puppet. In our previous example, we created a package named httpd. If we wanted to refer to this package later, we would capitalize its type (package) as follows: Package['httpd'] To refer to a class, for example, the something::somewhere class, which has already been included/defined in your manifest, you can reference it with the full path as follows: Class['something::somewhere'] When you have a defined type, for example the following defined type: example::thing {'one':} The preceding resource may be referenced later as follows: Example::Thing['one'] Knowing how to reference previously defined resources is necessary for the next section on metaparameters and ordering. Learning metaparameters and ordering All the manifests that will be used to define a node are compiled into a catalog. A catalog is the code that will be applied to configure a node. It is important to remember that manifests are not applied to nodes sequentially. There is no inherent order to the application of manifests. With this in mind, in the previous httpd example, what if we wanted to ensure that the httpd process started after the httpd package was installed? We couldn't rely on the httpd service coming after the httpd package in the manifests. What we have to do is use metaparameters to tell Puppet the order in which we want resources applied to the node. Metaparameters are parameters that can be applied to any resource and are not specific to any one resource type. They are used for catalog compilation and as hints to Puppet but not to define anything about the resource to which they are attached. When dealing with ordering, there are four metaparameters used: before require notify subscribe The before and require metaparameters specify a direct ordering; notify implies before and subscribe implies require. The notify metaparameter is only applicable to services; what notify does is tell a service to restart after the notifying resource has been applied to the node (this is most often a package or file resource). In the case of files, once the file is created on the node, a notify parameter will restart any services mentioned. The subscribe metaparameter has the same effect but is defined on the service; the service will subscribe to the file. Trifecta The relationship between package and service previously mentioned is an important and powerful paradigm of Puppet. Adding one more resource-type file into the fold, creates what puppeteers refer to as the trifecta. Almost all system administration tasks revolve around these three resource types. As a system administrator, you install a package, configure the package with files, and then start the service. Diagram of Trifecta (Files require package for directory, service requires files and package) Idempotency A key concept of Puppet is that the state of the system when a catalog is applied to a node cannot affect the outcome of Puppet run. In other words, at the end of Puppet run (if the run was successful), the system will be in a known state and any further application of the catalog will result in a system that is in the same state. This property of Puppet is known as idempotency. Idempotency is the property that no matter how many times you do something, it remains in the same state as the first time you did it. For instance, if you had a light switch and you gave the instruction to turn it on, the light would turn on. If you gave the instruction again, the light would remain on. Installing, configuring, and starting a service There are many examples of this pattern online. In our simple example, we will create an Apache configuration file under /etc/httpd/conf.d/cookbook.conf. The /etc/httpd/conf.d directory will not exist until the httpd package is installed. After this file is created, we would want httpd to restart to notice the change; we can achieve this with a notify parameter. How to do it... We will need the same definitions as our last example; we need the package and service installed. We now need two more things. We need the configuration file and index page (index.html) created. For this, we follow these steps: As in the previous example, we ensure the service is running and specify that the service requires the httpd package: service {'httpd': ensure => running, require => Package['httpd'], } We then define the package as follows: package {'httpd': ensure => installed, } Now, we create the /etc/httpd/conf.d/cookbook.conf configuration file; the /etc/httpd/conf.d directory will not exist until the httpd package is installed. The require metaparameter tells Puppet that this file requires the httpd package to be installed before it is created: file {'/etc/httpd/conf.d/cookbook.conf': content => "<VirtualHost *:80>nServername cookbooknDocumentRoot /var/www/cookbookn</VirtualHost>n", require => Package['httpd'], notify => Service['httpd'], } We then go on to create an index.html file for our virtual host in /var/www/cookbook. This directory won't exist yet, so we need to create this as well, using the following code: file {'/var/www/cookbook': ensure => directory, } file {'/var/www/cookbook/index.html': content => "<html><h1>Hello World!</h1></html>n", require => File['/var/www/cookbook'], } How it works… The require attribute to the file resources tell Puppet that we need the /var/www/cookbook directory created before we can create the index.html file. The important concept to remember is that we cannot assume anything about the target system (node). We need to define everything on which the target depends. Anytime you create a file in a manifest, you have to ensure that the directory containing that file exists. Anytime you specify that a service should be running, you have to ensure that the package providing that service is installed. In this example, using metaparameters, we can be confident that no matter what state the node is in before running Puppet, after Puppet runs, the following will be true: httpd will be running The VirtualHost configuration file will exist httpd will restart and be aware of the VirtualHost file The DocumentRoot directory will exist An index.html file will exist in the DocumentRoot directory Using regular expressions in if statements Another kind of expression you can test in if statements and other conditionals is the regular expression. A regular expression is a powerful way to compare strings using pattern matching. How to do it… This is one example of using a regular expression in a conditional statement. Add the following to your manifest: if $::architecture =~ /64/ { notify { '64Bit OS Installed': } } else { notify { 'Upgrade to 64Bit': } fail('Not 64 Bit') } How it works… Puppet treats the text supplied between the forward slashes as a regular expression, specifying the text to be matched. If the match succeeds, the if expression will be true and so the code between the first set of curly braces will be executed. In this example, we used a regular expression because different distributions have different ideas on what to call 64bit; some use amd64, while others use x86_64. The only thing we can count on is the presence of the number 64 within the fact. Some facts that have version numbers in them are treated as strings to Puppet. For instance, $::facterversion. On my test system, this is 2.0.1, but when I try to compare that with 2, Puppet fails to make the comparison: Error: comparison of String with 2 failed at /home/thomas/.puppet/manifests/version.pp:1 on node cookbook.example.com If you wanted instead to do something if the text does not match, use !~ rather than =~: if $::kernel !~ /Linux/ { notify { 'Not Linux, could be Windows, MacOS X, AIX, or ?': } } There's more… Regular expressions are very powerful, but can be difficult to understand and debug. If you find yourself using a regular expression so complex that you can't see at a glance what it does, think about simplifying your design to make it easier. However, one particularly useful feature of regular expressions is the ability to capture patterns. Capturing patterns You can not only match text using a regular expression, but also capture the matched text and store it in a variable: $input = 'Puppet is better than manual configuration' if $input =~ /(.*) is better than (.*)/ { notify { "You said '${0}'. Looks like you're comparing ${1} to ${2}!": } } The preceding code produces this output: You said 'Puppet is better than manual configuration'. Looks like you're comparing Puppet to manual configuration! The variable $0 stores the whole matched text (assuming the overall match succeeded). If you put brackets around any part of the regular expression, it creates a group, and any matched groups will also be stored in variables. The first matched group will be $1, the second $2, and so on, as shown in the preceding example. Regular expression syntax Puppet's regular expression syntax is the same as Ruby's, so resources that explain Ruby's regular expression syntax will also help you with Puppet. You can find a good introduction to Ruby's regular expression syntax at this website: http://www.tutorialspoint.com/ruby/ruby_regular_expressions.htm. Using selectors and case statements Although you could write any conditional statement using if, Puppet provides a couple of extra forms to help you express conditionals more easily: the selector and the case statement. How to do it… Here are some examples of selector and case statements: Add the following code to your manifest: $systemtype = $::operatingsystem ? { 'Ubuntu' => 'debianlike', 'Debian' => 'debianlike', 'RedHat' => 'redhatlike', 'Fedora' => 'redhatlike', 'CentOS' => 'redhatlike', default => 'unknown', } notify { "You have a ${systemtype} system": } Add the following code to your manifest: class debianlike { notify { 'Special manifest for Debian-like systems': } } class redhatlike { notify { 'Special manifest for RedHat-like systems': } } case $::operatingsystem { 'Ubuntu', 'Debian': { include debianlike } 'RedHat', 'Fedora', 'CentOS', 'Springdale': { include redhatlike } default: { notify { "I don't know what kind of system you have!": } } } How it works… Our example demonstrates both the selector and the case statement, so let's see in detail how each of them works. Selector In the first example, we used a selector (the ? operator) to choose a value for the $systemtype variable depending on the value of $::operatingsystem. This is similar to the ternary operator in C or Ruby, but instead of choosing between two possible values, you can have as many values as you like. Puppet will compare the value of $::operatingsystem to each of the possible values we have supplied in Ubuntu, Debian, and so on. These values could be regular expressions (for example, for a partial string match, or to use wildcards), but in our case, we have just used literal strings. As soon as it finds a match, the selector expression returns whatever value is associated with the matching string. If the value of $::operatingsystem is Fedora, for example, the selector expression will return the redhatlike string and this will be assigned to the variable $systemtype. Case statement Unlike selectors, the case statement does not return a value. case statements come in handy when you want to execute different code depending on the value of some expression. In our second example, we used the case statement to include either the debianlike or redhatlike class, depending on the value of $::operatingsystem. Again, Puppet compares the value of $::operatingsystem to a list of potential matches. These could be regular expressions or strings, or as in our example, comma-separated lists of strings. When it finds a match, the associated code between curly braces is executed. So, if the value of $::operatingsystem is Ubuntu, then the code including debianlike will be executed. There's more… Once you've got a grip of the basic use of selectors and case statements, you may find the following tips useful. Regular expressions As with if statements, you can use regular expressions with selectors and case statements, and you can also capture the values of the matched groups and refer to them using $1, $2, and so on: case $::lsbdistdescription { /Ubuntu (.+)/: { notify { "You have Ubuntu version ${1}": } } /CentOS (.+)/: { notify { "You have CentOS version ${1}": } } default: {} } Defaults Both selectors and case statements let you specify a default value, which is chosen if none of the other options match (the style guide suggests you always have a default clause defined): $lunch = 'Filet mignon.' $lunchtype = $lunch ? { /fries/ => 'unhealthy', /salad/ => 'healthy', default => 'unknown', } notify { "Your lunch was ${lunchtype}": } The output is as follows: t@mylaptop ~ $ puppet apply lunchtype.pp Notice: Your lunch was unknown Notice: /Stage[main]/Main/Notify[Your lunch was unknown]/message: defined 'message' as 'Your lunch was unknown' When the default action shouldn't normally occur, use the fail() function to halt the Puppet run. How to do it… The following steps will show you how to use the in operator: Add the following code to your manifest: if $::operatingsystem in [ 'Ubuntu', 'Debian' ] { notify { 'Debian-type operating system detected': } } elsif $::operatingsystem in [ 'RedHat', 'Fedora', 'SuSE', 'CentOS' ] { notify { 'RedHat-type operating system detected': } } else { notify { 'Some other operating system detected': } } Run Puppet: t@cookbook:~/.puppet/manifests$ puppet apply in.pp Notice: Compiled catalog for cookbook.example.com in environment production in 0.03 seconds Notice: Debian-type operating system detected Notice: /Stage[main]/Main/Notify[Debian-type operating system detected]/message: defined 'message' as 'Debian-type operating system detected'Notice: Finished catalog run in 0.02 seconds There's more… The value of an in expression is Boolean (true or false) so you can assign it to a variable: $debianlike = $::operatingsystem in [ 'Debian', 'Ubuntu' ] if $debianlike { notify { 'You are in a maze of twisty little packages, all alike': } } Creating a centralized Puppet infrastructure A configuration management tool such as Puppet is best used when you have many machines to manage. If all the machines can reach a central location, using a centralized Puppet infrastructure might be a good solution. Unfortunately, Puppet doesn't scale well with a large number of nodes. If your deployment has less than 800 servers, a single Puppet master should be able to handle the load, assuming your catalogs are not complex (take less than 10 seconds to compile each catalog). If you have a larger number of nodes, I suggest a load balancing configuration described in Mastering Puppet, Thomas Uphill, Packt Publishing. A Puppet master is a Puppet server that acts as an X509 certificate authority for Puppet and distributes catalogs (compiled manifests) to client nodes. Puppet ships with a built-in web server called WEBrick, which can handle a very small number of nodes. In this section, we will see how to use that built-in server to control a very small (less than 10) number of nodes. Getting ready The Puppet master process is started by running puppet master; most Linux distributions have start and stop scripts for the Puppet master in a separate package. To get started, we'll create a new debian server named puppet.example.com. How to do it... Install Puppet on the new server and then use Puppet to install the Puppet master package: # puppet resource package puppetmaster ensure='installed' Notice: /Package[puppetmaster]/ensure: created package { 'puppetmaster': ensure => '3.7.0-1puppetlabs1', } Now start the Puppet master service and ensure it will start at boot: # puppet resource service puppetmaster ensure=true enable=true service { 'puppetmaster': ensure => 'running', enable => 'true', } How it works... The Puppet master package includes the start and stop scripts for the Puppet master service. We use Puppet to install the package and start the service. Once the service is started, we can point another node at the Puppet master (you might need to disable the host-based firewall on your machine). From another node, run puppet agent to start a puppet agent, which will contact the server and request a new certificate: t@ckbk:~$ sudo puppet agent -t Info: Creating a new SSL key for cookbook.example.com Info: Caching certificate for ca Info: Creating a new SSL certificate request for cookbook.example.com Info: Certificate Request fingerprint (SHA256): 06:C6:2B:C4:97:5D:16:F2:73:82:C4:A9:A7:B1:D0:95:AC:69:7B:27:13:A9:1A:4C:98:20:21:C2:50:48:66:A2 Info: Caching certificate for ca Exiting; no certificate found and waitforcert is disabled Now on the Puppet server, sign the new key: root@puppet:~# puppet cert list pu "cookbook.example.com" (SHA256) 06:C6:2B:C4:97:5D:16:F2:73:82:C4:A9:A7:B1:D0:95:AC:69:7B:27:13:A9:1A:4C:98:20:21:C2:50:48:66:A2 root@puppet:~# puppet cert sign cookbook.example.com Notice: Signed certificate request for cookbook.example.com Notice: Removing file Puppet::SSL::CertificateRequestcookbook.example.com at'/var/lib/puppet/ssl/ca/requests/cookbook.example.com.pem' Return to the cookbook node and run Puppet again: t@ckbk:~$ sudo puppet agent –vt Info: Caching certificate for cookbook.example.com Info: Caching certificate_revocation_list for caInfo: Caching certificate for cookbook.example.comInfo: Retrieving pluginfactsInfo: Retrieving pluginInfo: Caching catalog for cookbookInfo: Applying configuration version '1410401823'Notice: Finished catalog run in 0.04 seconds There's more... When we ran puppet agent, Puppet looked for a host named puppet.example.com (since our test node is in the example.com domain); if it couldn't find that host, it would then look for a host named Puppet. We can specify the server to contact with the --server option to puppet agent. When we installed the Puppet master package and started the Puppet master service, Puppet created default SSL certificates based on our hostname. Creating certificates with multiple DNS names By default, Puppet will create an SSL certificate for your Puppet master that contains the fully qualified domain name of the server only. Depending on how your network is configured, it can be useful for the server to be known by other names. In this recipe, we'll make a new certificate for our Puppet master that has multiple DNS names. Getting ready Install the Puppet master package if you haven't already done so. You will then need to start the Puppet master service at least once to create a certificate authority (CA). How to do it... The steps are as follows: Stop the running Puppet master process with the following command: # service puppetmaster stop [ ok ] Stopping puppet master. Delete (clean) the current server certificate: # puppet cert clean puppet Notice: Revoked certificate with serial 6 Notice: Removing file Puppet::SSL::Certificate puppet at '/var/lib/puppet/ssl/ca/signed/puppet.pem' Notice: Removing file Puppet::SSL::Certificate puppet at '/var/lib/puppet/ssl/certs/puppet.pem' Notice: Removing file Puppet::SSL::Key puppet at '/var/lib/puppet/ssl/private_keys/puppet.pem' Create a new Puppet certificate using Puppet certificate generate with the --dns-alt-names option: root@puppet:~# puppet certificate generate puppet --dns-alt-names puppet.example.com,puppet.example.org,puppet.example.net --ca-location local Notice: puppet has a waiting certificate request true Sign the new certificate: root@puppet:~# puppet cert --allow-dns-alt-names sign puppet Notice: Signed certificate request for puppet Notice: Removing file Puppet::SSL::CertificateRequest puppet at '/var/lib/puppet/ssl/ca/requests/puppet.pem' Restart the Puppet master process: root@puppet:~# service puppetmaster restart[ ok ] Restarting puppet master. How it works... When your puppet agents connect to the Puppet server, they look for a host called Puppet, they then look for a host called Puppet.[your domain]. If your clients are in different domains, then you need your Puppet master to reply to all the names correctly. By removing the existing certificate and generating a new one, you can have your Puppet master reply to multiple DNS names. Summary Configuration management has become a requirement for system administrators. Knowing how to use configuration management tools, such as Puppet, enables administrators to take full advantage of automated provisioning systems and cloud resources. There is a natural progression from performing a task, scripting a task to creating a module in Puppet, or Puppetizing a task. Resources for Article: Further resources on this subject: Designing Puppet Architectures [article] External Tools and the Puppet Ecosystem [article] Puppet: Integrating External Tools [article]

0
0
3903

Packt

20 Feb 2015

13 min read

The Spark Programming Model

Packt

20 Feb 2015

13 min read

In this article by Nick Pentreath, author of the book Machine Learning with Spark, we will delve into a high-level overview of Spark's design, we will introduce the SparkContext object as well as the Spark shell, which we will use to interactively explore the basics of the Spark programming model. While this section provides a brief overview and examples of using Spark, we recommend that you read the following documentation to get a detailed understanding:Spark Quick Start: http://spark.apache.org/docs/latest/quick-start.htmlSpark Programming guide, which covers Scala, Java, and Python: http://spark.apache.org/docs/latest/programming-guide.html (For more resources related to this topic, see here.) SparkContext and SparkConf The starting point of writing any Spark program is SparkContext (or JavaSparkContext in Java). SparkContext is initialized with an instance of a SparkConf object, which contains various Spark cluster-configuration settings (for example, the URL of the master node). Once initialized, we will use the various methods found in the SparkContext object to create and manipulate distributed datasets and shared variables. The Spark shell (in both Scala and Python, which is unfortunately not supported in Java) takes care of this context initialization for us, but the following lines of code show an example of creating a context running in the local mode in Scala: val conf = new SparkConf().setAppName("Test Spark App").setMaster("local[4]")val sc = new SparkContext(conf) This creates a context running in the local mode with four threads, with the name of the application set to Test Spark App. If we wish to use default configuration values, we could also call the following simple constructor for our SparkContext object, which works in exactly the same way: val sc = new SparkContext("local[4]", "Test Spark App") The Spark shell Spark supports writing programs interactively using either the Scala or Python REPL (that is, the Read-Eval-Print-Loop, or interactive shell). The shell provides instant feedback as we enter code, as this code is immediately evaluated. In the Scala shell, the return result and type is also displayed after a piece of code is run. To use the Spark shell with Scala, simply run ./bin/spark-shell from the Spark base directory. This will launch the Scala shell and initialize SparkContext, which is available to us as the Scala value, sc. Your console output should look similar to the following screenshot: To use the Python shell with Spark, simply run the ./bin/pyspark command. Like the Scala shell, the Python SparkContext object should be available as the Python variable sc. You should see an output similar to the one shown in this screenshot: Resilient Distributed Datasets The core of Spark is a concept called the Resilient Distributed Dataset (RDD). An RDD is a collection of "records" (strictly speaking, objects of some type) that is distributed or partitioned across many nodes in a cluster (for the purposes of the Spark local mode, the single multithreaded process can be thought of in the same way). An RDD in Spark is fault-tolerant; this means that if a given node or task fails (for some reason other than erroneous user code, such as hardware failure, loss of communication, and so on), the RDD can be reconstructed automatically on the remaining nodes and the job will still complete. Creating RDDs RDDs can be created from existing collections, for example, in the Scala Spark shell that you launched earlier: val collection = List("a", "b", "c", "d", "e")val rddFromCollection = sc.parallelize(collection) RDDs can also be created from Hadoop-based input sources, including the local filesystem, HDFS, and Amazon S3. A Hadoop-based RDD can utilize any input format that implements the Hadoop InputFormat interface, including text files, other standard Hadoop formats, HBase, Cassandra, and many more. The following code is an example of creating an RDD from a text file located on the local filesystem: val rddFromTextFile = sc.textFile("LICENSE") The preceding textFile method returns an RDD where each record is a String object that represents one line of the text file. Spark operations Once we have created an RDD, we have a distributed collection of records that we can manipulate. In Spark's programming model, operations are split into transformations and actions. Generally speaking, a transformation operation applies some function to all the records in the dataset, changing the records in some way. An action typically runs some computation or aggregation operation and returns the result to the driver program where SparkContext is running. Spark operations are functional in style. For programmers familiar with functional programming in Scala or Python, these operations should seem natural. For those without experience in functional programming, don't worry; the Spark API is relatively easy to learn. One of the most common transformations that you will use in Spark programs is the map operator. This applies a function to each record of an RDD, thus mapping the input to some new output. For example, the following code fragment takes the RDD we created from a local text file and applies the size function to each record in the RDD. Remember that we created an RDD of Strings. Using map, we can transform each string to an integer, thus returning an RDD of Ints: val intsFromStringsRDD = rddFromTextFile.map(line => line.size) You should see output similar to the following line in your shell; this indicates the type of the RDD: intsFromStringsRDD: org.apache.spark.rdd.RDD[Int] = MappedRDD[5] at map at <console>:14 In the preceding code, we saw the => syntax used. This is the Scala syntax for an anonymous function, which is a function that is not a named method (that is, one defined using the def keyword in Scala or Python, for example). The line => line.size syntax means that we are applying a function where the input variable is to the left of the => operator, and the output is the result of the code to the right of the => operator. In this case, the input is line, and the output is the result of calling line.size. In Scala, this function that maps a string to an integer is expressed as String => Int.This syntax saves us from having to separately define functions every time we use methods such as map; this is useful when the function is simple and will only be used once, as in this example. Now, we can apply a common action operation, count, to return the number of records in our RDD: intsFromStringsRDD.count The result should look something like the following console output: 14/01/29 23:28:28 INFO SparkContext: Starting job: count at <console>:17...14/01/29 23:28:28 INFO SparkContext: Job finished: count at <console>:17, took 0.019227 sres4: Long = 398 Perhaps we want to find the average length of each line in this text file. We can first use the sum function to add up all the lengths of all the records and then divide the sum by the number of records: val sumOfRecords = intsFromStringsRDD.sumval numRecords = intsFromStringsRDD.countval aveLengthOfRecord = sumOfRecords / numRecords The result will be as follows: aveLengthOfRecord: Double = 52.06030150753769 Spark operations, in most cases, return a new RDD, with the exception of most actions, which return the result of a computation (such as Long for count and Double for sum in the preceding example). This means that we can naturally chain together operations to make our program flow more concise and expressive. For example, the same result as the one in the preceding line of code can be achieved using the following code: val aveLengthOfRecordChained = rddFromTextFile.map(line => line.size).sum / rddFromTextFile.count An important point to note is that Spark transformations are lazy. That is, invoking a transformation on an RDD does not immediately trigger a computation. Instead, transformations are chained together and are effectively only computed when an action is called. This allows Spark to be more efficient by only returning results to the driver when necessary so that the majority of operations are performed in parallel on the cluster. This means that if your Spark program never uses an action operation, it will never trigger an actual computation, and you will not get any results. For example, the following code will simply return a new RDD that represents the chain of transformations: val transformedRDD = rddFromTextFile.map(line => line.size).filter(size => size > 10).map(size => size * 2) This returns the following result in the console: transformedRDD: org.apache.spark.rdd.RDD[Int] = MappedRDD[8] at map at <console>:14 Notice that no actual computation happens and no result is returned. If we now call an action, such as sum, on the resulting RDD, the computation will be triggered: val computation = transformedRDD.sum You will now see that a Spark job is run, and it results in the following console output: ...14/11/27 21:48:21 INFO SparkContext: Job finished: sum at <console>:16, took 0.193513 scomputation: Double = 60468.0 The complete list of transformations and actions possible on RDDs as well as a set of more detailed examples are available in the Spark programming guide (located at http://spark.apache.org/docs/latest/programming-guide.html#rdd-operations), and the API documentation (the Scala API documentation) is located at http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.rdd.RDD). Caching RDDs One of the most powerful features of Spark is the ability to cache data in memory across a cluster. This is achieved through use of the cache method on an RDD: rddFromTextFile.cache Calling cache on an RDD tells Spark that the RDD should be kept in memory. The first time an action is called on the RDD that initiates a computation, the data is read from its source and put into memory. Hence, the first time such an operation is called, the time it takes to run the task is partly dependent on the time it takes to read the data from the input source. However, when the data is accessed the next time (for example, in subsequent queries in analytics or iterations in a machine learning model), the data can be read directly from memory, thus avoiding expensive I/O operations and speeding up the computation, in many cases, by a significant factor. If we now call the count or sum function on our cached RDD, we will see that the RDD is loaded into memory: val aveLengthOfRecordChained = rddFromTextFile.map(line => line.size).sum / rddFromTextFile.count Indeed, in the following output, we see that the dataset was cached in memory on the first call, taking up approximately 62 KB and leaving us with around 270 MB of memory free: ...14/01/30 06:59:27 INFO MemoryStore: ensureFreeSpace(63454) called with curMem=32960, maxMem=31138775014/01/30 06:59:27 INFO MemoryStore: Block rdd_2_0 stored as values to memory (estimated size 62.0 KB, free 296.9 MB)14/01/30 06:59:27 INFO BlockManagerMasterActor$BlockManagerInfo:Added rdd_2_0 in memory on 10.0.0.3:55089 (size: 62.0 KB, free: 296.9 MB)... Now, we will call the same function again: val aveLengthOfRecordChainedFromCached = rddFromTextFile.map(line => line.size).sum / rddFromTextFile.count We will see from the console output that the cached data is read directly from memory: ...14/01/30 06:59:34 INFO BlockManager: Found block rdd_2_0 locally... Spark also allows more fine-grained control over caching behavior. You can use the persist method to specify what approach Spark uses to cache data. More information on RDD caching can be found here: http://spark.apache.org/docs/latest/programming-guide.html#rdd-persistence. Broadcast variables and accumulators Another core feature of Spark is the ability to create two special types of variables: broadcast variables and accumulators. A broadcast variable is a read-only variable that is made available from the driver program that runs the SparkContext object to the nodes that will execute the computation. This is very useful in applications that need to make the same data available to the worker nodes in an efficient manner, such as machine learning algorithms. Spark makes creating broadcast variables as simple as calling a method on SparkContext as follows: val broadcastAList = sc.broadcast(List("a", "b", "c", "d", "e")) The console output shows that the broadcast variable was stored in memory, taking up approximately 488 bytes, and it also shows that we still have 270 MB available to us: 14/01/30 07:13:32 INFO MemoryStore: ensureFreeSpace(488) called with curMem=96414, maxMem=31138775014/01/30 07:13:32 INFO MemoryStore: Block broadcast_1 stored as values to memory(estimated size 488.0 B, free 296.9 MB)broadCastAList: org.apache.spark.broadcast.Broadcast[List[String]] = Broadcast(1) A broadcast variable can be accessed from nodes other than the driver program that created it (that is, the worker nodes) by calling value on the variable: sc.parallelize(List("1", "2", "3")).map(x => broadcastAList.value ++ x).collect This code creates a new RDD with three records from a collection (in this case, a Scala List) of ("1", "2", "3"). In the map function, it returns a new collection with the relevant record from our new RDD appended to the broadcastAList that is our broadcast variable. Notice that we used the collect method in the preceding code. This is a Spark action that returns the entire RDD to the driver as a Scala (or Python or Java) collection. We will often use collect when we wish to apply further processing to our results locally within the driver program. Note that collect should generally only be used in cases where we really want to return the full result set to the driver and perform further processing. If we try to call collect on a very large dataset, we might run out of memory on the driver and crash our program.It is preferable to perform as much heavy-duty processing on our Spark cluster as possible, preventing the driver from becoming a bottleneck. In many cases, however, collecting results to the driver is necessary, such as during iterations in many machine learning models. On inspecting the result, we will see that for each of the three records in our new RDD, we now have a record that is our original broadcasted List, with the new element appended to it (that is, there is now either "1", "2", or "3" at the end): ...14/01/31 10:15:39 INFO SparkContext: Job finished: collect at <console>:15, took 0.025806 sres6: Array[List[Any]] = Array(List(a, b, c, d, e, 1), List(a, b, c, d, e, 2), List(a, b, c, d, e, 3)) An accumulator is also a variable that is broadcasted to the worker nodes. The key difference between a broadcast variable and an accumulator is that while the broadcast variable is read-only, the accumulator can be added to. There are limitations to this, that is, in particular, the addition must be an associative operation so that the global accumulated value can be correctly computed in parallel and returned to the driver program. Each worker node can only access and add to its own local accumulator value, and only the driver program can access the global value. Accumulators are also accessed within the Spark code using the value method. For more details on broadcast variables and accumulators, see the Shared Variables section of the Spark Programming Guide: http://spark.apache.org/docs/latest/programming-guide.html#shared-variables. Summary In this article, we learned the basics of Spark's programming model and API using the interactive Scala console. Resources for Article: Further resources on this subject: Ridge Regression [article] Clustering with K-Means [article] Machine Learning Examples Applicable to Businesses [article]

0
0
4922

article-image-aggregators-file-exchange-over-ftpftps-social-integration-and-enterprise-messaging

Packt

20 Feb 2015

26 min read

Aggregators, File exchange Over FTP/FTPS, Social Integration, and Enterprise Messaging

Packt

20 Feb 2015

26 min read

0
0
5607

How-To Tutorials

article-image-android-and-udoo-home-automation

Packt

20 Feb 2015

6 min read

Android and UDOO for Home Automation

Packt

20 Feb 2015

6 min read

This article written by Emanuele Palazzetti, the author of Getting Started with UDOO, will teach us about home automation and using sensors to monitor the CO2 emission by the wood heater in our home. (For more resources related to this topic, see here.) During the last few years, the maker culture greatly improved the way hobbyists, students, and, more in general, technology enthusiasts create new hardware devices. The advent of prototyping boards such as Arduino together with a widespread open source philosophy, changed how and where ideas are realized. If you're a maker or you have a friend that really like building prototypes and devices, probably both of you have already transformed the garage or the personal studio in a home lab. This process is so spontaneous that nowadays newcomers in the Makers Community begin to create their first device at home. UDOO and Home Automation Like other communities, the makers family self-sustained their ideas joining their passion with the Do It Yourself (DIY) philosophy, which led makers to build and use creative devices in their everyday life. The DIY movement was the key factor that triggered the Home Automation made with open source platforms, making this process funnier and maker-friendly. Indeed, if we take a look at some projects released on the Internet we can find thousands of prototypes, usually composed by an Arduino together with a Raspberry Pi, or any other combinations of similar prototyping boards. However, in this scenario, we have a new alternative proposed by the UDOO board—a stand alone computer that provides a multidevelopment-platform solution for Linux and Android operating systems with an integrated Arduino Due. Thanks to its flexibility, UDOO combines the best from two different worlds: the hardware makers and the software programmers. Applying the UDOO board to home automation is a natural process because of its power and versatility. Furthermore, the capability to install the Android operating system increases the capabilities of this board, because it offers out-of-the-box mobile applications ecosystem that can be easily reused in our appliances, enhancing the user experience of our prototype. Since 2011, Android supports the interaction with an Arduino compatible device through the Android Accessory Development Kit (ADK) reference. Designed to implement compelling accessories that could be connected to Android-powered devices, this reference defines the Android Open Accessory (AOA) protocol used to communicate with an Arduino device through an USB On-The-Go (OTG) port or via Bluetooth. UDOO makes use of this protocol and opens a serial communication between two processors soldered in the same board, which run the Android operating system and the microcontroller actions respectively. Mastering the ADK is not so easy, especially during the first approach. For this reason, the community has developed an Android library that simplifies a lot the use of the ADK: the Android ADKToolkit (http://adktoolkit.org). To put our hands-on code, we can imagine a scenario in which we want to monitor the carbon dioxide emissions (CO2) produced by the wood heater in our home. The first step is to connect the CO2 sensor to our board and use the manufacturer library, if available, in the Arduino sketch to retrieve the detected CO2. Using the ADK implementation, we can send this value back to the main UDOO's processor that runs the Android operating system, so it can use its powerful APIs, and a more powerful processor, to visualize or compute the collected data. The following example is a part of an Arduino sketch used to send sensor data back to the Android application using the ADK implementation: uint8_t buffer[128]; int co2Sensor = 0; void loop() { Usb.Task(); if (adk.isReady()) { // Hypothetical function that // gets the value from the sensor co2Sensor = readFromCO2Sensor(); buffer[0] = co2Sensor; // Sends the value to Android adk.write(1, buffer); delay(1000); } } On the Android side, using a traditional AsyncTask method or a more powerful ExecutorService method, we should read this value from the buffer. This task is greatly simplified through the ADKToolkit library that requires the initialization of the AdkManager class that holds the connection and exposes a handler to open or close the communication with Arduino. This instance could be initialized during the onCreate() activity callback with the following code: private AdkManager mAdkManager;@Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.hello_world); mAdkManager = new AdkManager(this); mAdkManager.open(); } In a separated thread, we can use the mAdkManager instance to read data from the ADK buffer. The following is an example that reads the message from the buffer and uses the getInt() method to parse the value: private class SensorThread implements Runnable { @Override public void run() { // Reads detected CO2 from ADK AdkMessage response = mAdkManager.read(); int co2 = response.getInt(); // Continues the execution doSomething(co2); } } Once we retrieve the detected CO2 value from the sensor, we can use it in our application. For instance, we may plot this value using an Android open source chart library or send collected data to an external web service. These preceding snippets are just examples to show how it's easy to implement the communication in UDOO between the Android operating system and the onboard Arduino. With a powerful operating system such as Android and one of the most widespread prototyping platform provided in a single board, UDOO can play a key role in our home automation projects. Summary As makers, if we manage to make enough experience into the home automation field, chances are that we will be able to develop and build a high-end system for our own house, flexible enough to be easily extended without any further knowledge. When I wrote Getting started with UDOO, the idea was to make a comprehensive guide and a collection of examples, to help developers grabbing quickly the key concepts of this prototyping board, focusing in the Android development to bring back to light the advantages provided by this widespread platform, if used in our prototypes and not only in our mobile devices. Resources for Article: Further resources on this subject: Android Virtual Device Manager [Article] Writing Tag Content [Article] The Arduino Mobile Robot [Article]

0
0
20627

How-To Tutorials

Packt

19 Feb 2015

17 min read

Visualize This!

Packt

19 Feb 2015

17 min read

This article is written by Michael Phillips, the author of the book TIBCO Spotfire: A Comprehensive Primer, discusses that human beings are fundamentally visual in the way they process information. The invention of writing was as much about visually representing our thoughts to others as it was about record keeping and accountancy. In the modern world, we are bombarded with formalized visual representations of information, from the ubiquitous opinion poll pie chart to clever and sophisticated infographics. The website http://data-art.net/resources/history_of_vis.php provides an informative and entertaining quick history of data visualization. If you want a truly breathtaking demonstration of the power of data visualization, seek out Hans Rosling's The best stats you've ever seen at http://ted.com. (For more resources related to this topic, see here.) We will spend time getting to know some of Spotfire's data capabilities. It's important that you continue to think about data; how it's structured, how it's related, and where it comes from. Building good visualizations requires visual imagination, but it also requires data literacy. This article is all about getting you to think about the visualization of information and empowering you to use Spotfire to do so. Apart from learning the basic features and properties of the various Spotfire visualization types, there is much more to learn about the seamless interactivity that Spotfire allows you to build in to your analyses. We will be taking a close look at 7 of the 16 visualization types provided by Spotfire, but these 7 visualization types are the most commonly used. We will cover the following topics: Displaying information quickly in tabular form Enriching your visualizations with color categorization Visualizing categorical information using bar charts Dividing a visualization across a trellis grid Key Spotfire concept—marking Visualizing trends using line charts Visualizing proportions using pie charts Visualizing relationships using scatter plots Visualizing hierarchical relationships using treemaps Key Spotfire concept—filters Enhancing tabular presentations using graphical tables Now let's have some fun! Displaying information quickly in tabular form While working through the data examples, we used the Spotfire Table visualization, but now we're going to take a closer look. People will nearly always want to see the "underlying data", the details behind any visualization you create. The Table visualization meets this need. It's very important not to confuse a table in the general data sense with the Spotfire Table visualization; the underlying data table remains immutable and complete in the background. The Table visualization is a highly manipulatable view of the underlying data table and should be treated as a visualization, not a data table. The data used here is BaseballPlayerData.xls There is always more than one way to do the same thing in Spotfire, and this is particularly true for the manipulation of visualizations. Let's start with some very quick manipulations: First, insert a table visualization by going to the Insert menu, selecting New Visualization, and then Table. To move a column, left-click on the column name, hold, and drag it. To sort by a column, left-click on the column name. To sort by more than one column, left-click on the first column name and then press Shift + left-click on the subsequent columns in order of sort precedence. To widen or narrow a column, hover the mouse over the right-hand edge of the column title until you see the cursor change to a two-way arrow, and then click and drag it. These and other properties of the Table visualization are also accessed via visualization properties. As you work through the various Spotfire visualizations, you'll notice that some types have more options than others, but there are common trends and an overall consistency in conventions. Visualization properties can be opened in a number of ways: By right-clicking on the visualization, a table in this case, and selecting Properties. By going to the Edit menu and selecting Visualization Properties. By clicking on the Visualization Properties icon, as shown in the following screenshot, in the icon tray below the main menu bar. It's beyond the scope of this book to explore every property and option. The context-sensitive help provided by Spotfire is excellent and explains all the options in glorious detail. I'd like to highlight four important properties of the Table visualization: The General property allows you to change the table visualization title, not the name of the underlying data table. It also allows you to hide the title altogether. The Data property allows you to switch the underlying data table, if you have more than one table loaded into your analysis. The Columns property allows you to hide columns and order the columns you do want to show. The Show/Hide Items property allows you to limit what is shown by a rule you define, such as top five hitters. After clicking on the Add button, you select the relevant column from a dropdown list, choose Rule type (Top), and finally, choose Value for the rule (5). The resulting visualization will only show the rows of data that meet the rule you defined. Enriching your visualizations with color categorization Color is a strong feature in Spotfire and an important visualization tool, often underestimated by report creators. It can be seen as merely a nice-to-have customization, but paying attention to color can be the difference between creating a stimulating and intuitive data visualization rather than an uninspiring and even confusing corporate report. Take some pride and care in the visual aesthetics of your analytics creations! Let's take a look at the color properties of the Table visualization. Open the Table visualization properties again, select Colors, and then Add the column Runs. Now, you can play with a color gradient, adding points by clicking on the Add Point button and customizing the colors. It's as easy as left-clicking on any color box and then selecting from a prebuilt palette or going into a full RGB selection dialog by choosing More Colors…. The result is a heatmap type effect for runs scored, with yellow representing low run totals, transitioning to green as the run total approaches the average value in the data, and becoming blue for the highest run totals. Visualizing categorical information using bar charts We saw how the Table visualization is perfect for showing and ordering detailed information. It's quite similar to a spreadsheet. The Bar Chart visualization is very good for visualizing categorical information, that is, where you have categories with supporting hard numbers—sales by region, for example. The region is the category, whereas the sales is the hard number or fact. Bar charts are typically used to show a distribution. Depending on your data or your analytic requirement, the bars can be ordered by value, placed side by side, stacked on top of each other, or arranged vertically or horizontally. There is a special case of the category and value combination and that is where you want to plot the frequencies of a set of numerical values. This type of bar chart is referred to as a histogram, and although it is number against number, it is still, in essence, a distribution plot. It is very common in fact to transform the continuous number range in such cases into a set of discrete bins or categories for the plot. For example, you could take some demographic data and plot age as the category and the number of people at that age as the value (the frequency) on a bar chart. The result, for a general population, would approach a bell-shaped curve. Let's create a bar chart using the baseball data. The data we will use is BaseballPlayerData.xls, which you can download from http://www.insidespotfire.com. Create a new page by right-clicking on any page tab and selecting New Page. You can also select New Page from the Insert menu or click on the new page icon in the icon bar below the main menu. Create a Bar Chart visualization by left-clicking on the bar chart icon or by selecting New Visualization and then Bar Chart from the Insert menu. Spotfire will automatically create a default chart, that is, rarely exactly what you want, so the next step is to configure the chart. Two distributions might be interesting to look at: the distribution of home runs across all the teams and the distribution of player salaries across all the teams. The axes are easy to change; simply use the axes selectors. If the bars are vertical, it means that the category—Team, in our case—should be on the horizontal axis, with the value—Home Runs or Salary—on the vertical axis, representing the height of the bars. We're going to pick Home Runs from the vertical axis selector and then an appropriate aggregation dropdown, which is highlighted in red in the screenshot. Sum would be a valid option, but let's go with Avg (Average). Similarly, select Team from the horizontal axis dropdown selector. The vertical, or value, axis must be an aggregation because there is more than one home run value for each category. You must decide if you want a sum, an average, a minimum, and so on. You can modify the visualization properties just as you did for the Table visualization. Some of the options are the same; some are specific to the bar chart. We're going to select the Sort bars by value option in the Appearance property. This will order the bars in descending order of value. We're also going to check the option Vertically under Scale labels | Show labels for the Category Axis property. There are two more actions to perform: create an identical bar chart except with average salary as the value axis, and give each bar chart an appropriate title (Visualization Properties|General|Title:). To copy an existing visualization, simply right-click on it and select Duplicate Visualization. We can now compare the distribution of home run average and salary average across all the baseball teams, but there's a better way to do this in a single visualization using color. Close the salary distribution bar chart by left-clicking on X in the upper right-hand corner of the visualization (X appears when you hover the mouse) or right-clicking on the visualization and selecting Close. Now, open the home run bar chart visualization properties, go to the Colors property, and color by Avg(Salary). Select a Gradient color mode, and add a median point by clicking on the Add Point button and selecting Median from the dropdown list of options on the added point. Finally, choose a suitable heat map range of colors; something like blue (min) through pale yellow (median) through red (max). You will still see the distribution of home runs across the baseball teams, but now you will have a superimposed salary heat map. Texas and Cleveland appear to be getting much more bang for their buck than the NY Yankees. Dividing a visualization across a trellis grid Trellising, whereby you divide a series of visualizations into individual panels, is a useful technique when you want to subdivide your analysis. In the example we've been working with, we might, for instance, want to split the visualization by league. Open the visualization properties for the home runs distribution bar chart colored by salary and select the Trellis property. Go to Panels and split by League (use the dropdown column selector). Spotfire allows you to build layers of information with even basic visualizations such as the bar chart. In one chart, we see the home run distribution by team, salary distribution by team, and breakdown by league. Key Spotfire concept – marking It's time to introduce one of the most important Spotfire concepts, called marking, which is central to the interactivity that makes Spotfire such a powerful analysis tool. Marking refers to the action of selecting data in a visualization. Every element you see is selectable, or markable, that is, a single row or multiple rows in a table, a single bar or multiple bars in a bar chart. You need to understand two aspects to marking. First, there is the visual effect, or color(s) you see, when you mark (select) visualization elements. Second, there is the behavior that follows marking: what happens to data and the display of data when you mark something. How to change the marking color From Spotfire v5.5 onward, you can choose, on a visualization-by-visualization basis, two distinct visual effects for marking: Use a separate color for marked items: all marked items are uniformly colored with the marking color, and all unmarked items retain their existing color. Keep existing color attributes and fade out unmarked items: all marked items keep their existing color, and all unmarked items also keep their existing color but with a high degree of color fade applied, leaving the marked items strongly highlighted. The second option is not available in versions older than v5.5 but is the default option in Versions 5.5 onward. The setting is made in the visualization's Appearance property by checking or unchecking the option Use separate color for marked items. The default color when using a separate color for marked items is dark green, but this can be changed by going to Edit|Document Properties|Markings|Edit. The new option has the advantage of retaining any underlying coloring you defined, but you might not like how the rest of the chart is washed out. Which approach you choose depends on what information you think is critical for your particular situation. When you create a new analysis, a default marking is created and applied to every visualization you create by default. You can change the color of the marking in Document Properties, which is found in the Edit menu. Just open Document Properties, click on the Markings tab, select the marking, click on the Edit button, and change the color. You can also create as many markings as you need, giving them convenient names for reference purposes, but we'll just focus on using one for now. How to set the marking behavior of a visualization Marking behavior depends fundamentally on data relationships. The data within a single data table is intrinsically related; the data in separate data tables must be explicitly related before you configure marking behavior for visualizations based on separate datasets. When you mark something in a visualization, five things can happen depending on the data involved and how you configured your visualizations: Conditions Behavior Two visualizations with the same underlying data table (they can be on different pages in the analysis file) and the same marking scheme applied. Marking data on one visualization will automatically mark the same data on the other. Two visualizations with related underlying data tables and the same marking scheme applied. The same as the previous condition's behavior, but subject to differences in data granularity. For example, marking a baseball team in one visualization will mark all the team's players in another visualization that is based on a more detailed table related by team. Two visualizations with the same or related data tables where one has been configured with data dependency on the marking in the other. Nothing will display in the marking-dependent visualization other than what is marked in the reference visualization. Visualizations with unrelated underlying data tables. No marking interaction will occur, and the visualizations will mark completely independently of one another. Two visualizations with the same underlying data table or related data tables and with different marking schemes applied. Marking data on one visualization will not show on the other because the marking schemes are different. Here's how we set these behaviors: Open the visualization properties of the bar chart we have been working with and navigate to the Data property. You'll notice that two settings refer to marking: Marking and Limit data using markings. Use the dropdown under Marking to select the marking to be used for the visualization. Having no marking is an option. Visualizations with the same marking will display synchronous selection, subject to the data relation conditions described earlier. The options under Limit data using markings determine how the visualization will be limited to marking elsewhere in the analysis. The default here is no dependency. If you select a marking, then the visualization will only display data selected elsewhere with that marking. It's not good to have the same marking for Marking and Limit data using markings. If you are using the limit data setting, select no marking, or create a second marking and select it under Marking. You're possibly a bit confused by now. Fortunately, marking is much harder to describe than to use! Let's build a tangible example. We'll start a new analysis, so close any analysis you have open and create a new one, loading the player-level baseball data (BaseballPlayerData.xls). Add two bar charts and a table. You can rearrange the layout by left-clicking on the title bar of a visualization, holding, and dragging it. Position the visualizations any way you wish, but you can place the two bar charts side by side, with the table below them spanning both. Save your analysis file at this point and at regular intervals. It's good behavior to save regularly as you build an analysis. It will save you a lot of grief if your PC fails in any way. There is no autosave function in Spotfire. For the first bar chart, set the following visualization properties: Property Value General | Title Home Runs Data | Marking Marking Data | Limit data using markings Nothing checked Appearance | Orientation Vertical bars Appearance | Sort bars by value Check Category Axis | Columns Team Value Axis | Columns Avg(Home Runs) Colors | Columns Avg(Salary) Colors | Color mode Gradient Add Point for median Max = strong red; Median = pale yellow; Min = strong blue Labels | Show labels for Marked Rows Labels | Types of labels | Complete bar Check For the second bar chart, set the following visualization properties: Property Value General | Title Roster Data | Marking Marking Data | Limit data using markings Nothing checked Appearance | Orientation Horizontal bars Appearance | Sort bars by value Check Category Axis | Columns Team Value Axis | Columns Count(Player Name) Colors | Columns Position Colors | Color mode Categorical For the table, set the following visualization properties: Property Value General | Title Details Data | Marking (None) Data | Limit data using markings Check Marking Columns Team, Player Name, Games Played, Home Runs, Salary, Position Now start selecting visualization elements with your mouse. You can click on elements such as bars or segments of bars, or you can click and drag a rectangular block around multiple elements. When you select a bar on the Home Runs bar chart, the corresponding team bar automatically selects the Roster bar chart, and details for all the players in that team display in the Details table. When you select a bar segment on the Roster bar chart, the corresponding team bar automatically selects on the Home Runs bar chart and only players in the selected position for the team selected appear in the details. There are some very useful additional functions associated with marking, and you can access these by right-clicking on a marked item. They are Unmark, Invert, Delete, Filter To, and Filer Out. You can also unmark by left-clicking on any blank space in the visualization. Play with this analysis file until you are comfortable with the marking concept and functionality. Summary This article is a small taste of the book TIBCO Spotfire: A comprehensive primer. You've seen how the Table visualization is an easy and traditional way to display detailed information in tabular form and how the Bar Chart visualization is excellent for visualizing categorical information, such as distributions. You've learned how to enrich visualizations with color categorization and how to divide a visualization across a trellis grid. You've also been introduced to the key Spotfire concept of marking. Apart from gaining a functional understanding of these Spotfire concepts and techniques, you should have gained some insight into the science and art of data visualization. Resources for Article: Further resources on this subject: The Spotfire Architecture Overview [article] Interacting with Data for Dashboards [article] Setting Up and Managing E-mails and Batch Processing [article]

0
0
6769

Christmas Light Sequencer

The Raspberry Pi and Raspbian

Static Data Management

Agile data modeling with Neo4j

Introducing Web Application Development in Rails

URL Routing and Template Rendering

Actors and Pawns

Selenium Testing Tools

Customizing App Controller

Sprites in Action

Trending Topics

Puppet Language and Style

The Spark Programming Model

Aggregators, File exchange Over FTP/FTPS, Social Integration, and Enterprise Messaging

Android and UDOO for Home Automation

Visualize This!

Create a Free Account To Continue Reading

Sign in to activate your 7-day free access