Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Save more on your purchases! discount-offer-chevron-icon
Savings automatically calculated. No voucher code required.
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Events
Videos
Audiobooks
Packt Hub
Free Learning
Arrow right icon
timer SALE ENDS IN
0 Days
:
00 Hours
:
00 Minutes
:
00 Seconds

How-To Tutorials

7019 Articles
article-image-spatial-data-services
Packt
19 Nov 2013
6 min read
Save for later

Spatial Data Services

Packt
19 Nov 2013
6 min read
(For more resources related to this topic, see here.) So far we have worked with relatively small sets of data; for larger collections, Bing Maps provide Spatial Data Services. They offer the Data Source Management API to load large datasets into Bing Maps servers, and the Query API to query the data. In this article, we will use the Geocode Dataflow API, which provides geocoding and reverse geocoding of large datasets. Geocoding is the process of finding geographic coordinates from other geographic data, such as, street addresses or postal codes. Reverse geocoding is the opposite process, where the coordinates are used to find their associated textual locations, such as, addresses and postal codes. Bing Maps implement these processes by creating jobs on Bing Maps servers, and querying them later. All the process can be automated, which is ideal for huge amounts of data. Please note that strict rules of data usage apply to the Spatial Data Services (please refer http://msdn.microsoft.com/en-us/library/gg585136.aspx for full details). At the moment of writing, a user with a basic account can set up to 5 jobs in a 24-hour period. Our task in this article is to geocode the addresses of ten of the biggest technology companies in the world, such as Microsoft, Google, Apple, Facebook, and so on, and then display them on the map. The first step is to prepare the file with the companies' addresses. Geocoding dataflow input data The input and output data can be supplied in the following formats: XML (content type application/xml) Comma separated values (text/plain) Tab-delimited values (text/plain) Pipe-delimited values (text/plain) Binary (application/octet-stream) used with Blob Service REST API We will use the XML format, for its clearer declarative structure. Now, let's open Visual Studio and create a new C# Console project named LBM.Geocoder. We then add a Datafolder, which will contain all the data files and samples with which we'll work in this article, starting with the data.xml file we need to upload to the Spatial Data servers to be geocoded. <?xml version="1.0" encoding="utf-8"?><GeocodeFeedVersion="2.0"><GeocodeEntity Id="001"><GeocodeRequest Culture="en-US" IncludeNeighborhood="1"><Address AddressLine="1 Infinite Loop"AdminDistrict="CA" Locality="Cupertino" PostalCode="95014" /></GeocodeRequest></GeocodeEntity><GeocodeEntity Id="002"><GeocodeRequest Culture="en-US" IncludeNeighborhood="1"><Address AddressLine="185 Berry St"AdminDistrict="NY" Locality="New York" PostalCode="10038"/></GeocodeRequest></GeocodeEntity> The listing above is a fragment of that file with the addresses of the companies' headquarters. Please note that the more addressing information we provide to the API, the better quality geocoding we receive. In production, this file would be created programmatically, probably based on an Addresses database. The ID of GeocodeEntity, could also be stored, so that the data is matched easier once fetched from the servers. (You can find the Geocode Dataflow Data Schema, Version 2, at http://msdn.microsoft.com/en-us/library/jj735477.aspx.) The job Let's add a Jobclass to our project: public class Job{private readonly string dataFilePath;public Job(string dataFilePath){this.dataFilePath = dataFilePath;}} The dataFilePath argument is the path to the data.xml file we created earlier. Creating the job is as easy as calling a REST URL: public void Create(){var uri =String.Format("{0}?input=xml&output=xml&key={1}",Settings.DataflowUri, Settings.Key);var data = File.ReadAllBytes(dataFilePath);try{var wc = new WebClient();wc.Headers.Add("Content-Type", "application/xml");var receivedBytes = wc.UploadData(uri, "POST",data);ParseJobResponse(receivedBytes);}catch (WebException e){var response = (HttpWebResponse)e.Response;var status = response.StatusCode;}} We place all the API URLs and other settings in the Settings class: public class Settings{public static string Key = "[YOUR BING MAPS KEY];public static string DataflowUri ="https://spatial.virtualearth.net/REST/v1/dataflows/geocode";public static XNamespace XNamespace ="http://schemas.microsoft.com/search/local/ws/rest/v1";public static XNamespace GeocodeFeedXNamespace ="http://schemas.microsoft.com/search/local/2010/5/geocode";} To create the job, we need to build the Dataflow URL template with a Bing Maps Key, and parameters such as input and output formats. We specify the latter to be XML. Next, we use a WebClient instance to load the data with a POST protocol. Then, we parse the server response: private void ParseJobResponse(byte[] response){using (var stream = new MemoryStream(response)){var xDoc = XDocument.Load(stream);var job = xDoc.Descendants(Settings.XNamespace +"DataflowJob").FirstOrDefault();var linkEl = job.Element(Settings.XNamespace +"Link");if (linkEl != null) Link = linkEl.Value;}} Here, we pass the stream created with the bytes received from the server to the XDocument.load method. This produces an XDocument instance, which we will use to extract the data we need. We will apply a similar process throughout the article to parse XML content. Note that the appropriate XNamespace needs to be supplied in order to navigate through the document nodes. You can find a sample of the response inside the Data folder (jobSetupResponse.xml), which shows that the link to the job created is found under a Link element within the DataflowJob node. Getting job status Once we have set up a job, we can store the link on a data store, such as a database, and check for its status later. The data will be available on the Microsoft servers up to 14 days after creation. Let's see how we can query the job status: public static JobStatus CheckStatus(string jobUrl){var result = new JobStatus();var uri = String.Format("{0}?output=xml&key={1}", jobUrl,Settings.Key);var xDoc = XDocument.Load(uri);var job = xDoc.Descendants(Settings.XNamespace +"DataflowJob").FirstOrDefault();if (job != null){var linkEls = job.Elements(Settings.XNamespace +"Link").ToList();foreach (var linkEl in linkEls){var nameAttr = linkEl.Attribute("name");if (nameAttr != null){if (nameAttr.Value == "succeeded") result.SucceededLink = linkEl.Value;if (nameAttr.Value == "failed") result.FailedLink= linkEl.Value;}}var statusEl = job.Elements(Settings.XNamespace +"Status").FirstOrDefault();if (statusEl != null) result.Status = statusEl.Value;}return result;} Now, we know that to query a Data API we need to first build the URL template. We do this by attaching the Bing Maps Key and an output parameter to the job link. The response we get from the server, stores the job status within a Link element of a DataflowJob node (the jobResponse.xml file inside the Data folder contains an example). The link we need has a name attribute with the value succeeded. Summary When it comes to large amounts of data, the Spatial Data Services offer a number of interfaces to store, and query user data; geocode addresses or reverse geocode geographical coordinates. The services perform these tasks by means of background jobs, which can be set up and queried through REST URLs. Resources for Article: Further resources on this subject: NHibernate 2: Mapping relationships and Fluent Mapping [Article] Using Maps in your Windows Phone App [Article] What is OpenLayers? [Article]
Read more
  • 0
  • 0
  • 2391

article-image-setting-woocommerce
Packt
19 Nov 2013
6 min read
Save for later

Setting Up WooCommerce

Packt
19 Nov 2013
6 min read
(For more resources related to this topic, see here.) So, you're already familiar with WordPress and know how to use plugins, widgets, and themes? Your next step is to expand your existing WordPress website or blog with an online store? In that case you've come to the right place! WooCommerce is a versatile plugin for WordPress, that gives the possibility for everyone with a little WordPress knowledge to start their own online store. In case you are not familiar with WordPress at all, this book is not the first one you should read. No worries though, WordPress isn't that hard to learn and there are tons of online possibilities to learn about the WordPress solution very quickly. Or just turn to one of the many printed books on WordPress that are available. These are the topics we'll be covering in this article: Installing and activating WooCommerce Learn everything about setting up WooCommerce correctly Preparing for takeoff Before we start, remember that it's only possible to install your own plugins if you're working in your own WordPress installation. This means that users running a website on WordPress.com will not be able to follow along. It's simply impossible in that environment to install plugins yourself. Although installing WooCommerce on top of WordPress isn't difficult, we highly recommend that you set up a test environment first. Without going too much into depth, this is what you need to do: Create a backup copy of your complete WordPress environment using FTP. Alternatively use a plugin to store a copy into your Dropbox folder automatically. There are tons of solutions available, just pick your own favorite. UpDraftPlus is one of the possibilities and delivers a complete backup solution: http://wordpress.org/plugins/updraftplus/. Don't forget to backup your WordPress database as well. You may do this using a tool like phpMyAdmin and create an export from there. But also in this case, there are plugins that make life easier. The UpDraftPlus plugin mentioned previously can perform this task as well. Once your backups are complete, install XAMPP on a local (Windows) machine that can be downloaded from http://www.apachefriends.org. Although XAMPP is available for Mac users, MAMP is a widely used alternative for this group. MAMP can be downloaded from http://www.mamp.info/en/index.html. Restore your WordPress backup on your test server and start following the remaining part of this book in your new test environment. Alternatively, install a copy of your WordPress website as a temporary subdomain at your hosting provider. For instance, if my website is http://www.example.com, I could easily create a copy of my site in http://test.example.com. Possibilities may vary, depending on the package you have with your hosting provider. If in your situation it isn't needed to add WooCommerce to an existing WordPress site, of course you may also start from scratch. Just install WordPress on a local test server or install it at your hosting provider. To keep our instructions in this book as clear as possible we did just that. We created a fresh installation of WordPress Version 3.6. Next, you see a screenshot of our fresh WordPress installation: Are these short instructions just too much for you at this moment? Do you need a more detailed step-by-step guide to create a test environment for your WordPress website? Look at the following tutorials: For Max OSX users: http://wpmu.org/local-wordpresstest-environment-mamp-osx/ For Windows users: http://www.thegeekscope.com/howto-copy-a-live-wordpress-website-to-local-indowsenvironment/ More tutorials will also be available on our website: http://www.joomblocks.com Don't forget to sign up for the free Newsletter, that will bring you even more news and tutorials on WordPress, WooCommerce, and other open source software solutions! Once ready, we'll be able to take the next step and install the WooCommerce plugin. Let's take a look at our WordPress backend. In our situation we can open this by browsing to http://localhost/wp36/wp-admin. Depending on the choices you made previously for your test environment, your URL could be different. Well, this should all be pretty familiar for you already. Again, your situation might look different, depending on your theme or the number of plugins already active for your website. Installing WooCommerce Installing a plugin is a fairly simple task: Click on Plugins in the menu on the left and click on Add New. Next, simply enter woocommerce in the Search field and click on Search Plugins. Verify if the correct plugin is shown on top and click on Install Now. Confirm the warning message that appears by clicking on OK. Click on Activate Plugin. Note that in the following screenshot, we're installing Version 2.0.13 of WooCommerce. New versions will follow rather quickly, so you might already see a higher version number. WooCommerce needs to have a number of specific WordPress pages, that it automatically will setup for you. Just click on the Install WooCommerce Pages button and make sure not to forget this step! In our example project, we're installing the English version of WooCommerce. But you might need a different language. By default, WooCommerce is already delivered in a number of languages. This means the installation will automatically follow the language of your WordPress installation. If you need something else, just browse through the plugin directory on WordPress.org to find any additional translations. Once we have created the necessary pages, the WooCommerce welcome screen will appear and you will see a new menu item has been added to the main menu on the left. Meanwhile the plugin created the necessary pages, that you can access by clicking on Pages in the menu on the left. Note that if you open a page that was automatically created by WooCommerce, you'll only see a shortcode, which is used to call the needed functionality. Do not delete the shortcodes, or WooCommerce might stop working. However, it's still possible to add your own content before or after the shortcode on these pages. WooCommerce also added some widgets to your WordPress dashboard, giving an overview of the latest product and sales statistics. At this moment this is all still empty of course. Summary In this article, we learned about the basics of WooCommerce and installing the same. We also learned that WooCommerce is a free but versatile plugin for WordPress, that you may use to easily set up your own online store. Resources for Article: Further resources on this subject: Django 1.2 E-commerce: Generating PDF Reports from Python using ReportLab [Article] Increasing sales with Brainshark slideshows/documents [Article] Implementing OpenCart Modules [Article]
Read more
  • 0
  • 0
  • 10147

article-image-configuring-ovirt
Packt
19 Nov 2013
9 min read
Save for later

Configuring oVirt

Packt
19 Nov 2013
9 min read
(For more resources related to this topic, see here.) Configuring the NFS storage NFS storage is a fairly common type of storage that is quite easy to set up and run even without special equipment. You can take the server with large disks and create NFS directory.But despite the apparent simplicity of NFS, setting s should be done with attention to details. Make sure that the NFS directory is suitable for use; go to the procedure of connecting storage to the data center. The following options are displayed after you click on the Configure Storage dialog box in which we specify the basic storage configuration: Name and Data Center: It is used to specify a name and target of the data center for storage Domain Function/Storage Type: It is used to choose a data function and NFS type Use Host: It is used to enter the host that will make the initial connection to the storage and a host who will be in the role of SPM Export Path: It is used to enter the storage server name and path of the exported directory Advanced Parameters: It provides additional connection options, such as NFS version, number of retransmissions and timeout, that are recommended to be changed only in exceptional cases Fill in the required storage settings and click on the OK button; this will start the process of connecting storage. The following image shows the New Storage dialog box with the connecting NFS storage: Configuring the iSCSI storage This section will explain how to connect the iSCSI storage to the data center with the type of storage as iSCSI. You can skip this section if you do not use iSCSI storage. iSCSI is a technology for building SAN (Storage Area Network). A key feature of this technology is the transmission of SCSI commands over the IP networks. Thus, there is a transfer of block data via IP. By using the IP networks, data transfer can take place over long distances and through network equipment such as routers and switches. These features make the iSCSI technology good for construction of low-cost SAN. oVirt supports iSCSI and iSCSI storages that can be connected to oVirt data centers. Then begin the process of connecting the storage to the data center. After you click on the Configure Storage dialog box in which you specify the basic storage configuration, the following options are displayed: Name and Data Center: It is used to specify the name and target of the data center. Domain Function/Storage Type: It is used to specify the domain function and storage type. In this case, the data function and iSCSI type. Use Host: It is used to specify the host to which the storage (SPM) will be attached. The following options are present in the search box for iSCSI targets: Address and Port: It is used to specify the address and port of the storage server that contains the iSCSI target User Authentication: Enable this checkbox if authentication is to be used on the iSCSI target CHAP username and password: It is used to specify the username and password for authentication Click on the Discover button and oVirt Engine connects to the specified server for the searching of iSCSI targets. In the resulting list, click on the designated targets, we click on the Login button to authenticate. Upon successful completion of the authentication, the display target LUN will be displayed; check it and click on OK to start connection to the data center. New storage will automatically connect to the data center. If it does not, select the location from the list and click on the Attach button in the detail pane where we choose a target data center. Configuring the Fibre Channel storage If you have selected Fibre Channel when creating the data center, we should create a Fibre Channel storage domain. oVirt supports Fibre Channel storage based on multiple preconfigured Logical Unit Numbers (LUN). Skip this section if you do not use Fibre Channel equipment. Begin the process of connecting the storage to the data center. Open the Guide Me wizard and click on the Configure Storage dialog box where you specify the basic storage configuration: Name and Data Center: It is used to specify the name and data center Domain Function/Storage Type: Here we need to specify the data function and Fibre Channel type Use Host: It specifies the address of the virtualization host that will act as the SPM In the area below, the list of LUNs are displayed, enable the Add LUN checkbox on the selected LUN to use it as Fibre Channel data storage. Click on the OK button and this will start the process of connecting storage to the data centers. In the Storage tab and in the list of storages, we can see created Fibre Channel storage. In the process of connecting, its status will change and at the end new storage will be activated and connected to the data center. The connection process can also be seen in the event pane. The following screenshot shows the New Storage dialog box with Fibre Channel storage type: Configuring the GlusterFS storage GlusterFS is a distributed, parallel, and linearly scalable filesystem. GlusterFS can combine the data storage that are located on different servers into a parallel network filesystem. GlusterFS's potential is very large, so developers directed their efforts towards the implementation and support of GlusterFS in oVirt (GlusterFS documentation is available at http://www.gluster.org/community/documentation/index.php/Main_Page). oVirt 3.3 has a complete data center with the GlusterFS type of storage. Configuring the GlusterFS volume Before attempting to connect GlusterFS storage into the data center, we need to create the volume. The procedure of creating GlusterFS volume is common in all versions. Select the Volumes tab in the resource pane and click on Create Volume. In the open window, fill the volume settings: Data Center: It is used to specify the data center that will be attached to the GlusterFS storage. Volume Cluster: It is used to specify the name of the cluster that will be created. Name: It is used to specify a name for the new volume. Type: It is used to specify the type of GlusterFS volume available to choose from, there are seven types of volume that implement various strategic placements of data on the filesystem. Base types are Distribute, Replicate, and Stripe and other combination of these types: Distributed Replicate, Distributed Stripe, Striped Replicate, and Distributed Striped Replicate (additional info can be found at the link: http://gluster.org/community/documentation/index.php/GlusterFS_Concepts). Bricks: With this button, a list of bricks will be collected from the filesystem. Brick is a separate piece with which volume will be built. These bricks are distributed across the hosts. As bricks use a separate directory, it should be placed on a separate partition. Access Protocols: It defines basic access protocols that can be used to gain access to the following: Gluster: It is a native protocol access to volumes GlusterFS, enabled by default. NFS: It is an access protocol based on NFS. CIFS: It is an access protocol based on CIFS. Allow Access From: It allows us to enter a comma-separated IP address, hostnames, or * for all hosts that are allowed to access GlusterFS volume. Optimize for oVirt Store: Enabling this checkbox will enable extended options for created volume. The following screenshot shows the dialog box of Create Volume: Fill in the parameters, click on the Bricks button, and go to the new window to add new bricks with the following properties: Volume Type: This is used to change the previously marked type of the GlusterFS volume Server: It is used to specify a separate server that will export GlusterFS brick Brick Directory: It is used to specify the directory to use Specify the server and directory and click on Add. Depending on the type of volume, specify multiple bricks. After completing the list with bricks, click on the OK button to add volume and return to the menu. Click on the OK button to create GlusterFS volumes with the specified parameters. The following screenshot shows the Add Bricks dialog box: Now that we have GlusterFS volume, we select it from the list and click on Start. Configuring the GlusterFS storage oVirt 3.3 has support for creating data centers with the GlusterFS storage type: The GlusterFS storage type requires a preconfigured data center. A pre-created cluster should be present inside the data center. The enabled Gluster service is required. Go to the Storage section in resource pane and click on New Domain. In the dialog box that opens, fill in the details of our storage. The details are given as follows: Name and Data Center: It is used to specify the name and data center Domain Function/Storage Type: It is used to specify the data function and GlusterFS type Use Host: It is used to specify the host that will connect to the SPM Path: It is used to specify the path to the location in the format hostname:volume_name VFS Type: Leave it as glusterfs and leave Mount Option blank Click on the OK button; this will start the process of creating the repository. The created storage automatically connects to the specified data centers. If not, select the repository created in the list, and in the subtab named Data Center in the detail pane, click on the Attach button and choose our data center. After you click on OK, the process of connecting storage to the data center starts. The following screenshot shows the New Storage dialog box with the GlusterFS storage type. Summary In this article we learned how to configure NFS Storage, iSCSI Storage, FC storages, and GlusterFS Storage. Resources for Article: Further resources on this subject: Tips and Tricks on Microsoft Application Virtualization 4.6 [Article] VMware View 5 Desktop Virtualization [Article] Qmail Quickstarter: Virtualization [Article]
Read more
  • 0
  • 0
  • 10991

article-image-automated-testing-using-robotium
Packt
19 Nov 2013
10 min read
Save for later

Automated testing using Robotium

Packt
19 Nov 2013
10 min read
(For more resources related to this topic, see here.) Robotium framework Robotium is an open source automation testing framework that is used to write a robust and powerful black box for Android applications (the emphasis is mostly on black box test cases). It fully supports testing for native and hybrid applications. Native apps are live on the device, that is, designed for a specific platform and can be installed from the Google Play Store, whereas Hybrid apps are partly native and partly web apps. These can also be installed from the app store, but require the HTML to be rendered in the browser. Robotium is mostly used to automate UI test cases and internally uses run-time binding to Graphical User Interface (GUI) components. Robotium is released under the Apache License 2.0. It is free to download and can be easily used by individuals and enterprises and is built on Java and JUnit 3. It will be more appropriate to call Robotium an extension of the Android Test Unit Framework, available at http://developer.android.com/tools/testing/testing_android.html. Robotium can also work without the application, under the test's source code. The test cases written using Robotium can either be executed on the Android Emulator (Android Virtual Device (AVD))—we will see how to create an AVD during installation in the following section—or on a real Android device. Developers can write function, system, and acceptance test scenarios across multiple activities. It is currently the world's leading Automation Testing Framework, and many open source developers are contributing to introduce more and more exciting features in subsequent releases. The following screenshot is of the git repository website for the Robotium project: As Robotium is an open source project, anyone can contribute for the purpose of development and help in enhancing the framework with many more features. The Robotium source code is maintained at GitHub and can be accessed using the following link: https://github.com/jayway/robotium You just need to fork the project. Make all your changes in a clone project and click on Pull Request on your repository to tell core team members which changes to bring in. If you are new to the git environment, you can refer to the GitHub tutorial at the following link: https://help.github.com/ Robotium is like Selenium but for Android. This project was started in January 2010 by Renas Reda. He is the founder and main developer for Robotium. The project initiated with v1.0 and continues to be followed up with new releases due to new requirements. It has support for Android features such as activities, toasts, menus, context menus, web views, and remote controls. Let's see most of the Robotium features and benefits for Android test case developers. Features and benefits Automated testing using Robotium has many features and benefits. The triangularization workflow diagram between the user, Robotium, and the Android device clearly explains use cases between them: The features and benefits of Robotium are as follows: Robotium helps us to quickly write powerful test cases with minimal knowledge of the application under test. Robotium offers APIs to directly interact with UI controls within the Android application such as EditText, TextView, and Button. Robotium officially supports Android 1.6 and above versions. The Android platform is not modified by Robotium. The Robotium test can also be executed using command prompt. Robotium can be integrated smoothly with Maven or Ant. This helps to add Robotium to your project's build automation process. Screenshots can be captured in Robotium (an example screenshot is shown as follows): The test application project and the application project run on the same JVM, that is, Dalvik Virtual Machine (DVM). It's possible to run Robotium without a source code. Robotium can work with other code coverage measurement tools, such as Cobertura and Emma. Robotium can detect the messages that are shown on the screen (Toasts). Robotium supports Android features such as activities, menu, and context menu. Robotium automated tests can be implemented quickly. Robotium is built on JUnit, because of which it inherits all JUnit's features. The Robotium framework automatically handles multiple activities in an Android application. Robotium test cases are prominently readable, in comparison to standard instrumentation tests. Scrolling activity is automatically handled by the Robotium framework. Recent versions of Robotium support hybrid applications. Hybrid applications use WebViews to present the HTML and JavaScript files in full screen, using the native browser rendering engine. API set Web support has been added to the Robotium framework since Robotium 4.0 released. Robotium has full support for hybrid applications. There are some key differences between native and hybrid applications. Let's go through them one by one, as follows: Native Application Hybrid Application Platform dependent Cross platform Run on the device's internal software and hardware Built using HTML5 and JavaScript and wrapped inside a thin native container that provides access to native platform features Need more developers to build apps on different platforms and learning time is more Save development cost and time Excellent performance Less performance The native and hybrid applications are shown as follows: Let's see some of the existing methods in Robotium that support access to web content. They are as follows: searchText (String text) scrollUp/Down () clickOnText (String text) takeScreenshot () waitForText (String text) In the methods specifically added for web support, the class By is used as a parameter. It is an abstract class used as a conjunction with the web methods. These methods are used to select different WebElements by their properties, such as ID and name. The element used in a web view is referred to as a WebElement. It is similar to the WebDriver implemented in Selenium. The following table lists all the methods inside the class By: Method Description className (String className) Select a WebElement by its class name cssSelector (String selectors) Select a WebElement by its CSS selector getValue () Return the value id (String id) Select a WebElement by its id name (String name) Select a WebElement by its name tagName (String tagName) Select a WebElement by its tag name textContent (String textContent) Select a WebElement by its text content xpath (String xpath) Select a WebElement by its xpath Some of the important methods in the Robotium framework, that aim at direct communication with web content in Android applications, are listed as follows: clickOnWebElement(By by): It clicks on the WebElement matching the specified By class object. waitForWebElement(By by): It waits for the WebElement matching the specified By class object. getWebElement(By by, int index): It returns a WebElement matching the specified By class object and index. enterTextInWebElement(By by, String text): It enters the text in a WebElement matching the specified By class object. typeTextInWebElement(By by): It types the text in a WebElement matching the specified By class object. In this method, the program actually types the text letter by letter using the keyboard, whereas enterTextInWebElement directly enters the text in the particular. clearTextInWebElement(By by): It clears the text in a WebElement matching the specified By class object. getCurrentWebElements(By by): It returns the ArrayList of WebElements displayed in the active web view matching the specified By class object. Before actually looking into the hybrid test example, let's gain more information about WebViews. You can get an instance of WebView using the Solo class as follows: WebView wb = solo.getCurrentViews(WebView.class).get(0); Now that you have control of WebView, you can inject your JavaScript code as follows: Wb.loadUrl("<JavaScript>"); This is very powerful, as we can call every function on the current page; thus, it helps automation. Robotium Remote Control using SAFS SAFS tests are not wrapped up as JUnit tests and the SAFS Remote Control of Robotium uses an implementation that is NOT JUnit based. Also, there is no technical requirement for a JUnit on the Remote-Control side of the test. The test setup and deployment of the automation of the target app can be achieved using the SDK tools. These tools are used as part of the test runtime such as adb and aapt. The existing packaging tools can be used to repackage a compiled Robotium test with an alternate AndroidManifest.xml file, which can change the target application at runtime. SAFS is a general-purpose, data-driven framework. The only thing that should be provided by the user is the target package name or APK path arguments. The test will extract and redeploy the modified packages automatically and then launch the actual test. Traditional JUnit/Robotium users might not have, or see the need for, this general-purpose nature, but that is likely because it was necessary for the previous Android tests to be JUnit tests. It is required for the test to target one specific application. The Remote Control application is application specific. That's why the test app with the Remote Control installed in the device no longer needs to be an application. The Remote Control in Robotium means there are two test applications to build for any given test. They are as follows: Traditional on-device Robotium/JUnit test app Remote Control app These two build projects have entirely different dependencies and build scripts. The on-device test app has the traditional Robotium/Android/JUnit dependencies and build scripts, while the Remote Control app only has dependencies on the TCP sockets for communications and Robotium Remote Control API. The implementation for the remote-controlled Robotium can be done in the following two pieces: On Device: ActivityInstrumentationTestCase2.setup() is initialized when Robotium's Solo class object is to be used for the RobotiumTestRunner (RTR). The RTR has a Remote Control Listener and routes remote control calls and data to the appropriate Solo class methods and returns any results, as needed, to the Remote Controller. The on-device implementation may exploit test-result asserts if that is desirable. Remote Controller: The RemoteSolo API duplicates the traditional Solo API, but its implementation largely pushes the data through the Remote Control to the RTR, and then receives results from the Remote Controller. The Remote Control implementation may exploit any number of options for asserting, handling, or otherwise reporting or tracking the test results for each call. As you can see, the Remote-Control side only requires a RemoteSolo API without any specific JUnit context. It can be wrapped in a JUnit context if the tester desires it, but it is not necessary to be in a JUnit context. The sample code and installation of Robotium Remote Control can be accessed in the following link: http://code.google.com/p/robotium/wiki/RemoteControl Summary Thus this article introduced us to the Robotium framework, its different features, its benefits in the world of automated testing, the API set of the Robotium Framework, and how to implement the Robotium Remote Control using SAFS. Resources for Article: Further resources on this subject: So, what is Spring for Android? [Article] Android Native Application API [Article] Introducing an Android platform [Article]
Read more
  • 0
  • 0
  • 6648

Packt
19 Nov 2013
6 min read
Save for later

Code interlude – signals and slots

Packt
19 Nov 2013
6 min read
(For more resources related to this topic, see here.) Qt offers a better way: signals and slots. Like an event, the sending component generates a signal—in Qt parlance, the object emits a signal—which recipient objects may receive in a slot for the purpose. Qt objects may emit more than one signal, and signals may carry arguments; in addition, multiple Qt objects can have slots connected to the same signal, making it easy to arrange one-to-many notifications. Equally important, if no object is interested in a signal, it can be safely ignored, and no slots connected to the signal. Any object that inherits from QObject, Qt's base class for objects, can emit signals or provide slots for connection to signals. Under the hood, Qt provides extensions to C++ syntax for declaring signals and slots. A simple example will help make this clear. The classic example you find in the Qt documentation is an excellent one, and we'll use it again it here, with some extension's. Imagine you have the need for a counter, that is, a container that holds an integer. In C++, you might write: class Counter { public: Counter() { m_value = 0; } int value() const { return m_value; } void setValue(int value); private: int m_value; }; The Counter class has a single private member, m_value, bearing its value. Clients can invoke the value to obtain the counter's value, or set its value by invoking setValue with a new value. In Qt, using signals and slots, we write the class this way: #include <QObject> class Counter : public QObject { Q_OBJECT public: Counter() { m_value = 0; } int value() const { return m_value; } public slots: void setValue(int value); void increment(); void decrement(); signals: void valueChanged(int newValue); private: int m_value; }; This Counter class inherits from QObject, the base class for all Qt objects. All QObject subclasses must include the declaration Q_OBJECT as the first element of their definition; this macro expands to Qt code implementing the subclass-specific glue necessary for the Qt object and signal-slot mechanism. The constructor remains the same, initializing our private member to zero. Similarly, the accessor method value remains the same, returning the current value for the counter. An object's slots must be public, and are declared using the Qt extension to C++ public slots. This code defines three slots: a setValue slot, which accepts a new value for the counter, and the increment and decrement slots, which increment and decrement the value of the counter. Slots may take arguments, but do not return them; the communication between a signal and its slots is one way, initiating with the signal and terminating with the slot(s) connected to the signal. The counter offers a single signal. Like slots, signals are also declared using a Qt extension to C++, signals. In the example above, a Counter object emits the signal valueChanged with a single argument, which is the new value of the counter. A signal is a function signature, not a method; Qt's extensions to C++ use the type signature of signals and slots to ensure type safety between signal-slot connections, a key advantage signals and slots have over other decoupled messaging schemes. As the developers, it's our responsibility to implement each slot in our class with whatever application logic makes sense. The Counter class's slots look like this: void Counter::setValue(int newValue) { if (newValue != m_value) { m_value = newValue; emit valueChanged(newValue); } } void Counter::increment() { setValue(value() + 1); } void Counter::decrement() { setValue(value() – 1); } We use the implementation of the setValue slot as a method, which is what all slots are at their heart. The setValue slot takes a new value and assigns the new value to the Counter class's private member variable if they aren't the same. Then, the signal emits the valueChanged signal, using the Qt extension emit, which triggers an invocation to the slots connected to the signal. This is a common pattern for signals that handle object properties: testing the property to be set for equality with the new value, and only assigning and emitting a signal if the values are unequal. If we had a button, say QPushButton, we could connect its clicked signal to the increment or decrement slot, so that a click on the button incremented or decremented the counter. I'd do that using the QObject::connect method, like this: QPushButton* button = new QPushButton(tr("Increment"), this); Counter* counter = new Counter(this); QObject::connect(button, SIGNAL(clicked(void)), Counter, SLOT(increment(void)); We first create the QPushButton and Counter objects. The QPushButton constructor takes a string, the label for the button, which we denote to be the string Increment or its localized counterpart. Why do we pass this to each constructor? Qt provides a parent-child memory management between QObjects and their descendants, easing clean-up when you're done using an object. When you free an object, Qt also frees any children of the parent object, so you don't have to. The parent-child relationship is set at construction time; I'm signaling to the constructors that when the object invoking |this code is freed, the push button and counter may be freed as well. (Of course, the invoking method must also be a subclass of QObject for this to work.) Next, I call QObject::connect, passing first the source object and the signal to be connected, and then the receiver object and the slot to which the signal should be sent. The types of the signal and the slot must match, and the signals and slots must be wrapped in the SIGNAL and SLOT macros, respectively. Signals can also be connected to signals, and when that happens, the signals are chained and trigger any slots connected to the downstream signals. For example, I could write: Counter a, b; QObject::connect(&a, SIGNAL(valueChanged(int)), &b, SLOT(setValue(int))); This connects the counter b with the counter a, so that any change in value to the counter a also changes the value of the counter b. Signals and slots are used throughout Qt, both for user interface elements and to handle asynchronous operations, such as the presence of data on network sockets and HTTP transaction results. Under the hood, signals and slots are very efficient, boiling down to function dispatch operations, so you shouldn't hesitate to use the abstraction in your own designs. Qt provides a special build tool, the meta-object compiler, which compiles the extensions to C++ that signals and slots require and generates the additional code necessary to implement the mechanism. Summary In this article we learned the usage events for the purpose of coupling different objects; components offering data encapsulate that data in an event, and an event loop (or, more recently, an event listener) catches the event and performs some action. Resources for Article: Further resources on this subject: One-page Application Development [Article] Android 3.0 Application Development: Multimedia Management [Article] Creating and configuring a basic mobile application [Article]
Read more
  • 0
  • 0
  • 1863

article-image-derivatives-pricing
Packt
18 Nov 2013
10 min read
Save for later

Derivatives Pricing

Packt
18 Nov 2013
10 min read
(For more resources related to this topic, see here.) Derivatives are financial instruments which derive their value from (or are dependent on) the value of another product, called the underlying. The three basic types of derivatives are forward and futures contracts, swaps, and options. In this article we will focus on this latter class and show how basic option pricing models and some related problems can be handled in R. We will start with overviewing how to use the continuous Black-Scholes model and the binomial Cox-Ross-Rubinstein model in R, and then we will proceed with discussing the connection between these models. Furthermore, with the help of calculating and plotting of the Greeks, we will show how to analyze the most important types of market risks that options involve. Finally, we will discuss what implied volatility means and will illustrate this phenomenon by plotting the volatility smile with the help of real market data. The most important characteristics of options compared to futures or swaps is that you cannot be sure whether the transaction (buying or selling the underlying) will take place or not. This feature makes option pricing more complex and requires all models to make assumptions regarding the future price movements of the underlying product. The two models we are covering here differ in these assumptions: the Black-Scholes model works with a continuous process while the Cox-Ross-Rubinstein model works with a discrete stochastic process. However, the remaining assumptions are very similar and we will see that the results are close too. The Black-Scholes model The assumptions of the Black-Scholes model (Black and Sholes, 1973, see also Merton, 1973) are as follows: The price of the underlying asset (S) follows geometric Brownian motion: Here μ (drift) and σ (volatility) are constant parameters and W is a standard Wiener process. The market is arbitrage-free. The underlying is a stock paying no dividends. Buying and (short) selling the underlying asset is possible in any (even fractional) amount. There are no transaction costs. The short-term interest rate (r) is known and constant over time. The main result of the model is that under these assumptions, the price of a European call option (c) has a closed form: Here X is the strike price, T-tis the time to maturity of the option, and N denotes the cumulative distribution function of the standard normal distribution. The equation giving the price of the option is usually referred to as the Black-Scholes formula. It is easy to see from put-call parity that the price of a European put option (p) with the same parameters is given by: Now consider a call and put option on a Google stock in June 2013 with a maturity of September 2013 (that is, with 3 months of time to maturity).Let us assume that the current price of the underlying stock is USD 900, the strike price is USD 950, the volatility of Google is 22%, and the risk-free rate is 2%. We will calculate the value of the call option with the GBSOption function from the fOptions package. Beyond the parameters already discussed, we also have to set the cost of carry (b); in the original Black-Scholes model, (with underlying paying no dividends) it equals the risk-free rate. > library(fOptions) > GBSOption(TypeFlag = "c", S = 900, X =950, Time = 1/4, r = 0.02, + sigma = 0.22, b = 0.02) Title: Black Scholes Option Valuation Call: GBSOption(TypeFlag = "c", S = 900, X = 950, Time = 1/4, r = 0.02, b = 0.02, sigma = 0.22) Parameters: Value: TypeFlag c S 900 X 950 Time 0.25 r 0.02 b 0.02 sigma 0.22 Option Price: 21.79275 Description: Tue Jun 25 12:54:41 2013 This prolonged output returns the passed parameters with the result just below the Option Price label. Setting the TypeFlag to p would compute the price of the put option and now we are only interested in the results (found in the price slot—see the str of the object for more details) without the textual output: > GBSOption(TypeFlag = "p", S = 900, X =950, Time = 1/4, r = 0.02, sigma = 0.22, b = 0.02)@price [1] 67.05461 We also have the choice to compute the preceding values with a more user-friendly calculator provided by the GUIDE package. Running the blackscholes() function would trigger a modal window with a form where we can enter the same parameters. Please note that the function uses the dividend yield instead of cost of carry, which is zero in this case. The Cox-Ross-Rubinstein model The Cox-Ross-Rubinstein(CRR) model (Cox, Ross and Rubinstein, 1979) assumes that the price of the underlying asset follows a discrete binomial process. The price might go up or down in each period and hence changes according to a binomial tree illustrated in the following plot, where u and dare fixed multipliers measuring the price changes when it goes up and down. The important feature of the CRR model is that u=1/d and the tree is recombining; that is, the price after two periods will be the same if it first goes up and then goes down or vice versa, as shown in the following figure: To build a binomial tree, first we have to decide how many steps we are modeling (n); that is, how many steps the time to maturity of the option will be divided into. Alternatively, we can determine the length of one time step ∆t,(measured in years) on the tree: If we know the volatility (σ) of the underlying, the parameters u and dare determined according to the following formulas: And consequently: When pricing an option in a binomial model, we need to determine the tree of the underlying until the maturity of the option. Then, having all the possible prices at maturity, we can calculate the corresponding possible option values, simply given by the following formulas: To determine the option price with the binomial model, in each node we have to calculate the expected value of the next two possible option values and then discount it. The problem is that it is not trivial what expected return to use for discounting. The trick is that we are calculating the expected value with a hypothetic probability, which enables us to discount with the risk-free rate. This probability is called risk neutral probability (pn) and can be determined as follows: The interpretation of the risk-neutral probability is quite plausible: if the probability that the underlying price goes up from any of the nodes was pn, then the expected return of the underlying would be the risk-free rate. Consequently, an expected value calculated with pn can be discounted by rand the price of the option in any node of the tree is determined as: In the preceding formula, g is the price of an option in general (it may be call or put as well) in a given node, gu and gd are the values of this derivative in the two possible nodes one period later. For demonstrating the CRR model in R, we will use the same parameters as in the case of the Black-Scholes formula. Hence, S=900, X=950, σ=22%, r=2%, b=2%, T-t=0.25. We also have to set n, the number of time steps on the binomial tree. For illustrative purposes, we will work with a 3-period model: > CRRBinomialTreeOption(TypeFlag = "ce", S = 900, X = 950, + Time = 1/4, r = 0.02, b = 0.02, sigma = 0.22, n = 3)@price [1] 20.33618 > CRRBinomialTreeOption(TypeFlag = "pe", S = 900, X = 950, + Time = 1/4, r = 0.02, b = 0.02, sigma = 0.22, n = 3)@price [1] 65.59803 It is worth observing that the option prices obtained from the binomial model are close to (but not exactly the same as) the Black-Scholes prices calculated earlier. Apart from the final result, that is, the current price of the option, we might be interested in the whole option tree as well: > CRRTree <- BinomialTreeOption(TypeFlag = "ce", S = 900, X = 950, + Time = 1/4, r = 0.02, b = 0.02, sigma = 0.22, n = 3) > BinomialTreePlot(CRRTree, dy = 1, xlab = "Time steps", + ylab = "Number of up steps", xlim = c(0,4)) > title(main = "Call Option Tree") Here we first computed a matrix by BinomialTreeOption with the given parameters and saved the result in CRRTree that was passed to the plot function with specified labels for both the x and y axis with the limits of the x axis set from 0 to 4, as shown in the following figure. The y-axis (number of up steps) shows how many times the underlying price has gone up in total. Down steps are defined as negative up steps. The European put option can be shown similarly by changing the TypeFlag to pe in the previous code: Connection between the two models After applying the two basic option pricing models, we give some theoretical background to them. We do not aim to give a detailed mathematical derivation, but we intend to emphasize (and then illustrate in R) the similarities of the two approaches. The financial idea behind the continuous and the binomial option pricing is the same: if we manage to hedge the option perfectly by holding the appropriate quantity of the underlying asset, it means we created a risk-free portfolio. Since the market is supposed to be arbitrage-free, the yield of a risk-free portfolio must equal the risk-free rate. One important observation is that the correct hedging ratio is holding underlying asset per option. Hence, the ratio is the partial derivative (or its discrete correspondent in the binomial model) of the option value with respect to the underlying price. This partial derivative is called the delta of the option. Another interesting connection between the two models is that the delta-hedging strategy and the related arbitrage-free argument yields the same pricing principle: the value of the derivative is the risk-neutral expected value of its future possible values, discounted by the risk-free rate. This principle is easily tractable on the binomial tree where we calculated the discounted expected values node by node; however, the continuous model has the same logic as well, even if the expected value is mathematically more complicated to compute. This is the reason why we gave only the final result of this argument, which was the Black-Scholes formula. Now we know that the two models have the same pricing principles and ideas (delta-hedging and risk-neutral valuation), but we also observed that their numerical results are not equal. The reason is that the stochastic processes assumed to describe the price movements of the underlying asset are not identical. Nevertheless, they are very similar; if we determine the value of u and d from the volatility parameter as we did it in The Cox-Ross-Rubinstein model section, the binomial process approximates the geometric Brownian motion. Consequently, the option price of the binomial model converges to that of the Black-Scholes model if we increase the number of time steps (or equivalently, decrease the length of the steps). To illustrate this relationship, we will compute the option price in the binomial model with increasing numbers of time steps. In the following figure, we compare the results with the Black-Scholes price of the option: The plot was generated by a loop running N from 1 to 200 to compute CRRBinomialTreeOption with fixed parameters: > prices <- sapply(1:200, function(n) { + CRRBinomialTreeOption(TypeFlag = "ce", S = 900, X = 950, + Time = 1/4, r = 0.02, b = 0.02, sigma = 0.22, n = n)@price + }) Now the prices variable holds 200 computed values: > str(prices) num [1:200] 26.9 24.9 20.3 23.9 20.4... Let us also compute the option with the generalized Black-Scholes option: > price <- GBSOption(TypeFlag = "c", S = 900, X = 950, Time = 1/4, r = 0.02, sigma = 0.22, b = 0.02)@price And show the prices in a joint plot with the GBS option rendered in red: > plot(1:200, prices, type='l', xlab = 'Number of steps', + ylab = 'Prices') > abline(h = price, col ='red') > legend("bottomright", legend = c('CRR-price', 'BS-price'), + col = c('black', 'red'), pch = 19)
Read more
  • 0
  • 0
  • 21748
Unlock access to the largest independent learning library in Tech for FREE!
Get unlimited access to 7500+ expert-authored eBooks and video courses covering every tech area you can think of.
Renews at €18.99/month. Cancel anytime
article-image-3d-printing
Packt
18 Nov 2013
9 min read
Save for later

3D Printing

Packt
18 Nov 2013
9 min read
(For more resources related to this topic, see here.) Function The software starts by taking a .stl or .obj file along, with all our settings, and converts it into GCode. Think of the GCode like an instruction set to our printer, which includes where to move, how fast to move, whether or not to extrude material, extruder temperature, lower platform, and so on. The following screenshot shows an example of GCode produced by the ReplicatorG slicing engine Skeinforge at the start of a print: Start of GCode The slicing engine is what tells your printer what to make and exactly how to make it. The algorithm involved directly relates to print quality, and thus gets the most attention from developers. It's at this stage we can see the tradeoff between software and hardware: maybe the printer has the capability to print with greater resolution but current software might only break the model down so far? Or perhaps it's the opposite, where the software can break down the model finer than the hardware is capable of producing? It turns out this is a major factor in what separates personal and commercial printers, and even a $200 and $2,000 printer—precision. More precision costs more money in both hardware and software development. The hardware must be able to handle the precision calculated in software, and where it cannot, and then software solutions must be implemented in circumvention. It's these reasons why algorithms improve by leaps and bounds with every software update, and why newer released printers outperform their predecessors. To make it easier on the microcontroller, in the printer the GCode is converted into the .s3g or .x3g code, which is essentially just optimized GCode. From here it is used to generate motor steps and direction pulses, which are sent to the motor controller and then to the motors. It's at this stage we realize that the process of 3D printing is just a handful of motors moving in a set pattern combined with a heater to melt the plastic material. The magic of 3D printing happens behind the scenes inside the slicing algorithm in order to create those explicit patterns. MakerWare You may be thinking, "This software is made by MakerBot and is intended to use with MakerBot printers that sounds like the best option"; well, you are mostly correct. MakerWare is currently still in beta, but is released for general use. The software is always in revision, and has made major improvements in a very short period time. Logically, it would make sense to use the software explicitly designed for use with your printer, but up until MakerWare v2.2, the ReplicatorG software had been superior. With all the improvements made in MakerWare's latest release v2.3.1, I would argue that the MakerWare software surpasses the ReplicatorG for use with a MakerBot 3D printer. This is one of the joys of being involved with MakerBot and the 3D printing environment- The products are always evolving and always improving. In a period of five years, MakerBot went from an idea to being acquired by Stratasys for $403 million. This speaks for how fast the industry is moving, and how fast the technology is advancing. For those interested, visit http://www.makerbot.com/blog/category/makerbot-software-updates/ to see a detailed description (lots of pictures) of the improvements in each MakerWare update. ReplicatorG You might be wondering why we would even consider the ReplicatorG software. The simple answer is with the release of MakerWare v2.3.1 and the purchase of a MakerBot, we wouldn't. The ReplicatorG software undoubtedly served as a building block for many of the features in MakerWare, and was the leading software for personal/hobbyist 3D printing for many years. The MakerWare software will meet all the needs for our designs, but if you are interested in learning more about open source 3D printing, I would suggest checking out this software. We have chosen to use MakerWare (v2.3.1) for the examples in the article, as this software is most tailored to our needs. Visit http://www.makerbot.com/makerware/ to download your own beta copy. MakerWare options and settings The first step after opening MakerWare is to add a model to the build platform by clicking on the Add button. Let's add Mr Jaws (by navigating to File | Examples | Mr_Jaws.stl) to our build platform. Once the model has been added, it needs to be selected by left-clicking it. This should highlight the model in yellow as shown in the following screenshot: Mr Jaws is selected Notice the buttons on the left-hand side, which are intuitively labeled Look, Move, Turn, and Scale. Clicking these buttons allows us to orient our model. Let's move Mr Jaws to the top-right corner, spin him 180 degrees in the Z-plane, and scale him to 110 percent. The result can be seen in the following screenshot: Mr Jaws is moved, rotated, and scaled Once we are satisfied with the orientation, click on Make located at the top of the screen to open up the print options. Ensure that your model is sitting on the platform by hitting the On Platform button (by navigating to Move | On Platform); else, you will print many layers of supporting material before finally reaching your part (if your part is floating above the platform), or you will damage your nozzle and platform (if your part is floating below the platform). Print options The default options are fairly straightforward, and will modify all the advanced options automatically. We will use the following table to describe each of the default options: Option Description I want to: This gives the option where to save the file or to send it directly to your MakerBot, if its plugged in by USB. Export For: This helps you select your MakerBot. Material: This helps you select the PLA that is recommended. Resolution: MakerWare has 3 quick set profiles which are Low, Standard, and High. These profiles reference the desired print resolution and directly control the Z-layer height. Remember that higher resolution requires longer print times. Raft: A raft is a surface slightly larger than the part which is built between the bottom of the part and the build platform. Rafts help reduce warping by having more surface area adhered to the build platform. Once a part has been printed, the raft is easily broken away. We will always be using rafts to help reduce errors in our models brought about by poor adhesion and warping. Supports: Supports are used to support sharp overhangs. We will always have the support box checked, as the software will determine when and where to use supports and will not print them if they are not needed. This ensures we will never run into errors from floating layers. By evaluating the advanced options, we are able to observe the result from changing the default options and gain a little more insight into settings that influence our print. The following is a table describing these advanced options: Advanced options Description Profile: Profiles handle all the settings of a print. Profiles are a grouping of preselected options. By default, there are 4 profiles: Low, Standard, and High but you can create your own custom profiles. Slicer: The options are between MakerBot Slicer and Skeinforge slicer, and can be changed by creating a new profile. It is recommended that we use the MakerBot slicing engine because it has been optimized for use with MakerBot printers as mentioned earlier. Quality | Infill: Infill is the density of the object measured by a percentage. By default, the amount of infill is low (less than 20 percent) to save both time and material. The slicing engine will automatically create a pattern for the infill (most commonly honeycomb). Quality | Number of Shells: The number of shells will represent the perimeter thickness of your model. One shell corresponds to one layer (which is approximately 0.4 mm the width of the nozzle opening in XY, and will depend on the next property, Quality | Layer Height, for the Z thickness). All the profiles default to two shells. If strength is a concern for your model, it is suggested to increase the number of shells rather than infill. Adding shells will also increase print times. Quality | Layer Height: Layer height is the height of each individual cross-sectional layer. MakerBot Replicator 2 is capable of heights as low as 0.1 mm. This corresponds to 10 layers for a model of 1 mm in height. Lowering the height will increase print times. Temperature | Extruders: The temperature for the extruder by default sets to 230 C. Greater temperatures can improve adhesion but may require slower printing. Every individual has their own "magic number" for temperature, which they feel works best but its best to say within +/- 10 C range of the default. For our models we will be using the default 230 C. Temperature | Build Plate A heated build plate is only required if we are printing in ABS, in order to reduce warping. The default is 110 C, and slightly higher temperatures will improve adhesion but also risk greater warping upon cooling. Speed | Speed while Extruding: Extruder speed and temperature are directly linked to one another; the speed needs to be slow enough to allow the layer currently being extruded to bond with the layer underneath. Greater speeds have the potential to reduce accuracy but will decrease print time. Be extremely cautious while modifying this parameter, as it takes experience to match increased speed and temperatures properly. Speed | Speed while Travelling: When not extruding, the extruder head is capable of faster travel. It is recommended to leave this parameter as set. Preview before printing This is an amazing tool and you should check it off every time. Preview before printing allows you to see an approximate material use and time estimate to print the given model. Summary This article thus explained some basic steps to build a 3D model in MakerBot and to get it ready for 3D printing. Resources for Article: Further resources on this subject: Design Tools and Basics [Article] Materials with Ogre 3D [Article] Importing 3D Formats into Away3D [Article]
Read more
  • 0
  • 0
  • 10638

article-image-icinga-object-configuration
Packt
18 Nov 2013
9 min read
Save for later

Icinga Object Configuration

Packt
18 Nov 2013
9 min read
(For more resources related to this topic, see here.) A localhost monitoring setup Let us take a close look at our current setup, which we created, for monitoring a localhost. Icinga by default comes with object configuration for a localhost. The object configuration files are inside /etc/icinga/objects for default installations. $ ls /etc/icinga/objects commands.cfg notifications.cfg templates.cfg contacts.cfg printer.cfg timeperiods.cfg localhost.cfg switch.cfg windows.cfg There are several configuration files with object definitions. Together, these object definitions define the monitoring setup for monitoring some services on a localhost. Let's first look at localhost.cfg, which has most of the relevant configuration. We have a host definition: define host{ use linux-server host_name localhost alias localhost address 127.0.0.1 } The preceding object block defines one object, that is, the host that we want to monitor, with details such as the hostname, alias for the host, and the address of the server—which is optional, but is useful when you don't have DNS record for the hostname. We have a localhost host object defined in Icinga with the preceding object configuration. The localhost.cfg file also has a hostgroup defined which is as follows: define hostgroup { hostgroup_name linux-servers alias Linux Servers members localhost // host_name of the host object } The preceding object defines a hostgroup with only one member, localhost, which we will extend later to include more hosts. The members directive specifies the host members of the hostgroup. The value of this directive refers to the value of the host_name directive in the host definitions. It can be a comma-separated list of several hostnames. There is also a directive called hostgroups in the host object, where you can give a comma-separated list of names of the hostgroups that we want the host to be part of. For example, in this case, we could have omitted the members directive in the hostgroup definition and specified a hostgroups directive, which has the value linux-servers, in the localhost host definition. At this point, we have a localhost host and a linux-servers hostgroup, and localhost is a member of linux-servers. This is illustrated in the following figure: Going further into localhost.cfg, we have a bunch of service object definitions that follow. Each of these definitions indicate the service on a localhost that we want to monitor with the host_name directive. define service { use local-service host_name localhost service_description PING check_command check_ping!100.0,20%!500.0,60% } This is one of the service definitions. The object defines a PING service check that monitors the reachability. The host_name directive specifies the host that this service check should be associated with, which in this case is localhost. Again, the value of the host_name directive here should reflect the value of the host_name directive defined in the host object definition. So, we have a PING service check defined for a localhost, which is illustrated by following figure: There are several such service definitions that are placed on a localhost. Each service has a check_command directive that specifies the command for monitoring that service. Note that the exclamation marks in the check_command values are the command argument separators. So, cmd!foo!bar indicates that the command is cmd with foo as its first argument and bar as the second. It is important to remember that the check_ping part in check_command in the preceding example does not mean the check_ping executable that is in /usr/lib64/nagios/plugins/check_ping for most installations; it refers to the Icinga object of type command. In our setup, all command object definitions are inside commands.cfg. The commands.cfg file has the command object definition for check_ping. define command { command_name check_ping command_line $USER1$/check_ping -H $HOSTADDRESS$ -w $ARG1$ -c $ARG2$ -p 5 } The check_command value in the PING service definition refers to the preceding command object, which indicates the exact command to be executed for performing the service check. $USER1$ is a user-defined Icinga macro. Macros in Icinga are like variables that can be used in various object definitions to wrap data inside these variables. Some macros are predefined, while some are user defined. These user macros are usually defined in /etc/icinga/resources.cfg: $USER1$=/usr/lib64/nagios/plugins So replace the $USER1$ macro with its value, and execute: $ value/of/USER1/check_ping --help This command will print the usual usage string with all the command-line options available. $ARG1$ and $ARG2$ in the command definition are macros referring to the arguments passed in the check_command value in the service definition, which are 100.0,20% and 500.0,60% respectively for the PING service definition. We will come to this later. As noted earlier, the status of the service is determined by the exit code of the command that is specified in the command_line directive in command definition. We have many such service definitions for a localhost in localhost.cfg, such as Root Partition (monitors disk space), Total Processes, Current Load, HTTP, along with command definitions in commands.cfg for check_commands of each of these service definitions. So, we have a host definition for localhost, a hostgroup definition linux-servers having localhost as its member, several service check definitions for localhost with check commands, and the command definitions specifying the exact command with arguments to execute for the checks. This is illustrated with the example Ping check in the following figure: This completes the basic understanding of how our localhost monitoring is built up from plain-text configuration. Notifications We would, as is the point of having monitoring systems, like to get alerted when something actually goes down. We don't want to keep monitoring the Icinga web interface screen, waiting for something to go down. Icinga provides a very generic and flexible way of sending out alerts. We can have any alerting script triggered when something goes wrong, which in turn may run commands for sending e-mails, SMS, Jabber messages, Twitter tweets, or practically anything that can be done from within a script. The default localhost monitoring setup has an e-mail alerting configuration. The way these notifications work is that we define contact objects where we give the contact name, e-mail addresses, pager numbers, and other necessary details. These contact names are specified in the host/service templates or the objects themselves. So, when Icinga detects that a host/service has gone down, it will use this contact object to send contact details to the alerting script. The contact object definition also has the host_notification_commands and service_notification_commands directives. These directives specify the command objects that should be used to send out the notifications for that particular contact. The former directive is used when the host goes down, and the latter is used when a service goes down. The respective command objects are then looked up and the value of their command_line directive is executed. This command object is the same as the one we looked at previously for executing checks. The same command object type is used to also define notification commands. We can also define contact groups and specify them in the host/service object definitions to alert a bunch of contacts at the same time. We can also give a comma-separated list of contact names instead of a contact group. Let's have a look at our current setup for notification configuration. The host/service template objects have the admin contact group specified, whose definition is in contacts.cfg: define contactgroup { contactgroup_name admins alias Icinga Administrators members icingaadmin } The group has the icingaadmin member contact, which is again defined in the same file: define contact { contact_name icingaadmin use generic-contact alias Icinga Admin email your@email.com } The contacts.cfg file has your e-mail address. The contact object inherits the generic-contact template contact object. define contact{ name generic-contact service_notification_period 24x7 host_notification_period 24x7 service_notification_options w,u,c,r,f,s host_notification_options d,u,r,f,s service_notification_commands notify-service-by-email host_notification_commands notify-host-by-email register 0 } This template object has the host_notification_commands and service_notification_commands directives defined as notify-host-by-email and notify-service-by-email respectively. These are commands similar to what we use in service definitions. These commands are defined in commands.cfg: define command { command_name notify-host-by-email command_line /usr/bin/printf "%b" "***** Icinga *****nnNotification Type: $NOTIFICATIONTYPE$nHost: $HOSTNAME$nState: $HOSTSTATE$nAddress: $HOSTADDRESS$nInfo: $HOSTOUTPUT$nnDate/Time: $LONGDATETIME$n" | /bin/mail -s "** $NOTIFICATIONTYPE$ Host Alert: $HOSTNAME$ is $HOSTSTATE$ **" $CONTACTEMAIL$ } define command { command_name notify-service-by-email command_line /usr/bin/printf "%b" "***** Icinga *****nnNotification Type: $NOTIFICATIONTYPE$nnService: $SERVICEDESC$nHost: $HOSTALIAS$nAddress: $HOSTADDRESS$nState: $SERVICESTATE$nnDate/Time: $LONGDATETIME$nnAdditional Info:nn$SERVICEOUTPUT$n" | /bin/mail -s "** $NOTIFICATIONTYPE$ Service Alert: $HOSTALIAS$/$SERVICEDESC$ is $SERVICESTATE$ **" $CONTACTEMAIL$ } These commands are eventually executed to send out e-mail notifications to the supplied e-mail addresses. Notice that command_lines uses the /bin/mail command to send e-mails, which is why we need a working setup of a SMTP server. Similarly, we could use any command/script path to send out custom alerts, such as SMS and Jabber. We could also change the above e-mail command to change the content format to suit our requirements. The following figure illustrates the contact and notification configuration: The correlation between hosts/services and contacts/notification commands is shown below: Summary In this article, we analyzed our current configuration for the Icinga setup which monitors a localhost. We can replicate this to monitor a number of other servers using the desired service checks. We also looked at how the alerting configuration works to send out notifications when something goes down. Resources for Article: Further resources on this subject: Troubleshooting Nagios 3.0 [Article] Notifications and Events in Nagios 3.0-part1 [Article] BackTrack 4: Target Scoping [Article]
Read more
  • 0
  • 0
  • 11410

article-image-getting-started-ansible
Packt
18 Nov 2013
8 min read
Save for later

Getting Started with Ansible

Packt
18 Nov 2013
8 min read
(For more resources related to this topic, see here.) First steps with Ansible Ansible modules take arguments in key value pairs that look similar to key=value, perform a job on the remote server, and return information about the job as JSON. The key value pairs allow the module to know what to do when requested. The data returned from the module lets Ansible know if anything changed or if any variables should be changed or set afterwards. Modules are usually run within playbooks as this lets you chain many together, but they can also be used on the command line. Previously, we used the ping command to check that Ansible had been correctly setup and was able to access the configured node. The ping module only checks that the core of Ansible is able to run on the remote machine but effectively does nothing. A slightly more useful module is called setup. This module connects to the configured node, gathers data about the system, and then returns those values. This isn't particularly handy for us while running from the command line, however, in a playbook you can use the gathered values later in other modules. To run Ansible from the command line, you need to pass two things, though usually three. First is a host pattern to match the machine that you want to apply the module to. Second you need to provide the name of the module that you wish to run and optionally any arguments that you wish to give to the module. For the host pattern, you can use a group name, a machine name, a glob, and a tilde (~), followed by a regular expression matching hostnames, or to symbolize all of these, you can either use the word all or simply *. To run the setup module on one of your nodes, you need the following command line: $ ansible machinename -u root -k -m setup The setup module will then connect to the machine and give you a number of useful facts back. All the facts provided by the setup module itself are prepended with ansible_ to differentiate them from variables. The following is a table of the most common values you will use, example values, and a short description of the fields: Field Example Description ansible_architecture x86_64 The architecture of the managed machine ansible_distribution CentOS The Linux or Unix Distribution on the managed machine ansible_distribution_version 6.3 The version of the preceding distribution ansible_domain example.com The domain name part of the server's hostname ansible_fqdn machinename.example.com This is the fully qualified domain name of the managed machine. ansible_interfaces ["lo", "eth0"] A list of all the interfaces the machine has, including the loopback interface ansible_kernel 2.6.32-279.el6.x86_64 The kernel version installed on the managed machine ansible_memtotal_mb 996 The total memory in megabytes available on the managed machine ansible_processor_count 1 The total CPUs available on the managed machine ansible_virtualization_role guest Whether the machine is a guest or a host machine ansible_virtualization_type kvm The type of virtualization setup on the managed machine These variables are gathered using Python from the host system; if you have facter or ohai installed on the remote node, the setup module will execute them and return their data as well. As with other facts, ohai facts are prepended with ohai_ and facter facts with facter_. While the setup module doesn't appear to be too useful on the command line, once you start writing playbooks, it will come into its own. If all the modules in Ansible do as little as the setup and the ping module, we will not be able to change anything on the remote machine. Almost all of the other modules that Ansible provides, such as the file module, allow us to actually configure the remote machine. The file module can be called with a single path argument; this will cause it to return information about the file in question. If you give it more arguments, it will try and alter the file's attributes and tell you if it has changed anything. Ansible modules will almost always tell you if they have changed anything, which becomes more important when you are writing playbooks. You can call the file module, as shown in the following command, to see details about /etc/fstab: $ ansible machinename -u root -k -m file -a 'path=/etc/fstab' The preceding command should elicit a response like the following code: machinename | success >> { "changed": false, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/fstab", "size": 779, "state": "file" } Or like the following command to create a new test directory in /tmp: $ ansible machinename -u root -k -m file -a 'path=/tmp/test state=directory mode=0700 owner=root' The preceding command should return something like the following code: machinename | success >> { "changed": true, "group": "root", "mode": "0700", "owner": "root", "path": "/tmp/test", "size": 4096, "state": "directory" } The second command will have the changed variable set to true, if the directory doesn't exist or has different attributes. When run a second time, the value of changed should be false indicating that no changes were required. There are several modules that accept similar arguments to the file module, and one such example is the copy module. The copy module takes a file on the controller machine, copies it to the managed machine, and sets the attributes as required. For example, to copy the /etc/fstabfile to /tmp on the managed machine, you will use the following command: $ ansible machinename -m copy -a 'path=/tmp/fstab mode=0700 owner=root' The preceding command, when run the first time, should return something like the following code: machinename | success >> { "changed": true, "dest": "/tmp/fstab", "group": "root", "md5sum": "fe9304aa7b683f58609ec7d3ee9eea2f", "mode": "0700", "owner": "root", "size": 637, "src": "/root/.ansible/tmp/ansible-1374060150.96- 77605185106940/source", "state": "file" } There is also a module called command that will run any arbitrary command on the managed machine. This lets you configure it with any arbitrary command, such as a preprovided installer or a self-written script; it is also useful for rebooting machines. Please note that this module does not run the command within the shell, so you cannot perform redirection, use pipes, and expand shell variables or background commands. Ansible modules strive to prevent changes being made when they are not required. This is referred to as idempotency and can make running commands against multiple servers much faster. Unfortunately, Ansible cannot know if your command has changed anything or not, so to help it be more idempotent you have to give it some help. It can do this either via the creates or the removes argument. If you give a creates argument, the command will not be run if the filename argument exists. The opposite is true of the removes argument; if the filename exists, the command will be run. You run the command as follows: $ ansible machinename -m command -a 'rm -rf /tmp/testing removes=/tmp/testing' If there is no file or directory named /tmp/testing, the command output will indicate that it was skipped, as follows: machinename | skipped Otherwise, if the file did exist, it will look as follows: ansibletest | success | rc=0 >> Often it is better to use another module in place of the command module. Other modules offer more options and can better capture the problem domain they work in. For example, it would be much less work for Ansible and also the person writing the configurations to use the file module in this instance, since the file module will recursively delete something if the state is set to absent. So, this command would be equivalent to the following command: $ ansible machinename -m file -a 'path=/tmp/testing state=absent' If you need to use features usually available in a shell while running your command, you will need the shell module. This way you can use redirection, pipes, or job backgrounding. You can pick which shell to use with the executable argument. However, when you write the code, it also supports the creates argument but does not support the removes argument. You can use the shell module as follows: $ ansible machinename -m shell -a '/opt/fancyapp/bin/installer.sh > /var/log/fancyappinstall.log creates=/var/log/fancyappinstall.log' Summary In this article, we have covered which installation type to choose, installing Ansible, and how to build an inventory file to reflect your environment. After this, we saw how to use Ansible modules in an ad hoc style for simple tasks. Finally, we discussed how to learn which modules are available on your system and how to use the command line to get instructions for using a module. Resources for Article: Further resources on this subject: Configuring Manage Out to DirectAccess Clients [Article] Creating and configuring a basic mobile application [Article] Deploying Applications and Software Updates on Microsoft System Center 2012 Configuration Manager [Article]
Read more
  • 0
  • 0
  • 12321

article-image-css3-animation
Packt
18 Nov 2013
7 min read
Save for later

CSS3 Animation

Packt
18 Nov 2013
7 min read
(For more resources related to this topic, see here.) The websites, we see today, are complex and complicated. By complex and complicated, we are referring to the development of these websites and not the webpage itself. We see animations and complex features. Prior to HTML5 and CSS3, JavaScript was used extensively for this purpose. HTML was incorrectly used for styling when it was expected to design the structural markup of the page. However, with the advent of CSS, it is a good practice to use HTML for markup and CSS for styling. CSS3 brings along transforms, transition elements, and animation features that make it easier to develop awesome features. In transition, we can view the change from a single state to other but when it comes to multiple states, Animation is the solution. Let's discuss the various properties of CSS3 Animations and then we will incorporate all of that in a code to understand it better. @keyframes The points at which the transition should take place can be defined using the @keyframes property. As of now, we need to add a vendor prefix to the @keyframes property as it is still in its development state. In future, when it is accepted as a standard, then we do not have to use a vendor prefix. We can use percentage or from and to keywords to implement the change in state from one CSS style to another. animation-name We need to apply animation to an element. This property enables us to do so by applying it to the animation name defined in the keyframes rule. However, it cannot be a standalone property and has to be used in conjunction with other animation properties. animation-duration Using this feature, we can define the duration of the animation. If we specify the animation-duration to 5 seconds, changes in the CSS defined states will need to be completed within 5 seconds. animation-delay Similar to the delay property in transition, the delay feature will delay the animation by the time period specified. animation-timing-function Similar to the timing function, this property decides the speed of transition. It behaves the same way as the transition timing function that we have seen earlier. animation-iteration-count We can decide the number of iteration carried out in the animation phase using this property. Setting this property to infinite will mean that the animation will never stop. animation-direction We can decide the direction of the animation using this property. We can use values like reverse, alternate to define the direction of the element to be animated. animation-play-state Using this feature, we can determine whether the animation would be running or paused accordingly. Now that we had a look at these properties, we will now incorporate some of these properties in a code and understand the functionality in a better way. Hence, to gain a practical insight, let's look at the following code. <!DOCTYPE html> <html> <head> <style> body { background:#000; color:#fff; } #trigger { width:100px; height:100px; position:absolute; top:50%; margin:-50px 0 0 -50px; left:50%; background: black; border-radius:50px; /*set the animation*/ /*[animation name] [animation duration] [animation timing function] [animation delay] [animation iterations count] [animation direction]*/ animation: glowness 5s linear 0s 5 alternate; -moz-animation: glowness 5s linear 0s 5 alternate; /* Firefox */ -webkit-animation: glowness 5s linear 0s 5 alternate; /* Safari and Chrome */ -o-animation: glowness 5s linear 0s 5 alternate; /* Opera */ -ms-animation: glowness 5s linear 0s 5 alternate; /* IE10 */ } #trigger:hover { animation-play-state: paused; -moz-animation-play-state: paused; -webkit-animation-play-state: paused; -o-animation-play-state: paused; -ms-animation-play-state: paused; } /*animation keyframes*/ @keyframes glowness { 0% {box-shadow: 0 0 80px orange;} 25% {box-shadow: 0 0 150px red;} 50% {box-shadow: 0 0 70px pink;} 75% {box-shadow: 0 0 50px violet;} 100% {box-shadow: 0 0 100px yellow;} } @-moz-keyframes glowness /* Firefox */ { 0% {box-shadow: 0 0 80px orange;} 25% {box-shadow: 0 0 150px red;} 50% {box-shadow: 0 0 70px pink;} 75% {box-shadow: 0 0 50px violet;} 100% {box-shadow: 0 0 100px yellow;} } @-webkit-keyframes glowness /* Safari and Chrome */ { 0% {box-shadow: 0 0 80px orange;} 25% {box-shadow: 0 0 150px red;} 50% {box-shadow: 0 0 70px pink;} 75% {box-shadow: 0 0 50px violet;} 100% {box-shadow: 0 0 100px yellow;} } @-o-keyframes glowness /* Opera */ { 0% {box-shadow: 0 0 80px orange;} 25% {box-shadow: 0 0 150px red;} 50% {box-shadow: 0 0 70px pink;} 75% {box-shadow: 0 0 50px violet;} 100% {box-shadow: 0 0 100px yellow;} } @-ms-keyframes glowness /* IE10 */ { 0% {box-shadow: 0 0 20px green;} 25% {box-shadow: 0 0 150px red;} 50% {box-shadow: 0 0 70px pink;} 75% {box-shadow: 0 0 50px violet;} 100% {box-shadow: 0 0 100px yellow;} } </style> <script> // animation started (buggy on firefox) $('#trigger').on('animationstart mozanimationstart webkitAnimationStart oAnimationStart msanimationstart',function() { $('p').html('animation started'); }) // animation paused $('#trigger').on('mouseover',function(){ $('p').html('animation paused'); }) // animation re-started $('#trigger').on('mouseout',function(){ $('p').html('animation re-started'); }) // animation ended $('#trigger').on('animationend mozanimationend webkitAnimationEnd oAnimationEnd msanimationend',function() { $('p').html('animation ended'); }) //iteration count var i =0; $('#trigger').on('animationiteration mozanimationiteration webkitAnimationIteration oAnimationIteration msanimationiteration', function() { i++; $('p').html('animation iteration='+i); }) </script> </head> <body> <div id="trigger"></div> </body> </html> The output of the code on execution would be as follows: We have used –webkit as the prefix in this example as we are executing the code in Google Chrome. Please us –moz prefix for Firefox and –o- for Opera. Comments are added in the code so that we can understand it easily. Apart from HTML5 and CSS3, we have used a bit of JQuery. Let’s go through the animation part of the code to understand it better. In the CSS3 styles, we have mentioned the animation direction as alternate as a result of which the animation would be in a different direction after the first iteration. We have used the hover property. In this code, whenever we hover over the object, the animation is paused. We have also defined the glowness of the object in keyframes. We have also mentioned how the colors change and defined a box-shadow attribute for the animation in keyframes. We have defined the <script> tag in which we have included the JavaScript and JQuery code. We have used the trigger attribute. The trigger() method triggers a particular event and the default behavior of an event with regards to the chosen elements. We have used mouseover and mouseout properties. The mouseover and mouseout event fires when the user moves the mouse pointer over an element and out of an element respectively. We have used those events in conjunction with the start, end and pausing of the animation. Therefore, we can create complex animations using CSS3. Coding is an art which gets better with practice. Hence, we need to implement it practically in order to know the subtle nuances of HTML5 and CSS3. However, we can achieve that after a considerable amount of practice. However, we are just on the shore; the sea of knowledge is far beyond. In this article, we have covered a lot of HTML5 and CSS3 features. Instead of wading through loads of theory, the concepts in this article are explained in a practical manner using code samples to demonstrate the new features of HTML5 and CSS3. The code samples are such that you can copy the code (the entire code is written instead of code snippets) and execute it for better understanding. Transition, transformation, and animation are also explained in a lucid manner, and there is a gradual increase in the difficulty level throughout the article. By the end of the book, you will be thoroughly acquainted with HTML5 and CSS3, enabling you to design a web page using the included code samples with ease. Click on the following link to have a look at the book: http://www.packtpub.com/html5-and-css3-for-transition-transformation-animation/book Summary This article has discussed how HTML5 and CSS3 features can be used used in websites. There is a detailed discussion on the animations used in the websites offered by CSS3. Resources for Article: Further resources on this subject: Mobiles First – How and Why [Article] Creating an Animated Gauge with CSS3 [Article] HTML5 Canvas [Article]
Read more
  • 0
  • 0
  • 1999
article-image-exploring-financial-reporting-and-analysis
Packt
18 Nov 2013
3 min read
Save for later

Exploring Financial Reporting and Analysis

Packt
18 Nov 2013
3 min read
(For more resources related to this topic, see here.) The financial reporting is one of the basic implementation corner stone. The main principles for reporting are reliability of business information and the ability to get the right information at the right time for the right person. Reports that analyze ERP data in an expressive way represent the output of the ERP implementation; it is considered the cream of the implementation, the next level of value that solution stakeholders should target for. This ultimate outcome results from building all reports based on a single point of information. Planning reporting needs for ERP The Microsoft Dynamics AX implementation teamwork should challenge the management's reporting needs in the analysis phase of the implementation, with a particular focus on exploring the data required to build reports. These data requirements should then be cross-checked with the real data entry activities that end users will execute to ensure that business users will get vital information from the reports. The reporting levels are as follows: Operational Management Middle Management Top Management Understanding the information technology value chain The model of a management information system is most applicable to a businesses' Information Technology (IT) manager or Chief Information Officer (CIO). Business owners likely don't care as much about the specifics as long as these aspects of the solution deliver the required results. The basic layers of the value chain are as follows: Infrastructure Database Management Business Processes Business Intelligence Frontend Understanding Microsoft Dynamics AX information source blocks In this section, we will explore the information sources that eventually determine the strategic value of BI reporting and analytics. These are divided into three blocks: Detailed Transactions Block Business Intelligence Block Executive Dashboard Block Discovering Microsoft Dynamics AX reporting options The reporting options offered by Microsoft Dynamics AX are as follows: Inquiry Forms SQL Reporting Services (SSRS) Reports Original Transaction Original Document Audit Trail Reporting Currency Companies report their transactions in a specific currency that is known as accounting currency or local currency. It is normal to post transactions in a different currency, and this amount of money is translated to the home currency using the current exchange rate. Autoreports The autoreport wizard is a user-friendly tool. The end user can easily generate a report starting from every form in Microsoft Dynamics AX. The autoreport wizard helps the user to create a report based on the information in the form and save this report. Summary In this article, we have covered financial reporting from planning to consideration of reporting levels. We also covered important points that affect reporting quality. Resources for Article: Further resources on this subject: Types of services in Microsoft Dynamics AX 2012 [Article] Integrating with Microsoft Dynamics AX 2009 using BizTalk Adapter [Article] Working with Microsoft Dynamics AX and .NET: Part 1 [Article]
Read more
  • 0
  • 0
  • 5636

article-image-jaas-based-security-authentication-jsps
Packt
18 Nov 2013
9 min read
Save for later

JAAS-based security authentication on JSPs

Packt
18 Nov 2013
9 min read
(For more resources related to this topic, see here.) The deployment descriptor is the main configuration file of all the web applications. The container first looks out for the deployment descriptor before starting any application. The deployment descriptor is an XML file, web.xml, inside the WEB-INF folder. If you look at the XSD of the web.xml file, you can see the security-related schema. The schema can be accessed using the following URL: http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd. The following is the schema element available in the XSD: <xsd:element name="security-constraint" type="j2ee:securityconstraintType"/> <xsd:element name="login-config" type="j2ee:login-configType"/> <xsd:element name="security-role "type="j2ee:security-roleType"/> Getting ready You will need the following to demonstrate authentication and authorization: JBoss 7 Eclipse Indigo 3.7 Create a dynamic web project and name it Security Demo Create a package, com.servlets Create an XML file in the WebContent folder, jboss-web.xml Create two JSP pages, login.jsp and logoff.jsp How to do it... Perform the following steps to achieve JAAS-based security for JSPs: Edit the login.jsp file with the input fields j_username, j_password, and submit it to SecurityCheckerServlet: <%@ page contentType="text/html; charset=UTF-8" %> <%@ page language="java" %> <html > <HEAD> <TITLE>PACKT Login Form</TITLE> <SCRIPT> function submitForm() { var frm = document. myform; if( frm.j_username.value == "" ) { alert("please enter your username, its empty"); frm.j_username.focus(); return ; } if( frm.j_password.value == "" ) { alert("please enter the password,its empty"); frm.j_password.focus(); return ; } frm.submit(); } </SCRIPT> </HEAD> <BODY> <FORM name="myform" action="SecurityCheckerServlet" METHOD=get> <TABLE width="100%" border="0" cellspacing="0" cellpadding= "1" bgcolor="white"> <TABLE width="100%" border="0" cellspacing= "0" cellpadding="5"> <TR align="center"> <TD align="right" class="Prompt"></TD> <TD align="left"> <INPUT type="text" name="j_username" maxlength=20> </TD> </TR> <TR align="center"> <TD align="right" class="Prompt"> </TD> <TD align="left"> <INPUT type="password" name="j_password" maxlength=20 > <BR> <TR align="center"> <TD align="right" class="Prompt"> </TD> <TD align="left"> <input type="submit" onclick="javascript:submitForm();" value="Login"> </TD> </TR> </TABLE> </FORM> </BODY> </html> The j_username and j_password are the indicators of using form-based authentication. Let's modify the web.xml file to protect all the files that end with .jsp. If you are trying to access any JSP file, you would be given a login form, which in turn calls a SecurityCheckerServlet file to authenticate the user. You can also see role information is displayed. Update the web.xml file as shown in the following code snippet. We have used 2.5 xsd. The following code needs to be placed in between the webapp tag in the web.xml file: <display-name>jaas-jboss</display-name> <welcome-file-list> <welcome-file>index.html</welcome-file> <welcome-file>index.htm</welcome-file> <welcome-file>index.jsp</welcome-file> <welcome-file>default.html</welcome-file> <welcome-file>default.htm</welcome-file> <welcome-file>default.jsp</welcome-file> </welcome-file-list> <security-constraint> <web-resource-collection> <web-resource-name>something</web-resource-name> <description>Declarative security tests</description> <url-pattern>*.jsp</url-pattern> <http-method>HEAD</http-method> <http-method>GET</http-method> <http-method>POST</http-method> <http-method>PUT</http-method> <http-method>DELETE</http-method> </web-resource-collection> <auth-constraint> <role-name>role1</role-name> </auth-constraint> <user-data-constraint> <description>no description</description> <transport-guarantee>NONE</transport-guarantee> </user-data-constraint> </security-constraint> <login-config> <auth-method>FORM</auth-method> <form-login-config> <form-login-page>/login.jsp</form-login-page> <form-error-page>/logoff.jsp</form-error-page> </form-login-config> </login-config> <security-role> <description>some role</description> <role-name>role1</role-name> </security-role> <security-role> <description>packt managers</description> <role-name>manager</role-name> </security-role> <servlet> <description></description> <display-name>SecurityCheckerServlet</display-name> <servlet-name>SecurityCheckerServlet</servlet-name> <servlet-class>com.servlets.SecurityCheckerServlet </servlet-class> </servlet> <servlet-mapping> <servlet-name>SecurityCheckerServlet</servlet-name> <url-pattern>/SecurityCheckerServlet</url-pattern> </servlet-mapping> JAAS Security Checker and Credential Handler: Servlet is a security checker. Since we are using JAAS, the standard framework for authentication, in order to execute the following program you need to import org.jboss.security.SimplePrincipal and org.jboss.security.auth.callback.SecurityAssociationHandle and add all the necessary imports. In the following SecurityCheckerServlet, we are getting the input from the JSP file and passing it to the CallbackHandler. We are then passing the Handler object to the LoginContext class which has the login() method to do the authentication. On successful authentication, it will create Subject and Principal for the user, with user details. We are using iterator interface to iterate the LoginContext object to get the user details retrieved for authentication. In the SecurityCheckerServlet Class: package com.servlets; public class SecurityCheckerServlet extends HttpServlet { private static final long serialVersionUID = 1L; public SecurityCheckerServlet() { super(); } protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { char[] password = null; PrintWriter out=response.getWriter(); try { SecurityAssociationHandler handler = new SecurityAssociationHandler(); SimplePrincipal user = new SimplePrincipal(request.getParameter ("j_username")); password=request.getParameter("j_password"). toCharArray(); handler.setSecurityInfo(user, password); System.out.println("password"+password); CallbackHandler myHandler = new UserCredentialHandler(request.getParameter ("j_username"),request.getParameter ("j_password")); LoginContext lc = new LoginContext("other", handler); lc.login(); Subject subject = lc.getSubject(); Set principals = subject.getPrincipals(); List l=new ArrayList(); Iterator it = lc.getSubject().getPrincipals(). iterator(); while (it.hasNext()) { System.out.println("Authenticated: " + it.next().toString() + "<br>"); out.println("<b><html><body><font color='green'>Authenticated: " + request.getParameter("j_username")+" <br/>"+it.next().toString() + "<br/></font></b></body></html>"); } it = lc.getSubject().getPublicCredentials (Properties.class).iterator(); while (it.hasNext()) System.out.println(it.next().toString()); lc.logout(); } catch (Exception e) { out.println("<b><font color='red'>failed authenticatation.</font>-</b>"+e); } } protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { } } Create the UserCredentialHandler file: package com.servlets; class UserCredentialHandler implements CallbackHandler { private String user, pass; UserCredentialHandler(String user, String pass) { super(); this.user = user; this.pass = pass; } @Override public void handle(Callback[] callbacks) throws IOException, UnsupportedCallbackException { for (int i = 0; i < callbacks.length; i++) { if (callbacks[i] instanceof NameCallback) { NameCallback nc = (NameCallback) callbacks[i]; nc.setName(user); } else if (callbacks[i] instanceof PasswordCallback) { PasswordCallback pc = (PasswordCallback) callbacks[i]; pc.setPassword(pass.toCharArray()); } else { throw new UnsupportedCallbackException (callbacks[i], "Unrecognized Callback"); } } } } In the jboss-web.xml file: <?xml version="1.0" encoding="UTF-8"?> <jboss-web> <security-domain>java:/jaas/other</security-domain> </jboss-web> Other is the name of the application policy defined in the login-config.xml file. All these will be packed in as a .war file. Configuring the JBoss Application Server. Go to jboss-5.1.0.GAserver defaultconflogin-config.xml in JBoss. If you look at the file, you can see various configurations for database LDAP and a simple one using the properties file, which I have used in the following code snippet: <application-policy name="other"> <!-- A simple server login module, which can be used when the number of users is relatively small. It uses two properties files: users.properties, which holds users (key) and their password (value). roles.properties, which holds users (key) and a commaseparated list of their roles (value). The unauthenticatedIdentity property defines the name of the principal that will be used when a null username and password are presented as is the case for an unauthenticated web client or MDB. If you want to allow such users to be authenticated add the property, e.g., unauthenticatedIdentity="nobody" --> <authentication> <login-module code="org.jboss.security.auth.spi.UsersRoles LoginModule" flag="required"/> <module-option name="usersProperties"> users.properties</module-option> <module-option name="rolesProperties"> roles.properties</module-option> <module-option name="unauthenticatedIdentity"> nobody</module-option> </authentication> </application-policy> Create the users.properties file in the same folder. The following is the Users. properties file with username mapped with role. User.properties anjana=anjana123 roles.properties anjana=role1 Restart the server. How it works... JAAS consists of a set of interfaces to handle the authentication process. They are: The CallbackHandler and Callback interfaces The LoginModule interface LoginContext The CallbackHandler interface gets the user credentials. It processes the credentials and passes them to LoginModule, which authenticates the user. JAAS is container specific. Each container will have its own implementation, here we are using JBoss application server to demonstrate JAAS. In my previous example, I have explicitly called JASS interfaces. UserCredentialHandler implements the CallbackHandler interfaces. So, CallbackHandlers are storage spaces for the user credentials and the LoginModule authenticates the user. LoginContext bridges the CallbackHandler interface with LoginModule. It passes the user credentials to LoginModule interfaces for authentication: CallbackHandler myHandler = new UserCredentialHandler(request. getParameter("j_username"),request.getParameter("j_password")); LoginContext lc = new LoginContext("other", handler); lc.login() The web.xml file defines the security mechanisms and also points us to the protected resources in our application. The following screenshot shows a failed authentication window: The following screenshot shows a successful authentication window: Summary We introduced the JAAs-based mechanism of applying security to authenticate and authorize the users to the application Resources for Article: Further resources on this subject: Spring Security 3: Tips and Tricks [Article] Spring Security: Configuring Secure Passwords [Article] Migration to Spring Security 3 [Article]
Read more
  • 0
  • 0
  • 15486

Packt
15 Nov 2013
5 min read
Save for later

RESTful Web Services – Server-Sent Events (SSE)

Packt
15 Nov 2013
5 min read
Getting started Generally, the flow of web services is initiated by the client by sending a request for the resource to the server. This is the traditional way of consuming web services. Traditional Flow Here, the browser or Jersey client initiates the request for data from the server, and the server provides a response along with the data. Every time a client needs to initiate a request for the resource, the server may not have the capability to generate the data. This becomes difficult in an application where real-time data needs to be shown. Even though there is no new data over the server, the client needs to check for it every time. Nowadays, there is a requirement that the server needs to send some data without the client's request. For this to happen the client and server need to be connected, and the server can push the data to the client. This is why it is termed as Server-Sent Events. In these events, the connections created initially between the client and server is not released after the request. The server maintains the connection and pushes the data to the respective client when required. Server-Sent Event Flow In the Server-Sent Event Flow diagram initially, when a browser or a Jersey client initiates a request to establish a connection with the server using EventSource, the server is always in a listening mode for the new connection to be established. When a new connection from any EventSource is received, the server opens a new connection and maintains it in a queue. Maintaining a connection depends upon the implementation of business logic. SSE creates a single unidirectional connection. So, only a single connection is established between the client and server. After the connection is successfully established, the client is in the listening mode for new events from the server. Whenever any new event occurs on the server side, it will broadcast the event, along with the data to a specific open HTTP connection. In modern browsers that support HTML5, the onmessage method of EventSource is responsible for handling new events received from the server; whereas, in the case of Jersey clients, we have the onEvent method of EventSource, which handles new events from the server. Implementing Server-Sent Events (SSE) To use SSE, we need to register SseFeature on both the client and server sides. By doing so, the client/server gets connected to SseFeature to be used while traversing data over the network. SSE: Internal Working In the SSE: Internal Working diagram, we assume that the client/server is connected. When any new event is generated, the server initiates an OutboundEvent instance that will be responsible to have chunked output, which in turn will have a serialized data format. OutboundEventWriter is responsible to serialize the data on the server side. We need to specify the media type of the data in OutboundEvent. There are no restrictions of providing specific media types only. However, on the client side, InboundEvent is responsible for handling the incoming data from the server. Here, InboundEvent receives the chunked input that contains serialized data format. Using InbounEventReader, data is deserialized. Using SSEBroadCaster, we are able to broadcast events to multiple clients that are connected to the server. Let's look at the example, which shows how to create SSE web services and broadcast the events: @ApplicationPath("services") public class SSEApplication extends ResourceConfig { publicSSEApplication() { super(SSEResource.class, SseFeature.class); } } Here, we registered the SseFeature module and the SSEResource root-resource class to the server. private static final SseBroadcaster BROADCASTER = new SseBroadcaster(); …… @GET @Path("sseEvents") @Produces(SseFeature.SERVER_SENT_EVENTS) public EventOutput getConnection() { final EventOutput eventOutput = new EventOutput(); BROADCASTER.add(eventOutput); return eventOutput; } …… In the SSEResource root class, we need to create a resource method that will allow clients to establish the connection and persist accordingly. Here, we are maintaining the connection into the BROADCASTER instance in the SseBroadcaster class. EventOutput manages specific client connections. SseBroadcaster is simply responsible for accommodating a group of EventOutput; that is, the client's connection. …… @POST @Consumes(MediaType.APPLICATION_FORM_URLENCODED) public void post(@FormParam("name") String name) { BROADCASTER .broadcast(new OutboundEvent.Builder() .data(String.class, name) .build()); } …… When any post method is consumed, we create a new event and broadcast it to the client available in the BROADCASTER instance. The OutboundEvent instance will contain the data (MediaType, Object) method that is initialized with a specific media type and actual data. We can provide any media type to send data. By using the build() method, data is being serialized with the OutBoundEventWriter class internally. When the broadcast (OutboundEvent) is called, internally SseBroadcaster pushes data on all registered EventOutputs; that is, on clients connected to SseBroadcaster. At times, there's a scenario where the client/server has been connected and after sometime, the client gets disconnected. So, in this case, SseBroadcaster automatically handles the client connection; that is, it determines whether the connection needs to be maintained. When any client connection is closed, the broadcaster detects EventOutput and frees the connection and resources obtained by that EventOutput connection. Summary Thus we learned the difference between the traditional web service flow and SSE web service flow. We also covered how to create the SSE web services and implement the Jersey client in order to consume the SSE using different programmatic models. Useful Links: Setting up the most Popular Journal Articles in your Personalized Community in Liferay Portal Understanding WebSockets and Server-sent Events in Detail RESS - The idea and the Controversies
Read more
  • 0
  • 0
  • 6093
article-image-dom-and-qtp
Packt
15 Nov 2013
7 min read
Save for later

DOM and QTP

Packt
15 Nov 2013
7 min read
(For more resources related to this topic, see here.) QuickTest Object property for a Web object allows us to get a reference to the DOM object, and can perform any operation on a DOM object. For example, the following code shows that the object on the page allows retrieving the element by name username, that is, the textbox in the next step assigns the value as ashish. Set Obj = Browser("Tours").Page("Tours").Object.getElementsByName("userName) 'Get the length of the objects Print obj.length ' obj(0).value="ashish" The following code snippet shows various operations that can be performed using the Object property of the web object: 'Retrieve all the link elements in a web page Set links = Browser ("Mercury Tours").Page("Mercury Tours").Object.links 'Length property provide total number of links For i =0 to links.length -1 Print links(i).toString() 'Print the value of the links Next Get the web edit object and set the focus on it Set MyWebEdit = Browser("Tours").Page("Tours").WebEdit("username"). Object MyWebEdit.focus 'Retrieve the html element by its name obj = Browser("Tours").Page("Tours").Object. getElementsByName("userName" ) 'set the value to the retrieved object obj.value ="ashish" 'Retrieve the total number of the images in the web pages set images = Browser("Mercury Tours").Page("Mercury Tours").Object.images print images.length 'Clicking on the Image obj = Browser("Tours").Page("Mercury Tours").Object.getElementsByName("login") obj.click 'Selecting the value from the drop down using selectedIndex method Browser("Flight").Page("Flight).WebList("pass.0.meal").Object. selectedIndex =1 'Click on the check box Browser("Flight").Page("Flight").WebCheckBox("ticketLess").Object. click Firing an event QTP allows firing of the events on the web objects: Browser("The Fishing Website fishing").Page("The Fishing Website fishing").Link("Link").FireEvent "onmouseover" The following example uses the FireEvent method to trigger the onpropertychange event on a form: Browser("New Page").Page("New Page").WebElement("html tag:=Form").FireEvent "onpropertychange" QTP allows executing JavaScript code. There are two JavaScript functions that allow us to interact with web pages. We can retrieve objects and perform the actions on them or we can retrieve the properties from the element on the pages: RunScript executes the JavaScript code, passed as an argument to this function. The following example shows how the RunScript method calls the ImgCount method, which returns the number of images in the page: length = Browser("Mercury Tours").Page("Mercury Tours").RunScript("ImgCount(); function ImageCount() {var list = document.getElementsByTagName('img'); return list.length;}") print "The total number of images on the page is " & length RunScriptsFormFile uses the full path of the JavaScript files to execute it. The location can be an absolute or relative filesystem path or a quality center path. The following is a sample JavaScript file (logo.js): var myNode = document.getElementById("lga"); myNode.innerHTML = ''; Use the logo.js file, as shown in the following code: Browser("Browser").Page("page").RunScriptFromFile "c:logo.js" 'Check that the Web page behaves correctly If Browser("Browser").Page("page").Image("Image").Exist Then Reporter.ReportEvent micFail, "Failed to remove logo" End If The above example uses the RunScriptFromFile method to remove a DOM element from a web page and checks if the page still behaves correctly when the DOM element has been removed. Using XPath XPath allows navigating and finding elements and attributes in an HTML document. XPath uses path expressions to navigate in HTML documents. QTP allows XPath to create the object description; for example: xpath:=//input[@type='image' and contains(@name,'findFlights') In the following section, we will learn the various XPath terminologies and methodologies to find the objects using XPath. XPath terminology XPath uses various terms to define elements and their relationships among HTML elements, as shown in the following table: Atomic values Atomic values are nodes with no children or parent Ancestors A node's parent, parent's parent, and so on Descendants A node's children, children's children, and so on Parent Each element and attribute has one parent Children Element nodes may have zero, one, or more children Siblings Nodes that have the same parent Selecting nodes A path expression allows selecting nodes in a document. The commonly used path expressions are shown in the following table: Symbol Meaning /(slash) Select elements relative to the root node //(double slash) Select nodes in the document from the current node that match the selection irrespective of its position .(dot) Represents the current node .. Represents the parent of the current node @ Represents an attribute nodename Selects all nodes with the name "nodename" Slash (/) is used in the beginning and it defines an absolute path; for example, /html/head/title returns the title tag. It defines ancestor and descendant relationships if used in the middle; for example, //div/table that returns the div containing a table. Double slash (//) is used to find a node in any location; for example, //table returns all the tables. It defines a descendant relationship if used in the middle; for example, /html//title returns the title tag, which is descendant of the html tag. Refer to the following table to see a few more examples with their meanings: Expression Meaning //a Find all anchor tags //a//img List the images that are inside a link //img/@alt Show all the alt tags //a/@href Show the href attribute for every link //a[@*] Anchor tab with any attribute //title/text() or /html/head/title/text() Get the title of a page //img[@alt] List the images that have alt tags //img[not(@alt)] List the images that don't have alt tags //*[@id='mainContent'] Get an element with a particular CSS ID //div [not(@id="div1")] Make an array element from the XPath //p/.. Selects the parent element of p (paragraph) child XXX[@att] Selects all the child elements of XXX with an attribute named att ./@* for example, //script/./@* Finds all attribute values of current element Predicates A predicate is embedded in square brackets and is used to find out specific node(s) or a node that contains a specific value: //p[@align]: This allows finding all the tags that have align attribute value as center //img[@alt]: This allows finding all the img (image) tags that contain the alt tag //table[@border]: This allows finding all the table tags that contain border attributes //table[@border >1]: This finds the table with border value greater than 1 Retrieve the table row using the complete path: //body/div/table/tbody/tr[1] Get the name of the parent of //body/div/table/.. (parent of the table tag) //body/div/table/..[name()] Path expression Result //div/p[1] Selects the first paragraph element that is the child of the div element //div/p [last()] Selects the last paragraph element that is the child of the div element //div/p[last()-1] Selects the second last paragraph element that is the child of the div element //div/p[position()<3] Selects the first two paragraph elements that are children of the div element //script[@language] Selects all script element(s) with an attribute named as language //script[@language='javascript'] Selects all the script elements that have an attribute named language with a value of JavaScript //div/p[text()>45.00] Selects all the paragraph elements of the div element that have a text element with a value greater than 45.00 Selecting unknown nodes Apart from selecting the specific nodes in XPath, XPath allows us to select the group of HTML elements using *, @, and node() functions. * represents an element node @ represents the attribute node() represents any node The previous mentioned elements allow selecting the unknown nodes; for example: /div/* selects all the child nodes of a div element //* selects all the elements in a document //script[@*] selects all the title elements which contain attributes
Read more
  • 0
  • 0
  • 6022

article-image-application-performance
Packt
15 Nov 2013
8 min read
Save for later

Application Performance

Packt
15 Nov 2013
8 min read
(For more resources related to this topic, see here.) Data sizing The cost of abstractions in terms of data size plays an important role. For example, whether or not a data element can fit into a processor cache line depends directly upon its size. On a Linux system, we can find out the cache line size and other parameters by inspecting the values in the files under /sys/devices/system/cpu/cpu0/cache/. Another concern we generally find with data sizing is how much data we are holding at a time in the heap. GC has direct consequences on the application's performance. While processing data, often we do not really need all the data we hold on to. Consider the example of generating a summary report of sold items for a certain period (months) of time. After the subperiod (month wise), summary data is computed. We do not need the item details anymore, hence it's better to remove the unwanted data while we add the summaries. This is shown in the following example: (defn summarize [daily-data] ; daily-data is a map (let [s (items-summary (:items daily-data))] (-> daily-data (select-keys [:digest :invoices]) ; we keep only the required key/val pairs (assoc :summary s)))) ;; now inside report generation code (-> (fetch-items period-from period-to :interval-day) (map summarize) generate-report) Had we not used select-keys in the preceding summarize function, it would have returned a map with extra summary data along with all the other existing keys in the map. Now, such a thing is often combined with lazy sequences. So, for this scheme to work, it is important not to hold on to the head of the lazy sequence. Reduced serialization An I/O channel is a common source of latency. The perils of over-serialization cannot be overstated. Whether we read or write data from a data source over an I/O channel, all of that data needs to be prepared, encoded, serialized, de-serialized, and parsed before being worked on. It is better for every step to have less data involved in order to lower the overhead. Where there is no I/O involved, such as in-process communication, it generally makes no sense to serialize. A common example of over-serialization is encountered while working with SQL databases. Often, there are common SQL query functions that fetch all columns of a table or a relation—they are called by various functions that implement the business logic. Fetching data that we do not need is wasteful and detrimental to the performance for the same reason that we discussed in the preceding paragraph. While it may seem more work to write one SQL statement and one database query function for each use case, it pays off with better performance. Code that uses NoSQL databases is also subject to this anti-pattern—we have to take care to fetch only what we need even though it may lead to additional code. There's a pitfall to be aware of when reducing serialization. Often, some information needs to be inferred in absence of the serialized data. In such cases where some of the serialization is dropped so that we can infer other information, we must compare the cost of inference versus the serialization overhead. The comparison may not be necessarily done per operation, but rather on the whole. Then, we can consider the resources we can allocate in order to achieve capacities for various parts of our systems. Chunking to reduce memory pressure What happens when we slurp a text file regardless of its size? The contents of the entire file will sit in the JVM heap. If the file is larger than the JVM heap capacity, the JVM will terminate by throwing OutOfMemoryError. If the file is large but not large enough to force the JVM into an OOM error, it leaves a relatively smaller JVM heap space for other operations in the application to continue. A similar situation takes place when we carry out any operation disregarding the JVM heap capacity. Fortunately, this can be fixed by reading data in chunks and processing them before reading further. Sizing for file/network operations Let us take the example of a data ingestion process where a semi-automated job uploads large Comma Separated File (CSV) files via the File Transfer Protocol (FTP) to a file server, and another automated job, which is written in Clojure, runs periodically to detect the arrival of files via the Network File System (NFS). After detecting a new file, the Clojure program processes the file, updates the result in a database, and archives the file. The program detects and processes several files concurrently. The size of the CSV files is not known in advance, but the format is predefined. As per the preceding description, one potential problem is that since there could be multiple files being processed concurrently, how do we distribute the JVM heap among the concurrent file-processing jobs? Another issue could be that the operating system imposes a limit on how many files can be opened at a time; on Unix-like systems, you can use the ulimit command to extend the limit. We cannot arbitrarily slurp the CSV file contents—we must limit each job to a certain amount of memory and also limit the number of jobs that can run concurrently. At the same time, we cannot read a very small number of rows from a file at a time because this may impact performance. (def ^:const K 1024) ;; create the buffered reader using custom 128K buffer-size (-> filename java.io.FileInputStream java.io.InputStreamReader (java.io.BufferedReader (* K 128))) Fortunately, we can specify the buffer size when reading from a file or even from a network stream so as to tune the memory usage and performance as appropriate. In the preceding code example, we explicitly set the buffer size of the reader to facilitate the same. Sizing for JDBC query results Java's interface standard for SQL databases, JDBC (which is technically not an acronym), supports fetch-size for fetching query results via JDBC drivers. The default fetch size depends on the JDBC driver. Most JDBC drivers keep a low default value so as to avoid high memory usage and attain internal performance optimization. A notable exception to this norm is the MySQL JDBC driver that completely fetches and stores all rows in memory by default. (require '[clojure.java.jdbc :as jdbc]) ;; using prepare-statement directly (we rarely use it directly, shown just for demo) (with-open [stmt (jdbc/prepare-statement conn sql :fetch-size 1000 max-rows 9000) rset (resultset-seq (.executeQuery stmt))] (vec rset)) ;; using query (query db [{:fetch-size 1000} "SELECT empno FROM emp WHERE country=?" 1]) When using the Clojure Contrib library java.jdbc (https://github.com/clojure/java.jdbc as of Version 0.3.0), the fetch size can be set while preparing a statement as shown in the preceding example. The fetch size does not guarantee proportional latency; however, it can be used safely for memory sizing. We must test any performance-impacting latency changes due to fetch size at different loads and use cases for the particular database and JDBC driver. Besides fetch-size, we can also pass the max-rowsargument to limit the maximum rows to be returned by a query. However, this implies that the extra rows will be truncated from the result, not that the database will internally limit the number of rows to realize. Resource pooling There are several types of resources on the JVM that are rather expensive to initialize. Examples are HTTP connections, execution threads, JDBC connections, and so on. The Java API recognizes such resources and has built-in support for creating a pool of some of those resources so that the consumer code borrows a resource from a pool when required and at the end of the job simply returns it to the pool. Java's thread pools and JDBC data sources are prominent examples. The idea is to preserve the initialized objects for reuse. Even when Java does not support pooling of a resource type directly, you can always create a pool abstraction around custom expensive resources. The pooling technique is common in I/O activities, but it can be equally applicable to non-I/O purposes where the initialization cost is high. Summary Designing an application for performance should be based on the use cases and patterns of anticipated system load and behavior. Measuring performance is extremely important to guide optimization in the process. Fortunately, there are several well-known optimization patterns to tap into, such as resource pooling, and data sizing. Thus we analysed the performance optimization using these patterns. Resources for Article: Further resources on this subject: Improving Performance with Parallel Programming [Article] Debugging Java Programs using JDB [Article] IBM WebSphere Application Server Security: A Threefold View [Article]
Read more
  • 0
  • 0
  • 7320
Modal Close icon
Modal Close icon