Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Save more on your purchases! discount-offer-chevron-icon
Savings automatically calculated. No voucher code required.
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Newsletter Hub
Free Learning
Arrow right icon
timer SALE ENDS IN
0 Days
:
00 Hours
:
00 Minutes
:
00 Seconds

How-To Tutorials

7010 Articles
article-image-execution-test-plans
Packt
28 Oct 2014
23 min read
Save for later

Execution of Test Plans

Packt
28 Oct 2014
23 min read
In this article by Bayo Erinle, author of JMeter Cookbook, we will cover the following recipes: Using the View Results Tree listener Using the Aggregate Report listener Debugging with Debug Sampler Using Constant Throughput Timer Using the JSR223 postprocessor Analyzing Response Times Over Time Analyzing transactions per second (For more resources related to this topic, see here.) One of the critical aspects of performance testing is knowing the right tools to use to attain your desired targets. Even when you settle on a tool, it is helpful to understand its features, component sets, and extensions, and appropriately apply them when needed. In this article, we will go over some helpful components that will aid you in recording robust and realistic test plans while effectively analyzing reported results. We will also cover some components to help you debug test plans. Using the View Results Tree listener One of the most often used listeners in JMeter is the View Results Tree listener. This listener shows a tree of all sample responses, giving you quick navigation of any sample's response time, response codes, response content, and so on. The component offers several ways to view the response data, some of which allow you to debug CSS/jQuery, regular expressions, and XPath queries, among other things. In addition, the component offers the ability to save responses to file, in case you need to store them for offline viewing or run some other processes on them. Along with the various bundled testers, the component provides a search functionality that allows you to quickly search for the responses of relevant items. How to do it… In this recipe, we will cover how to add the View Results Tree listener to a test plan and then use its in-built testers to test the response and derive expressions that we can use in postprocessor components. Perform the following steps: Launch JMeter. Add Thread Group to the test plan by navigating to Test Plan | Add | Threads (Users) | Thread Group. Add HTTP Request to the thread group by navigating to Thread Group | Add | Sampler | HTTP Request. Fill in the following details:    Server Name or IP: dailyjs.com Add the View Results Tree listener to the test plan by navigating to Test Plan | Add | Listener | View Results Tree. Save and run the test plan. Once done, navigate to the View Results Tree component and click on the Response Data tab. Observe some of the built-in renders. Switch to the HTML render view by clicking on the dropdown and use the search textbox to search for any word on the page. Switch to the HTML (download resources) render view by clicking on the dropdown. Switch to the XML render view by clicking on the dropdown. Notice the entire HTML DOM structure is presented as the XML node elements. Switch to the RegExp Tester render view by clicking on the dropdown and try out some regular expression queries. Switch to the XPath Query Tester render view and try out some XPath queries. Switch to the CSS/jQuery Tester render view and try out some jQuery queries, for example, selecting all links inside divs marked with a class preview (Selector: div.preview a, Attribute: href, CSS/jQuery Implementation: JSOUP). How it works… As your test plans execute, the View Result Tree listener reports each sampler in your test plans individually. The Sampler Result tab of the component gives you a summarized view of the request and response including information such as load time, latency, response headers, body content sizes, response code and messages, response header content, and so on. The Request tab shows the actual request that got fulfilled by the sampler, which could be any of the acceptable requests the server can fulfill (for example, GET, POST, PUT, DELETE, and so on) along with details of the request headers. Finally, the Response Data tab gives the rendered view of the response received back from the server. The component includes several built-in renders along with tester components (CSS/JQuery, RegExp, and XPath) that allow us to test and come up with the right expressions or queries needed to use in postprocessor components within our test plans. This is a huge time saver as it means we don't have to exercise the same tests repeatedly to nail down such expressions. There's more… As with most things bundled with JMeter, additional view renders can be added to the View Result Tree component. The defaults included are Document, HTML, HTML (download resources), JSON, Text, and XML. Should any of these not suit your needs, you can create additional ones by implementing org.apache.jmeter.visualizers.ResultRender interface and/or extending org.apache.jmeter.visualizers.SamplerResultTab abstract class, bundling up the compiled classes as a JAR file and placing them in the $JMETER_HOME/lib/ext directory to make them available for JMeter. The View Result Tree listener consumes a lot of memory and CPU resources, and should not be used during load testing. Use it only to debug and validate the test plans. See also The Debugging with Debug Sampler recipe The detailed component reference for the View Results Tree listener can be found at http://jmeter.apache.org/usermanual/component_reference.html#View_Results_Tree Using the Aggregate Report listener Another often used listener in JMeter is the Aggregate Report listener. This listener creates a row for each uniquely named request in the test plan. Each row gives a summarized view of useful information including Request Count, Average, Median, Min, Max, 90% Line, Error Rate, Throughput, Requests/second, and KB/sec. The 90% Line column is particularly worth paying close attention to as you execute your tests. This figure gives you the time it takes for the majority of threads/users to execute a particular request. It is measured in milliseconds. Higher numbers here are indicative of slow requests and/or components within the application under test. Equally important is the Error % column, which reports the failure rate of each sampled request. It is reasonable to have some level of failure when exercising test runs, but too high a number is an indication of either errors in scripts or certain components in the application under test. Finally, of interest to stack holders might be the number of requests per second, which the Throughput column reports. The throughput values are approximate and let you know just how many requests per second the server is able to handle. How to do it… In this recipe, we will cover how to add an Aggregate Report listener to a test plan and then see the summarized view of our execution: Launch JMeter. Open the ch7_shoutbox.jmx script bundled with the code samples. Alternatively, you can download it from https://github.com/jmeter-cookbook/bundled-code/scripts/ch7/ch7_shoutbox.jmx. Add the Aggregate Report listener to Thread Group by navigating to Thread Group | Add | Listener | Aggregate Report. Save and run the test plan. Observe the real-time summary of results in the listener as the test proceeds. How it works… As your test plans execute, the Aggregate Report listener reports each sampler in your test plan on a separate row. Each row is packed with useful information. The Label column reflects the sample name, # Samples gives a count of each sampler, and Average, Mean, Min, and Max all give you the respective times of each sampler. As mentioned earlier, you should pay close attention to the 90% Line and Error % columns. This can help quickly pinpoint problematic components within the application under test and/or scripts. The Throughput column gives an idea of the responsiveness of the application under test and/or server. This can also be indicative of the capacity of the underlying server that the application under test runs on. This entire process is demonstrated in the following screenshot: Using the Aggregate Report listener See also http://jmeter.apache.org/usermanual/component_reference.html#Summary_Report Debugging with Debug Sampler Often, in the process of recording a new test plan or modifying an existing one, you will need to debug the scripts to finally get your desired results. Without such capabilities, the process will be a mix of trial and error and will become a time-consuming exercise. Debug Sampler is a nifty little component that generates a sample containing the values of all JMeter variables and properties. The generated values can then be seen in the Response Data tab of the View Results Tree listener. As such, to use this component, you need to have a View Results Tree listener added to your test plan. This component is especially useful when dealing with postprocessor components as it helps to verify the correct or expected values that were extracted during the test run. How to do it… In this recipe, we will see how we can use Debug Sampler to debug a postprocessor in our test plans. Perform the following steps: Launch JMeter. Open the prerecorded script ch7_debug_sampler.jmx bundled with the book. Alternatively, you can download it from http://git.io/debug_sampler. Add Debug Sampler to the test Thread Group by navigating to Thread Group | Add | Sampler | Debug Sampler. Save and run the test. Navigate to the View Results Tree listener component. Switch to RegExp Tester by clicking on the dropdown. Observe the response data of the Get All Requests sampler. What we want is a regular expression that will help us extract the ID of entries within this response. After a few attempts, we settle at "id":(d+). Enable all the currently disabled samplers, that is, Request/Create Holiday Request, Modify Holiday, Get All Requests, and Delete Holiday Request. You can achieve this by selecting all the disabled components, right-clicking on them, and clicking on Enable. Add the Regular Expression Extractor postprocessor to the Request/Create Holiday Request sampler by navigating to Request/Create Holiday Request | Add | Post Processors | Regular Expression Extractor. Fill in the following details:    Reference Name: id    Regular Expression: "id":(d+)    Template: $1$    Match No.: 0    Default Value: NOT_FOUND Save and rerun the test. Observe the ID of the newly created holiday request and whether it was correctly extracted and reported in Debug Sampler. How it works… Our goal was to test a REST API endpoint that allows us to list, modify, and delete existing resources or create new ones. When we create a new resource, the identifier (ID) is autogenerated from the server. To perform any other operations on the newly created resource, we need to grab its autogenerated ID, store that in a JMeter variable, and use it further down the execution chain. In step 7, we were able to observe the format of the server response for the resource when we executed the Get All Requests sampler. With the aid of RegExp Tester, we were able to nail down the right regular expression to use to extract the ID of a resource, that is, "id":(d+). Armed with this information, we added a Regular Expression Extractor postprocessor component to the Request/Create Holiday Request sampler and used the derived expression to get the ID of the newly created resource. We then used the ID stored in JMeter to modify and delete the resource down the execution chain. After test completion, with the help of Debug Sampler, we were able to verify whether the resource ID was properly extracted by the Regular Expression Extractor component and stored in JMeter as an ID variable. Using Constant Throughput Timer While running test simulations, it is sometimes necessary to be able to specify the throughput in terms of the number of requests per minute. This is the function of Constant Throughput Timer. This component introduces pauses to the test plan in such a way as to keep the throughput as close as possible to the target value specified. Though the name implies it is constant, various factors affect the behavior, such as server capacity, other timers or time-consuming elements in the test plan, and so on. As a result, the targeted throughput could be lowered. How to do it… In this recipe, we will add Constant Throughput Timer to our test plan and see how we can specify the expected throughput with it. Perform the following steps: Launch JMeter. Open the prerecorded script ch7_constant_throughput.jmx bundled with the book. Alternatively, you can download it from http://git.io/constant_throughput. Add Constant Throughput Timer to Thread Group by navigating to Thread Group | Add | Timer | Constant Throughput Timer. Fill in the following details:    Target throughput (in samples per minute): 200    Calculate Throughput based on: this thread only Save and run the test plan. Allow the test to run for about 5 minutes. Observe the result in the Aggregate Result listener as the test is going on. Stop the test manually as it is currently set to run forever. How it works… The goal of the Constant Throughput Timer component is to get your test plan samples as close as possible to a specified desired throughput. It achieves this by introducing variable pauses to the test plan in such a manner that will keep numbers as close as possible to the desired throughput. That said, throughput will be lowered if the server resources of the system under test can't handle the load. Also, other elements (for example, other timers, the number of specified threads, and so on) within the test plan can affect attaining the desired throughput. In our recipe, we have specified the throughput rate to be calculated based on a single thread, but Constant Throughput Timer also allows throughput to be calculated based on all active threads and all active threads in the current thread group. Each of these settings can be used to alter the behavior of the desired throughput. As a rule of thumb, avoid using other timers at the same time you use Constant Throughput Timer, since you'll not achieve the desired throughput. See also The Using Throughput Shaping Timer recipe http://jmeter.apache.org/usermanual/component_reference.html#timers Using the JSR223 postprocessor The JSR223 postprocessor allows you to use precompiled scripts within test plans. The fact that the scripts are compiled before they are actually used brings a significant performance boost compared to other postprocessors. This also allows a variety of programming languages to be used, including Java, Groovy, BeanShell, JEXL, and so on. This allows us to harness the powerful language features in those languages within our test plans. JSR223 components, for example, could help us tackle preprocessor or postprocessor elements and samplers, allowing us more control over how elements are extracted from responses and stored as JMeter variables. How to do it… In this recipe, we will see how to use a JSR223 postprocessor within our test plan. We have chosen Groovy (http://groovy.codehaus.org/) as our choice of scripting language, but any of the other supporting languages will do: Download the standard set of plugins from http://jmeter-plugins.org/. Install the plugins by doing the following:    Extract the ZIP archive to the location of your chosen directory    Copy the lib folder in the extracted directory into the $JMETER_HOME directory Download the groovy-all JAR file from http://devbucket-afriq.s3.amazonaws.com/jmeter-cookbook/groovy-all-2.3.3.jar and add it to the $JMETER_HOME/lib directory. Launch JMeter. Add Thread Group by navigating to Test Plan | Add | Threads(Users) | Thread Group. Add Dummy Sampler to Thread Group by navigating to Thread Group | Add | Sampler | jp@gc - Dummy Sampler. In the Response Data text area, add the following content: <records>   <car name='HSV Maloo' make='Holden' year='2006'>       <country>Australia</country>       <record type='speed'>Production Pickup Truck with speed of 271kph</record>   </car>   <car name='P50' make='Peel' year='1962'>       <country>Isle of Man</country>       <record type='size'>Smallest Street-Legal Car at 99cm wide and 59 kg in weight</record>   </car>   <car name='Royale' make='Bugatti' year='1931'>       <country>France</country>       <record type='price'>Most Valuable Car at $15 million</record>   </car></records> Download the Groovy script file from http://git.io/8jCXMg to any location of your choice. Alternatively, you can get it from the code sample bundle accompanying the book (ch7_jsr223.groovy). Add JSR223 PostProcessor as a child of Dummy Sampler by navigating to jp@gc - Dummy Sampler | Add | Post Processors | JSR223 PostProcessor. Select Groovy as the language of choice in the Language drop-down box. In the File Name textbox, put in the absolute path to where the Groovy script file is, for example, /tmp/scripts/ch7/ch7_jsr223.groovy. Add the View Results Tree listener to the test plan by navigating to Test Plan | Add | Listener | View Results Tree. Add Debug Sampler to Thread Group by navigating to Thread Group | Add | Sampler | Debug Sampler. Save and run the test. Observe the Response Data tab of Debug Sampler and see how we now have the JMeter variables car_0, car_1, and car_2, all extracted from the Response Data tab and populated by our JSR223 postprocessor component. How it works… JMeter exposes certain variables to the JSR223 component, allowing it to get hold of sample details and information, perform logic operations, and store the results as JMeter variables. The exposed attributes include Log, Label, Filename, Parameters, args[], ctx, vars, props, prev, sampler, and OUT. Each of these allows access to important and useful information that can be used during the postprocessing of sampler responses. The log gives access to Logger (an instance of an Apache Commons Logging log instance; see http://bit.ly/1xt5dmd), which can be used to write log statements to the logfile. The Label and Filename attributes give us access to the sample label and script file name respectively. The Parameters and args[] attributes give us access to parameters sent to the script. The ctx attribute gives access to the current thread's JMeter context (http://bit.ly/1lM31MC). vars gives access to write values into JMeter variables (http://bit.ly/1o5DDBr), exposing them to the result of the test plan. The props attribute gives us access to JMeterProperties. The sampler attribute gives us access to the current sampler while OUT allows us to write log statements to the standard output, that is, System.out. Finally, the prev sample gives access to previous sample results (http://bit.ly/1rKn8Cs), allowing us to get useful information such as the response data, headers, assertion results, and so on. In our script, we made use of the prev and vars attributes. With prev, we were able to get hold of the XML response from the sample. Using Groovy's XmlSlurper (http://bit.ly/1AoRMnb), we were able to effortlessly process the XML response and compose the interesting bits, storing them as JMeter variables using the vars attribute. Using this technique, we are able to accomplish tasks that might have otherwise been cumbersome to achieve using any other postprocessor elements we have seen in other recipes. We are able to take full advantage of the language features of any chosen scripting language. In our case, we used Groovy, but any other supported scripting languages you are comfortable with will do as well. See also http://jmeter.apache.org/api http://jmeter.apache.org/usermanual/component_reference.html#BSF_PostProcessor http://jmeter.apache.org/api/org/apache/jmeter/threads/JMeterContext.html http://jmeter.apache.org/api/org/apache/jmeter/threads/JMeterVariables.html http://jmeter.apache.org/api/org/apache/jmeter/samplers/SampleResult.html Analyzing Response Times Over Time An important aspect of performance testing is the response times of the application under test. As such, it is often important to visually see the response times over a duration of time as the test plan is executed. Out of the box, JMeter comes with the Response Time Graph listener for this purpose, but it is limited and lacks some features. Such features include the ability to focus on a particular sample when viewing chat results, controlling the granularity of timeline values, selectively choosing which samples appear or not in the resulting chart, controlling whether to use relative graphs or not, and so on. To address all these and more, the Response Times Over Time listener extension from the JMeter plugins project comes to the rescue. It shines in areas where the Response Time Graph falls short. How to do it… In this recipe, we will see how to use the Response Times Over Time listener extension in our test plan and get the response times of our samples over time. Perform the following steps: Download the standard set of plugins from http://jmeter-plugins.org/. Install the plugins by doing the following:    Extract the ZIP archive to the location of your chosen directory    Copy the lib folder in the extracted directory into the $JMETER_HOME directory Launch JMeter. Open any of your existing prerecorded scripts or record a new one. Alternatively, you can open the ch7_response_times_over_time.jmx script accompanying the book or download it from http://git.io/response_times_over_time. Add the Response Times Over Time listener to the test plan by navigating to Test Plan | Add | Listener | jp@gc - Response Times Over Time. Save and execute the test plan. View the resulting chart in the tab by clicking on the Response Times Over Time component. Observe the time elapsed on the x axis and the response time in milliseconds on the y axis for all samples contained in the test plan. Navigate to the Rows tab and exclude some of the samples from the chart by unchecking the selection boxes next to the samples. Switch back to the Chart tab and observe that the chart now reflects your changes, allowing you to focus in on interested samples. Switch to the Settings tab and see all the available configuration options. Change some options and repeat the test execution. This is shown in the following screenshot: Analyzing Response Times Over Time How it works… Just like its name implies, the Response Times Over Time listener extension displays the average response time in milliseconds for each sampler in the test plan. It comes with various configuration options that allow you to customize the resulting graph to your heart's content. More importantly, it allows you to focus in on specific samples in your test plan, helping you pinpoint potential bottlenecks or problematic modules within the application under test. For graphs to be more meaningful, it helps to give samples sensible descriptive names and tweak the granularity of the elapsed time to a higher number in the Settings tab if you have long running tests. After test execution, data of any chart can also be exported to a CSV file for further analysis or use as you desire. Any listener that charts results will have some impact on performance and shouldn't be used during high volume load testing. Analyzing transactions per second Sometimes we are tasked with testing backend services, application program interfaces (APIs), or some other components that may not necessarily have a graphical user interface (GUI) attached to it, for example, a classic web application. At such times, the measure of the responsiveness of the module, for example, will be how many transactions per second it can withstand before slowness is observed. For example, Transactions Per Second (TPS) is useful information for stakeholders who are providing services that can be consumed by various third-party components or other services. Good examples of these include the Google search engine, which can be consumed by third-parties, and the Twitter and Facebook APIs, which allow developers to integrate their application with Twitter and Facebook respectively. The Transactions Per Second listener extension component from the JMeter plugins project allows us to measure the transactions per second. It plots a chart of the transactions per second over an elapsed duration of time. How to do it… In this recipe, we will see how to use the Transactions Per Second listener extension in our test plan and get the transactions per second for a test API service: Download the standard set of plugins from http://jmeter-plugins.org/. Install the plugins by doing the following:    Extract the ZIP archive to the location of your chosen directory    Copy the lib folder in the extracted directory into the $JMETER_HOME directory Launch JMeter. Open the ch7_transaction_per_sec.jmx script accompanying the book or download it from http://git.io/trans_per_sec. Add the Transactions Per Second listener to the test plan by navigating to Test Plan | Add | Listener | jp@gc - Transactions per Second. Save and execute the test plan. View the resulting chart in the tab by clicking on the Transactions Per Second component. Observe the time elapsed on the x axis and the transactions/sec on the y axis for all samples contained in the test plan. Navigate to the Rows tab and exclude some of the samples from the chart by unchecking the selection boxes next to the samples. Switch back to the Chart tab and observe that the chart now reflects your changes, allowing you to focus in on interesting samples. Switch to the Settings tab and see all the available configuration options. Change some options and repeat the test execution. How it works… The Transactions Per Second listener extension displays the transactions per second for each sample in the test plan by counting the number of successfully completed transactions each second. It comes with various configuration options that allow you to customize the resulting graph. Such configurations allow you to focus in on specific samples of interest in your test plan, helping you to get at impending bottlenecks within the application under test. It is helpful to give your samples sensible descriptive names to help make better sense of the resulting graphs and data points. This is shown in the following screenshot: Analyzing Transactions per Second Summary In this article, you learned how to build a test plan using the steps mentioned in the recipe. Furthermore, you saw how to debug and analyze the result of a test plan after building it. Resources for Article: Further resources on this subject: Functional Testing with JMeter [article] Performance Testing Fundamentals [article] Common performance issues [article]
Read more
  • 0
  • 0
  • 3356

article-image-scaling-friendly-font-rendering-distance-fields
Packt
28 Oct 2014
8 min read
Save for later

Scaling friendly font rendering with distance fields

Packt
28 Oct 2014
8 min read
This article by David Saltares Márquez and Alberto Cejas Sánchez, the authors of Libgdx Cross-platform Game Development Cookbook, describes how we can generate a distance field font and render it in Libgdx. As a bitmap font is scaled up, it becomes blurry due to linear interpolation. It is possible to tell the underlying texture to use the nearest filter, but the result will be pixelated. Additionally, until now, if you wanted big and small pieces of text using the same font, you would have had to export it twice at different sizes. The output texture gets bigger rather quickly, and this is a memory problem. (For more resources related to this topic, see here.) Distance field fonts is a technique that enables us to scale monochromatic textures without losing out on quality, which is pretty amazing. It was first published by Valve (Half Life, Team Fortress…) in 2007. It involves an offline preprocessing step and a very simple fragment shader when rendering, but the results are great and there is very little performance penalty. You also get to use smaller textures! In this article, we will cover the entire process of how to generate a distance field font and how to render it in Libgdx. Getting ready For this, we will load the data/fonts/oswald-distance.fnt and data/fonts/oswald.fnt files. To generate the fonts, Hiero is needed, so download the latest Libgdx package from http://libgdx.badlogicgames.com/releases and unzip it. Make sure the samples projects are in your workspace. Please visit the link https://github.com/siondream/libgdx-cookbook to download the sample projects which you will need. How to do it… First, we need to generate a distance field font with Hiero. Then, a special fragment shader is required to finally render scaling-friendly text in Libgdx. Generating distance field fonts with Hiero Open up Hiero from the command line. Linux and Mac users only need to replace semicolons with colons and back slashes with forward slashes: java -cp gdx.jar;gdx-natives.jar;gdx-backend-lwjgl.jar;gdx-backend-lwjgl-natives.jar;extensions gdx-toolsgdx-tools.jar com.badlogic.gdx.tools.hiero.Hiero Select the font using either the System or File options. This time, you don't need a really big size; the point is to generate a small texture and still be able to render text at high resolutions, maintaining quality. We have chosen 32 this time. Remove the Color effect, and add a white Distance field effect. Set the Spread effect; the thicker the font, the bigger should be this value. For Oswald, 4.0 seems to be a sweet spot. To cater to the spread, you need to set a matching padding. Since this will make the characters render further apart, you need to counterbalance this by the setting the X and Y values to twice the negative padding. Finally, set the Scale to be the same as the font size. Hiero will struggle to render the charset, which is why we wait until the end to set this property. Generate the font by going to File | Save BMFont files (text).... The following is the Hiero UI showing a font texture with a Distance field effect applied to it: Distance field fonts shader We cannot use the distance field texture to render text for obvious reasons—it is blurry! A special shader is needed to get the information from the distance field and transform it into the final, smoothed result. The vertex shader found in data/fonts/font.vert is simple. The magic takes place in the fragment shader, found in data/fonts/font.frag and explained later. First, we sample the alpha value from the texture for the current fragment and call it distance. Then, we use the smoothstep() function to obtain the actual fragment alpha. If distance is between 0.5-smoothing and 0.5+smoothing, Hermite interpolation will be used. If the distance is greater than 0.5+smoothing, the function returns 1.0, and if the distance is smaller than 0.5-smoothing, it will return 0.0. The code is as follows: #ifdef GL_ES precision mediump float; precision mediump int; #endif   uniform sampler2D u_texture;   varying vec4 v_color; varying vec2 v_texCoord;   const float smoothing = 1.0/128.0;   void main() {    float distance = texture2D(u_texture, v_texCoord).a;    float alpha = smoothstep(0.5 - smoothing, 0.5 + smoothing, distance);    gl_FragColor = vec4(v_color.rgb, alpha * v_color.a); } The smoothing constant determines how hard or soft the edges of the font will be. Feel free to play around with the value and render fonts at different sizes to see the results. You could also make it uniform and configure it from the code. Rendering distance field fonts in Libgdx Let's move on to DistanceFieldFontSample.java, where we have two BitmapFont instances: normalFont (pointing to data/fonts/oswald.fnt) and distanceShader (pointing to data/fonts/oswald-distance.fnt). This will help us illustrate the difference between the two approaches. Additionally, we have a ShaderProgram instance for our previously defined shader. In the create() method, we instantiate both the fonts and shader normally: normalFont = new BitmapFont(Gdx.files.internal("data/fonts/oswald.fnt")); normalFont.setColor(0.0f, 0.56f, 1.0f, 1.0f); normalFont.setScale(4.5f);   distanceFont = new BitmapFont(Gdx.files.internal("data/fonts/oswald-distance.fnt")); distanceFont.setColor(0.0f, 0.56f, 1.0f, 1.0f); distanceFont.setScale(4.5f);   fontShader = new ShaderProgram(Gdx.files.internal("data/fonts/font.vert"), Gdx.files.internal("data/fonts/font.frag"));   if (!fontShader.isCompiled()) {    Gdx.app.error(DistanceFieldFontSample.class.getSimpleName(), "Shader compilation failed:n" + fontShader.getLog()); } We need to make sure that the texture our distanceFont just loaded is using linear filtering: distanceFont.getRegion().getTexture().setFilter(TextureFilter.Linear, TextureFilter.Linear); Remember to free up resources in the dispose() method, and let's get on with render(). First, we render some text with the regular font using the default shader, and right after this, we do the same with the distance field font using our awesome shader: batch.begin(); batch.setShader(null); normalFont.draw(batch, "Distance field fonts!", 20.0f, VIRTUAL_HEIGHT - 50.0f);   batch.setShader(fontShader); distanceFont.draw(batch, "Distance field fonts!", 20.0f, VIRTUAL_HEIGHT - 250.0f); batch.end(); The results are pretty obvious; it is a huge win of memory and quality over a very small price of GPU time. Try increasing the font size even more and be amazed at the results! You might have to slightly tweak the smoothing constant in the shader code though: How it works… Let's explain the fundamentals behind this technique. However, for a thorough explanation, we recommend that you read the original paper by Chris Green from Valve (http://www.valvesoftware.com/publications/2007/SIGGRAPH2007_AlphaTestedMagnification.pdf). A distance field is a derived representation of a monochromatic texture. For each pixel in the output, the generator determines whether the corresponding one in the original is colored or not. Then, it examines its neighborhood to determine the 2D distance in pixels, to a pixel with the opposite state. Once the distance is calculated, it is mapped to a [0, 1] range, with 0 being the maximum negative distance and 1 being the maximum positive distance. A value of 0.5 indicates the exact edge of the shape. The following figure illustrates this process: Within Libgdx, the BitmapFont class uses SpriteBatch to render text normally, only this time, it is using a texture with a Distance field effect applied to it. The fragment shader is responsible for performing a smoothing pass. If the alpha value for this fragment is higher than 0.5, it can be considered as in; it will be out in any other case: This produces a clean result. There's more… We have applied distance fields to text, but we have also mentioned that it can work with monochromatic images. It is simple; you need to generate a low resolution distance field transform. Luckily enough, Libgdx comes with a tool that does just this. Open a command-line window, access your Libgdx package folder and enter the following command: java -cp gdx.jar;gdx-natives.jar;gdx-backend-lwjgl.jar;gdx-backend-lwjgl-natives.jar;extensionsgdx-tools gdx-tools.jar com.badlogic.gdx.tools.distancefield.DistanceFieldGenerator The distance field font generator takes the following parameters: --color: This parameter is in hexadecimal RGB format; the default is ffffff --downscale: This is the factor by which the original texture will be downscaled --spread: This is the edge scan distance, expressed in terms of the input Take a look at this example: java […] DistanceFieldGenerator --color ff0000 --downscale 32 --spread 128 texture.png texture-distance.png Alternatively, you can use the gdx-smart-font library to handle scaling. It is a simpler but a bit more limited solution (https://github.com/jrenner/gdx-smart-font). Summary In this article, we have covered the entire process of how to generate a distance field font and how to render it in Libgdx. Further resources on this subject: Cross-platform Development - Build Once, Deploy Anywhere [Article] Getting into the Store [Article] Adding Animations [Article]
Read more
  • 0
  • 0
  • 8868

article-image-clustering-k-means
Packt
27 Oct 2014
9 min read
Save for later

Clustering with K-Means

Packt
27 Oct 2014
9 min read
In this article by Gavin Hackeling, the author of Mastering Machine Learning with scikit-Learn, we will discuss an unsupervised learning task called clustering. Clustering is used to find groups of similar observations within a set of unlabeled data. We will discuss the K-Means clustering algorithm, apply it to an image compression problem, and learn to measure its performance. Finally, we will work through a semi-supervised learning problem that combines clustering with classification. Clustering, or cluster analysis, is the task of grouping observations such that members of the same group, or cluster, are more similar to each other by some metric than they are to the members of the other clusters. As with supervised learning, we will represent an observation as an n-dimensional vector. For example, assume that your training data consists of the samples plotted in the following figure: Clustering might reveal the following two groups, indicated by squares and circles. Clustering could also reveal the following four groups: Clustering is commonly used to explore a data set. Social networks can be clustered to identify communities, and to suggest missing connections between people. In biology, clustering is used to find groups of genes with similar expression patterns. Recommendation systems sometimes employ clustering to identify products or media that might appeal to a user. In marketing, clustering is used to find segments of similar consumers. In the following sections we will work through an example of using the K-Means algorithm to cluster a data set. Clustering with the K-Means Algorithm The K-Means algorithm is a clustering method that is popular because of its speed and scalability. K-Means is an iterative process of moving the centers of the clusters, or the centroids, to the mean position of their constituent points, and re-assigning instances to their closest clusters. The titular K is a hyperparameter that specifies the number of clusters that should be created; K-Means automatically assigns observations to clusters but cannot determine the appropriate number of clusters. K must be a positive integer that is less than the number of instances in the training set. Sometimes the number of clusters is specified by the clustering problem's context. For example, a company that manufactures shoes might know that it is able to support manufacturing three new models. To understand what groups of customers to target with each model, it surveys customers and creates three clusters from the results. That is, the value of K was specified by the problem's context. Other problems may not require a specific number of clusters, and the optimal number of clusters may be ambiguous. We will discuss a heuristic for estimating the optimal number of clusters called the elbow method later in this article. The parameters of K-Means are the positions of the clusters' centroids and the observations that are assigned to each cluster. Like generalized linear models and decision trees, the optimal values of K-Means' parameters are found by minimizing a cost function. The cost function for K-Means is given by the following equation: where µk is the centroid for cluster k. The cost function sums the distortions of the clusters. Each cluster's distortion is equal to the sum of the squared distances between its centroid and its constituent instances. The distortion is small for compact clusters, and large for clusters that contain scattered instances. The parameters that minimize the cost function are learned through an iterative process of assigning observations to clusters and then moving the clusters. First, the clusters' centroids are initialized to random positions. In practice, setting the centroids' positions equal to the positions of randomly selected observations yields the best results. During each iteration, K-Means assigns observations to the cluster that they are closest to, and then moves the centroids to their assigned observations' mean location. Let's work through an example by hand using the training data shown in the following table. Instance X0 X1 1 7 5 2 5 7 3 7 7 4 3 3 5 4 6 6 1 4 7 0 0 8 2 2 9 8 7 10 6 8 11 5 5 12 3 7 There are two explanatory variables; each instance has two features. The instances are plotted in the following figure. Assume that K-Means initializes the centroid for the first cluster to the fifth instance and the centroid for the second cluster to the eleventh instance. For each instance, we will calculate its distance to both centroids, and assign it to the cluster with the closest centroid. The initial assignments are shown in the “Cluster” column of the following table. Instance X0 X1 C1 distance C2 distance Last Cluster Cluster Changed? 1 7 5 3.16228 2 None C2 Yes 2 5 7 1.41421 2 None C1 Yes 3 7 7 3.16228 2.82843 None C2 Yes 4 3 3 3.16228 2.82843 None C2 Yes 5 4 6 0 1.41421 None C1 Yes 6 1 4 3.60555 4.12311 None C1 Yes 7 0 0 7.21110 7.07107 None C2 Yes 8 2 2 4.47214 4.24264 None C2 Yes 9 8 7 4.12311 3.60555 None C2 Yes 10 6 8 2.82843 3.16228 None C1 Yes 11 5 5 1.41421 0 None C2 Yes 12 3 7 1.41421 2.82843 None C1 Yes C1 centroid 4 6           C2 centroid 5 5           The plotted centroids and the initial cluster assignments are shown in the following graph. Instances assigned to the first cluster are marked with “Xs”, and instances assigned to the second cluster are marked with dots. The markers for the centroids are larger than the markers for the instances. Now we will move both centroids to the means of their constituent instances, re-calculate the distances of the training instances to the centroids, and re-assign the instances to the closest centroids. Instance X0 X1 C1 distance C2 distance Last Cluster New Cluster Changed? 1 7 5 3.492850 2.575394 C2 C2 No 2 5 7 1.341641 2.889107 C1 C1 No 3 7 7 3.255764 3.749830 C2 C1 Yes 4 3 3 3.492850 1.943067 C2 C2 No 5 4 6 0.447214 1.943067 C1 C1 No 6 1 4 3.687818 3.574285 C1 C2 Yes 7 0 0 7.443118 6.169378 C2 C2 No 8 2 2 4.753946 3.347250 C2 C2 No 9 8 7 4.242641 4.463000 C2 C1 Yes 10 6 8 2.720294 4.113194 C1 C1 No 11 5 5 1.843909 0.958315 C2 C2 No 12 3 7 1 3.260775 C1 C1 No C1 centroid 3.8 6.4           C2 centroid 4.571429 4.142857           The new clusters are plotted in the following graph. Note that the centroids are diverging, and several instances have changed their assignments. Now we will move the centroids to the means of their constituents' locations again, and re-assign the instances to their nearest centroids. The centroids continue to diverge, as shown in the following figure. None of the instances' centroid assignments will change in the next iteration; K-Means will continue iterating until some stopping criteria is satisfied. Usually, this criteria is either a threshold for the difference between the values of the cost function for subsequent iterations, or a threshold for the change in the positions of the centroids between subsequent iterations. If these stopping criteria are small enough, K-Means will converge on an optimum. This optimum will not necessarily be the global optimum. Local Optima Recall that K-Means initially sets the positions of the clusters' centroids to the positions of randomly selected observations. Sometimes the random initialization is unlucky, and the centroids are set to positions that cause K-Means to converge to a local optimum. For example, assume that K-Means randomly initializes two cluster centroids to the following positions: K-Means will eventually converge on a local optimum like that shown in the following figure. These clusters may be informative, but it is more likely that the top and bottom groups of observations are more informative clusters. To avoid local optima, K-Means is often repeated dozens or hundreds of times. In each iteration, it is randomly initialized to different starting cluster positions. The initialization that minimizes the cost function best is selected. The Elbow Method If K is not specified by the problem's context, the optimal number of clusters can be estimated using a technique called the elbow method. The elbow method plots the value of the cost function produced by different values of K. As K increases, the average distortion will decrease; each cluster will have fewer constituent instances, and the instances will be closer to their respective centroids. However, the improvements to the average distortion will decline as K increases. The value of K at which the improvement to the distortion declines the most is called the elbow. Let's use the elbow method to choose the number of clusters for a data set. The following scatter plot visualizes a data set with two obvious clusters. We will calculate and plot the mean distortion of the clusters for each value of K from one to ten with the following: >>> import numpy as np>>> from sklearn.cluster import KMeans>>> from scipy.spatial.distance import cdist>>> import matplotlib.pyplot as plt>>> cluster1 = np.random.uniform(0.5, 1.5, (2, 10))>>> cluster2 = np.random.uniform(3.5, 4.5, (2, 10))>>> X = np.hstack((cluster1, cluster2)).T>>> X = np.vstack((x, y)).T>>> K = range(1, 10)>>> meandistortions = []>>> for k in K:>>> kmeans = KMeans(n_clusters=k)>>> kmeans.fit(X)>>> meandistortions.append(sum(np.min(cdist(X, kmeans.cluster_centers_, 'euclidean'), axis=1)) / X.shape[0])>>> plt.plot(K, meandistortions, 'bx-')>>> plt.xlabel('k')>>> plt.ylabel('Average distortion')>>> plt.title('Selecting k with the Elbow Method')>>> plt.show() The average distortion improves rapidly as we increase K from one to two. There is little improvement for values of K greater than two. Now let's use the elbow method on the following data set with three clusters: The following is the elbow plot for the data set. From this we can see that the rate of improvement to the average distortion declines the most when adding a fourth cluster. That is, the elbow method confirms that K should be set to three for this data set. Summary In this article we explained what clustering is and we talked about the two methods available for clustering Resources for Article: Further resources on this subject: Machine Learning in IPython with scikit-learn [Article] Machine Learning in Bioinformatics [Article] Specialized Machine Learning Topics [Article]
Read more
  • 0
  • 0
  • 10283

article-image-emr-architecture
Packt
27 Oct 2014
6 min read
Save for later

The EMR Architecture

Packt
27 Oct 2014
6 min read
This article is written by Amarkant Singh and Vijay Rayapati, the authors of Learning Big Data with Amazon Elastic MapReduce. The goal of this article is to introduce you to the EMR architecture and EMR use cases. (For more resources related to this topic, see here.) Traditionally, very few companies had access to large-scale infrastructure to build Big Data applications. However, cloud computing has democratized the access to infrastructure allowing developers and companies to quickly perform new experiments without worrying about the need for setting up or scaling infrastructure. A cloud provides an infrastructure as a service platform to allow businesses to build applications and host them reliably with scalable infrastructure. It includes a variety of application-level services to help developers to accelerate their development and deployment times. Amazon EMR is one of the hosted services provided by AWS and is built on top of a scalable AWS infrastructure to build Big Data applications. The EMR architecture Let's get familiar with the EMR. This section outlines the key concepts of EMR. Hadoop offers distributed processing by using the MapReduce framework for execution of tasks on a set of servers or compute nodes (also known as a cluster). One of the nodes in the Hadoop cluster will be controlling the distribution of tasks to other nodes and it's called the Master Node. The nodes executing the tasks using MapReduce are called Slave Nodes: Amazon EMR is designed to work with many other AWS services such as S3 for input/output data storage, DynamoDB, and Redshift for output data. EMR uses AWS CloudWatch metrics to monitor the cluster performance and raise notifications for user-specified alarms. We can create on-demand Hadoop clusters using EMR while storing the input and output data in S3 without worrying about managing a 24*7 cluster or HDFS for data storage. The Amazon EMR job flow is shown in the following diagram: Types of nodes Amazon EMR provides three different roles for the servers or nodes in the cluster and they map to the Hadoop roles of master and slave nodes. When you create an EMR cluster, then it's called a Job Flow, which has been created to execute a set of jobs or job steps one after the other: Master node: This node controls and manages the cluster. It distributes the MapReduce tasks to nodes in the cluster and monitors the status of task execution. Every EMR cluster will have only one master node in a master instance group. Core nodes: These nodes will execute MapReduce tasks and provide HDFS for storing the data related to task execution. The EMR cluster will have core nodes as part of it in a core instance group. The core node is related to the slave node in Hadoop. So, basically these nodes have two-fold responsibility: the first one is to execute the map and reduce tasks allocated by the master and the second is to hold the data blocks. Task nodes: These nodes are used for only MapReduce task execution and they are optional while launching the EMR cluster. The task node is related to the slave node in Hadoop and is part of a task instance group in EMR. When you scale down your clusters, you cannot remove any core nodes. This is because EMR doesn't want to let you lose your data blocks. You can remove nodes from a task group while scaling down your cluster. You should also be using only task instance groups to have spot instances, as spot instances can be taken away as per your bid price and you would not want to lose your data blocks. You can launch a cluster having just one node, that is, with just one master node and no other nodes. In that case, the same node will act as both master and core nodes. For simplicity, you can assume a node as EC2 server in EMR. EMR use cases Amazon EMR can be used to build a variety of applications such as recommendation engines, data analysis, log processing, event/click stream analysis, data transformations (ETL), fraud detection, scientific simulations, genomics, financial analysis, or data correlation in various industries. The following section outlines some of the use cases in detail. Web log processing We can use EMR to process logs to understand the usage of content such as video, file downloads, top web URLs accessed by end users, user consumption from different parts of the world, and many more. We can process any web or mobile application logs using EMR to understand specific business insights relevant for your business. We can move all our web access application or mobile logs to Amazon S3 for analysis using EMR even if we are not using AWS for running our production applications. Clickstream analysis By using clickstream analysis, we can segment users into different groups and understand their behaviors with respect to advertisements or application usage. Ad networks or advertisers can perform clickstream analysis on ad-impression logs to deliver more effective campaigns or advertisements to end users. Reports generated from this analysis can include various metrics such as source traffic distribution, purchase funnel, lead source ROI, and abandoned carts among others. Product recommendation engine Recommendation engines can be built using EMR for e-commerce, retail, or web businesses. Many of the e-commerce businesses have a large inventory of products across different categories while regularly adding new products or categories. It will be very difficult for end users to search and identify the products quickly. With recommendation engines, we can help end users to quickly find relevant products or suggest products based on what they are viewing and so on. We may also want to notify users via an e-mail based on their past purchase behavior. Scientific simulations When you need distributed processing with large-scale infrastructure for scientific or research simulations, EMR can be of great help. We can quickly launch large clusters in a matter of minutes and install specific MapReduce programs for analysis using EMR. AWS also offers genomics datasets for free on S3. Data transformations We can perform complex extract, transform, and load (ETL) processes using EMR for either data analysis or data warehousing needs. It can be as simple as transforming XML file data into JSON data for further usage or moving all financial transaction records of a bank into a common date-time format for archiving purposes. You can also use EMR to move data between different systems in AWS such as DynamoDB, Redshift, S3, and many more. Summary In this article, we learned about the EMR architecture. We understood the concepts related to EMR for various node types in detail. Resources for Article: Further resources on this subject: Introduction to MapReduce [Article] Understanding MapReduce [Article] HDFS and MapReduce [Article]
Read more
  • 0
  • 0
  • 7334

article-image-animation-and-unity3d-physics
Packt
27 Oct 2014
5 min read
Save for later

Animation and Unity3D Physics

Packt
27 Oct 2014
5 min read
In this article, written by K. Aava Rani, author of the book, Learning Unity Physics, you will learn to use Physics in animation creation. We will see that there are several animations that can be easily handled by Unity3D's Physics. During development, you will come to know that working with animations and Physics is easy in Unity3D. You will find the combination of Physics and animation very interesting. We are going to cover the following topics: Interpolate and Extrapolate Cloth component and its uses in animation (For more resources related to this topic, see here.) Developing simple and complex animations As mentioned earlier, you will learn how to handle and create simple and complex animations using Physics, for example, creating a rope animation and hanging ball. Let's start with the Physics properties of a Rigidbody component, which help in syncing animation. Interpolate and Extrapolate Unity3D offers a way that its Rigidbody component can help in the syncing of animation. Using the interpolation and extrapolation properties, we sync animations. Interpolation is not only for animation, it works also with Rigidbody. Let's see in detail how interpolation and extrapolation work: Create a new scene and save it. Create a Cube game object and apply Rigidbody on it. Look at the Inspector panel shown in the following screenshot. On clicking Interpolate, a drop-down list that contains three options will appear, which are None, Interpolate, and Extrapolate. By selecting any one of them, we can apply the feature. In interpolation, the position of an object is calculated by the current update time, moving it backwards one Physics update delta time. Delta time or delta timing is a concept used among programmers in relation to frame rate and time. For more details, check out http://docs.unity3d.com/ScriptReference/Time-deltaTime.html. If you look closely, you will observe that there are at least two Physics updates, which are as follows: Ahead of the chosen time Behind the chosen time Unity interpolates between these two updates to get a position for the update position. So, we can say that the interpolation is actually lagging behind one Physics update. The second option is Extrapolate, which is to use for extrapolation. In this case, Unity predicts the future position for the object. Although, this does not show any lag, but incorrect prediction sometime causes a visual jitter. One more important component that is widely used to animate cloth is the Cloth component. Here, you will learn about its properties and how to use it. The Cloth component To make animation easy, Unity provides an interactive component called Cloth. In the GameObject menu, you can directly create the Cloth game object. Have a look at the following screenshot: Unity also provides Cloth components in its Physics sections. To apply this, let's look at an example: Create a new scene and save it. Create a Plane game object. (We can also create a Cloth game object.) Navigate to Component | Physics and choose InteractiveCloth. As shown in the following screenshot, you will see the following properties in the Inspector panel: Let's have a look at the properties one by one. Blending Stiffness and Stretching Stiffness define the blending and stretching stiffness of the Cloth while Damping defines the damp motion of the Cloth. Using the Thickness property, we decide thickness of the Cloth, which ranges from 0.001 to 10,000. If we enable the Use Gravity property, it will affect the Cloth simulation. Similarly, if we enable Self Collision, it allows the Cloth to collide with itself. For a constant or random acceleration, we apply the External Acceleration and Random Acceleration properties, respectively. World Velocity Scale decides movement of the character in the world, which will affect the Cloth vertices. The higher the value, more movement of the character will affect. World Acceleration works similarly. The Interactive Cloth component depends on the Cloth Renderer components. Lots of Cloth components in a game reduces the performance of game. To simulate clothing in characters, we use the Skinned Cloth component. Important points while using the Cloth component The following are the important points to remember while using the Cloth component: Cloth simulation will not generate tangents. So, if you are using a tangent dependent shader, the lighting will look wrong for a Cloth component that has been moved from its initial position. We cannot directly change the transform of moving Cloth game object. This is not supported. Disabling the Cloth before changing the transform is supported. The SkinnedCloth component works together with SkinnedMeshRenderer to simulate clothing on a character. As shown in the following screenshot, we can apply Skinned Cloth: As you can see in the following screenshot, there are different properties that we can use to get the desired effect: We can disable or enable the Skinned Cloth component at any time to turn it on or off. Summary In this article, you learned about how interpolation and extrapolation work. We also learned about Cloth component and its uses in animation with an example. Resources for Article: Further resources on this subject: Animations in Cocos2d-x [article] Unity Networking – The Pong Game [article] The physics engine [article]
Read more
  • 0
  • 0
  • 17974

article-image-data-visualization
Packt
27 Oct 2014
8 min read
Save for later

Data visualization

Packt
27 Oct 2014
8 min read
Data visualization is one of the most important tasks in data science track. Through effective visualization we can easily uncover underlying pattern among variables with doing any sophisticated statistical analysis. In this cookbook we have focused on graphical analysis using R in a very simple way with each independent example. We have covered default R functionality along with more advance visualization techniques such as lattice, ggplot2, and three-dimensional plots. Readers will not only learn the code to produce the graph but also learn why certain code has been written with specific examples. R Graphs Cookbook Second Edition written by Jaynal Abedin and Hrishi V. Mittal is such a book where the user will learn how to produce various graphs using R and how to customize them and finally how to make ready for publication. This practical recipe book starts with very brief description about R graphics system and then gradually goes through basic to advance plots with examples. Beside the R default graphics this recipe book introduces advance graphic system such as lattice and ggplot2; the grammar of graphics. We have also provided examples on how to inspect large dataset using advanced visualization such as tableplot and three dimensional visualizations. We also cover the following topics: How to create various types of bar charts using default R functions, lattice and ggplot2 How to produce density plots along with histograms using lattice and ggplot2 and customized them for publication How to produce graphs of frequency tabulated data How to inspect large dataset by simultaneously visualizing numeric and categorical variables in a single plot How to annotate graphs using ggplot2 (For more resources related to this topic, see here.) This recipe book is targeted to those reader groups who already exposed to R programming and want to learn effective graphics with the power of R and its various libraries. This hands-on guide starts with very short introduction to R graphics system and then gets straight to the point – actually creating graphs, instead of just theoretical learning. Each recipe is specifically tailored to full fill reader’s appetite for visually representing the data in the best way possible. Now, we will present few examples so that you can have an idea about the content of this recipe book: The ggplot2 R package is based on The Grammar of Graphics by Leland Wilkinson, Springer). Using this package, we can produce a variety of traditional graphics, and the user can produce their customized graphs as well. The beauty of this package is in its layered graphics facilities; through the use of layered graphics utilities, we can produce almost any kind of data visualization. Recently, ggplot2 is the most searched keyword in the R community, including the most popular R blog (www.r-bloggers.com). The comprehensive theme system allows the user to produce publication quality graphs with a variety of themes of choice. If we want to explain this package in a single sentence, then we can say that if whatever we can think about data visualization can be structured in a data frame, the visualization is a matter of few seconds. In the specific chapter on ggplot2 , we will see different examples and use themes to produce publication quality graphs. However, in this introductory chapter, we will show you one of the important features of the ggplot2 package that produces various types of graphs. The main function is ggplot(), but with the help of a different geom function, we can easily produce different types of graphs, such as the following: geom_point(): This will create scatter plot geom_line(): This will create a line chart geom_bar(): This will create a bar chart geom_boxplot(): This will create a box plot geom_text(): This will write certain text inside the plot area Now, we will see a simple example of the use of different geom functions with the default R mtcars dataset: # loading ggplot2 library library(ggplot2) # creating a basic ggplot object p <- ggplot(data=mtcars) # Creating scatter plot of mpg and disp variable p1 <- p+geom_point(aes(x=disp,y=mpg)) # creating line chart from the same ggplot object but different # geom function p2 <- p+geom_line(aes(x=disp,y=mpg)) # creating bar chart of mpg variable p3 <- p+geom_bar(aes(x=mpg)) # creating boxplot of mpg over gear p4 <- p+geom_boxplot(aes(x=factor(gear),y=mpg)) # writing certain text into the scatter plot p5 <- p1+geom_text(x=200,y=25,label="Scatter plot") The visualization of the preceding five plot will look like the following figure: Visualizing an empirical Cumulative Distribution function The empirical Cumulative Distribution function (CDF) is the non-parametric maximum-likelihood estimation of the CDF. In this recipe, we will see how the empirical CDF can be produced. Getting ready To produce this plot, we need to use the latticeExtra library. We will use the simulated dataset as shown in the following code: # Set a seed value to make the data reproducible set.seed(12345) qqdata <-data.frame(disA=rnorm(n=100,mean=20,sd=3),                disB=rnorm(n=100,mean=25,sd=4),                disC=rnorm(n=100,mean=15,sd=1.5),                age=sample((c(1,2,3,4)),size=100,replace=T),                sex=sample(c("Male","Female"),size=100,replace=T),                 econ_status=sample(c("Poor","Middle","Rich"),                size=100,replace=T)) How to do it… To plot an empirical CDF, we first need to call the latticeExtra library (note that this library has a dependency on RColorBrewer). Now, to plot the empirical CDF, we can use the following simple code: library(latticeExtra) ecdfplot(~disA|sex,data=qqdata) Graph annotation with ggplot To produce publication-quality data visualization, we often need to annotate the graph with various texts, symbols, or even shapes. In this recipe, we will see how we can easily annotate an existing graph. Getting ready In this recipe, we will use the disA and disD variables from ggplotdata. Let's call ggplotdata for this recipe. We also need to call the grid and gridExtra libraries for this recipe. How to do it... In this recipe, we will execute the following annotation on an existing scatter plot. So, the whole procedure will be as follows: Create a scatter plot Add customized text within the plot Highlight certain region to indicate extreme values Draw a line segment with an arrow within the scatter plot to indicate a single extreme observation Now, we will implement each of the steps one by one: library(grid) library(gridExtra) # creating scatter plot and print it annotation_obj <- ggplot(data=ggplotdata,aes(x=disA,y=disD))+geom_point() annotation_obj # Adding custom text at (18,29) position annotation_obj1 <- annotation_obj + annotate(geom="text",x=18,y=29,label="Extreme value",size=3) annotation_obj1 # Highlight certain regions with a box annotation_obj2 <- annotation_obj1+ annotate("rect", xmin = 24, xmax = 27,ymin=17,ymax=22,alpha = .2) annotation_obj2 # Drawing line segment with arrow annotation_obj3 <- annotation_obj2+ annotate("segment",x = 16,xend=17.5,y=25,yend=27.5,colour="red", arrow = arrow(length = unit(0.5, "cm")),size=2) annotation_obj3 The preceding four steps are displayed in the following single graph: How it works... The annotate() function takes input of a geom such as “segment”, “text” etc, and then it takes another input regarding position of that geom that is where to draw or where to place.. In this particular recipe, we used three geom instances, such as text to write customized text within the plot, rect to highlight a certain region in the plot, and segment to draw an arrow. The alpha argument represents the transparency of the region and size argument to represent the size of the text and line width of the line segment. Summary This article just gives a sample recipe of what kind of recipes are included in the book, and how the structure of each recipe is. Resources for Article: Further resources on this subject: Using R for Statistics, Research, and Graphics [Article] First steps with R [Article] Aspects of Data Manipulation in R [Article]
Read more
  • 0
  • 0
  • 3747
Unlock access to the largest independent learning library in Tech for FREE!
Get unlimited access to 7500+ expert-authored eBooks and video courses covering every tech area you can think of.
Renews at €18.99/month. Cancel anytime
article-image-basic-concepts-proxmox-virtual-environment
Packt
27 Oct 2014
20 min read
Save for later

Basic Concepts of Proxmox Virtual Environment

Packt
27 Oct 2014
20 min read
 This article by Simon M.C. Cheng, author of the book Proxmox High Availability, will show you some basic concepts of Proxmox VE before actually using it, including the technology used, basic administration, and some options available during set up. The following topics are going to be covered in this chapter: An explanation of server virtualization used by Proxmox VE An introduction of basic administrative tools available in Proxmox VE An explanation of different virtualization modes and storage options (For more resources related to this topic, see here.) Introduction to server virtualization Have you ever heard about cloud computing? It is a hot topic in the IT industry and claims that you can allocate nearly unlimited computer resources in a pay-as-you-go basis. Are you not curious to know on how they are able to provide such a service? The underlying technology that allows them to be able to provide such a service is hardware virtualization. Depending on the kind of processor used, there are three different types of virtualizations available: full virtualization, para-virtualization, and hardware assisted virtualization. Full virtualization: In this the VMM is placed under Ring 0 while the virtualized guest OS is installed under Ring 1. However, some system calls can only be executed under Ring 0. Therefore, a process called binary translation is used to translate such system calls, and thus, the performance is degraded. In this mode, the guest OS does not know it is being virtualized, so it does not require kernel modification. Here is a simple structure for this type of virtualization: Para-virtualization: It is very similar to full virtualization, but custom drivers are installed on the guest OS in order to access CPU resources without downgrading to Ring 1. So, the performance of the guest OS is near to that of the physical machine because the translation process is not needed, but the guest OS requires a modified kernel. Thus, the guest cannot run operating system different from the host operation system. The following shows the structure of this virtualization: Hardware assisted virtualization: CPU manufacturers introduce a new functionality for a virtualized platform, Intel VT-x and AMD-V. The ring level 0 to 3 is categorized into non-root modes, and a new level, -1, is introduced as the root mode. The guest OS is now installed to Ring 0, which means it can access hardware directly. Because it does not need a custom API to make system calls under Ring 0, no kernel modification is needed. The following diagram shows you the structure of the virtualization mode: Comparing server virtualization software We have discussed why we need to learn server virtualization and how virtualization works, so are you curious on how many major virtualization software are in the market? What are the differences between them? Let's take a deep look at it: Proxmox VE: As mentioned earlier, Proxmox VE is an open source hypervisor based on GNU/Linux (Debian based) with a RHEL-based kernel and published under GNU AGPL v3. It differs from the alternatives virtualization software as Proxmox provides a central web-based management without further installation. The underlying technologies used are Open Virtuozzo (OpenVZ) and Kernel-based Virtual Machine (KVM), which will be described in Version 1.6. Subscription plans are available for accessing enterprise repository, software updates, and user support. XenServer: It is a native hypervisor based on GNU/Linux developed by the Xen Project and published under GNU AGPL v2 as open source. For XenServer, a concept of domain is used for all virtual machines, and the most privileged domain (for example, have direct access to hardware)—dom0, is used by the hypervisor to manage other domU virtual machines. It supports para-virtualization, which allows a user to run virtualized guests on a CPU without support for virtualization; for example, no Intel VT-x or AMD-V is needed. Amazon Web Service (AWS) is a production sample of using XenServer. VMware ESX/ESXi: This is a bare-metal hypervisor developed by VMware based on a customized kernel called vmkernel, which is a microkernel developed by VMware.The difference between ESX and ESXi is that ESXi is a free version of ESX with some resource limitations. ESX has a hardware compatibility list that includes many drivers for network cards and SCSI cards. An extra hardware driver can be added to the base installation media if needed. On top of the para-virtualization and hardware-assisted virtualization, ESX provides full virtualization as another option.There are two management tools available: vSphere client and vCenter server. VSphere client is enough for normal administration operation on one ESX while vCenter server allows the user to manage multiple ESXs, including the configuration of advanced features such as high availability and live migration. Hyper-V server: This is a proprietary virtualization platform produced by Microsoft Corporation running under Windows platform starting from Windows Server 2008. If you are mainly using a Windows platform as your virtualized guest, it is recommended that you use Hyper-V, especially if you have enabled an Active Directory domain services.Hyper-V provides better migration options to users; it not only provides live migration, but also provides unlimited guest movements between hosts. The benefit of the features of an Active Directory domain is that Hyper-V provides replica on virtual machines, which allows a user to copy a specific VM from the source site to the target site asynchronously via a WAN or a secure VPN. Virtualization options explained in Proxmox VE There are two types of virtualization available in Proxmox: OpenVZ and KVM. OpenVZ is an operating-system-level virtualization based on the GNU/Linux kernel and the host operation system. Theoretically, OpenVZ is not a type of virtualization but more like the jail concept in Linux. Since a patched Linux kernel is needed, only Linux guests can be created. All guests are called containers that share the same kernel and architecture as long as the host OS, while each container reserves a separate user space. Kernel-based Virtual Machine (KVM) is basically a hardware-assisted virtualization with the modified Linux kernel built with KVM module. KVM itself does not perform any emulation or virtualization. Instead, it simply exposes the /dev/kvm interface. QEMU is chosen as a software-based emulator to simulate hardware for the virtualized environment. Virtual disk options under Proxmox VE During the virtual machine creation, the following virtual disk options are available: RAW: This is a raw file format. The disk space is allocated during the creation and will use up the specified size. When compared with QCOW2, it gives a better overall performance. QCOW2: This is an enhanced version of QCOW, which offers a provisioning ability for disk storage used by QEMU. QCOW2 offers a capability to create multiple virtual machine snapshots compared to the older version. The following features are supported: Thin-provisioning: During the disk creation, a file smaller than the specified size is created, and the specified size will be configured as the maximum size of the disk image. The file size will grow according to the usage inside the guest system; this is called thin-provisioning. Snapshot: QCOW2 allows the user to create snapshots of the current system. With the use of the copy-on-write technology and a read-only base image, differential backup can be achieved. VMDK: This file format for a disk image is used by VMware. The virtual disk file can be imported back to VMware Player, ESX, and ESXi. It also provides a thin-provisioning function such as QCOW2. What is availability? What means availability? Let's take a look on the formula of calculating the availability; we need to divide the subtraction of Downtime duration (DD) from Expected uptime (EU) with Expected uptime (EU), then multiply it with 100. Availability is expressed as a percentage of uptime with a year. Here is the formula: Downtime duration (DU): Refers to the number of hours which the system is unavailable. Expected uptime (EU): Refers to the expected system availability, normally we expect the system is 365 x 24 x 7 hours available. Negative effects on system downtime What are the problems brought from downtime? Let's have a look: Lose of customer trust: This is a huge impact if your application is an online buying platform. When the user tries to pay for the products or services they have selected, the system responses with an error page or even worse a white screen. If you were the customer, would you still trust this platform as a reliable one? I think the answer is no. Customers tend to share bad experience to friends and thus company reputation is damaged. System recovery: At the background side, lots of system recovery and troubleshooting tasks must be made. Sometimes we have to wait for the support from vendor and there might not have essential service parts for older system. If that's the case, the repairing time would be longer than normal while you still have to pay for the rack rental fee in the data center. Reduce the productivity of internal staff: If the affected server contains an internal system, the daily operation of a group of staff is affected. For example, when it is a CRM system, sales staff cannot load customer information to process. When it is a financial system, the accountant cannot send and receive money from banks and customers. Introduction on DRBD DRBD is a short form for Distributed Replicated Block Device, it is intended to use under high availability environment. DRBD provides high availability by mirroring existing system to another machine including the disk storage, network card status and services running under existing system. So if the existing system is out of service, we can instantly switch to the backup system to avoid service interruption. Besides high availability, there are a few more functions provided by Proxmox cluster mode but the most important one is live migration. Unlike normal migration, in Proxmox cluster, migration can be processed without shutting down the virtual machine. Such approach is called live migration which greatly reduces the downtime of each virtual machine. The Proxmox Cluster file system (pmxcfs) Are you curious on how to manage multiple configuration files in Proxmox cluster mode? Proxmox Cluster file system (pmxcfs) is a built-in function which Proxmox cluster provided to synchronize configuration files between cluster member nodes. It is an essential component for Proxmox cluster as a version control on configuration files including cluster configuration, the virtual machine configuration, etc. It is basically a database-driven file system to store configuration files for all host servers and replicate in real time on all host nodes using corosync. The underlying file system is created by FUSE, with maximum size of 30 MB now. Here are the concepts for this file system: FUSE: It is a short form for Filesystem in Userspace which allows users to define their own device under their own userspace. With the use of FUSE, we don't have to worry the system will be crashed if we have mis-configured a file system because FUSE is not linked to system kernel. Corosync: It is a short form for Corosync Cluster Engine, a group communication system allowing clients to communicate with each other. The following diagram shows the structure for the Proxmox Cluster file system: An explanation on Logical Volume Manager (LVM) Unlike building a local RAID1 device by using mdadm command, we need to form LVM volume with dedicated local disk in multiple servers. LVM is used to simplify the disk management process of large hard disks. By adding an abstruction layer, users are able to add/replace their hard disks without downtime in combining with hot swapping. Besides, users are able to add/remove/resize their LVM volumes or even create a RAID volume easily. The structure of LVM is shown as follows: The Gluster file system GlusterFS is a distributed filesystem running in server-client architecture. It makes used of native Gluster protocol but also be seen as a NFS share or even work as an object-storage (Amazon S3-like networked key-value storage) with GlusterFS UFO. Gluster over LVM with iSCSI provides auto healing function. With auto healing, Gluster client would still be able to read/write files even if one Gluster server has failed which is similar to what RAID 1 offered. Let's check out how Gluster file system handles server failure: Initially, we need to haveat least two storage servers installed with Gluster server package in order to enjoy the functionality of auto healing. On the client side, we have configured to use Replicate mode and mount the file system to /glusterfs. The file content will be stored in both storages in this mode as follows: If Storage 1 is failed, Gluster client will redirect the request to Storage 2. When Storage 1 becomes available, the updated content will be synchronized from Storage 2. Therefore, client will not notice there is a server failure. This is shown in the following diagram: Thus, Gluster file system can provide high availability if we are using replication mode. For performance, we can distribute the file to more servers as follow: The Ceph filesystem Ceph is also a distributed filesystem providing petabyte-level storage but is more focused on eliminating a single point of failure. To ensure high availability, replicas are created on other storage nodes. Ceph is developed based on the concepts of RADOS (reliable autonomic distributed object store) with different accessing methods provided: Accessing method Support Platforms Usage Library packages C, C++, JAVA, Python, Ruby, PHP Programming RADOS gateway Amazon S3, Swift Cloud platform RBD KVM Virtualization CEPH file system Linux kernel, FUSE File system Here is a simple diagram demonstrating the structure of CEPH file system: What is fencing device? Fencing device, as name stated, is a virtual fence to prevent the communications between two nodes. It is used to separate the failed node from accessing shared resources. If there are two nodes accessing the shared resources at the same time, collision occurs which might corrupt the shared data which is the data inside virtual machines. Available fencing device options It is very important to protect our data without any corruption, what types of fencing devices are available and how can they build their fences during node failure? There are two approaches as listed below: Power fencing: Both nodes are added to fencing device for monitoring. If there is any suspicious failure on production node for a period of time, fencing device simply turns off the power of outlet which the affected server is connected to while the backup node will take over the position to provide services. For the failed node, power switch will send notification to system admin and manual recovery is required but no service interruption on the client side. Network fencing: Server nodes are connected to network switch instead of power switch. There are two types of network fencing: IPMI: It requires a separate IMPI card or onboard IPMI port to function. During the running operating system, periodically query checks can be performed to ensure the accessibility of the monitored server. If the query check failed for many times, IPMI message will be sent to this failed node to turn it off. Here is how it operates: SNMP: When there is suspicious failure on server node in a period of time, the network port between network switch and the failed server is disabled which prevents failed server to access shared storage. When operator requires turning the server back to service, manual configuration is required. Here is a diagram on how it operates: The concept for quorum device Since the voting system is a democratic system, which means there is one vote for each node. So if we only have two nodes, no one could win the race which causes the racing problem. As a result, we need to add a 3rd node joining this system (i.e. the quorum in our case). Here is a sample on why the racing problem appears and how we can fix it: Assumes we have a cluster system with only two nodes, the above diagram show the initial state for the cluster. We have marked Node 1 as the Primary node. Here, Node 1 is disconnected and therefore Node 2 would like to take over its position to become primary node. But it cannot be successful because two votes are needed for role switching operation. Therefore the cluster will become non-operational until Node 1 is recovered as follows: When Node 1 is recovered from failure, it tries to join back the cluster but failed because the cluster has stopped working. To solve the problem, an extra node is recommended to join the cluster in order to create a high availability environment. Here is an example when a node failed and Node 2 would like to be the primary node: The concept on bonding device For the network interface, a bonding device (Bond0 and Bond1) will be created in Proxmox. Bonding device is also called NIC teaming which is a native Linux kernel feature allowing user to double network speed performance or perform network redundancy. There are two options for network redundancy, 802.1ad and Active-backup. They have different response patterns when handling multiple sessions: These network redundancy options are explained in the following points: In 802.1ad, both network interfaces are active therefore sessions can be processed by different network card which is an active-active model. On the other hand, only one interface is in active state in Active-backup mode. The backup interface will become active only if the active one fails. The concepts of cluster manager The following points explain the concepts of cluster manager: Resource Group Manager (RGManager) combines with cluster manager (CMAN) and distributed lock manager (DLM) processes to manage and provide failover capabilities for collections of resources called services, resource groups, or resource trees. It is the essential process for high availability on services. If this service is turned off, HA function is disabled. Cluster Manager (CMAN) is the main process of the cluster architecture. CMAN manages the state of quorum and the status of different cluster members. In order to check the status of all cluster members, monitoring messages are sent periodically on all cluster nodes. If there is any status change on the cluster member, it will be distributed to all other cluster nodes. It is also responsible for the quorum management. When more than half node members are active, a cluster is said to be healthy. If the number of active member nodes is decreased to less than half, all cluster-related activity is blocked: Any change to cluster.conf file is not allowed Unable to start resource manager which disables HA function Any operation on VM creation is blocked NOTE: The operations of the existing virtual machines without high availability are not affected. Distributed Lock Manager (DLM) is used by resource group manager by applying different lock modes to resource to prevent multiple accesses. For details, please refer: http://en.wikipedia.org/wiki/Distributed_lock_manager I have prepared a simple diagram showing the relationship between them: Backup virtual machine data in Proxmox After we have made a copy of the container configurations, we are going to back the actual data inside the virtual machine. There are two different methods—manual backup with vzdump command for both KVM and openVZ guests and backup via GUI management console. Backup with vzdump for virtual machines There are three different backup approaches in the vzdump command: Stop mode: Stop the VM during backup which takes long backup time. Suspend mode: Use rsync command to copy data to a temporary location (defined in--tmpdir), then performs second rsync operation while suspending the container. When the second rsync operation completes, the suspended VM is resumed. Snapshot mode: This mode makes use of LVM2 snapshot function. It requires extra space within the LVM volume. Backup with web management console Except from manually backup the container under command-line interface, we can also make it under web management interface too. Here are the steps to perform backup with GUI: Log in to the web management console with root account information. Browse the left panel to locate the virtual machine to be backed up. Choose the Backup tab in the right panel and you will only see the latest backup files you have created in the previous steps: Then we can simply click on the Backup button to initialize the backup dialog:Notice that Proxmox uses TAR package as the compression method and make used of Snapshot mode by default. Therefore, make sure you have enough free space in your Volume Group which stores the data of virtual machines before using default values. By default, the volume group used is pve which is mounted at /var/lib/vz and you cannot place your dump file the same volume group. From the dialog, we can choose if the backup output file is compressed or not. To conserve disk space, here we choose GZIP as the compression method and choose snapshot to enjoy the zero downtime backup process as follow: Building up our own OS template There are two types of templates: one is OpenVZ template and another one is VM template. OpenVZ template: is only used for building up OpenVZ containers but not for KVM machine which limits the choice on operating system that it must be Linux platform. VM template: is introduced with Proxmox 3.x series, used to deploy KVM virtual machine which therefore escapes from the limitation on operating system. Here are the steps to download an OpenVZ template: Log in to the web interface of Proxmox, and find local storage from the panel on the left-hand side. Click on the Content tab and choose Templates as shown in the following screenshot: Next, we need to find a suitable template to download; for example, we can download a system with Drupal installed, as shown in the following screenshot: Then we click on the Download button. When the download completes, the template file is listed on the templates page, as shown in the following screenshot: Troubleshooting on system access problems Basically, it should not be difficult for you to install a Proxmox server from scratch. But after I have performed a few installations on different platforms, I noticed there are few scenarios which might cause you into trouble. Here are the problems I have found. Undefined video mode number Symptom: In some motherboards, you would receive the Undefined video mode number warning after you have pressed Enter to begin installation. It simply tells you that you cannot run the fancy installation wizard as below: Root cause: The main problem is the display chipset. When your motherboard is using display chipset which is not VESA2.0 compatible, this error message appears. To learn more about the VESA2.0, please find the following links: Solution: Then, you will be asked to press either <ENTER>, <SPACE> or wait for 30 seconds to continue. If you have pressed <ENTER>, the possible video modes available on your system will be shown: You can pick up a display mode number based on the list mentioned above. Normally, you can choose display mode 314 with 800 x 600 resolutions and 16-bit color depth or you can choose display mode 311 which provides you with 640 x 480 resolutions and 16-bit color depth. Then you should be able to continue the installation process. Prevention: I found that this problem normally happened in Nvidia display cards. If it is possible, you can try to replace it with Intel or ATI display cards during your installation. Summary In this article, we explained the concept of virtualization and compared Proxmox with other virtualization software. Resources for Article:  Further resources on this subject: A Virtual Machine for a Virtual World [article] Planning Desktop Virtualization [article] Setting up of Software Infrastructure on the Cloud [article]
Read more
  • 0
  • 2
  • 26773

article-image-creating-java-ee-applications
Packt
24 Oct 2014
16 min read
Save for later

Creating Java EE Applications

Packt
24 Oct 2014
16 min read
In this article by Grant Shipley author of Learning OpenShift we are going to learn how to use OpenShift in order to create and deploy Java-EE-based applications using the JBoss Enterprise Application Platform (EAP) application server. To illustrate and learn the concepts of Java EE, we are going to create an application that displays an interactive map that contains all of the major league baseball parks in the United States. We will start by covering some background information on the Java EE framework and then introduce each part of the sample application. The process for learning how to create the sample application, named mlbparks, will be started by creating the JBoss EAP container, then adding a database, creating the web services, and lastly, creating the responsive map UI. (For more resources related to this topic, see here.) Evolution of Java EE I can't think of a single programming language other than Java that has so many fans while at the same time has a large community of developers that profess their hatred towards it. The bad reputation that Java has can largely be attributed to early promises made by the community when the language was first released and then not being able to fulfill these promises. Developers were told that we would be able to write once and run anywhere, but we quickly found out that this meant that we could write once and then debug on every platform. Java was also perceived to consume more memory than required and was accused of being overly verbose by relying heavily on XML configuration files. Another problem the language had was not being able to focus on and excel at one particular task. We used Java to create thick client applications, applets that could be downloaded via a web browser, embedded applications, web applications, and so on. Having Java available as a tool that completes most projects was a great thing, but the implementation for each project was often confusing. For example, let's examine the history of the GUI development using the Java programming language. When the language was first introduced, it included an API called the Abstract Window Toolkit (AWT) that was essentially a Java wrapper around native UI components supplied by the operating system. When Java 1.2 was released, the AWT implementation was deprecated in the favor of the Swing API that contained GUI elements written in 100 percent Java. By this time, a lot of developers were quickly growing frustrated with the available APIs and a new toolkit called the Standard Widget Toolkit (SWT) was developed to create another UI toolkit for Java. SWT was developed at IBM and is the windowing toolkit in use by the Eclipse IDE and is considered by most to be the superior toolkit that can be used when creating applications. As you can see, rapid changes in the core functionality of the language coupled with the refusal of some vendors to ship the JRE as part of the operating system left a bad taste in most developers' mouths. Another reason why developers began switching from Java to more attractive programming languages was the implementation of Enterprise JavaBeans (EJB). The first Java EE release occurred in December, 1999, and the Java community is just now beginning to recover from the complexity introduced by the language in order to create applications. If you were able to escape creating applications using early EJBs, consider yourself one of the lucky ones, as many of your fellow developers were consumed by implementing large-scale systems using this new technology. It wasn't fun; trust me. I was there and experienced it firsthand. When developers began abandoning Java EE, they seemed to go in one of two directions. Developers who understood that the Java language itself was quite beautiful and useful adopted the Spring Framework methodology of having enterprise grade features while sticking with a Plain Old Java Object (POJO) implementation. Other developers were wooed away by languages that were considered more modern, such as Ruby and the popular Rails framework. While the rise in popularity of both Ruby and Spring was happening, the team behind Java EE continued to improve and innovate, which resulted in the creation of a new implementation that is both easy to use and develop with. I am happy to report that if you haven't taken a look at Java EE in the last few years, now is the time to do so. Working with the language after a long hiatus has been a rewarding and pleasurable experience. Introducing the sample application For the remainder of this article, we are going to develop an application called mlbparks that displays a map of the United States with a pin on the map representing the location of each major league baseball stadium. The requirements for the application are as follows: A single map that a user can zoom in and out of As the user moves the map around, the map must be updated with all baseball stadiums that are located in the shown area The location of the stadiums must be searchable based on map coordinates that are passed to the REST-based API The data should be transferred in the JSON format The web application must be responsive so that it is displayed correctly regardless of the resolution of the browser When a stadium is listed on the map, the user should be able to click on the stadium to view details about the associated team The end state application will look like the following screenshot: The user will also be able to zoom in on a specific location by double-clicking on the map or by clicking on the + zoom button in the top-left corner of the application. For example, if a user zooms the map in to the Phoenix, Arizona area of the United States, they will be able to see the information for the Arizona Diamondbacks stadium as shown in the following screenshot: To view this sample application running live, open your browser and type http://mlbparks-packt.rhcloud.com. Now that we have our requirements and know what the end result should look like, let's start creating our application. Creating a JBoss EAP application For the sample application that we are going to develop as part of this article, we are going to take advantage of the JBoss EAP application server that is available on the OpenShift platform. The JBoss EAP application server is a fully tested, stable, and supported platform for deploying mission-critical applications. Some developers prefer to use the open source community application server from JBoss called WildFly. Keep in mind when choosing WildFly over EAP that it only comes with community-based support and is a bleeding edge application server. To get started with building the mlbparks application, the first thing we need to do is create a gear that contains the cartridge for our JBoss EAP runtime. For this, we are going to use the RHC tools. Open up your terminal application and enter in the following command: $ rhc app create mlbparks jbosseap-6 Once the previous command is executed, you should see the following output: Application Options ------------------- Domain:     yourDomainName Cartridges: jbosseap-6 (addtl. costs may apply) Gear Size: default Scaling:   no Creating application 'mlbparks' ... done Waiting for your DNS name to be available ... done Cloning into 'mlbparks'... Your application 'mlbparks' is now available. URL:       http://mlbparks-yourDomainName.rhcloud.com/ SSH to:     5311180f500446f54a0003bb@mlbparks-yourDomainName.rhcloud.com Git remote: ssh://5311180f500446f54a0003bb@mlbparks-yourDomainName.rhcloud.com/~/git/mlbparks.git/ Cloned to: /home/gshipley/code/mlbparks  Run 'rhc show-app mlbparks' for more details about your app. If you have a paid subscription to OpenShift Online, you might want to consider using a medium- or large-size gear to host your Java-EE-based applications. To create this application using a medium-size gear, use the following command: $ rhc app create mlbparks jbosseap-6 -g medium Adding database support to the application Now that our application gear has been created, the next thing we want to do is embed a database cartridge that will hold the information about the baseball stadiums we want to track. Given that we are going to develop an application that doesn't require referential integrity but provides a REST-based API that will return JSON, it makes sense to use MongoDB as our database. MongoDB is arguably the most popular NoSQL database available today. The company behind the database, MongoDB, offers paid subscriptions and support plans for production deployments. For more information on this popular NoSQL database, visit www.mongodb.com. Run the following command to embed a database into our existing mlbparks OpenShift gear: $ rhc cartridge add mongodb-2.4 -a mlbparks Once the preceding command is executed and the database has been added to your application, you will see the following information on the screen that contains the username and password for the database: Adding mongodb-2.4 to application 'mlbparks' ... done  mongodb-2.4 (MongoDB 2.4) ------------------------- Gears:         Located with jbosseap-6 Connection URL: mongodb://$OPENSHIFT_MONGODB_DB_HOST:$OPENSHIFT_MONGODB_DB_PORT/ Database Name: mlbparks Password:       q_6eZ22-fraN Username:       admin MongoDB 2.4 database added. Please make note of these credentials:    Root User:     admin    Root Password: yourPassword    Database Name: mlbparks Connection URL: mongodb://$OPENSHIFT_MONGODB_DB_HOST:$OPENSHIFT_MONGODB_DB_PORT/ Importing the MLB stadiums into the database Now that we have our application gear created and our database added, we need to populate the database with the information about the stadiums that we are going to place on the map. The data is provided as a JSON document and contains the following information: The name of the baseball team The total payroll for the team The location of the stadium represented with the longitude and latitude The name of the stadium The name of the city where the stadium is located The league the baseball club belongs to (national or American) The year the data is relevant for All of the players on the roster including their position and salary A sample for the Arizona Diamondbacks looks like the following line of code: {   "name":"Diamondbacks",   "payroll":89000000,   "coordinates":[     -112.066662,     33.444799   ], "ballpark":"Chase Field",   "city":"Phoenix",   "league":"National League", "year":"2013",   "players":[     {       "name":"Miguel Montero", "position":"Catcher",       "salary":10000000     }, ………… ]} In order to import the preceding data, we are going to use the SSH command. To get started with the import, SSH into your OpenShift gear for the mlbparks application by issuing the following command in your terminal prompt: $ rhc app ssh mlbparks Once we are connected to the remote gear, we need to download the JSON file and store it in the /tmp directory of our gear. To complete these steps, use the following commands on your remote gear: $ cd /tmp $ wget https://raw.github.com/gshipley/mlbparks/master/mlbparks.json Wget is a software package that is available on most Linux-based operating systems in order to retrieve files using HTTP, HTTPS, or FTP. Once the file has completed downloading, take a quick look at the contents using your favorite text editor in order to get familiar with the structure of the document. When you are comfortable with the data that we are going to import into the database, execute the following command on the remote gear to populate MongoDB with the JSON documents: $ mongoimport --jsonArray -d $OPENSHIFT_APP_NAME -c teams --type json --file /tmp/mlbparks.json -h $OPENSHIFT_MONGODB_DB_HOST --port $OPENSHIFT_MONGODB_DB_PORT -u $OPENSHIFT_MONGODB_DB_USERNAME -p $OPENSHIFT_MONGODB_DB_PASSWORD If the command was executed successfully, you should see the following output on the screen: connected to: 127.7.150.130:27017 Fri Feb 28 20:57:24.125 check 9 30 Fri Feb 28 20:57:24.126 imported 30 objects What just happened? To understand this, we need to break the command we issued into smaller chunks, as detailed in the following table: Command/argument Description mongoimport This command is provided by MongoDB to allow users to import data into a database. --jsonArray This specifies that we are going to import an array of JSON documents. -d $OPENSHIFT_APP_NAME Specifies the database that we are going to import the data into the database. We are using a system environment variable to use the database that was created by default when we embedded the database cartridge in our application. -c teams This defines the collection to which we want to import the data. If the collection does not exist, it will be created. --type json This specifies the type of file we are going to import. --file /tmp/mlbparks.json This specifies the full path and name of the file that we are going to import into the database. -h $OPENSHIFT_MONGODB_DB_HOST This specifies the host of the MongoDB server. --port $OPENSHIFT_MONGODB_DB_PORT This specifies the port of the MongoDB server. -u $OPENSHIFT_MONGODB_DB_USERNAME This specifies the username to be used to be authenticated to the database. -p $OPENSHIFT_MONGODB_DB_PASSWORD This specifies the password to be authenticated to the database. To verify that data was loaded properly, you can use the following command that will print out the number of documents in the teams collections of the mlbparks database: $ mongo -quiet $OPENSHIFT_MONGODB_DB_HOST:$OPENSHIFT_MONGODB_DB_PORT/$OPENSHIFT_APP_NAME -u $OPENSHIFT_MONGODB_DB_USERNAME -p $OPENSHIFT_MONGODB_DB_PASSWORD --eval "db.teams.count()" The result should be 30. Lastly, we need to create a 2d index on the teams collection to ensure that we can perform spatial queries on the data. Geospatial queries are what allow us to search for specific documents that fall within a given location as provided by the latitude and longitude parameters. To add the 2d index to the teams collections, enter the following command on the remote gear: $ mongo $OPENSHIFT_MONGODB_DB_HOST:$OPENSHIFT_MONGODB_DB_PORT/$OPENSHIFT_APP_NAME --eval 'db.teams.ensureIndex( { coordinates : "2d" } );' Adding database support to our Java application The next step in creating the mlbparks application is adding the MongoDB driver dependency to our application. OpenShift Online supports the popular Apache Maven build system as the default way of compiling the source code and resolving dependencies. Maven was originally created to simplify the build process by allowing developers to specify specific JARs that their application depends on. This alleviates the bad practice of checking JAR files into the source code repository and allows a way to share JARs across several projects. This is accomplished via a pom.xml file that contains configuration items and dependency information for the project. In order to add the dependency for the MongoDB client to our mlbparks applications, we need to modify the pom.xml file that is in the root directory of the Git repository. The Git repository was cloned to our local machine during the application's creation step that we performed earlier in this article. Open up your favorite text editor and modify the pom.xml file to include the following lines of code in the <dependencies> block: <dependency> <groupId>org.mongodb</groupId> <artifactId>mongo-java-driver</artifactId> <version>2.9.1</version> </dependency> Once you have added the dependency, commit the changes to your local repository by using the following command: $ git commit -am "added MongoDB dependency" Finally, let's push the change to our Java application to include the MongoDB database drivers using the git push command: $ git push The first time the Maven build system builds the application, it downloads all the dependencies for the application and then caches them. Because of this, the first build will always that a bit longer than any subsequent build. Creating the database access class At this point, we have our application created, the MongoDB database embedded, all the information for the baseball stadiums imported, and the dependency for our database driver added to our application. The next step is to do some actual coding by creating a Java class that will act as the interface for connecting to and communicating with the MongoDB database. Create a Java file named DBConnection.java in the mlbparks/src/main/java/org/openshift/mlbparks/mongo directory and add the following source code: package org.openshift.mlbparks.mongo;  import java.net.UnknownHostException; import javax.annotation.PostConstruct; import javax.enterprise.context.ApplicationScoped; import javax.inject.Named; import com.mongodb.DB; import com.mongodb.Mongo;  @Named @ApplicationScoped public class DBConnection { private DB mongoDB; public DBConnection() {    super(); } @PostConstruct public void afterCreate() {    String mongoHost = System.getenv("OPENSHIFT_MONGODB_DB_HOST");    String mongoPort = System.getenv("OPENSHIFT_MONGODB_DB_PORT");    String mongoUser = System.getenv("OPENSHIFT_MONGODB_DB_USERNAME");    String mongoPassword = System.getenv("OPENSHIFT_MONGODB_DB_PASSWORD");    String mongoDBName = System.getenv("OPENSHIFT_APP_NAME");    int port = Integer.decode(mongoPort);    Mongo mongo = null;  try {     mongo = new Mongo(mongoHost, port);    } catch (UnknownHostException e) {     System.out.println("Couldn't connect to MongoDB: " + e.getMessage() + " :: " + e.getClass());    }    mongoDB = mongo.getDB(mongoDBName);    if (mongoDB.authenticate(mongoUser, mongoPassword.toCharArray()) == false) {     System.out.println("Failed to authenticate DB ");    } } public DB getDB() {    return mongoDB; } } The preceding source code as well as all source code for this article is available on GitHub at https://github.com/gshipley/mlbparks. The preceding code snippet simply creates an application-scoped bean that is available until the application is shut down. The @ApplicationScoped annotation is used when creating application-wide data or constants that should be available to all the users of the application. We chose this scope because we want to maintain a single connection class for the database that is shared among all requests. The next bit of interesting code is the afterCreate method that gets authenticated on the database using the system environment variables. Once you have created the DBConnection.java file and added the preceding source code, add the file to your local repository and commit the changes as follows: $ git add . $ git commit -am "Adding database connection class" Creating the beans.xml file The DBConnection class we just created makes use of Context Dependency Injection (CDI), which is part of the Java EE specification, for dependency injection. According to the official specification for CDI, an application that uses CDI must have a file called beans.xml. The file must be present and located under the WEB-INF directory. Given this requirement, create a file named beans.xml under the mlbparks/src/main/webapp/WEB-INF directory and add the following lines of code: <?xml version="1.0"?> <beans xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://jboss.org/schema/cdi/beans_1_0.xsd"/> After you have added the beans.xml file, add and commit it to your local Git repository: $ git add . $ git commit -am "Adding beans.xml for CDI" Summary In this article we learned about the evolution of Java EE, created a JBoss EAP application, and created the database access class. Resources for Article: Further resources on this subject: Using OpenShift [Article] Common performance issues [Article] The Business Layer (Java EE 7 First Look) [Article]
Read more
  • 0
  • 0
  • 3454

article-image-wireshark
Packt
24 Oct 2014
16 min read
Save for later

Wireshark

Packt
24 Oct 2014
16 min read
In this article by James H. Baxter, author of Wireshark Essentials, we will learn how to install Wireshark, perform a packet capture, use display filters to isolate traffic of interest, and save a filtered packet trace file. (For more resources related to this topic, see here.) Installing Wireshark Wireshark can be installed on machines running 32- and 64-bit Windows (XP, Win7, Win8.1, and so on), Mac OS X (10.5 and higher), and most flavors of Linux/Unix. Installation on Windows and Mac machines is quick and easy because installers are available from the Wireshark website download page. Wireshark is a standard package available on many Linux distributions, and there is a list of links to third-party installers provided on the Wireshark download page for a variety of popular *nix platforms. Alternatively, you can download the source code and compile Wireshark for your environment if a precompiled installation package isn't available. Wireshark relies on the WinPcap (Windows) or libpcap (Linux/Unix/Mac) libraries to provide the packet capture and capture filtering function; the appropriate library is installed during the Wireshark installation. You might need the administrator (Windows) or root (Linux/Unix/Mac) privileges to install Wireshark and the WinPcap/libpcap utilities on your workstation. Assuming that you're installing Wireshark on a Windows or Mac machine, you need to go to the Wireshark website (https://www.wireshark.org/) and click on the Download button at the top of the page. This will take you to the download page, and at the same time attempt to perform an autodiscovery of your operating system type and version from your browser info. The majority of the time, the correct Wireshark installation package for your machine will be highlighted, and you only have to click on the highlighted link to download the correct installer. If you already have Wireshark installed, an autoupdate feature will notify you of available version updates when you launch Wireshark. Installing Wireshark on Windows In the following screenshot, the Wireshark download page has identified that a 64-bit Windows installer is appropriate for this Windows workstation: Clicking on the highlighted link downloads a Wireshark-win64-1.10.8.exe file or similar executable file that you can save on your hard drive. Double-clicking on the executable starts the installation process. You need to follow these steps: Agree to the License Agreement. Accept all of the defaults by clicking on Next for each prompt, including the prompt to install WinPcap, which is a library needed to capture packets from the Network Interface Card (NIC) on your workstation. Early in the Wireshark installation, the process will pause and prompt you to click on Install and several Next buttons in separate windows to install WinPcap. After the WinPcap installation is complete, click through the remaining Next prompts to finish the Wireshark installation. Installing Wireshark on Mac OS X The process to install Wireshark on Mac is the same as the process for Windows, except that you will not be prompted to install WinPcap; libpcap, the packet capture library for Mac and *nix machines, gets installed instead (without prompting). There are, however, two additional requirements that may need to be addressed in a Mac installation: The first is to install X11, a windowing system library. If this is needed for your system, you will be informed and provided a link that ultimately takes you to the XQuartz project download page so you can install this package. The second requirement that might come up is if upon starting Wireshark, you are informed that there are no interfaces on which a capture can be done. This is a permissions issue on the Berkeley packet filter (BPF) that can be resolved by opening a terminal window and typing the following command: bash-3.2$ sudo chmod 644 /dev/bpf* If this process needs to be repeated each time you start Wireshark, you can perform a web search for more permanent permissions solution for your environment. Installing Wireshark on Linux/Unix The requirements and process to install Wireshark on a Linux or Unix platform can vary significantly depending on the particular environment. Performing your first packet capture When you first start Wireshark, you are presented with an initial Start Page as shown in the following screenshot: Don't get too fond of this screen. Although you'll see this every time you start Wireshark, once you do a capture, open a trace file, or perform any other function within Wireshark, this screen will be replaced with the standard Wireshark user interface and you won't see it again until the next time you start Wireshark. So, we won't spend much time here. Selecting a network interface If you have a number of network interfaces on your machine, you may not be sure which one to select to capture packets, but there's a fairly easy way to figure this out. On the Wireshark start page, click on Interface List (alternatively, click on Interfaces from the Capture menu or click on the first icon on the icon bar). The Wireshark Capture Interfaces window that opens provides a list and description of all the network interfaces on your machine, the IP address assigned to each one (if an address has been assigned), and a couple of counters, such as the total number of packets seen on the interface since this window starts and a packets/s (packets per second) counter. If an interface has an IPv6 address assigned (which may start with fe80:: and contain a number of colons) and this is being displayed, you can click on the IPv6 address and it will toggle to display the IPv4 address. This is shown in the following screenshot: On Linux/Unix/Mac platforms, you might also see a loopback interface that can be selected to capture packets being sent between applications on the same machine. However, in most cases, you'll only be interested in capturing packets from a network interface. The goal is to identify the active interface that will be used to communicate with the Internet when you open a browser and navigate to a website. If you have a wired local area network connection and the interface is enabled, that's probably the active interface, but you might also have a wireless interface that is enabled and you may or may not be the primary interface. The most reliable indicator of the active network interface is that it will have greater number of steadily increasing packets with a corresponding active number of packets/s (which will vary over time). Another possible indicator is if an interface has an IP address assigned and others do not. If you're still unsure, open a browser window and navigate to one of your favorite websites and watch the packets and packets/s counters to identify the interface that shows the greatest increase in activity. Performing the packet capture Once you've identified the correct interface, select the checkbox on the left-hand side of that interface and click on the Start button at the bottom of the Capture Interfaces window. Wireshark will start capturing all the packets that can be seen from that interface, including the packets sent to and from your workstation. If you don't see this, try a different interface. It's a bit amazing just how much background traffic there is on a typical network such as broadcast packets from devices advertising their names, addresses, and services and from other devices asking for addresses of stations they want to communicate with. Also, a fair amount of traffic is generated from your own workstation for applications and services that are running in the background, and you had no idea they were creating this much noise. Your Wireshark's Packet List pane may look similar to the following screenshot; however, we can ignore all this for now: We're ready to generate some traffic that we'll be interested in analyzing. Open a new Internet browser window, enter www.wireshark.org in the address box, and press Enter. When the https://www.wireshark.org/ home page finishes loading, stop the Wireshark capture by either selecting Stop from the Capture menu or by clicking on the red square stop icon that's between the View and Go menu headers. Wireshark user interface essentials Once you have completed your first capture, you will see the normal Wireshark user interface main screen. So before we go much further, a quick introduction to the primary parts of this user interface will be helpful so you'll know what's being referred to when as we continue the analysis process. There are eight significant sections or elements of the default Wireshark user interface as shown in the following screenshot: Let's look at the eight significant sections in detail: Title: This area reflects the interface from where a capture is being taken or the filename of an open packet trace file Menu: This is the standard row of main functions and subfunctions in Wireshark Main toolbar (icons): These provide a quick way to access the most useful Wireshark functions and are well worth getting familiar with and using Display filter toolbar: This allows you to quickly create, edit, clear, apply, and save filters to isolate packets of interest for analysis Packet list pane: This section contains a summary info line for each captured packet, as well as a packet number and relative timestamp Packet details pane: This section provides a hierarchical display of information about a single packet that has been selected in the packet list pane, which is divided into sections for the various protocols contained in a packet Packet bytes pane: This section displays the selected packets' contents in hex bytes or bits form, as well as an ASCII display of the data that can be helpful Status bar: This section provides an expert info indicator, edit capture comments icon, trace file path, name, and size information, data on the number of packets captured and displayed and other info, and a profile display and selection section. Filtering out the noise Somewhere in your packet capture, there are packets involved with loading the Wireshark home page—but how do you find and view just those packets out of all the background noise? The simplest and most reliable method is to determine the IP address of the Wireshark website and filter out all the packets except those flowing between that IP address and the IP address of your workstation by using a display filter. The best approach—and the one that you'll likely use as a first step for most of your post-capture analysis work in future—is to investigate a list of all the conversations by IP address and/or hostnames, sorted by the most active nodes, and identify your target hostname, website name, or IP address from this list. From the Wireshark menu, select Conversations from the Statistics menu and in the Conversations window that opens, select the IPv4 tab at the top. You'll see a list of network conversations identified by Address A and Address B, with columns for total Packets, Bytes, Packets A→B, Bytes A→B, Packets A←B, and Bytes A←B. Scrolling over to the right-hand side of this window, there are Relative Start values. These are the times when each particular conversation was first observed in the capture, relative to the start of the capture in seconds. The next column is Duration, which is how long this conversation persisted in the capture (first to last packet seen). Finally, there are average data rates in bits per second (bps) in each direction for each conversation, which is the network impact for this conversation. All these are shown in the following screenshot: We want to sort the list of conversations to get the busiest ones—called the Top Talkers in the network jargon—at the top of the list. Click on the Bytes column header and then click it again. Your list should look something like the preceding screenshot, and if you didn't get a great deal on other background traffic flowing to/from your workstation, the traffic from https://www.wireshark.org/ should have the greatest volume and therefore be at the top of the list. In this example, the conversation between IP addresses 162.159.241.165 and 192.168.1.116 has the greatest overall volume, and looking at the Bytes A->B column, it's apparent that the majority of the traffic was from the 162.159.241.165 address to the 192.168.1.116 address. However, at this point, how do we know if this is really the conversation that we're after? We will need to resolve the IP addresses from our list to hostnames or website addresses, and this can be done from within Wireshark by turning on Network Name Resolution and trying to get hostnames and/or website addresses resolved for those IP addresses using reverse DNS queries (using what is known as a pointer (PTR) DNS record type). If you just installed or started Wireshark, the Name Resolution option may not be turned on by default. This is usually a good thing, as Wireshark can create traffic of its own by transmitting the DNS queries trying to resolve all the IP addresses that it comes across during the capture, and you don't really want that going on during a capture. However, the Name Resolution option can be very helpful to resolve IP addresses to proper hostnames after a capture is complete. To enable Name Resolution, navigate to View | Name Resolution | Enable for Network Layer (click to turn on the checkmark) and make sure Use External Network Name Resolver is enabled as well. Wireshark will attempt to resolve all the IP addresses in the capture to their hostname or website address, and the resolved names will then appear (replacing the previous IP addresses) in the packet list as well as the Conversations window. Note that the Name Resolution option at the bottom of the Conversations window must be enabled as well (it usually is by default), and this setting affects whether resolved names or IP addresses appear in that Conversations window (if Name Resolution is enabled in the Wireshark main screen), as shown in the following screenshot: At this point, you should see the conversation pair between wireshark.org and your workstation at or near the top of the list, as shown in the following screenshot. Of course, your workstation will have a different name or may only appear as an IP address, but identifying the conversation to wireshark.org has been achieved. Applying a display filter You now want to see just the conversation between your workstation and wireshark.org, and get rid of all the extraneous conversations so you can focus on the traffic of interest. This is accomplished by creating a filter that only displays the desired traffic. Right-click on the line containing the wireshark.org entry and navigate to Apply as Filter | Selected | A<->B, as shown in the following screenshot: Wireshark will create and apply a display filter string that isolates the displayed traffic to just the conversation between the IP addresses of wireshark.org and your workstation, as shown in the following screenshot. Note that if you create or edit a display filter entry manually, you will need to click on Apply to apply the filter to the trace file (or Clear to clear it). This particular display filter syntax works with IP addresses, not with hostnames, and uses an ip.addr== (IP address equals) syntax for each node along with the && (and) logic operator to build a string that says display any packet that contains this IP address *and* that IP address. This is the type of display filter that you will be using a great deal for packet analysis. You'll notice as you scroll up and down in the Packet List pane that all the other packets, except those between your workstation and wireshark.org are gone. They're not gone in the strict sense, they're just hidden—as you can observe by inspecting the Packet No. column, there are gaps in the numbering sequence, those are for the hidden packets. Saving the packet trace Now that you've isolated the traffic of interest using a display filter, you can save a new packet trace file that contains just the filtered packets. This serves two purposes. Firstly, you can close Wireshark, come back to it later, open the filtered trace file, and pick up where you left off in your analysis, as well as have a record of the capture in case you need to reference it later such as in a troubleshooting scenario. Secondly, it's much easier and quicker to work in the various Wireshark screens and functions with a smaller, more focused trace file that contains just the packets that you want to analyze. To create new packet trace file containing just the filtered/displayed packets, select Export Specified Packets from the Wireshark File menu. You can navigate to and/or create a folder to hold your Wireshark trace files, and then enter a filename for the trace file that you want to save. In this example, the filename is wireshark_website.pcapng. By default, Wireshark will save the trace file in the pcapng format (which is the preferred and more recent format). If you don't specify a file extension with the filename, Wireshark will provide the appropriate extension based on the Save as type selection, as shown in the following screenshot: Also, by default, Wireshark will have the All packets option selected, and if a display filter is applied (as it is in this scenario), the Displayed option will be selected as opposed to the Captured option that saves all the packets regardless of whether a filter was applied. Having entered a filename and confirmed that all the save selections are correct, you can click on Save to save the new packet trace file. Note that when you have finished this trace file save activity, Wireshark still has all the original packets from the capture in memory, and they can still be viewed by clicking on Clear in the Display Filter Toolbar menu. If you want work further with the new trace file you just saved, you'll need to open it by clicking on Open in the File menu (or Open Recent in the File menu). Summary Congratulations! If you accomplished all the activities covered in this article, you have successfully installed Wireshark, performed a packet capture, created a filter to isolate and display just the packets you were interested in from all the extraneous noise, and created a new packet trace file containing just those packets so you can analyze them later. Moreover, in the process, you gained an initial familiarity with the Wireshark user interface and you learned how to use several of its most useful and powerful features. Resources for Article: Further resources on this subject: Wireshark: Working with Packet Streams [Article] The Kendo MVVM Framework [Article] Kali Linux – Wireless Attacks [Article]
Read more
  • 0
  • 0
  • 4670

article-image-understanding-context-bdd
Packt
22 Oct 2014
6 min read
Save for later

Understanding the context of BDD

Packt
22 Oct 2014
6 min read
In this article by Sujoy Acharya, author of Mockito Essentials, you will learn about the BDD concepts and BDD examples. You will also learn about how BDD can help you minimize project failure risks. (For more resources related to this topic, see here.) This section of the article deals with the software development strategies, drawbacks, and conquering the shortcomings of traditional approaches. The following strategies are applied to deliver software products to customers: Top-down or waterfall approach Bottom-up approach We'll cover these two approaches in the following sections. The following key people/roles/stakeholders are involved in software development: Customers: They explore the concept and identify the high-level goal of the system, such as automating the expense claim process Analysts: They analyze the requirements, work with the customer to understand the system, and build the system requirement specifications Designers/architects: They visualize the system, design the baseline architecture, identify the components, interact and handle the nonfunctional requirements, such as scalability and availability Developers: They construct the system from the design and specification documents Testers: They design test cases and verify the implementation Operational folks: They install the software as per the customer's environment Maintenance team: They handle bugs and monitor the system's health Managers: They act as facilitators and keep track of the progress and schedule Exploring the top-down strategy In the top-down strategy, analysts analyze the requirements and hand over the use cases / functional specifications to the designers and architects for designing the system. The architects/designers design the baseline architecture, identify the system components and interactions, and then pass the design over to the developers for implementation. The testers then verify the implementation (might report bugs for fixing), and finally, the software is deployed to the customer's environment. The following diagram depicts the top-down flow from requirement engineering to maintenance: The biggest drawback of this approach is the cost of rework. For instance, if the development team finds that a requirement is not feasible, they consult the design or analysis team. Then the architects or analysts look at the issue and rework the analysis or design. This approach has a cascading effect; the cost of rework is very high. Customers rarely know what they want before they see the system in action. Building everything all at once is a quick way to cause your requirements to change. Even without the difference in cost of requirement changes, you'll have fewer changes if you write the requirements later in the process, when you have a partially working product that the customer can see and everybody has more information about how the product will work. Exploring the bottom-up strategy In the bottom-up strategy, the requirement is broken into small chunks and each chunk is designed, developed, and unit tested separately, and finally, the chunks are integrated. The individual base elements of the system are first specified in great detail. These elements are then linked together to form larger subsystems, which in turn are linked until a complete top-level system is formed. Each subsystem is developed in isolation from the other subsystems, so integration is very important in the bottom-up approach. If integration fails, the cost and effort of building the subsystems gets jeopardized. Suppose you are building a healthcare system with three subsystems, namely, patient management, receivable management, and the claims module. If the patient module cannot talk to the claims module, the system fails. The effort of building the patient management and claims management subsystems is just wasted. Agile development methodology would suggest building the functionality feature by feature across subsystems, that is, building a very basic patient management and claims management subsystem to make the functionality work initially, and then adding more to both simultaneously, to support each new feature that is required. Finding the gaps In real-life projects, the following is the percentage of feature usage: 60 percent of features are never used 30 percent of features are occasionally used 10 percent of features are frequently used However, in the top-down approach, the analyst pays attention and brainstorms to create system requirements for all the features. In the top-down approach, time is spent to build a system where 90 percent of features are either not used or occasionally used. Instead, we can identify the high-value features and start building the features instead of paying attention to the low priority features, by using the bottom-up approach. In the bottom-up approach, subsystems are built in isolation from each other, and this causes integration problems. If we prioritize the requirements and start with the highest priority feature, design the feature, build it, unit test it, integrate it, and then show a demo to the stakeholders (customers, analysts, product managers, and so on), we can easily identify the gaps and reduce the risk of rework. We can then pick the next feature and follow the steps (designing, coding, testing, and getting feedback from the customers), and finally integrate the feature with the existing system. This reduces the integration issues of the bottom-up approach. The following figure represents the approach. Each feature is analyzed, designed, coded, tested, and integrated separately. An example of a requirement could be login failure error messages appear red and in bold, while a feature could be incorrect logins are rejected. Typically, it should be a little larger and a useful standalone bit of functionality, rather than a specific single requirement for that functionality. Another problem associated with software development is communication; each stakeholder has a different vocabulary and this causes issues for common understanding. The following are the best practices to minimize software delivery risks: Focus on high-value, frequently used features. Build a common vocabulary for the stakeholders; a domain-specific language that anybody can understand. No more big-fat upfront designing. Evolve the design with the requirements, iteratively. Code to satisfy the current requirement. Don't code for a future requirement, which may or may not be delivered. Follow the YAGNI (You Aren't Going to Need It) principle. Build test the safety net for each requirement. Integrate the code with the system and rerun the regression test. Get feedback from the stakeholders and make immediate changes. BDD suggests the preceding best approaches. Summary This article covered and taught you about the BDD concepts and BDD examples. Resources for Article: Further resources on this subject: Important features of Mockito [article] Progressive Mockito [article] Getting Started with Mockito [article]
Read more
  • 0
  • 0
  • 4997
article-image-creating-jsf-composite-component
Packt
22 Oct 2014
9 min read
Save for later

Creating a JSF composite component

Packt
22 Oct 2014
9 min read
This article by David Salter, author of the book, NetBeans IDE 8 Cookbook, explains how to create a JSF composite component in NetBeans. (For more resources related to this topic, see here.) JSF is a rich component-based framework, which provides many components that developers can use to enrich their applications. JSF 2 also allows composite components to be easily created, which can then be inserted into other JSF pages in a similar way to any other JSF components such as buttons and labels. In this article, we'll see how to create a custom component that displays an input label and asks for corresponding input. If the input is not validated by the JSF runtime, we'll show an error message. The component is going to look like this: The custom component is built up from three different standard JSF components. On the left, we have a <h:outputText/> component that displays the label. Next, we have a <h:inputText /> component. Finally, we have a <h:message /> component. Putting these three components together like this is a very useful pattern when designing input forms within JSF. Getting ready To create a JSF composite component, you will need to have a working installation of WildFly that has been configured within NetBeans. We will be using the Enterprise download bundle of NetBeans as this includes all of the tools we need without having to download any additional plugins. How to do it… First of all, we need to create a web application and then create a JSF composite component within it. Perform the following steps: Click on File and then New Project…. Select Java Web from the list of Categories and Web Application form the list of Projects. Click on Next. Enter the Project Name value as CompositeComp. Click on Next. Ensure that Add to Enterprise Application is set to <None>, Server is set to WildFly Application Server, Java EE Version is set to Java EE 7 Web, and Context Path is set to /CompositeComp. Click on Next. Click on the checkbox next to JavaServer Faces as we are using this framework. All of the default JSF configurations are correct, so click on the Finish button to create the project. Right-click on the CompositeComp project within the Projects explorer and click on New and then Other…. In the New File dialog, select JavaServer Faces from the list of Categories and JSF Composite Component from the list of File Types. Click on Next. On the New JSF Composite Component dialog, enter the File Name value as inputWithLabel and change the folder to resourcescookbook. Click on Finish to create the custom component. In JSF, custom components are created as Facelets files that are stored within the resources folder of the web application. Within the resources folder, multiple subfolders can exist, each representing a namespace of a custom component. Within each namespace folder, individual custom components are stored with filenames that match the composite component names. We have just created a composite component within the cookbook namespace called inputWithLabel. Within each composite component file, there are two sections: an interface and an implementation. The interface lists all of the attributes that are required by the composite component and the implementation provides the XHTML code to represent the component. Let's now define our component by specifying the interface and the implementation. Perform the following steps: The inputWithLabel.xhtml file should be open for editing. If not, double–click on it within the Projects explorer to open it. For our composite component, we need two attributes to be passed into the component. We need the text for the label and the expression language to bind the input box to. Change the interface section of the file to read:    <cc:attribute name="labelValue" />   <cc:attribute name="editValue" /></cc:interface> To render the component, we need to instantiate a <h:outputText /> tag to display the label, a <h:inputText /> tag to receive the input from the user, and a <h:message /> tag to display any errors that are entered for the input field. Change the implementation section of the file to read: <cc:implementation>   <style>   .outputText{width: 100px; }   .inputText{width: 100px; }   .errorText{width: 200px; color: red; }   </style>   <h:panelGrid id="panel" columns="3" columnClasses="outputText, inputText, errorText">       <h:outputText value="#{cc.attrs.labelValue}" />       <h:inputText value="#{cc.attrs.editValue}" id="inputText" />       <h:message for="inputText" />   </h:panelGrid></cc:implementation> Click on the lightbulb on the left-hand side of the editor window and accept the fix to add the h=http://><html       > We can now reference the composite component from within the Facelets page. Add the following code inside the <h:body> code on the page: <h:form id="inputForm">   <cookbook:inputWithLabel labelValue="Forename" editValue="#{personController.person.foreName}"/>   <cookbook:inputWithLabel labelValue="Last Name" editValue="#{personController.person.lastName}"/>   <h:commandButton type="submit" value="Submit" action="#{personController.submit}"/></h:form> This code instantiates two instances of our inputWithLabel composite control and binds them to personController. We haven't got one of those yet, so let's create one and a class to represent a person. Perform the following steps: Create a new Java class within the project. Enter Class Name as Person and Package as com.davidsalter.cookbook.compositecomp. Click on Finish. Add members to the class to represent foreName and lastName: private String foreName;private String lastName; Use the Encapsulate Fields refactoring to generate getters and setters for these members. To allow error messages to be displayed if the foreName and lastName values are inputted incorrectly, we will add some Bean Validation annotations to the attributes of the class. Annotate the foreName member of the class as follows: @NotNull@Size(min=1, max=25)private String foreName; Annotate the lastName member of the class as follows: @NotNull@Size(min=1, max=50)private String lastName; Use the Fix Imports tool to add the required imports for the Bean Validation annotations. Create a new Java class within the project. Enter Class Name as PersonController and Package as com.davidsalter.cookbook.compositecomp. Click on Finish. We need to make the PersonController class an @Named bean so that it can be referenced via expression language from within JSF pages. Annotate the PersonController class as follows: @Named@RequestScopedpublic class PersonController { We need to add a Person instance into PersonController that will be used to transfer data from the JSF page to the named bean. We will also need to add a method onto the bean that will redirect JSF to an output page after the names have been entered. Add the following to the PersonController class: private Person person = new Person();public Person getPerson() {   return person;}public void setPerson(Person person) {   this.person = person;}public String submit() {   return "results.xhtml";} The final task before completing our application is to add a results page so we can see what input the user entered. This output page will simply display the values of foreName and lastName that have been entered. Create a new JSF page called results that uses the Facelets syntax. Change the <h:body> tag of this page to read: <h:body>   You Entered:   <h:outputText value="#{personController.person.foreName}" />&nbsp;   <h:outputText value="#{personController.person.lastName}" /></h:body> The application is now complete. Deploy and run the application by right-clicking on the project within the Projects explorer and selecting Run. Note that two instances of the composite component have been created and displayed within the browser. Click on the Submit button without entering any information and note how the error messages are displayed: Enter some valid information and click on Submit, and note how the information entered is echoed back on a second page. How it works… Creating composite components was a new feature added to JSF 2. Creating JSF components was a very tedious job in JSF 1.x, and the designers of JSF 2 thought that the majority of custom components created in JSF could probably be built by adding different existing components together. As it is seen, we've added together three different existing JSF components and made a very useful composite component. It's useful to distinguish between custom components and composite components. Custom components are entirely new components that did not exist before. They are created entirely in Java code and build into frameworks such as PrimeFaces and RichFaces. Composite components are built from existing components and their graphical view is designed in the .xhtml files. There's more... When creating composite components, it may be necessary to specify attributes. The default option is that the attributes are not mandatory when creating a custom component. They can, however, be made mandatory by adding the required="true" attribute to their definition, as follows: <cc:attribute name="labelValue" required="true" /> If an attribute is specified as required, but is not present, a JSF error will be produced, as follows: /index.xhtml @11,88 <cookbook:inputWithLabel> The following attribute(s) are required, but no values have been supplied for them: labelValue. Sometimes, it can be useful to specify a default value for an attribute. This is achieved by adding the default="…" attribute to their definition: <cc:attribute name="labelValue" default="Please enter a value" /> Summary In this article, we have learned to create a JSF composite component using NetBeans. Resources for Article: Further resources on this subject: Creating a Lazarus Component [article] Top Geany features you need to know about [article] Getting to know NetBeans [article]
Read more
  • 0
  • 0
  • 6501

article-image-designing-and-building-horizon-view-60-infrastructure
Packt
22 Oct 2014
18 min read
Save for later

Designing and Building a Horizon View 6.0 Infrastructure

Packt
22 Oct 2014
18 min read
This article is written by Peter von Oven, the author of VMware Horizon View Essentials. In this article, we will start by taking a closer look at the design process. We will now look at the reference architecture and how we start to put together a design, building out the infrastructure for a production deployment. Proving the technology – from PoC to production In this section, we are going to discuss how to approach a VDI project. This is a key and very important piece of work that needs to be completed in the very early stages and is somewhat different from how you would typically approach an IT project. Our starting point is to focus on the end users rather than the IT department. After all, these are the people that will be using the solution on a daily basis and know what tools they need to get their jobs done. Rather than giving them what you think they need, let's ask them what they actually need and then, within reason, deliver this. It's that old saying of don't try and fit a square peg into a round hole. No matter how hard you try, it's just never going to fit. First and foremost we need to design the technology around the user requirements rather than building a backend infrastructure only to find that it doesn't deliver what the users require. Assessment Once you have built your business case and validated that against your EUC strategy and there is a requirement for delivering a VDI solution, the next stage is to run an assessment. It's quite fitting that this book is entitled "Essentials", as this stage of the project is exactly that, and is essential for a successful outcome. We need to build up a picture of what the current environment looks like, ranging from looking at what applications are being used to the type of access devices. This goes back to the earlier point about giving the users what they need and the only way to find that out is to conduct an assessment. By doing this, we are creating a baseline. Then, as we move into defining the success criteria and proving the technology, we have the baseline as a reference point to demonstrate how we have improved current working and delivered on the business case and strategy. There are a number of tools that can be used in the assessment phase to gather the information required, for example, Liquidware Labs Stratusphere FIT or SysTrack from Lakeside Software. Don't forget to actually talk to the users as well, so you are armed with the hard-and-fast facts from an assessment as well as the user's perspective. Defining the success criteria The key objective in defining the success criteria is to document what a "good" solution should look like for the project to succeed and become production-ready. We need to clearly define the elements that need to function correctly in order to move from proof of concept to proof of technology, and then into a pilot phase before deploying into production. You need to fully document what these elements are and get the end users or other project stakeholders to sign up to them. It's almost like creating a statement of work with a clearly defined list of tasks. Another important factor is to ensure that during this phase of the project, the criteria don't start to grow beyond the original scope. By that, we mean other additional things should not get added to the success criteria or at least not without discussion first. It may well transpire that something key was missed; however, if you have conducted your assessment thoroughly, this shouldn't happen. Another thing that works well at this stage is to involve the end users. Set up a steering committee or advisory panel by selecting people from different departments to act as sponsors within their area of business. Actively involve them in the testing phases, but get them on board early as well to get their input in shaping the solution. Too many projects fail when an end user tries something that didn't work. However, the thing that they tried is not actually a relevant use case or something that is used by the business as a critical line of business application and therefore shouldn't derail the project. If we have a set of success criteria defined up front that the end users have signed up to, anything outside that criteria is not in scope. If it's not defined in the document, it should be disregarded as not being part of what success should look like. Proving the technology Once the previous steps have been discussed and documented, we should be able to build a picture around what's driving the project. We will understand what you are trying to achieve/deliver and, based upon hard-and-fast facts from the assessment phase, be able to work on what success should look like. From there, we can then move into testing some form of the technology should that be a requirement. There are three key stages within the testing cycle to consider, and it might be the case that you don't need all of them. The three stages we are talking about are as follows: Proof of concept (PoC) Proof of technology (PoT) Pilot In the next sections, we are briefly going to cover what each of these stages mean and why you might or might not need them. Proof of concept A proof of concept typically refers to a partial solution, typically built on any old hardware kicking about, that involves a relatively small number of users usually within the confines of the IT department acting in business roles, to establish whether the system satisfies some aspect of the purpose it was designed for. Once proven, one or two things happen. Firstly nothing happens as it's just the IT department playing with technology and there wasn't a real business driver in the first place. This is usually down to the previous steps not having been defined. In a similar way, by not having any success criteria, it will also fail, as you don't know exactly what you are setting out to prove. The second outcome is that the project moves into a pilot phase that we will discuss in a later section. You could consider moving directly into this phase and bypassing the PoC altogether. Maybe a demonstration of the technology would suffice, and using a demo environment over a longer period would show you how the technology works. Proof of technology In contrast to the PoC, the objective of a proof of technology is to determine whether or not the proposed solution or technology will integrate into your existing environment and therefore demonstrate compatibility. The objective is to highlight any technical problems specific to your environment, such as how your bespoke systems might integrate. As with the PoC, a PoT is typically run by the IT department and no business users would be involved. A PoT is purely a technical validation exercise. Pilot A pilot refers to what is almost a small-scale roll out of the solution in a production-style environment that would target a limited scope of the intended final solution. The scope may be limited by the number of users who can access the pilot system, the business processes affected, or the business partners involved. The purpose of a pilot is to test, often in a production-like environment, whether the system is working, as it was designed while limiting business exposure and risk. It will also touch real users so as to gauge the feedback from what would ultimately become a live, production solution. This is a critical step in achieving success, as the users are the ones that have to interact with the system on a daily basis, and the reason why you should set up some form of working group to gather their feedback. That would also mitigate the project from failing, as the solution may deliver everything the IT department could ever wish for, but when it goes live and the first user logs on and reports a bad experience or performance, you may as well not be bothered. The pilot should be carefully scoped, sized, and implemented. We will discuss this in the next section. The pilot phase In this section, we are going to discuss the pilot phase in a bit more detail and break it down into three distinct stages. These are important, as the output from the pilot will ultimately shape the design of your production environment. The following diagram shows the workflow we will follow in defining our project: Phase 1 – pilot design The pilot infrastructure should be designed on the same hardware platforms that the production solution is going to be deployed, for example, the same servers and storage. This takes into account any anomalies between platforms and configuration differences that could affect things such as scalability or more importantly performance. Even at pilot stage, the design is absolutely key, and you should make sure you take into account the production design even at this stage. Why? Basically because many pilot solutions end up going straight into production and more and more users get added above and beyond those scoped for the pilot. It's great going live with the solution and not having to go back and rebuild it, but when you start to scale by adding more users and applications, you might have some issues due to the pilot sizing. It may sound obvious, but often with a successful pilot, the users just keep on using it and additional users get added. If it's only ever going to be a pilot, that's fine, but keep this in mind and ask the question; if you are planning on taking the pilot straight into production design it for production. It is always useful to work from a prerequisite document to understand the different elements that need consideration in the design. Key design elements include: Hardware sizing (servers – CPU, memory, and consolidation ratios) Pool design (based on user segmentation) Storage design (local SSD, SAN, and acceleration technologies) Image creation (rebuild from scratch and optimize for VDI) Network design (load balancing and external access) Antivirus considerations Application delivery (delivering virtually versus installing in core image) User profile management Floating or dedicated desktop assignments Persistent or non-persistent desktop builds (linked clone or full clone) Once you have all this information, you can start to deploy the pilot. Phase 2 – pilot deployment In the deployment phase of the pilot, we are going to start building out the infrastructure, deploying the test users, building the OS images, and then start testing. Phase 3 – pilot test During the testing phase, the key thing during this stage is to work closely with the end users and your sponsors, showing them the solution and how it works, closely monitoring the users, and assessing the solution as it's being used. This allows you to keep in contact with the users and give them the opportunity to continually provide real-time feedback. This also allows you to answer questions and make adjustments and enhancements on the fly rather than wait to the end of the project and then to be told it didn't work or they just simply didn't understand something. This then leads us onto the last section, the review. Phase 4 – pilot review This final stage sometimes tends to get forgotten. We have deployed the solution, the users have been testing it, and then it ends there for whatever reason. However, there is one very important last thing to do to enable the customer to move to production. We need to measure the user experience or the IT department's experience against the success criteria we set out at the start of this process. We need to get customer sign off and agreement that we have successfully met all the objectives and requirements. If this is not the case, we need to understand the reasons why. Have we missed something in the use case, have the user requirements changed, or is it simply a perception issue? Whatever the case, we need to cycle round the process again. Go back to the use case, understand and reevaluate the user requirements, (what it is that is seemingly failing or not behaving as expected), and then tweak the design or make the required changes and get them to test the solution again. We need to continue this process until we get acceptance and sign off; otherwise, we will not get to the final solution deployment phase. When the project has been signed off after a successful pilot test, there is no reason why you cannot deploy the technology in production. Now that we have talked about how to prove the technology and successfully demonstrated that it delivers against both our business case and user requirements, in the next sections, we are going to start looking at the design for our production environment. Designing a Horizon 6.0 architecture We are going to start this section by looking at the VMware reference architecture for Horizon View 6.0 before we go into more detail around the design considerations, best practice, and then sizing guidelines. The pod and block reference architecture VMware has produced a reference architecture model for deploying Horizon View, with the approach being to make it easy to scale the environment by adding set component pieces of infrastructure, known as View blocks. To scale the number of users, you add View blocks up to the maximum configuration of five blocks. This maximum configuration of five View blocks is called a View pod. The important numbers to remember are that each View block supports up to a maximum of 2,000 users, and a View pod is made up of up to five View blocks, therefore supporting a maximum of 10,000 users. The View block contains all the infrastructure required to host the virtual desktop machines, so appropriately sized ESXi hosts, a vCenter Server, and the associated networking and storage requirements. We will cover the sizing aspects later on in this article. The following diagram shows an individual View block: Apart from having a View block that supports the virtual desktop machines, there is also a management block for the supporting infrastructure components. The management block contains the management elements of Horizon View, such as the connection servers and security servers. These will also be virtual machines hosted on the vSphere platform but using separate ESXi hosts and vCenter servers from those being used to host the desktops. The following diagram shows a typical View management block: The management block contains the key Horizon View components to support the maximum configuration of 10,000 users or a View pod. In terms of connection servers, the management block consists of a maximum of seven connection servers. This is often written as 5 + 2, which can be misleading, but what it means is you can have five connection servers and two that serve as backups to replace a failed server. Each connection server supports one of the five blocks, with the two spare in reserve in the event of a failure. As we discussed previously, the View Security Servers are paired with one of the connection servers in order to provide external access to the users. In our example diagram, we have drawn three security servers meaning that these servers are configured for external access, while the others serve the internal users only. In this scenario, the View Connection Servers and View Security Servers are deployed as virtual machines, and are therefore controlled and managed by vCenter. The vCenter Server can run on a virtual machine, or you can use the vCenter Virtual Appliance. It can also run on a physical Windows Server, as it's just a Windows application. The entire infrastructure is hosted on a vSphere cluster that's separate, from that being used to host the virtual desktop machines. There are a couple of other components that are not shown in the diagram, and those are the databases required for View such as the events database and for View Composer. If we now look at the entire Horizon View pod and block architecture for up to 10,000 users, the architecture design would look something like the following diagram: One thing to note is that although a pod is limited to 10,000 users, you can deploy more than one pod should you need an environment that exceeds the 10,000 users. Bear in mind though that the pods do not communicate with each other and will effectively be completely separate deployments. As this is potentially a limitation in the scalability, but more so for disaster recovery purposes, where you need to have two pods across two sites for disaster recovery, there is a feature in Horizon View 6.0 that allows you to deploy pods across sites. This is called the Cloud Pod Architecture (CPA), and we will cover this in the next section. The Cloud Pod Architecture The Cloud Pod Architecture, also referred to as linked-mode View (LMV) or multidatacenter View (MDCV), allows you to link up to four View pods together across two sites, with a maximum number of supported users of up to 20,000. There are four key features available by deploying Horizon View using this architecture: Scalability: This hosts more than 10,000 users on a single site Multidatacenter support: This supports View across more than one data center Geo roaming: This supports roaming desktops for users moving across sites DR: This delivers resilience in the event of a data center failure Let's take a look at the Cloud Pod Architecture in the following diagram to explain the features and how it builds on the pod and block architecture we discussed previously: With the Cloud Pod Architecture, user information is replicated globally and the pods are linked using the View interpod API (VIPA)—the setup for which is command-line-based. For scalability, with the Cloud Pod Architecture model, you have the ability to entitle users across pools on both different pods and sites. This means that, if you have already scaled beyond a single pod, you can link the pods together to allow you to go beyond the 10,000 user limit and also administer your users from a single location. The pods can, apart from being located on the same site, also be on two different sites to deliver a mutlidatacenter configuration running as active/active. This also introduces DR capabilities. In the event of one of the data centers failing or losing connectivity, users will still be able to connect to a virtual desktop machine. Users don't need to worry about what View Connection Server they need to use to connect to their virtual desktop machine. The Cloud Pod Architecture supports a single namespace with access via a global URL. As users can now connect from anywhere, there are some configuration options that you need to consider as to how they access their virtual desktop machine and from where it gets delivered. There are three options that form part of the global user entitlement feature: Any: This is delivered from any pod as part of the global entitlement Site: This is delivered from any pod from the same site the user is connecting from Local: This is delivered only from the local pod that the user is connected to It's not just the users that get the global experience; the administrators can also be segregated in this way so that you can deliver delegated management. Administration of pods could be delegated to the local IT teams on a per region/geo basis, with some operations such as provisioning and patching performed locally on the local pods or maybe it's so that local language support can be delivered. It is only global policy that is managed globally, typically from an organizations global HQ. Now that we have covered some of the high-level architecture options, you should now be able to start to look at your overall design, factoring in locations and the number of users. In the next section, we will start to look at how to size some of these components. Sizing the infrastructure In this section, we are going to discuss the sizing of the components previously described in the architecture section. We will start by looking at the management blocks containing the connection servers, security servers, and then the servers that host the desktops before finishing off with the desktops themselves. The management block and the block hosting the virtual desktop machines should be run on separate infrastructure (ESXi hosts and vCenter Servers); the reason being due to the different workload patterns between servers and desktops and to avoid performance issues. It's also easier to manage, as you can determine what desktops are and what servers are, but more importantly it's also the way in which the products are licensed. With vSphere for desktop that comes with Horizon View, it only entitles you to run workloads that are hosting and managing the virtual desktop infrastructure. Summary In this article, you learned how to design a Horizon 6.0 architecture. Resources for Article: Further resources on this subject: Backups in the VMware View Infrastructure [Article] Setting up of Software Infrastructure on the Cloud [Article] Introduction to Veeam® Backup & Replication for VMware [Article]
Read more
  • 0
  • 0
  • 3660

article-image-implementing-stacks-using-javascript
Packt
22 Oct 2014
10 min read
Save for later

Implementing Stacks using JavaScript

Packt
22 Oct 2014
10 min read
 In this article by Loiane Groner, author of the book Learning JavaScript Data Structures and Algorithms, we will discuss the stacks. (For more resources related to this topic, see here.) A stack is an ordered collection of items that follows the LIFO (short for Last In First Out) principle. The addition of new items or the removal of existing items takes place at the same end. The end of the stack is known as the top and the opposite is known as the base. The newest elements are near the top, and the oldest elements are near the base. We have several examples of stacks in real life, for example, a pile of books, as we can see in the following image, or a stack of trays from a cafeteria or food court: A stack is also used by compilers in programming languages and by computer memory to store variables and method calls. Creating a stack We are going to create our own class to represent a stack. Let's start from the basics and declare our class: function Stack() {   //properties and methods go here} First, we need a data structure that will store the elements of the stack. We can use an array to do this: Var items = []; Next, we need to declare the methods available for our stack: push(element(s)): This adds a new item (or several items) to the top of the stack. pop(): This removes the top item from the stack. It also returns the removed element. peek(): This returns the top element from the stack. The stack is not modified (it does not remove the element; it only returns the element for information purposes). isEmpty(): This returns true if the stack does not contain any elements and false if the size of the stack is bigger than 0. clear(): This removes all the elements of the stack. size(): This returns how many elements the stack contains. It is similar to the length property of an array. The first method we will implement is the push method. This method will be responsible for adding new elements to the stack with one very important detail: we can only add new items to the top of the stack, meaning at the end of the stack. The push method is represented as follows: this.push = function(element){   items.push(element);}; As we are using an array to store the elements of the stack, we can use the push method from the JavaScript array class. Next, we are going to implement the pop method. This method will be responsible for removing the items from the stack. As the stack uses the LIFO principle, the last item that we added is the one that is removed. For this reason, we can use the pop method from the JavaScript array class. The pop method is represented as follows: this.pop = function(){   return items.pop();}; With the push and pop methods being the only methods available for adding and removing items from the stack, the LIFO principle will apply to our own Stack class. Now, let's implement some additional helper methods for our class. If we would like to know what the last item added to our stack was, we can use the peek method. This method will return the item from the top of the stack: this.peek = function(){   return items[items.length-1];}; As we are using an array to store the items internally, we can obtain the last item from an array using length - 1 as follows: For example, in the previous diagram, we have a stack with three items; therefore, the length of the internal array is 3. The last position used in the internal array is 2. As a result, the length - 1 (3 - 1) is 2! The next method is the isEmpty method, which returns true if the stack is empty (no item has been added) and false otherwise: this.isEmpty = function(){   return items.length == 0;}; Using the isEmpty method, we can simply verify whether the length of the internal array is 0. Similar to the length property from the array class, we can also implement length for our Stack class. For collections, we usually use the term "size" instead of "length". And again, as we are using an array to store the items internally, we can simply return its length: this.size = function(){   return items.length;}; Finally, we are going to implement the clear method. The clear method simply empties the stack, removing all its elements. The simplest way of implementing this method is as follows: this.clear = function(){   items = [];}; An alternative implementation would be calling the pop method until the stack is empty. And we are done! Our Stack class is implemented. Just to make our lives easier during the examples, to help us inspect the contents of our stack, let's implement a helper method called print that is going to output the content of the stack on the console: this.print = function(){   console.log(items.toString());}; And now we are really done! The complete Stack class Let's take a look at how our Stack class looks after its full implementation: function Stack() {    var items = [];    this.push = function(element){       items.push(element);   };    this.pop = function(){       return items.pop();   };    this.peek = function(){       return items[items.length-1];   };    this.isEmpty = function(){       return items.length == 0;   };    this.size = function(){       return items.length;   };    this.clear = function(){       items = [];   };    this.print = function(){       console.log(items.toString());   };} Using the Stack class Before we dive into some examples, we need to learn how to use the Stack class. The first thing we need to do is instantiate the Stack class we just created. Next, we can verify whether it is empty (the output is true because we have not added any elements to our stack yet): var stack = new Stack();console.log(stack.isEmpty()); //outputs true Next, let's add some elements to it (let's push the numbers 5 and 8; you can add any element type to the stack): stack.push(5);stack.push(8); If we call the peek method, the output will be the number 8 because it was the last element that was added to the stack: console.log(stack.peek()); // outputs 8 Let's also add another element: stack.push(11);console.log(stack.size()); // outputs 3console.log(stack.isEmpty()); //outputs false We added the element 11. If we call the size method, it will give the output as 3, because we have three elements in our stack (5, 8, and 11). Also, if we call the isEmpty method, the output will be false (we have three elements in our stack). Finally, let's add another element: stack.push(15); The following diagram shows all the push operations we have executed so far and the current status of our stack: Next, let's remove two elements from the stack by calling the pop method twice: stack.pop();stack.pop();console.log(stack.size()); // outputs 2stack.print(); // outputs [5, 8] Before we called the pop method twice, our stack had four elements in it. After the execution of the pop method two times, the stack now has only two elements: 5 and 8. The following diagram exemplifies the execution of the pop method: Decimal to binary Now that we know how to use the Stack class, let's use it to solve some Computer Science problems. You are probably already aware of the decimal base. However, binary representation is very important in Computer Science as everything in a computer is represented by binary digits (0 and 1). Without the ability to convert back and forth between decimal and binary numbers, it would be a little bit difficult to communicate with a computer. To convert a decimal number to a binary representation, we can divide the number by 2 (binary is base 2 number system) until the division result is 0. As an example, we will convert the number 10 into binary digits: This conversion is one of the first things you learn in college (Computer Science classes). The following is our algorithm: function divideBy2(decNumber){    var remStack = new Stack(),       rem,       binaryString = '';    while (decNumber > 0){ //{1}       rem = Math.floor(decNumber % 2); //{2}       remStack.push(rem); //{3}       decNumber = Math.floor(decNumber / 2); //{4} }    while (!remStack.isEmpty()){ //{5}       binaryString += remStack.pop().toString();   }    return binaryString;} In this code, while the division result is not zero (line {1}), we get the remainder of the division (mod) and push it to the stack (lines {2} and {3}), and finally, we update the number that will be divided by 2 (line {4}). An important observation: JavaScript has a numeric data type, but it does not distinguish integers from floating points. For this reason, we need to use the Math.floor function to obtain only the integer value from the division operations. And finally, we pop the elements from the stack until it is empty, concatenating the elements that were removed from the stack into a string (line {5}). We can try the previous algorithm and output its result on the console using the following code: console.log(divideBy2(233));console.log(divideBy2(10));console.log(divideBy2(1000)); We can easily modify the previous algorithm to make it work as a converter from decimal to any base. Instead of dividing the decimal number by 2, we can pass the desired base as an argument to the method and use it in the divisions, as shown in the following algorithm: function baseConverter(decNumber, base){    var remStack = new Stack(),        rem,       baseString = '',       digits = '0123456789ABCDEF'; //{6}    while (decNumber > 0){       rem = Math.floor(decNumber % base);       remStack.push(rem);       decNumber = Math.floor(decNumber / base);   }    while (!remStack.isEmpty()){       baseString += digits[remStack.pop()]; //{7}   }    return baseString;} There is one more thing we need to change. In the conversion from decimal to binary, the remainders will be 0 or 1; in the conversion from decimal to octagonal, the remainders will be from 0 to 8; but in the conversion from decimal to hexadecimal, the remainders can be 0 to 8 plus the letters A to F (values 10 to 15). For this reason, we need to convert these values as well (lines {6} and {7}). We can use the previous algorithm and output its result on the console as follows: console.log(baseConverter(100345, 2));console.log(baseConverter(100345, 8));console.log(baseConverter(100345, 16)); Summary In this article, we learned about the stack data structure. We implemented our own algorithm that represents a stack and we learned how to add and remove elements from it using the push and pop methods. We also covered a very famous example of how to use a stack. Resources for Article: Further resources on this subject: Organizing Backbone Applications - Structure, Optimize, and Deploy [article] Introduction to Modern OpenGL [article] Customizing the Backend Editing in TYPO3 Templates [article]
Read more
  • 0
  • 0
  • 4827
article-image-introduction-s4-classes
Packt
22 Oct 2014
36 min read
Save for later

Introduction to S4 Classes

Packt
22 Oct 2014
36 min read
In this article, by Kelly Black, the author of the R Object-oriented Programming book, will examine S4 classes. The approach associated with S3 classes is more flexible, and the approach associated with S4 classes is a more formal and structured definition. This article is roughly divided into four parts: Class definition: This section gives you an overview of how a class is defined and how the data (slots) associated with the class are specified Class methods: This section gives you an overview of how methods that are associated with a class are defined Inheritance: This section gives you an overview of how child classes that build on the definition of a parent class can be defined Miscellaneous commands: This section explains four commands that can be used to explore a given object or class (For more resources related to this topic, see here.) Introducing the Ant class We will introduce the idea of S4 classes, which is a more formal way to implement classes in R. One of the odd quirks of S4 classes is that you first define the class along with its data, and then, you define the methods separately. As a result of this separation in the way a class is defined, we will first discuss the general idea of how to define a class and its data. We will then discuss how to add a method to an existing class. Next, we will discuss how inheritance is implemented. Finally, we will provide a few notes about other options that do not fit nicely in the categories mentioned earlier. The approach associated with an S4 class is less flexible and requires a bit more forethought in terms of how a class is defined. We will take a different approach and create a complete class from the beginning. In this case, we will build on an idea proposed by Cole and Cheshire. The authors proposed a cellular automata simulation to mimic how ants move within a colony. As part of a simulation, we will assume that we need an Ant class. We will depart from the paper and assume that the ants are not homogeneous. We will then assume that there are male (drones) and female ants, and the females can be either workers or soldiers. We will need an ant base class, which is discussed in the first two sections of this article as a means to demonstrate how to create an S4 class. In the third section, we will define a hierarchy of classes based on the original Ant class. This hierarchy includes male and female classes. The worker class will then inherit from the female class, and the soldier class will inherit from the worker class. Defining an S4 class We will define the base Ant class called Ant. The class is represented in the following figure. The class is used to represent the fundamental aspects that we need to track for an ant, and we focus on creating the class and data. The methods are constructed in a separate step and are examined in the next section. A class is created using the setClass command. When creating the class, we specify the data in a character vector using the slots argument. The slots argument is a vector of character objects and represents the names of the data elements. These elements are often referred to as the slots within the class. Some of the arguments that we will discuss here are optional, but it is a good practice to use them. In particular, we will specify a set of default values (the prototype) and a function to check whether the data is consistent (a validity function). Also, it is a good practice to keep all of the steps necessary to create a class within the same file. To that end, we assume that you will not be entering the commands from the command line. They are all found within a single file, so the formatting of the examples will reflect the lack of the R workspace markers. The first step is to define the class using the setClass command. This command defines a new class by name, and it also returns a generator that can be used to construct an object for the new class. The first argument is the name of the class followed by the data to be included in the class. We will also include the default initial values and the definition of the function used to ensure that the data is consistent. The validity function can be set separately using the setValidity command. The data types for the slots are character values that match the names of the R data types which will be returned by the class command: # Define the base Ant class. Ant <- setClass(    # Set the name of the class    "Ant",    # Name the data types (slots) that the class will track    slots = c(        Length="numeric",           # the length (size) of this ant.               Position="numeric",         # the position of this ant.                                    # (a 3 vector!)               pA="numeric",               # Probability that an ant will                                    # transition from active to                                     # inactive.        pI="numeric",               # Probability that an ant will                                    # transition from inactive to                                    # active.          ActivityLevel="numeric"     # The ant's current activity                            # level.        ),    # Set the default values for the slots. (optional)    prototype=list(        Length=4.0,        Position=c(0.0,0.0,0.0),        pA=0.05,        pI=0.1,        ActivityLevel=0.5        ),    # Make a function that can test to see if the data is consistent.    # (optional)    validity=function(object)    {        # Check to see if the activity level and length is        # non-negative.        # See the discussion on the @ notation in the text below.        if(object@ActivityLevel<0.0) {            return("Error: The activity level is negative")        } else if (object@Length<0.0) {            return("Error: The length is negative")        }        return(TRUE)  }    ) With this definition, there are two ways to create an Ant object: one is using the new command and the other is using the Ant generator, which is created after the successful execution of the setClass command. Note that in the following examples, the default values can be overridden when a new object is created: > ant1 <- new("Ant") > ant1 An object of class "Ant" Slot "Length": [1] 4 Slot "Position": [1] 0 0 0 Slot "pA": [1] 0.05 Slot "pI": [1] 0.1 Slot "ActivityLevel": [1] 0.5 We can specify the default values when creating a new object. > ant2 <- new("Ant",Length=4.5) > ant2 An object of class "Ant" Slot "Length": [1] 4.5 Slot "Position": [1] 0 0 0 Slot "pA": [1] 0.05 Slot "pI": [1] 0.1 Slot "ActivityLevel": [1] 0.5 The object can also be created using the generator that is defined when creating the class using the setClass command. > ant3 <- Ant(Length=5.0,Position=c(3.0,2.0,1.0)) > ant3 An object of class "Ant" Slot "Length": [1] 5 Slot "Position": [1] 3 2 1 Slot "pA": [1] 0.05 Slot "pI": [1] 0.1 Slot "ActivityLevel": [1] 0.5 > class(ant3) [1] "Ant" attr(,"package") [1] ".GlobalEnv" > getClass(ant3) An object of class "Ant" Slot "Length": [1] 5 Slot "Position": [1] 3 2 1 Slot "pA": [1] 0.05 Slot "pI": [1] 0.1 Slot "ActivityLevel": [1] 0.5 When the object is created and a validity function is defined, the validity function will determine whether the given initial values are consistent: > ant4 <- Ant(Length=-1.0,Position=c(3.0,2.0,1.0)) Error in validObject(.Object) : invalid class “Ant” object: Error: The length is negative > ant4 Error: object 'ant4' not found In the last steps, the attempted creation of ant4, an error message is displayed. The new variable, ant4, was not created. If you wish to test whether the object was created, you must be careful to ensure that the variable name used does not exist prior to the attempted creation of the new object. Also, the validity function is only executed when a request to create a new object is made. If you change the values of the data later, the validity function is not called. Before we move on to discuss methods, we need to figure out how to get access to the data within an object. The syntax is different from other data structures, and we use @ to indicate that we want to access an element from within the object. This can be used to get a copy of the value or to set the value of an element: > adomAnt <- Ant(Length=5.0,Position=c(-1.0,2.0,1.0)) > adomAnt@Length [1] 5 > adomAnt@Position [1] -1 2 1 > adomAnt@ActivityLevel = -5.0 > adomAnt@ActivityLevel [1] -5 Note that in the preceding example, we set a value for the activity level that is not allowed according to the validity function. Since it was set after the object was created, no check is performed. The validity function is only executed during the creation of the object or if the validObject function is called. One final note: it is generally a bad form to work directly with an element within an object, and a better practice is to create methods that obtain or change an individual element within an object. It is a best practice to be careful about the encapsulation of an object's slots. The R environment does not recognize the idea of private versus public data, and the onus is on the programmer to maintain discipline with respect to this important principle. Defining methods for an S4 class When a new class is defined, the data elements are defined, but the methods associated with the class are defined on a separate stage. Methods are implemented in a manner similar to the one used for S3 classes. A function is defined, and the way the function reacts depends on its arguments. If a method is used to change one of the data components of an object, then it must return a copy of the object, just as we saw with S3 classes. The creation of new methods is discussed in two steps. We will first discuss how to define a method for a class where the method does not yet exist. Next, we will discuss some predefined methods that are available and how to extend them to accommodate a new class. Defining new methods The first step to create a new method is to reserve the name. Some functions are included by default, such as the initialize, print or show commands, and we will later see how to extend them. To reserve a new name, you must first use the setGeneric command. At the very least, you need to give this command the name of the function as a character string. As in the previous section, we will use more options as an attempt to practice safe programming. The methods to be created are shown in preceding figure. There are a number of methods, but we will only define four here. All of the methods are accessors; they are used to either get or set values of the data components. We will only define the methods associated with the length slot in this text, and you can see the rest of the code in the examples available on the website. The other methods closely follow the code used for the length slot. There are two methods to set the activity level, and those codes are examined separately to provide an example of how a method can be overloaded. First, we will define the methods to get and set the length. We will first create the method to get the length, as it is a little more straightforward. The first step is to tell R that a new function will be defined, and the name is reserved using the setGeneric command. The method that is called when an Ant object is passed to the command is defined using the setMethod command: setGeneric(name="GetLength",            def=function(antie)            {                standardGeneric("GetLength")            }            ) setMethod(f="GetLength",          signature="Ant",          definition=function(antie)          {              return(antie@Length)          }          ) Now that the GetLength function is defined, it can be used to get the length component for an Ant object: > ant2 <- new("Ant",Length=4.5) > GetLength(ant2) [1] 4.5 The method to set the length is similar, but there is one difference. The method must return a copy of the object passed to it, and it requires an additional argument: setGeneric(name="SetLength",            def=function(antie,newLength)            {                standardGeneric("SetLength")            }            ) setMethod(f="SetLength",          signature="Ant",          definition=function(antie,newLength)          {              if(newLength>0.0) {                  antie@Length = newLength              } else {                  warning("Error - invalid length passed");              }              return(antie)           }          ) When setting the length, the new object must be set using the object that is passed back from the function: > ant2 <- new("Ant",Length=4.5) > ant2@Length [1] 4.5 > ant2 <- SetLength(ant2,6.25) > ant2@Length [1] 6.25 Polymorphism The definition of S4 classes allows methods to be overloaded. That is, multiple functions that have the same name can be defined, and the function that is executed is determined by the arguments' types. We will now examine this idea in the context of defining the methods used to set the activity level in the Ant class. Two or more functions can have the same name, but the types of the arguments passed to them differ. There are two methods to set the activity level. One takes a floating point number and sets the activity level based to the value passed to it. The other takes a logical value and sets the activity level to zero if the argument is FALSE; otherwise, it sets it to a default value. The idea is to use the signature option in the setMethod command. It is set to a vector of class names, and the order of the class names is used to determine which function should be called for a given set of arguments. An important thing to note, though, is that the prototype defined in the setGeneric command defines the names of the arguments, and the argument names in both methods must be exactly the same and in the same order: setGeneric(name="SetActivityLevel",            def=function(antie,activity)            {                standardGeneric("SetActivityLevel")            }          ) setMethod(f="SetActivityLevel",          signature=c("Ant","logical"),          definition=function(antie,activity)          {              if(activity) {                  antie@ActivityLevel = 0.1              } else {                  antie@ActivityLevel = 0.0              }              return(antie)          }          ) setMethod(f="SetActivityLevel",          signature=c("Ant","numeric"),          definition=function(antie,activity)          {              if(activity>=0.0) {                  antie@ActivityLevel = activity              } else {                  warning("The activity level cannot be negative")              }              return(antie)          }          ) Once the two methods are defined, R will use the class names of the arguments to determine which function to call in a given context: > ant2 <- SetActivityLevel(ant2,0.1) > ant2@ActivityLevel [1] 0.1 > ant2 <- SetActivityLevel(ant2,FALSE) > ant2@ActivityLevel [1] 0 There are two additional data types recognized by the signature option: ANY and missing. These can be used to match any data type or a missing value. Also note that we have left out the use of ellipses (…) for the arguments in the preceding examples. The … argument must be the last argument and is used to indicate that any remaining parameters are passed as they appear in the original call to the function. Ellipses can make the use of the overloaded functions in a more flexible way than indicated. More information can be found using the help(dotsMethods) command. Extending the existing methods There are a number of generic functions defined in a basic R session, and we will examine how to extend an existing function. For example, the show command is a generic function whose behavior depends on the class name of the object passed to it. Since the function name is already reserved, the setGeneric command is not used to reserve the function name. The show command is a standard example. The command takes an object and converts it to a character value to be displayed. The command defines how other commands print out and express an object. In the preceding example, a new class called coordinate is defined; this keeps track of two values, x and y, for a coordinate, and we will add one method to set the values of the coordinate: # Define the base coordinates class. Coordinate <- setClass(    # Set the name of the class    "Coordinate",    # Name the data types (slots) that the class will track    slots = c(        x="numeric", # the x position      y="numeric"   # the y position        ),    # Set the default values for the slots. (optional)    prototype=list(        x=0.0,        y=0.0        ),    # Make a function that can test to see if the data    # is consistent.    # (optional)    # This is not called if you have an initialize    # function defined!    validity=function(object)    {        # Check to see if the coordinate is outside of a circle of        # radius 100        print("Checking the validity of the point")        if(object@x*object@x+object@y*object@y>100.0*100.0) {        return(paste("Error: The point is too far ",        "away from the origin."))       }        return(TRUE)    }    ) # Add a method to set the value of a coordinate setGeneric(name="SetPoint",            def=function(coord,x,y)            {                standardGeneric("SetPoint")            }            ) setMethod(f="SetPoint",          signature="Coordinate",          def=function(coord,x,y)          {              print("Setting the point")              coord@x = x              coord@y = y              return(coord)          }          ) We will now extend the show method so that it can properly react to a coordinate object. As it is reserved, we do not have to use the setGeneric command but can simply define it: setMethod(f="show",          signature="Coordinate",          def=function(object)          {              cat("The coordinate is X: ",object@x," Y: ",object@y,"n")          }          ) As noted previously, the signature option must match the original definition of a function that you wish to extend. You can use the getMethod('show') command to examine the signature for the function. With the new method in place, the show command is used to convert a coordinate object to a string when it is printed: > point <- Coordinate(x=1,y=5) [1] "Checking the validity of the point" > print(point) The coordinate is X: 1 Y: 5 > point The coordinate is X: 1 Y: 5 Another import predefined method is the initialize command. If the initialize command is created for a class, then it is called when a new object is created. That is, you can define an initialize function to act as a constructor. If an initialize function is defined for a class, the validator is not called. You have to manually call the validator using the validObject command. Also note that the prototype for the initialize command requires the name of the first argument to be an object, and the default values are given for the remaining arguments in case a new object is created without specifying any values for the slots: setMethod(f="initialize",          signature="Coordinate",          def=function(.Object,x=0.0,y=0.0)          {              print("Checking the point")              .Object = SetPoint(.Object,x,y)              validObject(.Object) # you must explicitly call              # the inspector              return(.Object)          }          ) Now, when you create a new object, the new initialize function is called immediately: > point <- Coordinate(x=2,y=3) [1] "Checking the point" [1] "Setting the point" [1] "Checking the validity of the point" > point The coordinate is X: 2 Y: 3 Using the initialize and validity functions together can result in surprising code paths. This is especially true when inheriting from one class and calling the initialize function of a parent class from the child class. It is important to test codes to ensure that the code is executing in the order that you expect. Personally, I try to use either validator or constructor, but not both. Inheritance The Ant class discussed in the first section of this article provided an example of how to define a class and then define the methods associated with the class. We will now extend the class by creating new classes that inherit from the base class. The original Ant class is shown in the preceding figure, and now, we will propose four classes that inherit from the base class. Two new classes that inherit from Ant are the Male and Female classes. The Worker class inherits from the Female class, while the Soldier class inherits from the Worker class. The relationships are shown in the following figure. The code for all of the new classes is included in our example codes available at our website, but we will only focus on two of the new classes in the text to keep our discussion more focused. Relationships between the classes that inherit from the base Ant class When a new class is created, it can inherit from an existing class by setting the contains parameter. This can be set to a vector of classes for multiple inheritance. However, we will focus on single inheritance here to avoid discussing the complications associated with determining how R finds a method when there are collisions. Assuming that the Ant base class given in the first section has already been defined in the current session, the child classes can be defined. The details for the two classes, Female and Worker, are discussed here. First, the FemaleAnt class is defined. It adds a new slot, Food, and inherits from the Ant class. Before defining the FemaleAnt class, we add a caveat about the Ant class. The base Ant class should have been a virtual class. We would not ordinarily create an object of the Ant class. We did not make it a virtual class in order to simplify our introduction. We are wiser now and wish to demonstrate how to define a virtual class. The FemaleAnt class will be a virtual class to demonstrate the idea. We will make it a virtual class by including the VIRTUAL character string in the contains parameter, and it will not be possible to create an object of the FemaleAnt class: # Define the female ant class. FemaleAnt <- setClass(    # Set the name of the class    "FemaleAnt",    # Name the data types (slots) that the class will track    slots = c(        Food ="numeric"     # The number of food units carried        ),    # Set the default values for the slots. (optional)    prototype=list(        Food=0        ),    # Make a function that can test to see if the data is consistent.    # (optional)    # This is not called if you have an initialize function defined!    validity=function(object)    {        print("Validity: FemaleAnt")        # Check to see if the number of offspring is non-negative.        if(object@Food<0) {            return("Error: The number of food units is negative")        }        return(TRUE)    },    # This class inherits from the Ant class    contains=c("Ant","VIRTUAL")    ) Now, we will define a WorkerAnt class that inherits from the FemaleAnt class: # Define the worker ant class. WorkerAnt <- setClass(    # Set the name of the class    "WorkerAnt",    # Name the data types (slots) that the class will track    slots = c(        Foraging ="logical",   # Whether or not the ant is actively                                # looking for food        Alarm = "logical"       # Whether or not the ant is actively                                # announcing an alarm.               ),    # Set the default values for the slots. (optional)    prototype=list(        Foraging = FALSE,        Alarm   = FALSE        ),    # Make a function that can test to see if the data is consistent.    # (optional)    # This is not called if you have an initialize function defined!    validity=function(object)    {        print("Validity: WorkerAnt")        return(TRUE)    },    # This class inherits from the FemaleAnt class    contains="FemaleAnt"    ) When a new worker is created, it inherits from the FemaleAnt class: > worker <- WorkerAnt(Position=c(-1,3,5),Length=2.5) > worker An object of class "WorkerAnt" Slot "Foraging": [1] FALSE Slot "Alarm": [1] FALSE Slot "Food": [1] 0 Slot "Length": [1] 2.5 Slot "Position": [1] -1 3 5 Slot "pA": [1] 0.05 Slot "pI": [1] 0.1 Slot "ActivityLevel": [1] 0.5 > worker <- SetLength(worker,3.5) > GetLength(worker) [1] 3.5 We have not defined the relevant methods in the preceding examples. The code is available in our set of examples, and we will not discuss most of it to keep this discussion more focused. We will examine the initialize method, though. The reason to do so is to explore the callNextMethod command. The callNextMethod command is used to request that R searches for and executes a method of the same name that is a member of a parent class. We chose the initialize method because a common task is to build a chain of constructors that initialize the data associated for the class associated with each constructor. We have not yet created any of the initialize methods and start with the base Ant class: setMethod(f="initialize",          signature="Ant",          def=function(.Object,Length=4,Position=c(0.0,0.0,0.0))          {              print("Ant initialize")              .Object = SetLength(.Object,Length)              .Object = SetPosition(.Object,Position)              #validObject(.Object) # you must explicitly call the inspector              return(.Object)          }          ) The constructor takes three arguments: the object itself (.Object), the length, and the position of the ant, and default values are given in case none are provided when a new object is created. The validObject command is commented out. You should try uncommenting the line and create new objects to see whether the validator can in turn call the initialize method. Another important feature is that the initialize method returns a copy of the object. The initialize command is created for the FemaleAnt class, and the arguments to the initialize command should be respected when the request to callNextMethod for the next function is made: setMethod(f="initialize",          signature="FemaleAnt",          def=function(.Object,Length=4,Position=c(0.0,0.0,0.0))          {              print("FemaleAnt initialize ")              .Object <- callNextMethod(.Object,Length,Position)              #validObject(.Object) # you must explicitly call              #the inspector              return(.Object)          }          ) The callNextMethod command is used to call the initialize method associated with the Ant class. The arguments are arranged to match the definition of the Ant class, and it returns a new copy of the current object. Finally, the initialize function for the WorkerAnt class is created. It also makes use of callNextMethod to ensure that the method of the same name associated with the parent class is also called: setMethod(f="initialize",          signature="WorkerAnt",          def=function(.Object,Length=4,Position=c(0.0,0.0,0.0))          {              print("WorkerAnt initialize")             .Object <- callNextMethod(.Object,Length,Position)              #validObject(.Object) # you must explicitly call the inspector              return(.Object)          }          ) Now, when a new object of the WorkerAnt class is created, the initialize method associated with the WorkerAnt class is called, and each associated method for each parent class is called in turn: > worker <- WorkerAnt(Position=c(-1,3,5),Length=2.5) [1] "WorkerAnt initialize" [1] "FemaleAnt initialize " [1] "Ant initialize" Miscellaneous notes In the previous sections, we discussed how to create a new class as well as how to define a hierarchy of classes. We will now discuss four commands that are helpful when working with classes: the slotNames, getSlots, getClass, and slot commands. Each command is briefly discussed in turn, and it is assumed that the Ant, FemaleAnt, and WorkerAnt classes that are given in the previous section are defined in the current workspace. The first command, the slotnames command, is used to list the data components of an object of some class. It returns the names of each component as a vector of characters: > worker <- WorkerAnt(Position=c(1,2,3),Length=5.6) > slotNames(worker) [1] "Foraging"     "Alarm"         "Food"         "Length"       [5] "Position"     "pA"           "pI"           "ActivityLevel" The getSlots command is similar to the slotNames command. The difference is that the argument is a character variable which is the name of the class you want to investigate: > getSlots("WorkerAnt")      Foraging         Alarm         Food       Length     Position    "logical"     "logical"     "numeric"     "numeric"     "numeric"            pA           pI ActivityLevel    "numeric"     "numeric"     "numeric" The getClass command has two forms. If the argument is an object, the command will print out the details for the object. If the argument is a character string, then it will print out the details for the class whose name is the same as the argument: > worker <- WorkerAnt(Position=c(1,2,3),Length=5.6) > getClass(worker) An object of class "WorkerAnt" Slot "Foraging": [1] FALSE Slot "Alarm": [1] FALSE Slot "Food": [1] 0 Slot "Length": [1] 5.6 Slot "Position": [1] 1 2 3 Slot "pA": [1] 0.05 Slot "pI": [1] 0.1 Slot "ActivityLevel": [1] 0.5 > getClass("WorkerAnt") Class "WorkerAnt" [in ".GlobalEnv"] Slots:                                                                            Name:       Foraging         Alarm         Food       Length     Position Class:       logical      logical       numeric       numeric       numeric                                                Name:             pA           pI ActivityLevel Class:       numeric       numeric       numeric Extends: Class "FemaleAnt", directly Class "Ant", by class "FemaleAnt", distance 2 Known Subclasses: "SoldierAnt" Finally, we will examine the slot command. The slot command is used to retrieve the value of a slot for a given object based on the name of the slot: > worker <- WorkerAnt(Position=c(1,2,3),Length=5.6) > slot(worker,"Position") [1] 1 2 3 Summary We introduced the idea of an S4 class and provided several examples. The S4 class is constructed in at least two stages. The first stage is to define the name of the class and the associated data components. The methods associated with the class are then defined in a separate step. In addition to defining a class and its method, the idea of inheritance was explored. A partial example was given in this article; it built on a base class defined in the first section of the article. Additionally, the method to call-associated methods in parent classes was also explored, and the example made use of the constructor (or initialize method) to demonstrate how to build a chain of constructors. Finally, four useful commands were explained. The four commands offered different ways to get information about a class or about an object of a given class. For more information, you can refer to Mobile Cellular Automata Models of Ant Behavior: Movement Activity of Leptothorax allardycei, Blaine J. Cole and David Cheshire, The American Naturalist. Resources for Article: Further resources on this subject: Using R for Statistics, Research, and Graphics [Article] Learning Data Analytics with R and Hadoop [Article] First steps with R [Article]
Read more
  • 0
  • 0
  • 2683

article-image-important-aspect-angularjs-ui-development
Packt
22 Oct 2014
20 min read
Save for later

Important Aspect of AngularJS UI Development

Packt
22 Oct 2014
20 min read
In this article by Matthias Nehlsen, the co-author of the book, AngularJS UI Development, has explained about managing client-side dependencies with Bower. He also explains how to build an application, running Protractor from Grunt, and managing the source code with Git. He will also explain about building AngularUI Utils and integrating AngularUI-Utils into our project. (For more resources related to this topic, see here.) Managing client-side dependencies with Bower We download AngularJS and placed the angular.js file in our directory structure manually. This was not so bad for a single file, but this process will very soon become tedious and error-prone when dealing with multiple dependencies. Luckily, there is a great tool for managing client-side dependencies, and it can be found at http://bower.io. Bower allows us to record which dependencies we need for an application. Then, after downloading the application for the first time, we can simply run bower install and it will download all the libraries and assets specified in the configuration file for us. First, we will need to install Bower (potentially with sudo): # npm install -g bower Now, let's try it out by running the following command: # bower install angular You will notice an error message when running the previous command on a machine that does not have Git installed. Okay, this will download AngularJS and place the files in the app/bower_components/ folder. However, our remaining sources are in the src/ folder, so let's store the Bower files here as well. Create a file named .bowerrc in the project root, with these three lines in it: { "directory": "src/bower" } This will tell Bower that we want the managed dependencies inside the src/bower/ folder. Now, remove the app/ folder and run the earlier bower install command one more time. You should see the AngularJS files in the src/bower/ folder now. Now, we said we wanted to record the dependencies in a configuration file so that we can later run bower install after downloading/checking out the application. Why can't we just store all the dependencies in our version control system? Of course, we can, but this would bloat the repository a lot. Instead, it is better to focus on the artifacts that we created ourselves and pull in the dependencies when we check out the application for the first time. We will create the configuration file now. We could do this by hand or let Bower do it for us. Let's do the latter and then examine the results: # bower init This will start a dialogue that guides us through the initial Bower project setup process. Give the application a name, version number, and description as you desire. The main file stays empty for now. In the module type selection, just press Enter on the first option. Whenever you see something in square brackets, for example, [MIT], this is the default value that will be used when you simply press Enter. When we confirm that the currently installed dependencies should be set as dependencies, AngularJS will automatically appear as a project dependency in the configuration file if you have followed the previous instructions. Finally, let's set the package as private. There is no need to have it accidentally appear in the Bower registry. Once we are done, the bower.json file should look roughly as follows: { name: 'Hello World', version: '0.0.0', homepage: 'https://github.com/matthiasn/AngularUI-Code', authors: ['Matthias Nehlsen <mn@nehlsen-edv.de>'], description: 'Fancy starter application', license: 'MIT', private: true, ignore: [    '**/.*',    'node_modules',    'bower_components',    'src/bower',    'test',    'tests' ], dependencies: {    angular: '~1.2.22' } } Now that we have a dependency management in place, we can use AngularJS from here and delete the manually downloaded version inside the js/vendor/ folder. Also, edit index.html to use the file from the bower/ directory. Find the following line of code:    <script src="js/vendor/angular.js"></script> Replace it with this:    <script src="bower/angular/angular.js"></script> Now, we have a package manager for client-side dependencies in place. We will use more of this in the next step. Building the application Preparing an application for production environments can be a tedious task. For example, it is highly recommended that you create a single file that contains all the JavaScript needed by the application instead of multiple small files, as this can dramatically reduce page load time, particularly on slow mobile connections with high latency. It is not feasible to do this by hand though, and it is definitely no fun. This is where a build system really shines. We are going to use Grunt for this. A single command in the Terminal will run the tests and then put the necessary files (with a single, larger JavaScript file) for the application in the dist/ folder. Many more tasks can be automated with Grunt, such as minifying the JavaScript, automating tasks by watching folders, running JsHint (http://www.jshint.com), but we can only cover a fairly basic setup here. Let's get started. We need to install Grunt first (possibly with sudo): # npm install –g grunt-cli Then, we create a file named package.json in the root of our application folder with the following content: { "name": "my-hello-world", "version": "0.1.0", "devDependencies": {    "grunt": "~0.4.5",    "grunt-contrib-concat": "~0.5.0",    "grunt-contrib-copy": "~0.5.0",    "grunt-targethtml": "~0.2.6",    "grunt-karma": "~0.8.3",    "karma-jasmine": "~0.1.5",    "karma-firefox-launcher": "~0.1.3",    "karma-chrome-launcher": "~0.1.4" } } This file defines the npm modules that our application depends on. These will be installed automatically by running npm install inside the root of our application folder. Next, we define which tasks Grunt should automate by creating Gruntfile.js, also in the root of our application folder, with the following content: module.exports = function(grunt) { grunt.initConfig({    pkg: grunt.file.readJSON('package.json'),    concat: {      options: {        separator: ';'      },      dist: {        src: ['src/js/vendor/*.js','src/js/*.js'],        dest: 'dist/js/<%= pkg.name %>.js'      }    },    copy: {      main: {        src: 'src/css/main.css',        dest: 'dist/css/main.css',      },    },    targethtml: {      dist: {        files: {          'dist/index.html': 'src/index.html'        }      }    },    karma: {      unit: {        configFile: 'conf/karma.conf.js',        singleRun: true      }    } }); grunt.loadNpmTasks('grunt-contrib-concat'); grunt.loadNpmTasks('grunt-contrib-copy'); grunt.loadNpmTasks('grunt-targethtml'); grunt.loadNpmTasks('grunt-karma'); grunt.registerTask('dist', ['karma', 'concat', 'targethtml', 'copy']); }; This file contains different sections that are of interest. In the first part, the configuration object is created, which defines the options for the individual modules. For example, the concat section defines that a semicolon should be used as a separator when it merges all JavaScript files into a single file inside the dist/js folder, with the name of the application. It is important that angular.js come before the application code inside this file, which is guaranteed by the order inside the src array. Files in the vendor subfolder are processed first with this order. The copy task configuration is straightforward; here, we only copy a single CSS file into the dist/css folder. We will do more interesting things with CSS later when talking about CSS frameworks. The targethtml task processes the HTML so that it only loads the one concatenated JavaScript file. For this to work, we need to modify index.html, as follows: <!DOCTYPE html> <head>    <meta charset="utf-8">    <title>Angular UI Template</title>    <link rel="stylesheet" href="css/main.css"> </head> <body data-ng-app="myApp">    <div data-ng-controller="helloWorldCtrl">      <div hello-world name="name"></div>    </div>    <!--(if target dev)><!-->    <script src="js/vendor/angular.js"></script>    <script src="js/app.js"></script>    <script src="js/controllers.js"></script>    <script src="js/directives.js"></script>      <!--<!(endif)-->    <!--(if target dist)><!-->    <script src="js/my-hello-world.js"></script>    <!--<!(endif)--> </body> </html> This, together with the configuration, tells the targethtml task to only leave the section for the dist task inside the HTML file, effectively removing the section that will load individual files. One might be tempted to think that it will not make much of a difference if one or multiple files need to be retrieved when the page is loaded. After all, the simple concatenation step does not reduce the overall size of what needs to be loaded. However, particularly on mobile networks, it makes a huge difference because of the latency of the network. When it takes 300 ms to get a single response, it soon becomes noticeable whether one or ten files need to be loaded. This is still true even when you get the maximum speed in 3G networks. LTE significantly reduces latency, so the difference is not quite as noticeable. The improvements with LTE only occur in ideal conditions, so it is best not to count on them. The karma section does nothing more than tell the karma task where to find the previously defined configuration file and that we want a single test run for now. Next, we tell Grunt to load the modules for which we have created the configuration, and then we define the dist task, consisting of all the previously described tasks. All that remains to be done is to run grunt dist in the command line when we want to test and build the application. The complete AngularJS web application can then be found in the dist/ folder. Running Protractor from Grunt Let's also run Protractor from our grunt task. First, we need to install it as follows: # npm install grunt-protractor-runner --save-dev This will not only install the grunt-protractor-runner module, but also add it as a dependency to package.json so that when you, for example, check out your application from your version control system (covered next) on a new computer and you want to install all your project's dependencies, you can simply run: # npm install If you follow along using the companion source code instead of typing the source code yourself, you will need to run npm install again in the last step's folder. Next, edit Gruntfile.js so that from the karma section, it looks as follows:    karma: {      unit: {        configFile: 'conf/karma.conf.js',        singleRun: true      }    },    protractor: {      e2e: {        options: {         configFile: 'conf/protractor.conf.js'        }      }    } }); grunt.loadNpmTasks('grunt-contrib-concat'); grunt.loadNpmTasks('grunt-contrib-copy'); grunt.loadNpmTasks('grunt-targethtml'); grunt.loadNpmTasks('grunt-karma'); grunt.loadNpmTasks('grunt-protractor-runner'); grunt.registerTask('dist', ['karma', 'protractor', 'concat', 'targethtml', 'copy']); }; Now, Protractor will also run every single time we call grunt dist to build our application. The build process will be stopped when either the karma or the protractor step reports an error, keeping us from ever finding code in the dist folder that fails tests. Note that we will need to have both the webdriver-manager and http-server modules running in separate windows for the grunt dist task to work. As a little refresher, these were started as follows: # webdriver-manager start # http-server –a localhost –p 8000 Both can also be managed by Grunt, but that makes the configuration more complex and it would also mean that the task runs longer because of startup times. These can also be part of a complex configuration that watches folders and runs and spawns all the required tasks automatically. Explore the Grunt documentation further for tailoring an environment specific to your exact needs. Now that you know how the build system works in general, you may already want to explore more advanced features. The project's website is a good place to start (http://gruntjs.com). Managing the source code with Git You are probably using Git already. If not, you really should. Git is a distributed Version Control System (VCS). I cannot imagine working without it, neither in a team nor when working on a project myself. We don't have the space to cover Git for team development here in either case; this topic can easily fill an entire book. However, if you are working in a team that uses Git, you will probably know how to use it already. What we can do in is go through the basics for a single developer. First, you need to install Git if you do not have it on your system yet. OS X On a Mac, again the easiest way to do this is using Homebrew (http://brew.sh). Run the following command in the Terminal after installing Homebrew: # brew install git Windows On Windows, the easiest way to install Git is to run the Windows installer from http://git-scm.com/downloads. Linux (Ubuntu) On Ubuntu, run the following command in the shell: # sudo apt-get install git Let's initialize a fresh repository in the current directory: git init Then, we create a hidden file named .gitignore, for now with only the following content: node_modules This tells Git to ignore the hundreds of files and directories in the node_modules folder. These don't need to be stored in our version control system because the modules can be restored by running npm install in the root folder of the application instead, as all the dependencies are defined in the package.json file. Next, we add all files in the current directory (and in all subdirectories): git add Next, we freeze the file system in its current state: git commit –m "initial commit" Now, we can edit any file, try things out, and accidentally break things without worry because we can always come back to anything that was committed into the VCS. This adds incredible peace of mind when you are playing around with the code. When you issue the git status command, you will notice that we are on a branch called master. A project can have multiple branches at the same time; these are really useful when working on a new feature. We should always keep master as the latest stable version; additional features (or bug fixes) should always be worked upon in a separate branch. Let's create a new feature branch called additional-feature: git branch additional-feature Once it is created, we need to check out the branch: git checkout additional-feature Now, when the new code is ready to be committed, the process is the same as above: git add . git commit –m "additional feature added" We should commit early and often, this habit will make it much easier to undo previous changes when things go wrong. Now, when everything is working in the new branch (all the tests pass), we can go back into the master branch: git checkout master Then, we can merge the changes back into the master branch: git merge additional-feature Being able to freely change between branches, for example, makes it very easy to go back to the master branch from whatever you are working on and do a quick bug fix (in a specialized branch, ideally) without having to think about what you just broke with the current changes in the new feature branch. Please don't forget to commit before you switch the branch though. You can merge the bug fix in this example back into the master, go back to the feature branch you were working on, and even pull those changes that were just done in the master branch into the branch you are working on. For this, when you are inside the feature branch, merge the changes: git merge master If these changes were in different areas of your files, this should run without conflicts. If they were in the same lines, you will need to manually resolve the conflicts. Using Git is really useful to manage source code. However, it is in no way limited to code files (or text files, for that matter). Any file can be placed under the source control. For example, this book was written with the heavy usage of Git, for any file involved. This is extremely useful when you are trying to go back to the previous versions of any file. Building AngularUI Utils With what we learned about package managing, let's build AngularUI-Utils ourselves before we start using it. We could just download the JavaScript file(s), but it will be more gratifying to do this ourselves. Learning how to use Grunt will also be very helpful in any larger project later on. For this, first of all, either clone or fork the repository or just download the zip file from https://github.com/angular-ui/ui-utils. For simplicity, I suggest that you download the zip file; you can find the link on the right-hand side of the GitHub project page. Once we have unpacked the zip file, we first need to install the dependencies. On your command line inside the project folder, run the following commands: $ npm install $ bower install This will install the necessary files needed for the build process. Let's first check whether all the tests are passing after the previous commands have run through: $ karma start --browsers=Chrome test/karma.conf.js --single-run=true Alternatively, you can also simply run: $ grunt The tests are part of the default task specified in gruntFile.js. Take a moment and familiarize yourself with the file by trying to find where the default task is specified. Note that one subtask is the karma:unit task. Try to locate this task further down in the file; it specifies which Karma configuration file to load. If all the tests pass, as they should, we can then build the suite using the following command: $ grunt build This will run the following tasks specified in gruntFile.js: The concat:tmp task concatenates the modules into a temporary JavaScript file, except modules/utils.js. Take a look at the configuration for this task; the easiest way is to search for concat within gruntFile.js. The concat:modules task concatenates the resulting temporary file from step one into the final JavaScript library file, which you can then find in the bower_components/angular-ui-docs/build directory. The configuration for the concat:modules task should be right below the previous one. Here, the difference is that there is no absolute file and path; instead, the name is resolved so that common parts, such as the repository name and such, are not repeated within the configuration file. This follows the DRY (don't repeat yourself) principle and makes the configuration file easier to maintain. The clean:rm_tmp task removes the temporary file, created previously. The uglify task finally creates a minified version of the JavaScript file for use in production because of the smaller file size. I highly recommend that you spend some time reading and following through the gruntFile.js file and the tasks specified therein. It is not strictly necessary that you follow along in this article, as simply building the suite (or even downloading it from the project website) would suffice, but knowing more about the Grunt build system will always be helpful for our own projects. Imagine this: you find an issue in some project and add it to the list of known issues on the GitHub project. Someone picks it up immediately and fixes it. Now, do you want to wait for someone (maybe an automated build system) to decide when it is a good time to publish a version that includes the said fix? I wouldn't; I'd much rather be able to build it myself. You never know when the next release will happen. Integrating AngularUI-Utils into our project In this step, we will take the ui-utils.js file we built in the previous section and use it in a sample project. The UI-Utils suite consists of a large and growing number of individual components. We will not be able to cover all of them here, but the one's we do cover should give you a good idea about what is available. Now, let's do the following edits: Copy the ui-utils.js file from the ui-utils/bower_components/angular-ui-docs/build/ folder to the src/js/vendor/ folder of the current project. Open the package.json file and change the name of the application as you wish, for example: { "name": "fun-with-ui-utils", "version": "0.1.0", "devDependencies": {    "grunt": "~0.4.1",    "grunt-contrib-concat": "~0.3.0",    "grunt-contrib-copy": "~0.4.1",    "grunt-targethtml": "~0.2.6",    "grunt-karma": "~0.6.2" } } Edit src/index.html by inserting a script tag right below the script tag for the angular.js file, and change the name of our concatenated project JavaScript file, as the name change above will result in a different filename. The body of the index.html file now looks as follows: <body ng-app="myApp"> <div ng-controller="helloWorldCtrl">    <h1 hello-world name="name" id="greeting"></h1> </div> <!--(if target dev)><!--> <script src="bower/angular/angular.js"></script> <script src="js/vendor/ui-utils.js"></script> <script src="js/app.js"></script> <script src="js/controllers.js"></script> <script src="js/directives.js"></script>  <!--<!(endif)--> <!--(if target dist)><!-->      <script src="js/fun-with-ui-utils.js"></script> <!--<!(endif)--> </body> Run grunt dist to see if our tests are still passing and if the project gets built without problems. Dist folder You will notice that the my-hello-world.js file is still in the dist/js/ folder, despite not being used any longer. You can safely remove it. You could also remove the entire dist folder and run the my-hello-world.js file again: $ grunt dist This will recreate the folder with only the require files. Deleting the folder before recreating it could become a part of the dist task by adding a clean task that runs first. Check out gruntFile.js of the UI-Utils project if you want to know how this is done. Grunt task concat Note that all JavaScript files get concatenated into one file during the concat task that runs in our project during the grunt dist task. These files need to be in the correct order, as the browser will read the file from beginning to end and it will complain when, for example, something references the AngularJS namespace without that being loaded already. So, while it might be tempting to use wildcards as we did so far for simplicity, we shall name individual files in the correct order. This might seem tedious at first, but once you are in the habit of doing it for each file the moment you create or add it, it will only take a few seconds for each file and will keep you from scratching your head later. Let's fix this right away. Find the source property of the concat task in Gruntfile.js of our project:        src: ['src/js/vendor/*.js','src/js/*.js'], Now, replace it with the following:        src: ['src/bower/angular/angular.js',              'src/js/vendor/ui-utils.js',              'src/js/app.js',              'src/js/controllers.js',              'src/js/directives.js'], We also need to edit the app module so that it loads UI-Utils. For this, edit app.js as follows: 'use strict'; angular.module('myApp', ['myApp.controllers', 'myApp.directives', 'ui.utils']) With these changes in place, we are in a pretty good shape to try out the different components in the UI-Utils suite. We will use a single project for all the different components; this will save us time by not having to set up separate projects for every single one of them. Summary In this article, we have learned about managing client-side dependencies with Bower. We also learned about building an application, running Protractor from Grunt, and managing the source code with Git. Then, later we learned about building AngularUI Utils and integrating AngularUI-Utils into our project. Resources for Article: Further resources on this subject: AngularJS Project [article] AngularJS [article] Working with Live Data and AngularJS [article]
Read more
  • 0
  • 0
  • 1443
Modal Close icon
Modal Close icon